Some backup questions

[mberryman]

I recently finished archiving roughly 24TB of data to tape (fodder for another discussion). My tape drive (and disks) are connected via fibre channel. The tape model is LTO7 which has a native transfer speed of up to 300 MB/sec and a native capacity of 6TB.

My Itanium system can write to tape a little faster than the Alpha but my initial attempt couldn't write much faster than about 80 MB/sec and consumed most of the CPU. Turning off CRC more than doubled the write speed to around 184 MB/sec while consuming a fraction of the CPU.

Question 1: does the CRC routine used by backup (I am assuming LIB$CRC) use hardware CRC instructions? If not, can it be changed to do so?

I was using a block size of 65024. Writing that block size at 184 MB/sec means writing just under 3000 blocks a second (assuming my math is correct). Running a loop of LIB$CRC that many times in a test program gave the following completion times:

DS15 @ 1GHz: 1.29 secs
RX2620 @ 1.4GHz: 1.42 secs
x86 @ 3.2GHz: 1.11 secs

So, if it takes longer to compute CRC than it takes to write the block out, it is not surprising that a backup with CRC outputs at less than 1/2 the speed of a backup without it. Hopefully, something can be done about this.

Question 2:
As mentioned, I was rolling out a 24TB volume set. When checking progress with control-T I noticed that the amount saved would wrap at 2TB would indicate that this variable is being truncated or masked to 32 bits on output. Any chance of changing this to 64 bits?

Question 3:
Backups of this size currently take multiple days. My x86 system is faster than my other systems in every way and I am hoping that a backup using it will come closer to the 300 MB/sec speed the tape drive is capable of than the systems that currently have SAN tape support. How soon can we expect to see SAN tape support on x86?

Thank you.

[arne_v]

re HW)

VAX had a CRC instruction.

I am pretty sure Alpha did not. And I assume Itanium did not either.

x86-64 has the _mm_crc32_* thingies, but I don't think they are VAX CRC compatible.

So to get HW support on VMS x86-64 we will probably need /NOCRC, /CRC=VAX and /CRC=x64 to be able to both read old backups and to use HW for future tapes.

[imiller]

Are you using /GROUP=0 ? ( this is the default on recent versions of VMS )
What process quotas do you have for the process running BACKUP?

Have you looked at the documentation section 'Setting Software Parameters for Efficient Backups' https://docs.vmssoftware.com/vsi-openvm ... /#PROC_SEC ?

[jonesd]

re HW)

VAX had a CRC instruction.

I am pretty sure Alpha did not. And I assume Itanium did not either.

x86-64 has the _mm_crc32_* thingies, but I don't think they are VAX CRC compatible.

So to get HW support on VMS x86-64 we will probably need /NOCRC, /CRC=VAX and /CRC=x64 to be able to both read old backups and to use HW for future tapes.

VAX has a CRC instruction, and the backup utility originally used it. Then they discovered that a computing CRC could be done on the VAX faster when implemented with the less complex instructions, so backup replaced using the CRC instruction with the alternate implementation. Alpha's have always computed the the CRC without a specialized instruction, so X86 could continue the same way.

Added in 25 minutes 38 seconds:

Are you using /GROUP=0 ? ( this is the default on recent versions of VMS )
What process quotas do you have for the process running BACKUP?

Have you looked at the documentation section 'Setting Software Parameters for Efficient Backups' https://docs.vmssoftware.com/vsi-openvm ... /#PROC_SEC ?

I just tried it on 9.2-1, the default block size it created the save set with was 10 (same as described in the documention you referenced). Back in the day, redundancy groups were great for tapes since it could recover from a bad spot on the tape (after several retries to read it). Savesets on disk, however, never got a benefit from them because RMS would refuse to read past a bad block and backup only used sequential reads.

[arne_v]

re HW)

VAX had a CRC instruction.

VAX has a CRC instruction, and the backup utility originally used it. Then they discovered that a computing CRC could be done on the VAX faster when implemented with the less complex instructions, so backup replaced using the CRC instruction with the alternate implementation.

That rings a bell. I remember hearing that.

But was it because they came up with a smart software algorithm that was faster than the algorithm used in micro-code to implement the hardware instruction or did the hardware instruction become emulated so it was a case of software being faster that having a trap/exception call software?