Past my knowledge level using SUBMIT - Any help would be appreciated

Everything about buying, using, and managing OpenVMS systems not covered by other sections.

Topic author
mpiazza
Newbie
Posts: 3
Joined: Thu Apr 20, 2023 9:00 pm
Reputation: 0
Status: Offline

Past my knowledge level using SUBMIT - Any help would be appreciated

Post by mpiazza » Thu Apr 20, 2023 9:03 pm

We are currently running an older agent version of Data Protector which is providing backup capability for our aging VMS and OpenVMS systems. This old version of Data Protector ties us to more EOL equipment in the form of LT04/LT05 storage tape juke boxes. The enterprise has moved on to Veeam for the Windows and Linux systems to take advantage of intermediate disc caching but there is no Veeam support for VMS. In order for us to continue providing a backup solution we need to somehow migrate our VMS backups to the enterprise Veeam solution. Our solution was to fall back to using the built-in BACKUP capabilities native to VMS. We would then provide an intermediate Windows storage server where our VMS created save sets could be transferred and staged for Veeam to record and store them. To solve the problem of maintaining the VMS file attributes we added in a third-party solution called KnapSac that provides a fast backup utility combining file compression and file handing operations to enable storage of VMS file objects as pacsets. These pacsets provide a wrapper around the VMS files, in this case both image and non-image save sets, maintaining the VMS file attributes which are ultimately unaffected by Windows or Veeam’s Linux NFS file transfer and manipulations.

We run a DCL script in our SYS$BATCH queue called VMS_BACKUP_SCHEDULER which emulates the backup schedule that is running in Data Protector. After some initialization it resubmits itself successfully to the batch queue with the following command:

Code: Select all

$ SUBMIT/QUEUE=SYS$BATCH/AFTER="TOMORROW00:00:10"/NOPRINTER/LOG=$1$DGA322: - 
[VEEAM_VMS_BACKUPS.LOGS]'DATE'_'THISNODE'_VMS_BACKUP_SCHEDULER.LOG - 
$1$DGA322:[VEEAM_VMS_BACKUPS]VMS_BACKUP_SCHEDULER.COM
As you can see in the following snippet, the VMS_BACKUP_SCHEDULER.COM script is submitting the actual selected backup script job to the SYS$BATCH queue exactly as the
main SCHEDULER script has done with the system echoing the successful status of the job submittal.

Code: Select all

$ DISPLAY "SUBMITTING WEEKLY_SATSS1_INCR_BACKUP ON ''DATE'" -
SUBMITTING WEEKLY_SATSS1_INCR_BACKUP ON 19-APR-2023 00:00:04.47
$ SUBMIT/QUEUE=SYS$BATCH/AFTER="+3"/NOPRINTER/LOG=$1$DGA322:[VEEAM_VMS_BACKUPS.LOGS]'DATE'_'THISNODE'_WEEKLY_INCR_BACKUP.LOG $1$DGA322:[VEEAM_VMS_BACKUPS]WEEKLY_SATSS1_INCR_BACKUP.COM
Job WEEKLY_SATSS1_INCR_BACKUP (queue SATSS1$BATCH, entry 402) holding until 19-APR-2023 03:00
When the system eventually executes the batch job nothing happens. There is no output log or any indication that the job executed except that it is no longer in the batch queue.

Looking for some kind of error I ran the following command:

Code: Select all

ANALYZE/AUDIT/FULL/NOINTERACTIVE SYS$MANAGER:SECURITY.AUDIT$JOURNAL
I found the following single relevant entry in the security audit log:

Code: Select all

                  Security Audit Analysis Utility

Security alarm (SECURITY) and security audit (SECURITY) on SATSS1, system id: 31
Auditable event:	Batch process login failure
Event time:		19-APR-2023 03:00:04.59
PID:			20D4F2D0
Process name:		BATCH 402
Username:		SYSTEM
Process owner:		[SYSTEM]
Image name:		SATSS1SDKAO:[SYSO.SYSCOMMON.][SYSEXE]LOGINOUT.EXE
Posix UID:		-2
Posix GID:		-2 (%XFFFFFFFE)
Status:			%RMS-E-CRE, ACP file create failed
It shows a batch process login error.

This is what makes no sense and has me completely stymied. Both the Scheduling script job and the backup script job submitted from the scheduler are both running as SYSTEM processes. At least that has always been my understanding of how things work. Obviously I’m missing something because the login is failing but I don’t understand why. The thing that is confusing me is the “%RMS-E-CRE, ACP file create failure” statement in the Audit Log. What is it trying to create that it can’t. Is it complaining because it is unable to create the LOG file? If so why doesn't it say that? I mean seriously, a security person looking through the log would like to know something like that I would think. Other than the Audit Log failure record shown above there is nothing to indicate the process ever existed. The error message “%RMS-E-CRE, ACP file create failure” is one step above useless! This has always been one of my complaints about VMS.

I have looked in all my books and all the VMS manuals I can find. I’ve searched online rephrasing the query as many different ways that I can think of and I can come up with no explanation of exactly what this is telling me with any relevance to my current issue. I thought maybe a password failure and when I checked I did find that the SYSTEM password had expired so I set the new one but that has had no effect. I’ve included the SYSTEM account info and file directory permissions info below as I thought maybe it was telling me it couldn’t open the log file due to permissions or something but you can see that as a SYSTEM level process it should have all the access it needs.

Code: Select all

UAF> sho system
Username: SYSTEM                        Owner: SYSTEM MANAGER #System
Account: LMCO                           UIC:	[1,4] ([SYSTEM])
CLI:	DCL                             Tables: DCLTABLES
Default: SYS$SYSROOT:[SYSMGR]
LGICMD:
Flags: PwdMix
Primary days:	Mon Tue Wed Thu Fri
Secondary days:	Sat Sun
No access restrictions
Expiration:	(none)	Pwdminimum: 14	Login Fails:	0
Pwdlifetime:	60 00:00	Pwdchange: 17-APR-2023 17:08 5-MAR-1999 12:02
Last Login: 17-APR-2023 17:08 (interactive), 19-APR-2023 14:25 (non-interactive)
Maxjobs:	0 Fillm:	5000 Bytlm:	250000
Maxacctjobs:	0 Shrfillm:	   0 Pbytlm:	     0
Maxdetach: 	0 BIOlm: 	2500 3Tquota:     8192
Prclm: 	    36372 DIOlm: 	8000 WSdef:       8192
Prio: 		4 AST1m: 	4096 WSquo:      64000
Queprio: 	0 TQElm: 	 200 WSextent:   32768
CPU: 	   (none) Enqlm: 	2000 Pgflquo:  7000000

Authorized Privileges:
  ACNT	       ALLSPOOL	    ALTPRI      AUDIT     BUGCHK    BYPASS 
  CMEXEC       CMKRNL	    DIAGNOSE    DOWNGRADE EXQUOTA   GROUP
  GRPNAM       GRPPRV	    IMPERSONATE IMPORT    LOG_IO    MOUNT
  NETMBX       OPER	    PFNMAP      PHY_IO    PRMCEB    PRMGBL
  PRMMBX       PSWAPM	    READALL     SECURITY  SETPRV    SHARE
  SHMEM	       SYSGBL	    SYSLCK      SYSNAM    SYSPRV    TMPMBX
  UPGRADE      VOLPRO 	    WORLD
Default Privileges:
  ACNT	       ALLSPOOL	    ALTPRI      AUDIT     BUGCHK    BYPASS 
  CMEXEC       CMKRNL	    DIAGNOSE    DOWNGRADE EXQUOTA   GROUP
  GRPNAM       GRPPRV	    IMPERSONATE IMPORT    LOG_IO    MOUNT
  NETMBX       OPER	    PFNMAP      PHY_IO    PRMCEB    PRMGBL
  PRMMBX       PSWAPM	    READALL     SECURITY  SETPRV    SHARE
  SHMEM	       SYSGBL	    SYSLCK      SYSNAM    SYSPRV    TMPMBX
  UPGRADE      VOLPRO	    WORLD	
Identifier

SATSS1> dir/full WEEKLY_SATSS1_INCR_BACKUP.COM; 

Directory $1$DGA322:[000000.VEEAM_VMS_BACKUPS]

WEEKLY _SATSS1_INCR_BACKUP.COM;5 File ID:	(3820,313,0)
Size:            6/200	      Owner:     [SYSTEM]
Created:    13-APR-2023 17:47:36.67
Revised:    15-APR-2023 00:16:31.74 (4)
Expires:    16-APR-2023 00:16:31.74
Backup:     14-APR-2023 11:11:59.47
Effective:  <None specified>
Recording:  <None specified>
Accessed:   <None specified>
Attributes: <None specified>
Modified:   <None specified>
Linkcount:  1
File organization:  Sequential
Shelved state:	    Online
Caching attribute:  Writethrough
File attributes:    Allocation: 200, Extend: 0, Global buffer count: 0
		    No version limit
Record format:      Variable length, maximum 255 bytes, longest 85 bytes
Record attributes:  Carriage return carriage control
RMS attributes:     None
Journaling enabled: None
File protection:    System:RWED, Owner:RWED, Group:RE, World:
Access Cntrl List:  None
Client attributes:  None
 
Total of 1 file, 6/200 blocks.


SATSS1> sho que sys$batch/full
Batch queue SATSS1$BATCH, idle, on SATSS1::
  /BASE_PRIORITY=3 /CPUMAXIMUM=INFINITE /JOB_LIMIT=24 /OWNER=[SYSTEM]
  /PROTECTION=(S:M,O:D,G:R,W:S) /WSQUOTA=1000

  Entry Jobname          Username             Status	
  -----  ------          --------             ------	
    401  VMS_BACKUP_SCHEDULER
                         SYSTEM               Holding until 20-APR-2023 00:00:00
         Submitted 19-APR-2023 00:00:04.47 /KEEP 
         /LOG=$1$DGA322:[VEEAM_VMS_BACKUPS.LOGS]19-APR-2023_SATSS1_VMS_BACKUP_SC 
         /NOPRINT /PRIORITY=3
         File: _$1$DGA322:[VEEAM_VMS_BACKUPS]VMS_BACKUP_SCHEDULER.COM;10

    403  DAILY_WS        SYSTEM               Holding until 20-APR-2023 00:00:00
         Submitted 19-APR-2023 00:00:04.66 /KEEP
         /L°G=SATSSISDKAO:[SYS0.][sYsMGR]DAILY_Ws.LOG; /PARAM=("DAILY_WS", 
         "00:00","SYS$BATCH","8") /NOPRINT /PRIORITY=3
         File: _$1$DGA300:[VMS$COMMON.MGRSW.OPSJOBS]SUBMIT_BATCH_30B.COM;123
Added in 7 minutes 17 seconds:
There are some typos I missed from the scanning/OCR process. Some $ signs in the device names were OCR'd as S's and I didn't notice until I had posted the text.
Last edited by marty.stu on Thu Apr 27, 2023 4:07 pm, edited 1 time in total.

User avatar

volkerhalle
Master
Posts: 196
Joined: Fri Aug 14, 2020 11:31 am
Reputation: 0
Status: Offline

Re: Past my knowledge level using SUBMIT - Any help would be appreciated

Post by volkerhalle » Fri Apr 21, 2023 8:18 am

Bob,

I would expect, that you can submit a job into a queue, which is running on a different cluster node and put the log file on a disk, which is not mounted on the node, where you issue the submit command. So there may be no way for SUBMIT to determine, whether the log file name is valid.

So there is no way to determine the validity of the log file name until the time, when the batch process is being created. And at least OpenVMS reports the error status (in Accounting, Audit log or via a broadcast notification message).

Volker.
Last edited by volkerhalle on Fri Apr 21, 2023 8:21 am, edited 1 time in total.


bobwilson
VSI Expert
Contributor
Posts: 15
Joined: Sat Sep 11, 2021 10:24 pm
Reputation: 0
Status: Offline

Re: Past my knowledge level using SUBMIT - Any help would be appreciated

Post by bobwilson » Fri Apr 21, 2023 8:24 am

I tried a ODS-5 disk and got some other, um, unexpected behavior:

Code: Select all

$ x = f$fao("!236*a")   ! a bunch of a's
$ submit sys$login:login.com /log=ddcu:[dir]'x' /hold /notify /retain=always
(no issues SUBMITing the job)

The /LOG in SHOW ENTRY looked like (hmmm):

Code: Select all

 /LOG=_$1$DGA9128:[SYSBLD]AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA~[0,0,0].LOG;
Releasing the job gave:

Code: Select all

Job LOGIN (queue <node>_BATCH, entry 118) terminated with error status
%RMS-F-SYN, file specification syntax error
Added in 20 minutes 5 seconds:
re: I would expect, that you can submit a job into a queue, which is running on a different cluster node and put the log file on a disk, which is not mounted on the node, where you issue the submit command. So there may be no way for SUBMIT to determine, whether the log file name is valid

Yes, I thought of that later, but it would still be interesting to know the exact file name of the log file of the job being submitted [in the original post] and the structure level of the disk that the log file was to be created on.

And, there *is* some amount of log file validation that SUBMIT does:

Code: Select all

$ submit /log=foo:[000000]x.log sys$login:login.com
%SUBMIT-F-INVLOGFIL, invalid log file specification
-RMS-F-DEV, error in device name or inappropriate device type for operation
$


Topic author
mpiazza
Newbie
Posts: 3
Joined: Thu Apr 20, 2023 9:00 pm
Reputation: 0
Status: Offline

Re: Past my knowledge level using SUBMIT - Any help would be appreciated

Post by mpiazza » Fri Apr 21, 2023 5:10 pm

A little voice in the back of my head kept telling me "look at the log file name length..." and after posting another entry to this thread yesterday I did just that. I reduced the log filename length in the command in the script in question by 7 characters and deleted the job from the queue and resubmitted it.

The job in question was scheduled to run again this morning and when I came into work I found that it had successfully started and, in fact, was still running as it backs up about 20 disks as separate save sets/.pac files.

So Volker and Bob, you were correct in your suggestions. My mistake was assuming I was dealing with ODS-5 filename lengths when, in fact, I am dealing with ODS-2. I didn't set these systems up and I've been away from VMS for a little over a decade. I can't believe how much I've forgotten in that time as I was only brought in to deal with them within the last year or so.

Thank you all for taking the time to help and your suggestions! I'm extracting all the suggestions and putting them into my personal "How To".

Cheers! Thanks again.

User avatar

volkerhalle
Master
Posts: 196
Joined: Fri Aug 14, 2020 11:31 am
Reputation: 0
Status: Offline

Re: Past my knowledge level using SUBMIT - Any help would be appreciated

Post by volkerhalle » Sat Apr 22, 2023 12:46 am

bobwilson wrote:
Fri Apr 21, 2023 8:44 am
And, there *is* some amount of log file validation that SUBMIT does:

$ submit /log=foo:[000000]x.log sys$login:login.com
%SUBMIT-F-INVLOGFIL, invalid log file specification
-RMS-F-DEV, error in device name or inappropriate device type for operation
$
Bob,

you're right. SUBMIT checks for the availability and mount status of the log file device and for the directory name, but it does not (can not ?) verify, if the given file name would be a legal file name on that device and if the user could actually create a log file in that directory.

Volker.

Post Reply