edit/sum (sumslp) crash for stream-lf input file

[malmberg]

This showed up as part of the unit tests validating the port of the gawk package of OpenVMS.

Demonstration script:

Code: Select all

$! sum_streamlf_test.com

$create stream_lf.txt
$set file stream_lf.txt /attr=(rfm:stmlf,lrl:32767,mrs:0,rat:cr)
$open/write slf stream_lf.txt
$write slf "line 1"
$write slf "line 2"
$write slf "line 3"
$close slf

edit/sum stream_lf.txt /update=sys$input: /output=stream_lf.out
-1,2
/
$ diff stream_lf.txt stream_lf.out

Running Script:

Code: Select all

ROBIN> set ver
ROBIN> @sum_streamlf_test.com
$! sum_streamlf_test.com

$create stream_lf.txt
$set file stream_lf.txt /attr=(rfm:stmlf,lrl:32767,mrs:0,rat:cr)
$open/write slf stream_lf.txt
$write slf "line 1"
$write slf "line 2"
$write slf "line 3"
$close slf

edit/sum stream_lf.txt /update=sys$input: /output=stream_lf.out
%SYSTEM-F-ACCVIO, access violation, reason mask=05, virtual address=FFFFFFFF8000
6695, PC=FFFF830007C0DD1A, PS=0000001B

  Improperly handled condition, image exit forced.
    Signal arguments:   Number = 0000000000000005
                        Name   = 000000000000000C
                                 0000000000000005
                                 FFFFFFFF80006695
                                 FFFF830007C0DD1A
                                 000000000000001B
    Register dump:
    RAX = FFFFFFFF80006695  RDI = FFFFFFFF80006695  RSI = 0000000000000005
    RDX = 000000000000002E  RCX = 0000000000000000  R8  = 000000007AC9F3A0
    R9  = 0000000000000000  RBX = 000000007AC9FF58  RBP = 000000007AC9F390
    R10 = 0000000000000010  R11 = 0000000000000206  R12 = 000000007AC9F598
    R13 = 0000000000000000  R14 = 000000007FFCDA98  R15 = 000000012B274603
    RIP = FFFF830007C0DD1A  RSP = 000000007AC9F368  SS  = 000000000000001B
%DCL-W-SKPDAT, image data (records not beginning with "$") ignored

[arne_v]

What VMS version(s) have the problem and which do not?

[malmberg]

Seen on OpenVMS 9.2-2 x86_64 community edition.

Not seen on my AXP or IA64 systems that are still at an 8.4 version.

[arne_v]

Hmm.

On VMS x86-64 9.2-3 I see:

$! sum_streamlf_test.com
$create stream_lf.txt
$set file stream_lf.txt /attr=(rfm:stmlf,lrl:32767,mrs:0,rat:cr)
$open/write slf stream_lf.txt
$write slf "line 1"
$write slf "line 2"
$write slf "line 3"
$close slf
$ edit/sum stream_lf.txt /update=sys$input: /output=stream_lf.out
-1,2
/
-1,2
-1,2
/
$ diff stream_lf.txt stream_lf.out
************
File DKA0:[arne]stream_lf.txt;3
1 line 1
2 line 2
3 line 3
******
File DKA0:[arne]stream_lf.out;2
1 line 3
************

Number of difference sections found: 1
Number of difference records found: 2

DIFFERENCES /MERGED=1-
DKA0:[arne]stream_lf.txt;3-
DKA0:[arne]stream_lf.out;2

Added in 1 minute 20 seconds:
Which of course raise the question whether it is a 9.2-2 vs 9.2-3 difference. Or if it is a your system vs my system difference.

Anyone with 9.2-2 that can check if they can recreate?

[m_detommaso]

Confirmed, the problem is easily replicated only on VMS x86 V9.2-2; with VMS x86 V9.2-3 everything works correctly

Code: Select all

system_rieman_opa0 >product sho product vms/full
------------------------------------ ----------- --------- ------------------------------------ ------------------------------------
PRODUCT                              KIT TYPE    STATE     MAINTENANCE                          REFERENCED BY
------------------------------------ ----------- --------- ------------------------------------ ------------------------------------
VSI X86VMS VMS V9.2-2                Oper System Installed VSI X86VMS VMS922X_DEBUG V2.0        VSI X86VMS DWMOTIF V1.8
                                                           VSI X86VMS VMS922X_DEBUG V1.0        VSI X86VMS KERBEROS V3.3-2A
                                                           VSI X86VMS VMS922X_PCSI V1.0         VSI X86VMS OPENVMS V9.2-2
                                                           VSI X86VMS VMS922X_UPDATE V2.0       VSI X86VMS TCPIP V6.0-26
                                                           VSI X86VMS VMS922X_UPDATE V1.0
------------------------------------ ----------- --------- ------------------------------------ ------------------------------------
1 item found

system_rieman_opa0 >type sum_streamlf_test.com
$! SUM_STREAMLF_TEST.COM
$! https://forum.vmssoftware.com/viewtopic.php?f=42&t=9342&sid=7fbda86652c3ce0ebb51f61223031942
$!
$ create stream_lf.txt
$ set file stream_lf.txt /attr=(rfm:stmlf,lrl:32767,mrs:0,rat:cr)
$ open/write slf stream_lf.txt
$ write slf "line 1"
$ write slf "line 2"
$ write slf "line 3"
$ close slf
$ exit

system_rieman_opa0 >@sum_streamlf_test
system_rieman_opa0 >dir

Directory DISK$RIEMAN_DUMP:[dcl_procedures.malmberg_edit_issue]

stream_lf.txt;2     stream_lf.txt;1     sum_streamlf_test.com;1

Total of 3 files.
system_rieman_opa0 >edit/sum stream_lf.txt /update=sys$input: /output=stream_lf.out
%SYSTEM-F-ACCVIO, access violation, reason mask=05, virtual address=FFFFFFFF80006695, PC=FFFF83000780DD1A, PS=0000001B

  Improperly handled condition, image exit forced.
    Signal arguments:   Number = 0000000000000005
                        Name   = 000000000000000C
                                 0000000000000005
                                 FFFFFFFF80006695
                                 FFFF83000780DD1A
                                 000000000000001B
    Register dump:
    RAX = FFFFFFFF80006695  RDI = FFFFFFFF80006695  RSI = 0000000000000005
    RDX = 000000000000002E  RCX = 0000000000000000  R8  = 000000007AB493A0
    R9  = 0000000000000000  RBX = 000000007AB49F58  RBP = 000000007AB49390
    R10 = 0000000000000010  R11 = 0000000000000206  R12 = 000000007AB49598
    R13 = 0000000000000000  R14 = 000000007FFCDA98  R15 = 000000003D974603
    RIP = FFFF83000780DD1A  RSP = 000000007AB49368  SS  = 000000000000001B

Since vms x86 V9.2-2 is listed as being in standard support until December 31, 2027, this issue should be resolved by VSI engineering.

https://vmssoftware.com/about/roadmap/

[amuir]

Which of course raise the question whether it is a 9.2-2 vs 9.2-3 difference. Or if it is a your system vs my system difference.

Anyone with 9.2-2 that can check if they can recreate?

Well, I can confirm the issue occurs on my on 9.2-1 system, but not on my 9.2-3 system. So, I think that rules out the "his system" possibility, and strongly suggests that the issue was fixed in 9.2-3.

[arne_v]

I believe there are a lot of good fixes in the 9.2-2 -> 9.2-3 upgrade.

If possible then I will suggest going 9.2-3.

[dgordon]

EDIT/SUM was fixed for V9.2-3. The code has static data addresses in a PSECT marked EXE. This was not a problem on previous architectures, but can be on x86. I suspect that if you test it on prior versions, EDIT/SUM with any file type will crash with an ACCVIO.

You should be able to pull SYS$SYSTEM:SUMSLP.EXE from any V9.2-3 system and use it on previous versions. If you really need a fix for earlier versions, you'll have to go through support if that's an option for you.

[hein]

Code: Select all

$! sum_streamlf_test.com
$create stream_lf.txt
$set file stream_lf.txt /attr=(rfm:stmlf,lrl:32767,mrs:0,rat:cr)
$open/write slf stream_lf.txt

This is NOT a STREAM_LF thing if the above is the actual reproducer.
I'm not arguing whether there is an issue with EDIT/SUM.
However, the test file as created below is a regular DCL: VFC, 2 byte header, maximum 0 bytes

Two things are incorrect.

1 - Using SET FILE as shown only works on EMPTY file, which was the case here, but do not get lured into believing that's the right way to change a file from standard VAR to STREAM_LF. Any existing record in there will create a mess.

2 - The OPEN/WRITE will create a fresh standard DCL file. You need OPEN/WRITE/APPEND to use an existing file.

To create a STREAM_LF file I recommend:

Code: Select all

$create/fdl="record; format stream_lf; size 32767" stream_lf.txt

To make an existing file STREAM_LF I recommend:

Code: Select all

$convert/stat/fdl="record; format stream_lf; size 32767" x.txt stream_lf.txt

fwiw,
Hein.




'

[hb]

... if the above is the actual reproducer.

It seems, it is a reproducer for a problem in EDIT/SUM, which is fixed in V9.2-3. The problem as seen on older versions of VMS on x86 seems independent of the record format. The initial report on this is hidden in the blog, referenced in https://forum.vmssoftware.com/viewtopic ... 138#p23138. It was said, that the file was created by "gawk". It seems creating this "reproducer" was easier than showing/extracting the actual commands with "gawk" - which, as far as I know, is not yet available on x86.

Anyone who reproduced the problem or confirmed the fix, probably noticed the difference. If you create and modify the empty file as show, then a "gawk" redirection the output with ">stream_lf.txt" will create a new version of the file with stream_lf. Same if you use "define/user sys$output" to point to this file - and "gawk" without output redirection. (This may have been the original usage, the thread creator probably knows.) But a "define sys$output" and running "gawk" will create a file with vfc. At least that's what I see on Alpha. Confusing?

I assume someone who can change the title of this thread will adjust it.