Building "The Halls of ZK" without BLISS

[jhamby]

So I just learned about a text adventure from 1985, "The Halls of ZK", which is written in VMS Pascal and was only available as a VAX binary until somewhat recently, when original sources were discovered and updated to generate MACRO-32 tables instead of VAX object files. The current source is on GitHub:

https://github.com/tesneddon/zk

I ran into one problem building it on x86: the lack of public availability of a BLISS compiler. There's exactly one source file written in BLISS (converted from MACRO-32, the version I would've preferred to have):

Code: Select all

        %TITLE 'IFC$RTL_MACRO - IFC Run-Time System code in BLISS'

MODULE IFC$RTL_BLISS(IDENT = 'X01.00-00'
%IF %BLISS(BLISS32V) %THEN
                     ,ADDRESSING_MODE(EXTERNAL = GENERAL)
%FI
                    ) =
BEGIN
!
! Edit History:
! 13-Sep-2009  TES   Converted the MACRO-32 module to BLISS.
!
!

GLOBAL ROUTINE IFC$MESSAGE : NOVALUE =
BEGIN
    EXTERNAL ROUTINE
        IFC$MESSAGE_LIST : NOVALUE;

    BUILTIN
        ARGPTR;

    IFC$MESSAGE_LIST(ARGPTR());
END;

GLOBAL ROUTINE IFC$MESSAGE_INDENT : NOVALUE =
BEGIN
    EXTERNAL ROUTINE
        IFC$MESSAGE_INDENT_LIST : NOVALUE;

    BUILTIN
        ARGPTR;

    IFC$MESSAGE_INDENT_LIST(ARGPTR());
END;

END
ELUDOM

The only thing the wrapper routines are doing is bypassing Pascal's argument type checking so that the Pascal callers can pass a variable number of arguments, printf()-style, and the Pascal implementation sees a list of unsigned ints with the first one being the parameter count. Here's how the BLISS routines are defined to Pascal, in ifc/ifcrtl_def.pas:

Code: Select all

[asynchronous, external(ifc$message)] function $message(
        %immed message_codes : [list] unsigned) : unsigned;
        extern;

[asynchronous, external(ifc$message_indent)] function $message_indent(
        %immed message_codes : [list] unsigned) : unsigned;
        extern;

The actual Pascal implementation in ifc/ifcrtl_screen.pas looks like:

Code: Select all

type ...
        $message_args = array[0..30] of integer;

[global] function ifc$message_indent_list(
        ap : $message_args) : unsigned;
var     i, fao_count, column : integer;
        message_code : unsigned;
begin
...
end;

[global] function ifc$message_list(
        var ap : $message_args) : unsigned;
var     i, fao_count : integer;
        message_code : unsigned;
begin
...
end;

The OpenVMS Calling Standard, 5.7.5.2. OpenVMS Variable Argument Lists, says there's a synthetic emulation of the VAX argument block layout for BLISS and Pascal programs that use ARGPTR(), the Pascal list syntax, or the va_count() extension to C, and presumably the BLISS compiler generates the appropriate conversion code for x86-64. In reality, most of the arguments are being passed by register, and not in memory so it's not efficient to emulate the VAX layout.

I tried to write some C functions to manually loop through va_list and copy the va_count and parameters into a list to pass to the implementation functions, and amazingly got ifc$message() to mostly work, but the indented messages, which I assume are going through ifc$message_indent(), aren't printing. So when I try to read objects in the game, it says "The sign reads:" and then nothing.

What would be the preferred way to do this Pascal variable argument hackery in an x86-friendly way? Can the wrappers be written efficiently in C with compiler intrinsics? Is the original MACRO-32 version available, and if so, would that actually be the shortest way to do the wrapper? Or can someone with expertise in this wizardry supply such a wrapper from the BLISS version? Thanks!

[imiller]

you can get bliss via https://vmssoftware.com/community/freeware/

[whcox53]

Of course, that doesn't include the x86 version. That should be released before the end of the year according to the roadmap.

bill

[jhamby]

Well, the good news is my hunch about MACRO generating the appropriate synthetic argument list like BLISS does was correct. The generated object code for this attempt looks decent, although I couldn't figure out a way to get both procedures to be surrounded by ".cfi_startproc" and ".cfi_endproc" pairs, only the first one. If I were sure that 6 arguments would be enough, it wouldn't need to copy from the Alpha argument block, and the boilerplate would be even smaller.

Code: Select all

.title 'IFC$RTL_MACRO - IFC Run-Time System code in MACRO-32'
.ident 'X01.00-00'
;
; Edit History:
; 13-Sep-2009  TES   Converted the MACRO-32 module to BLISS.
; 04-Nov-2023  JEH   Converted the BLISS module back to MACRO-32.
;

.EXTERNAL IFC$MESSAGE_LIST, IFC$MESSAGE_INDENT_LIST

.PSECT CODE,NOWRT,SHR,PIC,EXE,RD

IFC$MESSAGE:: .CALL_ENTRY PRESERVE=<>,MAX_ARGS=8,HOME_ARGS=TRUE,output=<r0>
        PUSHL AP
        CALLS #1,G^IFC$MESSAGE_LIST
        RET

IFC$MESSAGE_INDENT:: .CALL_ENTRY PRESERVE=<>,MAX_ARGS=8,HOME_ARGS=TRUE,output=<r0>
        PUSHL AP
        CALLS #1,G^IFC$MESSAGE_INDENT_LIST
        RET
        .END

The bad news is that I still have the problem of not seeing the output for the indented lines, but this time I see that the Pascal SMG$ output code is using its own definition for the system str$append with what looks like an incorrect second parameter.

I'm trying to modify the code to use the official Pascal bindings in SYS$LIBRARY, but it's annoying that those files hide all of the data types, so I can't define vars of type "$uword", "$ubyte", etc. without including the existing typedef file. Thankfully, Pascal doesn't complain about a hidden typedef and an unhidden one with the same name, and it silently converts between the two if they're identical. I should know shortly whether or not my hunch is correct.

Added in 5 hours 38 minutes 14 seconds:
I got the bug figured out and now I can play the game! I pushed my diffs to GitHub:

https://github.com/jhamby/vms-halls-of-zk

Besides having to convert the variable arg wrapper from BLISS back to MACRO-32, that same variable argument message procedure was being called in a few places in the code with extra 0's. I had to track down what code was printing the message "The sign reads:", and looking at the parameters, I saw the extra 0's before the value of the description to print.

Added in 36 minutes 42 seconds:
Correction: the $message() function should accept pairs of 0's and print a blank line by calling put_scroll_dx() with none of the parameters. That function should notice that none of the parameters are present, and then call smg$put_with_scroll(main_display).

I'll have to do some more investigating to see why removing the pair of 0's in the message list got it to work, even though it should have worked the way it was originally written.

[hb]

Well, the good news is my hunch about MACRO generating the appropriate synthetic argument list like BLISS does was correct. The generated object code for this attempt looks decent, although I couldn't figure out a way to get both procedures to be surrounded by ".cfi_startproc" and ".cfi_endproc" pairs, only the first one. ...

Can you show/say what you did and what you saw? The MACRO-32 Compiler generates the .cfi directives.

[jhamby]

Sure. For this file:

Code: Select all

.title 'IFC$RTL_MACRO - IFC Run-Time System code in MACRO-32'
.ident 'X01.00-00'
;
; Edit History:
; 13-Sep-2009  TES   Converted the MACRO-32 module to BLISS.
; 04-Nov-2023  JEH   Converted the BLISS module back to MACRO-32.
;

.EXTERNAL IFC$MESSAGE_LIST, IFC$MESSAGE_INDENT_LIST

.PSECT CODE,NOWRT,SHR,PIC,EXE,RD

IFC$MESSAGE:: .CALL_ENTRY PRESERVE=<>,MAX_ARGS=10,HOME_ARGS=TRUE,output=<r0>
        PUSHL AP
        CALLS #1,G^IFC$MESSAGE_LIST
        RET

IFC$MESSAGE_INDENT:: .CALL_ENTRY PRESERVE=<>,MAX_ARGS=10,HOME_ARGS=TRUE,output=<r0>
        PUSHL AP
        CALLS #1,G^IFC$MESSAGE_INDENT_LIST
        RET
        .END

"ana/obj/disass" gives:

Code: Select all

Analyze/Disassemble Object File                  5-NOV-2023 10:02:00.51
Page 1
DKA100:[home.jhamby.Projects.vms-halls-of-zk.x86_64.OBJ]IFC$RTL_MACRO.OBJ;1
ANALYZ X01-86

.MAIN.                                          Machine Code Listing             5-NOV-2023 09:48     XMAC X6.0-111
X01.00-00

                                                .section           CODE, "ax", "progbits"   # EXE,SHR
                                                .align             16
Section size 285 bytes

                                                .cfi_startproc
                                                IFC$MESSAGE::
                                 55  00000000:  pushq   %rbp
    # 000013
                           E5 89 48  00000001:  movq    %rsp,%rbp
                                 53  00000004:  pushq   %rbx
                              57 41  00000005:  pushq   %r15
                              56 41  00000007:  pushq   %r14
                              55 41  00000009:  pushq   %r13
                           DC B6 0F  0000000B:  movzbl  %ah,%ebx
                     00 00 00 00 E8  0000000E:  callq   LIB$ALPHA_REG_VECTOR_BASE@PLT
                              93 48  00000013:  xchgq   %rax,%rbx
                              54 41  00000015:  pushq   %r12
                        C8 C4 83 48  00000017:  addq    $-38,%rsp
                     18 24 4C 89 44  0000001B:  movl    %r9d,18(%rsp)
                     14 24 44 89 44  00000020:  movl    %r8d,14(%rsp)
                        10 24 4C 89  00000025:  movl    %ecx,10(%rsp)
                        0C 24 54 89  00000029:  movl    %edx,0C(%rsp)
                        08 24 74 89  0000002D:  movl    %esi,08(%rsp)
                        04 24 7C 89  00000031:  movl    %edi,04(%rsp)
                           24 04 89  00000035:  movl    %eax,(%rsp)
                           28 45 8B  00000038:  movl    28(%rbp),%eax
                        28 24 44 89  0000003B:  movl    %eax,28(%rsp)
                           20 45 8B  0000003F:  movl    20(%rbp),%eax
                        24 24 44 89  00000042:  movl    %eax,24(%rsp)
                           18 45 8B  00000046:  movl    18(%rbp),%eax
                        20 24 44 89  00000049:  movl    %eax,20(%rsp)
                           10 45 8B  0000004D:  movl    10(%rbp),%eax
                        1C 24 44 89  00000050:  movl    %eax,1C(%rsp)
                           E4 89 49  00000054:  movq    %rsp,%r12
                                                _$$L1:
                           E2 89 4D  00000057:  movq    %r12,%r10
    # 000066
                              52 41  0000005A:  pushq   %r10
                                 5F  0000005C:  popq    %rdi
    # 000067
               00 00 01 00 C0 C7 48  0000005D:  movq    $00000100,%rax
                     00 00 00 00 E8  00000064:  callq   IFC$MESSAGE_LIST@PLT
                           03 89 48  00000069:  movq    %rax,(%rbx)
                        08 53 89 48  0000006C:  movq    %rdx,08(%rbx)
                                                _$$_0:
                        D8 65 8D 48  00000070:  leaq    -28(%rbp),%rsp
    # 000068
               FE 00 00 00 F0 A3 80  00000074:  andb    $-02,000000F0(%rbx)
                           03 8B 48  0000007B:  movq    (%rbx),%rax
                        08 53 8B 48  0000007E:  movq    08(%rbx),%rdx
                              5C 41  00000082:  popq    %r12
                              5D 41  00000084:  popq    %r13
                              5E 41  00000086:  popq    %r14
                              5F 41  00000088:  popq    %r15
                                 5B  0000008A:  popq    %rbx

Analyze/Disassemble Object File                  5-NOV-2023 10:02:00.52
Page 2
DKA100:[home.jhamby.Projects.vms-halls-of-zk.x86_64.OBJ]IFC$RTL_MACRO.OBJ;1
ANALYZ X01-86

                                 5D  0000008B:  popq    %rbp
                                 C3  0000008C:  retq
                                                .cfi_endproc
                                                _$$_3:
                           90 90 90  0000008D:  .byte   0x90,0x90,0x90
                                                IFC$MESSAGE_INDENT::
                                 55  00000090:  pushq   %rbp
    # 000070
                           E5 89 48  00000091:  movq    %rsp,%rbp
                                 53  00000094:  pushq   %rbx
                              57 41  00000095:  pushq   %r15
                              56 41  00000097:  pushq   %r14
                              55 41  00000099:  pushq   %r13
                           DC B6 0F  0000009B:  movzbl  %ah,%ebx
                     00 00 00 00 E8  0000009E:  callq   LIB$ALPHA_REG_VECTOR_BASE@PLT
                              93 48  000000A3:  xchgq   %rax,%rbx
                              54 41  000000A5:  pushq   %r12
                        C8 C4 83 48  000000A7:  addq    $-38,%rsp
                     18 24 4C 89 44  000000AB:  movl    %r9d,18(%rsp)
                     14 24 44 89 44  000000B0:  movl    %r8d,14(%rsp)
                        10 24 4C 89  000000B5:  movl    %ecx,10(%rsp)
                        0C 24 54 89  000000B9:  movl    %edx,0C(%rsp)
                        08 24 74 89  000000BD:  movl    %esi,08(%rsp)
                        04 24 7C 89  000000C1:  movl    %edi,04(%rsp)
                           24 04 89  000000C5:  movl    %eax,(%rsp)
                           28 45 8B  000000C8:  movl    28(%rbp),%eax
                        28 24 44 89  000000CB:  movl    %eax,28(%rsp)
                           20 45 8B  000000CF:  movl    20(%rbp),%eax
                        24 24 44 89  000000D2:  movl    %eax,24(%rsp)
                           18 45 8B  000000D6:  movl    18(%rbp),%eax
                        20 24 44 89  000000D9:  movl    %eax,20(%rsp)
                           10 45 8B  000000DD:  movl    10(%rbp),%eax
                        1C 24 44 89  000000E0:  movl    %eax,1C(%rsp)
                           E4 89 49  000000E4:  movq    %rsp,%r12
                                                _$$L3:
                           E2 89 4D  000000E7:  movq    %r12,%r10
    # 000071
                              52 41  000000EA:  pushq   %r10
                                 5F  000000EC:  popq    %rdi
    # 000072
               00 00 01 00 C0 C7 48  000000ED:  movq    $00000100,%rax
                     00 00 00 00 E8  000000F4:  callq   IFC$MESSAGE_INDENT_LIST@PLT
                           03 89 48  000000F9:  movq    %rax,(%rbx)
                        08 53 89 48  000000FC:  movq    %rdx,08(%rbx)
                                                _$$_1:
                        D8 65 8D 48  00000100:  leaq    -28(%rbp),%rsp
    # 000073
               FE 00 00 00 F0 A3 80  00000104:  andb    $-02,000000F0(%rbx)
                           03 8B 48  0000010B:  movq    (%rbx),%rax
                        08 53 8B 48  0000010E:  movq    08(%rbx),%rdx
                              5C 41  00000112:  popq    %r12
                              5D 41  00000114:  popq    %r13
                              5E 41  00000116:  popq    %r14
                              5F 41  00000118:  popq    %r15
                                 5B  0000011A:  popq    %rbx
                                 5D  0000011B:  popq    %rbp
                                 C3  0000011C:  retq
                                                _$$_5:

The generated boilerplate code looks good to me, but there's only one set of .cfi_startproc / .cfi_endproc lines.

Added in 7 minutes 35 seconds:
This morning I figured out why the game code to print the description of signs, or any message containing a blank line, was failing. I found a VSI Pascal optimizer bug that gives bad behavior when some arguments aren't present. Here's a test case containing a similar function to the one that's behaving badly for me:

Code: Select all

(* Test argument passing with default params *)

program TestArgs(output);

[global] function put_scroll_dx(
        column_number : [truncate] integer;
        str : [truncate] string;
        new_attributes : [truncate] unsigned) : unsigned;
begin
  writeln('In put_scroll_dx:');
  if (present(new_attributes)) then
    begin
      write('new_attributes: ');
      write(new_attributes);
      writeln;
    end;
  if (present(column_number)) then
    begin
      write('column_number: ');
      write(column_number);
      writeln;
    end;
  if (present(str)) then
    begin
      write('string: ');
      write(str);
      writeln;
    end;
  put_scroll_dx := 1;
end;

begin
  writeln('Hello, world!');
  put_scroll_dx(1, 'test', 3);
  put_scroll_dx(1);
  put_scroll_dx(2, 'foo');
  put_scroll_dx;
end.

The code that's failing is calling "put_scroll_dx" with no parameters. It should print:

Code: Select all

Hello, world!
In put_scroll_dx:
new_attributes:          3
column_number:          1
string: test
In put_scroll_dx:
column_number:          1
In put_scroll_dx:
column_number:          2
string: foo
In put_scroll_dx:

When I build with /NoOpt, it works fine. When I build with the default optimization, it crashes:

Code: Select all

Hello, world!
In put_scroll_dx:
new_attributes:          3
column_number:          1
string: test
%SYSTEM-F-ACCVIO, access violation, reason mask=04, virtual address=0000000000000000, PC=00000000800001DC, PS=0000001B
%TRACE-F-TRACEBACK, symbolic stack dump follows
image     module    routine               line      rel PC           abs PC
testargs  testargs.pas;4  PUT_SCROLL_DX
                                             5 00000000000001DC 00000000800001DC
testargs  testargs.pas;4  TESTARGS          35 00000000000000B2 00000000800000B2
                                             0 FFFF8300081FC0A6 FFFF8300081FC0A6
DCL                                          0 000000008006778B 000000007ADEB78B
%TRACE-I-END, end of TRACE stack dump

It looks like I'll have to modify the Pascal code to work around this bug for now, since I'd prefer not to disable optimization if I can avoid it.

[hb]

Sure. For this file:
...
"ana/obj/disass" gives:
...
The generated boilerplate code looks good to me, but there's only one set of .cfi_startproc / .cfi_endproc lines.

That's a bug in the disassembler code of ANALYZE. Look at the .eh_frame section and you will see that all the necessary unwind information is present.

Or write an assembler source module with two functions and with .cfi_startproc / .cfi_endproc for each. The disassembler code will show ony one pair of .cfi directives.

A fix will probably be in one of the next versions of VMS.

[arne_v]

I can't but wonder.

You are already working with the code.

Instead of converting Bliss to Macro-32 wouldn't it make more sense to get rid of the Bliss/Macro-32?

VMS Pascal has two supported mechanisms for variable number of arguments:
* LIST and argument_list_length & argument functions
* TRUNCATE and present function

Use one of those. Modify both the calling and called code to use that mechanism and you have a nice pure Pascal solution.

Boy-scout rule: Always leave the code better than you found it.

[jhamby]

You're absolutely right, and in fact I already explored that route and ran into an issue I didn't have time to figure out how to deal with properly.

You mention TRUNCATE and PRESENT, and as I mentioned, I ran into a Pascal compiler bug with optional args which I'll have to file through the support portal instead of just mentioning it in the forum, so that it gets logged.

The problem I ran into with converting the code to use LIST and Argument is that it's not legal to take the address of, say, the 4th argument, then pass that address to a function like sys$faol in order to give it the 4th through 6th arguments. It also doesn't work if you try to.

To properly get rid of the varargs hack, I'll have to pass a smaller list with a subset of the original arguments to the print function that calls $faol, and it in turn will have to pass that smaller list to $faol in a form that it can accept.

While we're on the subject of leaving the code cleaner than I found it, I started an attempt to replace the game's local definitions of the system routines with the auto-generated STARLET and PASCAL$xxx_ROUTINES Pascal bindings in sys$library. After replacing everything and sprinkling "%immed", "%ref", and "%descr" on the lines that gave compiler errors until they compiled, I can build a version of the game that's badly broken (e.g. prints nothing, accepts no commands, etc).

I'm hesitant to spend more time on that effort because it will create very large diffs against the original source and, unlike getting rid of the ugly varags hack, including a different set of library bindings that do the same thing won't make a difference to the final product. Or would it? I know it would've caught the fact that the code was calling mth$random and treating the return as IEEE floating point and not VAX, and I had to replace that with a call to Pascal's RANDOM.

The code is also using an obsolete smg$ call, "smg$put_with_scroll", which appears to be essentially identical to smg$put_line, but with the args in a slightly different order. So I should probably change that call to use the documented API, independently of changing the calls to use the proper documented Pascal bindings with the correct "%" overrides.

Added in 8 minutes 21 seconds:
Speaking of bugs I need to file, there is an absolutely bizarre one with the Halls of ZK tree and VMS's GIT. There are a couple of files starting with z, "zk/zklink_time.opt" and "zk/zkversion.opt" that git can't deal with and prints:

Code: Select all

Bad news:
 file changed before we could read it [30]

I know it's those files because I turned on "set watch file/class=(all,nodump)" and it's scanning those files when it gives the error. Also, it starts working after I delete the offending files. As I workaround, I've been deleting them, doing my commits and diffs, and then checking them out again.

I don't know if it's the date or some other metadata that git doesn't like about those files in the repo, but it chokes on them when it scans the directory tree. I'll file a bug.

[arne_v]

The problem I ran into with converting the code to use LIST and Argument is that it's not legal to take the address of, say, the 4th argument, then pass that address to a function like sys$faol in order to give it the 4th through 6th arguments. It also doesn't work if you try to.

To properly get rid of the varargs hack, I'll have to pass a smaller list with a subset of the original arguments to the print function that calls $faol, and it in turn will have to pass that smaller list to $faol in a form that it can accept.

Switch from $FAOL to $FAO.