Does it exist Linker limits when linking with an object library (I64) ?


Topic author
l.cedric
Valued Contributor
Posts: 51
Joined: Thu Jul 18, 2019 8:18 am
Reputation: 0
Status: Offline

Does it exist Linker limits when linking with an object library (I64) ?

Post by l.cedric » Thu Sep 24, 2020 10:16 am

Hello,

we are trying to rework our build tools. We wish to use just 1 object library for the whole modules (~ 4000 modules and ~ 83000 symbols) of our application, which will make our link process easier.

But but we are facing this link (or cxxlink) issue :(
link.png
Remarks :
  • If needed, find the tools versions in the above screeenshoot
  • Note the size of our library.
Doubtless, i did not search enough (?), but for our I64 server, I did not find any limits neither for the size, nor number of module or symbol limits :?:

Perhaps a clue / Linker ? ($ sh mem/unit=byte)
linkMem.png
So twice ~ 1 GB ... before the crash ? Is it due to a sort of weird P0 space limit of the i64 Linker ? But I can't understand, as we are on I64 architecture.

Or perhaps a hidden switch to allow the Linker to work with a big library ?

Anyone have any ideas on what to do ?
Thanks for any help.

User avatar

volkerhalle
Master
Posts: 196
Joined: Fri Aug 14, 2020 11:31 am
Reputation: 0
Status: Offline

Re: Does it exist Linker limits when linking with an object library (I64) ?

Post by volkerhalle » Mon Sep 28, 2020 8:21 am

Cédric,

thanks for the clarification.

If I understand you correctly, your hope now is, that the %ILINK-F-MEMFUL error will go away for your application, once you've added all necessary modules in your library (without using /SELECTIVE_SEARCH).

Good luck,

Volker.


Topic author
l.cedric
Valued Contributor
Posts: 51
Joined: Thu Jul 18, 2019 8:18 am
Reputation: 0
Status: Offline

Re: Does it exist Linker limits when linking with an object library (I64) ?

Post by l.cedric » Mon Sep 28, 2020 9:51 am

Yes, i must confirm this => i'll keep you informed.

But we would have wished to use this selective_search option, as said in the (librarian & linker) documention : does not clutter the symbol table, so make the link more efficient, taking into account that I must be carefull about objects order (thanks to CLUSTER keyword, etc ...)

So, for me (if confirmed), with this option, the Linker should not stop with this %ILINK-F-MEMFUL error, but should work as without it, by giving the undefine symbols. If not, it is VERY difficult to find what happens. It was by luck i found this behaviour.

Cédric


Topic author
l.cedric
Valued Contributor
Posts: 51
Joined: Thu Jul 18, 2019 8:18 am
Reputation: 0
Status: Offline

Re: Does it exist Linker limits when linking with an object library (I64) ?

Post by l.cedric » Thu Oct 01, 2020 4:37 am

Hi Volker,

just to confirm that (whithout /selective_search option) :
  • after adding few missing modules in the library (for just some executables, so maybe not all those that are missing),
  • I was able to build these few executables without any "undefined symbols".
So I've tried again with the /selective_search option, but unfortunalety the %ILINK-F-MEMFUL error is still there :(

As I still have a lot of work to do before I finish our new build system, from now on I forget the /selective_search option.
Not possible to work with => For me, something in our library (with the /selective_search option) makes the Linker buggy

Later, if I find some clues I'll post there.
Cédric

User avatar

volkerhalle
Master
Posts: 196
Joined: Fri Aug 14, 2020 11:31 am
Reputation: 0
Status: Offline

Re: Does it exist Linker limits when linking with an object library (I64) ?

Post by volkerhalle » Sun Oct 04, 2020 2:56 am

Hi Cedric,

as you might currently be the only one, who can reproduce this error, please try the following, to collect some more information:

- start the LINK operation, which will cause the error
- find out the process-id of the process running the LINK
- login a separate process and SET DEFAULT to the same directory as the LINKer
- invoke @FREP0VA <pid-of-linker-process>
- after the link operation fails, look at the last lines of the output of the FREP0VA procedure

It should look similar to this:
I64VMS $ @frep0va 3434004C
4-OCT-2020 08:50:21.23 FREP0VA: 00900000
4-OCT-2020 08:50:36.23 FREP0VA: 00FD0000
4-OCT-2020 08:50:51.23 FREP0VA: 01678000
4-OCT-2020 08:51:06.23 FREP0VA: 01BAC000
4-OCT-2020 08:51:21.23 FREP0VA: 021A4000
4-OCT-2020 08:51:36.23 FREP0VA: 02778000
Linker terminated...
SSLOG: Finding failed $EXPREG_64 calls...
SYS$EXPREG_64 sts: 00002a2c acmode: U !08:51:43.85
image: IA64_LINK+00041f10 argct: 06
arg 1:000000007ad17918 2:0000000024736000 3:0000000000000003

In my test case, the linker fails with

%ILINK-F-MEMFUL, insufficient virtual address space to complete this link
-SYSTEM-F-EXPGFLQUOTA, exceeded pagefile quota

It would be interesting to see the values of FREP0VA and P2 of the $EXPREG_64 in your case, where the linker fails with SYSTEM-W-REGISFUL (should be: sts: 00000af8)

Volker.

Here is the contents of the FREP0VA.COM procedure:

Code: Select all

$!
$! FREP0VA.COM - Examine FREP0VA of process running IA64_LINK.EXE
$!
$! Input:       P1 - PID of process running LINK operation
$!
$ IF P1.EQS.""
$ THEN
$   WRITE SYS$OUTPUT "Please invoke with $ @FREP0VA <pid-of-linker-process>"
$   EXIT
$ ENDIF
$!
$ SET PROC/SSLOG=(STATE=ON)/ID='P1'     ! turn on SSLOG
$!
$loop:
$ image = F$GETJPI(P1,"IMAGNAME")
$!! SHOW SYMB image
$ IF image .EQS. "" THEN $ GOTO exit    ! Linker terminated ?
$ IF F$LOCATE("IA64_LINK",image).EQ.0 THEN $ GOTO exit ! Linker terminated ?
$!
$ WRITE SYS$OUTPUT F$TIME() + " FREP0VA: " + F$GETJPI(P1,"FREP0VA")
$!
$ wait 0:0:15
$ GOTO loop
$!
$exit:
$ WRITE SYS$OUTPUT "Linker terminated..."
$ SET PROC/SSLOG=(STATE=UNLOAD)/ID='P1' ! turn off SSLOG
$!
$ IF F$SEARCH("SSLOG.DAT").NES.""
$ THEN
$   WRITE SYS$OUTPUT "SSLOG: Finding failed $EXPREG_64 calls..."
$   ANALYZE/SSLOG/SELECT=STATUS sslog.dat /OUT=SSLOG.TXT
$   SEARCH SSLOG.TXT EXPREG/WINDOW=(0,2)
$ ENDIF

Last edited by volkerhalle on Sun Oct 04, 2020 2:58 am, edited 2 times in total.


Topic author
l.cedric
Valued Contributor
Posts: 51
Joined: Thu Jul 18, 2019 8:18 am
Reputation: 0
Status: Offline

Re: Does it exist Linker limits when linking with an object library (I64) ?

Post by l.cedric » Mon Oct 05, 2020 3:07 am

Hi Volker,

here's my command and the error :

Code: Select all

$link psf_scat_common_tools.OBJ, obj:pioche_totale.olb/lib
%ILINK-F-MEMFUL, insufficient virtual address space to complete this link
-SYSTEM-W-REGISFULL, specified region is full
And your script displays :

Code: Select all

MMS_GEN_NEO>@FREP0VA A4D
 5-OCT-2020 08:52:15.06 FREP0VA: 05A5E000
 5-OCT-2020 08:52:30.06 FREP0VA: 1C7EE000
 5-OCT-2020 08:52:45.06 FREP0VA: 2970E000
 5-OCT-2020 08:53:00.06 FREP0VA: 3268E000
 5-OCT-2020 08:53:15.06 FREP0VA: 39CAE000
 5-OCT-2020 08:53:30.06 FREP0VA: 40000000
Linker terminated...
SSLOG: Finding failed $EXPREG_64 calls...
SYS$EXPREG_64                           sts: 00000af8  acmode: U   !08:30:32.98
       image:                                IA64_LINK+00032080  argct: 00
        entry number: 00003B05    number at completion: 00003B05
***************
SYS$EXPREG_64                           sts: 00000af8  acmode: U   !08:30:32.98
       image:                                IA64_LINK+00032080  argct: 00
        entry number: 00003B0C    number at completion: 00003B0C

etc ...        
A lot of display like the above, and finished itself with :

Code: Select all

...
***************
SYS$EXPREG_64                           sts: 00000af8  acmode: U   !08:30:39.60
       image:                                IA64_LINK+00032080  argct: 00
        entry number: 00007B31    number at completion: 00007B31
***************
SYS$EXPREG_64                           sts: 00000af8  acmode: U   !08:30:39.60
       image:                                IA64_LINK+00032080  argct: 00
        entry number: 00007B32    number at completion: 00007B32
MMS_GEN_NEO>
Hope it helps
Cédric

User avatar

volkerhalle
Master
Posts: 196
Joined: Fri Aug 14, 2020 11:31 am
Reputation: 0
Status: Offline

Re: Does it exist Linker limits when linking with an object library (I64) ?

Post by volkerhalle » Mon Oct 05, 2020 3:23 am

Cédric,

P0 space is full as indicated by: FREP0VA: 40000000. The maximum possible address in P0 space is %x3fffffff

SSLOG shows SYS$EXPREG_64 calls with argct: 00, the required number of arguments is 6. And the time of the SSLOG entries (08:30) does not really match the time of the linker operation (08:53). Are these SSLOG data from the SAME LINK operation or maybe from a previous one ?

Now you at least know for certain, why the LINK operation fails -> P0 space completely exhausted. Why that happens, only HPE/VSI can answer. You would need to raise a call and probably provide your PIOCHE.OLB for reproducing the problem.

Volker.


Topic author
l.cedric
Valued Contributor
Posts: 51
Joined: Thu Jul 18, 2019 8:18 am
Reputation: 0
Status: Offline

Re: Does it exist Linker limits when linking with an object library (I64) ?

Post by l.cedric » Mon Oct 05, 2020 4:18 am

volkerhalle wrote:
Mon Oct 05, 2020 3:23 am
Are these SSLOG data from the SAME LINK operation or maybe from a previous one ?
Doubtless, as I've made few tries (nevertheless always the same issue), the 1st one at 08:30 ... Your script seems to display only the timestamps of the 1st try for the last messages
volkerhalle wrote:
Mon Oct 05, 2020 3:23 am
You would need to raise a call
Our Adminitrator already done that => no answer yet from there => I'm going to remind him ;)

Thanks for all Volker
Cédric

User avatar

volkerhalle
Master
Posts: 196
Joined: Fri Aug 14, 2020 11:31 am
Reputation: 0
Status: Offline

Re: Does it exist Linker limits when linking with an object library (I64) ?

Post by volkerhalle » Tue Oct 06, 2020 2:38 am

volkerhalle wrote:
Mon Oct 05, 2020 3:23 am
SSLOG shows SYS$EXPREG_64 calls with argct: 00
Explanation: the process running the linker must have CMEXEC, CMKRNL, or SETPRV privileges to log argument values.

Volker.


Topic author
l.cedric
Valued Contributor
Posts: 51
Joined: Thu Jul 18, 2019 8:18 am
Reputation: 0
Status: Offline

Re: Does it exist Linker limits when linking with an object library (I64) ?

Post by l.cedric » Tue Oct 06, 2020 3:45 am

I've tried with a more privileged login (which owns your 3 privileges) and ... anymore the %ILINK-F-MEMFUL error, but this one !

Code: Select all

MANCED-RESO«»link DISK$GIT:[users.cedric_git.mantis.build]psf_scat_common_tools.
obj, DISK$GIT:[users.cedric_git.mantis.build.obj]pioche_totale.olb/lib
%SYSTEM-F-ACCVIO, access violation, reason mask=04, virtual address=000000007AC6
DFF0, PC=FFFFFFFF8077FC31, PS=0000001B

  Improperly handled condition, image exit forced by last chance handler.
    Signal arguments:   Number = 0000000000000005
                        Name   = 000000000000000C
                                 0000000000000004
                                 000000007AC6DFF0
                                 FFFFFFFF8077FC31
                                 000000000000001B

    Register dump:
    R0  = 0000000000000000  R1  = FFFFFFFF8654E400  R2  = FFFFFFFF86001B58
    R3  = 000000000000000A  R4  = 000000007AC6E090  R5  = 0000000000000000
    R6  = 000000007AC6DFF8  R7  = 000000007AC6E3A8  R8  = 000000000000000A
    R9  = FFFFFFFF8632631C  R10 = 000000007AC6E13C  R11 = 000000007AC6E0F8
    SP  = 000000007AC70000  TP  = 000000007B4AC1C8  R14 = 000000000000000C
    R15 = 000000007AC6E000  R16 = 000000007AC6E0B0  R17 = 0000000000000053
    R18 = 0000000000000010  R19 = FFFFFFFF86326318  R20 = 000000007FFF0268
    R21 = 000000007AC6E180  R22 = FFFFFFFF86326315  R23 = 000000007AC6DFFC
    R24 = 0000000000060001  R25 = 0000000000000006  R26 = 0000000000000000
    R27 = 000000007AC6E040  R28 = 000000007AC6E008  R29 = 000000007AC6E010
    R30 = 0000000000000003  R31 = 0000000000000640  PC  = FFFFFFFF8077FC31
    BSP/STORE = 000007FDBFFD4C90 / 000007FDBFFD4AB8 PSR = 00001213080AE010
    IIPA = FFFFFFFF8077FC30
    B0  = FFFFFFFF8074C760  B6  = FFFFFFFF80000040  B7  = FFFFFFFF8077F9D0

    Interrupted Frame RSE Backing Store, Size = 38 registers

    R32 = 000000007AC6F0E0  R33 = 0000000000000000  R34 = FFFFFFFF86491C00
    R35 = 000000007FF43E80  R36 = 0000000000000000  R37 = FFFFFFFF86526A00
    R38 = 000000007FF43F48  R39 = 000000007FFB75E8  R40 = 000000007AC6E160
    R41 = 0000000000000000  R42 = 000000000789CB8C  R43 = 0000000000140010
    R44 = 000000007AC6F358  R45 = 00000000001710E0  R46 = 0000000000000000
    R47 = 0000000000000003  R48 = 0000000000000640  R49 = 000000007FFF0268
    R50 = 000000007AC6E130  R51 = 0000000000000000  R52 = 0000000000000003
    R53 = 0000000000000005  R54 = 000000007AC6E160  R55 = 0000000000000030
    R56 = 00000000000650AB  R57 = FFFFFFFF80746220  R58 = C000000000000B9F
    R59 = 000000007AC6E110  R60 = 000000007FFCF818  R61 = 000000000000000F
    R62 = 0000000000000000  R63 = 000000000014000E  R64 = 0000000000140000
    R65 = 0000000000140013  R66 = 0000000000000003  R67 = C0000000000011AB
    R68 = 00000000001710C8  R69 = 0000000000000000
MANCED-RESO«»
And your script displays :

Code: Select all

MANCED-RESO«»@DISK$GIT:[users.cedric_git.mantis.build]FREP0VA.COM 00000270
 6-OCT-2020 09:34:21.96 FREP0VA: 0502E000
 6-OCT-2020 09:34:36.96 FREP0VA: 143CE000
 6-OCT-2020 09:34:51.96 FREP0VA: 1ED2E000
 6-OCT-2020 09:35:06.96 FREP0VA: 264DE000
 6-OCT-2020 09:35:21.96 FREP0VA: 2C06E000
 6-OCT-2020 09:35:36.96 FREP0VA: 3104E000
 6-OCT-2020 09:35:51.97 FREP0VA: 3580E000
 6-OCT-2020 09:36:06.97 FREP0VA: 3993E000
 6-OCT-2020 09:36:21.97 FREP0VA: 3D2CE000
 6-OCT-2020 09:36:36.97 FREP0VA: 3FC6A000
Linker terminated...
SSLOG: Finding failed $EXPREG_64 calls...
%SEARCH-I-NOMATCHES, no strings matched
MANCED-RESO«»
Cédric

User avatar

volkerhalle
Master
Posts: 196
Joined: Fri Aug 14, 2020 11:31 am
Reputation: 0
Status: Offline

Re: Does it exist Linker limits when linking with an object library (I64) ?

Post by volkerhalle » Tue Oct 06, 2020 4:31 am

Cédric,

I believe we agree, that the OpenVMS I64 linker should never terminate like this ! The linker attempted to write into a P1 space address (7AC6DFF0) and failed with an ACCVIO.

Your link operation seems to be stressing the linker quite heavily !

Let's see what VSI has to say about this.

Volker.

Post Reply