CXX on X86 : corrupt .obj file created without any error messages.

[joukj]

Hi I get the following:

Code: Select all

rumba-jj) cxx/vers
clang version 10.0.1 (git@bitbucket.org:vms_software/llvm-10.0.1.git 1944c289616
bb7073af92c05d3ba6e7fd5d7ed0e)
Build date: 09-01-2023
Target: x86_64-OpenVMS
Thread model: posix
InstalledDir: RUMBA$DKA100:[SYS0.SYSCOMMON.][SYSEXE]
rumba-jj) define diplib []
rumba-jj) define eigen [.eigen]
%DCL-I-SUPERSEDE, previous value of EIGEN has been superseded
rumba-jj) cxx/stan=gnu14 matrix.cpp
rumba-jj) lib/crea l.olb matrix  
%LIBRAR-F-INSERTERR, error inserting _ZN5Eigen15PlainObjectBaseINS_6MatrixINSt3_
_17complexIdEELin1ELin1ELi0ELin1ELin1EEEE10resizeLikeINS_14TriangularViewIKNS_5B
lockIKS5_Lin1ELin1ELb0EEELj2EEEEEvRKNS_9EigenBaseIT_EE in $DSTM:[JOUKJ.test.bugs
.cxx.corrupt_ob
-RMS-E-EOF, end of file detected

I attached the files whith which I created this "reproducer". Sorry that it is not "simple"

(removed attachment)

[hb]

Can you double check the zip archive?

Code: Select all

$ unzip -v CORRUPT_OBJ.ZIP
Archive:  SYS$SYSROOT:[SYSMGR]CORRUPT_OBJ.ZIP;1
warning [SYS$SYSROOT:[SYSMGR]CORRUPT_OBJ.ZIP;1]:  2784 extra bytes at beginning or within zipfile
  (attempting to process anyway)
error [SYS$SYSROOT:[SYSMGR]CORRUPT_OBJ.ZIP;1]:  start of central directory not found;
  zipfile corrupt.
  (please check that you have transferred or created the zipfile in the
  appropriate BINARY mode--this includes ftp, Kermit, AND unzip'd zipfiles)
$ 

[joukj]

Sorry. Corrected .zip file attached. (something went wrong when copying from OpenVMS to my linux system.

[hb]

It doesn't look like the .OBJ file is corrupt. It seems to be a LIBRARIAN problem.

The object contains 11070 symbols (I didn't filter out the global ones) with an average name length of 100. Some symbol names are quite long. LIBRARIAN has a problem with creating the global symbol table. Whether that is caused by the number of symbols or the length of the symbol names is not obvious. A long name is shown in the error message, but there were longer ones which were already processed and didn't cause an error.

You can very likely link that object into an image. And you can insert it with /NOGLOBALS into an OLB.

[arne_v]

My guess would be on the name length.

Many names must have been seen before.

But something has changed regarding name length.

Alpha:

Code: Select all

$ type z.c
void c2345678901234567890123456789012345678901234567890123456789012345678901234567890()
{
}

$ cc z

void c2345678901234567890123456789012345678901234567890123456789012345678901234567890()
.....^
%CC-W-LONGEXTERN, The external identifier name exceeds 31 characters; truncated to "C234567890123456789012345678901".
at line number 1 in file DISK2:[ARNE]z.c;1
$ libr/obj/ins namlen z
%LIBRAR-W-COMCOD, compilation warnings in module Z file DISK2:[ARNE]z.OBJ;2
$ type zz.cxx
void cxx45678901234567890123456789012345678901234567890123456789012345678901234567890()
{
}

$ cxx zz
$ libr/obj/ins namlen zz
$ libr/obj/list/name namlen
Directory of ALPHA OBJECT library DISK2:[ARNE]namlen.OLB;2 on 16-SEP-2023 08:11:23
Creation date:  16-SEP-2023 08:11:22      Creator:  Librarian A09-32
Revision date:  16-SEP-2023 08:11:23      Library format:   3.0
Number of modules:      2                 Max. key length:  128
Other entries:          2                 Preallocated index blocks:    213
Recoverable deleted blocks:        0      Total index blocks used:        2
Max. Number history records:      20      Library history records:        2

Module Z
C234567890123456789012345678901

Module ZZ
CXX$CXX45678901234567890284RDHF

x86-64:

Code: Select all

$ type z.c
void c2345678901234567890123456789012345678901234567890123456789012345678901234567890()
{
}

$ cc z

void c2345678901234567890123456789012345678901234567890123456789012345678901234567890()
.....^
%CC-W-LONGEXTERN, The external identifier name exceeds 31 characters; truncated to "C234567890123456789012345678901".
at line number 1 in file DKA0:[arne]z.c;1
$ libr/obj/ins namlen z
%LIBRAR-W-COMCOD, compilation warnings in module Z file DKA0:[arne]z.OBJ;2
$ type zz.cxx
void cxx45678901234567890123456789012345678901234567890123456789012345678901234567890()
{
}

$ cxx zz
$ libr/obj/ins namlen zz
$ libr/obj/list/name namlen
Directory of ELF OBJECT library DKA0:[arne]namlen.OLB;2 on 16-SEP-2023 08:14:24
Creation date:  16-SEP-2023 08:14:23      Creator:  Librarian I01-47
Revision date:  16-SEP-2023 08:14:24      Library format:   6.0
Number of modules:      2                 Max. key length:  1024
Other entries:          2                 Preallocated index blocks:    213
Recoverable deleted blocks:        0      Total index blocks used:        2
Max. Number history records:      20      Library history records:        2

Module Z
C234567890123456789012345678901

Module ZZ
_Z80cxx45678901234567890123456789012345678901234567890123456789012345678901234567890v

[mberryman]

What I have seen is that clang neither shortens nor truncates long symbol names and neither does the C++ compiler based on it. However, the x86_64 C compiler does. This is not an issue in all C++ code but I got bitten by it when building some packages that mix C and C++ code. I ended up having to compile the C modules with long symbols or routine names in common with the C++ code with clang.

I did not hit any length limits in the librarian as none of the long names I was dealing with were longer than 40 or 50 bytes. By default, the maximum key size in the librarian is 1024 bytes. Perhaps that is also the maximum symbol length?

[arne_v]

It is not just the length.

500 long names get inserted fine.

A 1030 long name fails with %LIBRAR-E-SYMNAMLNG.

But I still consider it likely that the problem somehow relates to long names.

It would be easy to test if one could enable shorten/trunc but it looks like CXX ignores /name=shorten and /name=trunc.

[hb]

Anyone can guess, once, twice, or more. Even if the second guess is more vague than the first. You can even guess that someone at VSI already looked at it with running librarian in the debugger.

Anyway, yes, the name length of global symbols is limited to 1024. (I would have expected a test for 1023, 1024, and 1025 instead of 1030) And yes, you can add some options to /create. Have you tried it? For example,

Code: Select all

$ libr/cre=(key=750,gl=40000) x matrix

? Or

Code: Select all

$ libr/cre=(key=800,gl=40000) x matrix

?

The problem may be related to the name length, but it's not obvious what's going on. Oh, I already said something like that.

[arne_v]

And yes, you can add some options to /create. Have you tried it? For example,

Code: Select all

$ libr/cre=(key=750,gl=40000) x matrix

? Or

Code: Select all

$ libr/cre=(key=800,gl=40000) x matrix

?

No.

But apparently you have.

Since they result in different error messages:

Code: Select all

$ libr/obj/crea=(keysiz:750,glob=40000) z
$ libr/obj/ins z matrix
%LIBRAR-E-SYMNAMLNG, symbol _ZN5Eigen8internal21dense_assignment_loopINS0_31generic_dense_assignment_kernelINS0_9evaluatorINS_5Block
INS_6MatrixINSt3__17complexIdEELin1ELin1ELi1ELin1ELin1EEELin1ELi1ELb0EEEEENS3_INS_13CwiseBinaryOpINS0_17scalar_product_opIS
$ libr/obj/crea=(keysiz:800,glob=40000) z
$ libr/obj/ins z matrix
%LIBRAR-F-INSERTERR, error inserting _ZN5Eigen15PlainObjectBaseINS_6MatrixINSt3__17complexIdEELin1ELin1ELi0ELin1ELin1EEEE10resizeLik
eINS_7ProductINS_14TriangularViewIKNS_12CwiseUnaryOpINS_8internal19scalar_conjugate_opIS4_EEKNS_9TransposeIKNS_5BlockIS5_Li
-RMS-E-EOF, end of file detected

Which seems to indicate that the longest key is somewhere between 750 and 800.

But if this stuff did work on Alpha & Itanium but give this error on x86-64 what has changed besides the length of the symbol names??

[jreagan]

You can use /NAMES=SHORTENED to get the old CRC-suffix behavior