MOP circuit configuration fails on V9.2-1 -- %SYSTEM-W-POOLEXPF, et al.

[sms]

Code: Select all

   Has anyone done anything (successful) with MOP on V9.2-1?  I've had
problems.  Around here:

R86 $ tcpip show vers

  VSI TCP/IP Services for OpenVMS x86_64 Version 66.0
  on a VMware, Inc. VMware7,1 running OpenVMS V9.2-1  

R86 $ show net decnet /full
[...]
    Implementation                    = 
       {
          [
          Name = OpenVMS x86_64 ,
          Version = "V9.2-1  "
          ] ,
          [
          Name = VSI DECnet-Plus for OpenVMS ,
          Version = "V9.2-E  8-MAY-2023 13:53:12.64"
          ]
       }
[...]


   The first symptom I noticed was this, when trying to talk to a
DECserver (this was on E9.2-1):

V87 $ set host /mop an2
%CCR-F-WRONGSTATE, wrong circuit state

   Looking at the circuit revealed an empty "Functions" list:

R86 $ ncl show mop circ * all
[...]
Node 0 MOP Circuit CSMACD-0
[...]
Status

    UID                               = 58795AA6-2878-11EE-8926-AA0004006408
    Functions                         = 
       {
       }
[...]

which would normally contain more (as on a working IA64 system):

    Functions                         = 
       {
          Loop Requester ,
          Console Requester ,
          Load Server ,
          Dump Server
       }

   A manual attempt also failed:

R86 $ ncl ENABLE NODE 0 MOP CIRCUIT CSMACD-0 FUNCTION = -
 {LOAD SERVER, DUMP SERVER, CONSOLE REQUESTER, LOOP REQUESTER}

Node 0 MOP Circuit CSMACD-0
at 2023-07-22-11:00:31.331-05:00Iinf

command failed due to:
 no resources available

   That seemed a bit vague, but sys$manager:net$mop_output.log said:

MOP$LOG_MODE = 0 or undefined, Mode logging disabled
%%% Management #3  22-JUL-2023 10:13:04.08 %%%
%MOP-E-XCREVCI, failed to create VCI port CSMACD-0(1)
-SYSTEM-F-INSFMEM, insufficient dynamic memory

   Later, I noticed the following at system start:

[...]
%NET$STARTUP-I-EXECUTESCRIPT, executing NCL script SYS$SYSROOT:[SYSMGR]NET$APPLI
CATION_STARTUP.NCL;
%NET-I-LOADED, executive image NET$LOOP_APPLICATION.EXE loaded
%NET$STARTUP-I-STARTPROCESS, starting process MOP
%RUN-S-PROC_ID, identification of created process is 0000041A
%NET$STARTUP-I-EXECUTESCRIPT, executing NCL script SYS$SYSROOT:[SYSMGR]NET$MOP_C
LIENT_STARTUP.NCL;
%SYSTEM-W-POOLEXPF, Pool expansion failed -- insufficient NPAGEVIR

%NET$STARTUP-I-OPERSTATUS, DECnet-Plus for OpenVMS operational status is RUNNING
-ALL
[...]

   "help /mess POOLEXPF" suggested increasing NPAGEDYN or buying more
memory.  The VM has 16GB, and this stuff worked on VAXes which couldn't
even _spell_ "giga", so I let AUTOGEN work its will.  That seemed to
boost a few parameters to values pretty close to what I see in a working
IA64 system.  For example:

R86 $ sysgen show NPAGE
Parameter Name            Current    Default     Min.       Max.   Unit  Dynamic
--------------            -------    -------   -------    -------  ----  -------
NPAGEDYN                 23601152    4194304    163840 1879048192 Bytes      
NPAGEVIR                126410752   16777216    163840 1879048192 Bytes      
NPAGECALC                       0          1         0          2 Coded-valu 
NPAGERAD                        0          0         0         -1 Bytes      
NPAGEDYN_S2                     6          2         2       1024 MBytes     
NPAGEXPVIR_S2                   8          4         4       2048 MBytes     

All of which had no obvious effect.

   Am I missing something obvious here, or am I the first person to try
SET HOST /MOP from a x86_64 system, or what?  Is something in the 
x86_64 MOP code asking for all the memory in the world?

   Note: I'm currently away from my usual LAN, leaving me out of touch
with my DECservers, so I can't do more testing than, say, those NCL
commands shown above.

[pocketprobe]

This may be a silly newbie offer, but I do have a DECServer 90M here. I could try to connect with it via MOP, or let you poke at my VMS rig and see if it behaves.

[sms]

Code: Select all

> This may be a silly newbie offer, [...]

   You're welcome to run the experiment on your system, but from what
I've done I'm reasonably satisfied that there's a new-on-x86_64 bug.

   I first observed it on an E9.2-1 system ("V87", which I can't contact
at the moment), and I've replicated the configuration failure on a
V9.2-1 system ("R86", which is on a LAN with no actual MOP clients).

   So, although I can't run a realistic test of SET HOST /MOP at my
current location, I'm willing to wait for someone to get a command like
that "NCL ENABLE NODE 0 MOP CIRCUIT [...]" to work.

[dgordon]

Not to ask the bleeding obvious but have you executed AUTOGEN with Feedback and rebooted?

[sms]

Code: Select all

> Not to ask the bleeding obvious [...]

   Thanks for looking.  Many things might be less obvious to me than
they should be.

> [...] but have you executed AUTOGEN with Feedback and rebooted?

   I may have done it with NOFEEDBACK shortly after installing V9.2-1. 
I probably requested NOFEEDBACK if it complained about being up for less
than 24 hours.

   Now that it's been up a few days, I've done it with feedback, and
some parameters have jumped more:

R86 $ sysgen show NPAGE
Parameter Name            Current    Default     Min.       Max.   Unit  Dynamic
--------------            -------    -------   -------    -------  ----  -------
NPAGEDYN                130023424    4194304    163840 1879048192 Bytes      
NPAGEVIR                478150656   16777216    163840 1879048192 Bytes      
NPAGECALC                       0          1         0          2 Coded-valu 
NPAGERAD                        0          0         0         -1 Bytes      
NPAGEDYN_S2                     7          2         2       1024 MBytes     
NPAGEXPVIR_S2                   9          4         4       2048 MBytes     

   But the bad behavior remains.  That is, %SYSTEM-W-POOLEXPF at system
start, and the interactive command failure:

R86 $ ncl ENABLE NODE 0 MOP CIRCUIT CSMACD-0 FUNCTION = -
 {LOAD SERVER, DUMP SERVER, CONSOLE REQUESTER, LOOP REQUESTER}

Node 0 MOP Circuit CSMACD-0
at 2023-07-24-14:10:42.770-05:00Iinf

command failed due to:
 no resources available

[dgordon]

That narrows down the likelihood that tuning is the source of the problem. Still a slim possibility, but sounds more like a pool leak to me at this point if the POOLEXPF error is accurate.

I've poked the maintainer to make him aware of the issue.

[roberbrooks]

I'm the DECnet maintainer Doug mentioned in his reply.

It would be interesting to try this via LANCP, getting DECnet out of the picture as much as possible.

I'll also try to give DECnet Phase IV a shot at this as well.


-- Rob

Added in 40 minutes 7 seconds:
Our DECnet support person works from 7pm to 3AM Eastern time.

He's begun the investigation on this . . . stay tuned!

[sms]

Code: Select all

> I'm the DECnet maintainer [...]

> It would be interesting to try this via LANCP, getting DECnet out of
> the picture as much as possible. [...]

   (Because someone _else_ is the LANCP maintainer?)

   I may not know enough to do that.  At the moment, the only x86_64
system with which I can communicate is on a LAN with no DECservers, so I
ran a brief experiment on an IA64 system (on a LAN with some
DECservers):

ITS $ tcpip show vers

  HP TCP/IP Services for OpenVMS Industry Standard 64 Version V5.7 - ECO 5
  on an HP rx2600  (1.50GHz/6.0MB) running OpenVMS V8.4-2L3

ITS $ lancp connect node /device = ewa 08-00-2B-A1-49-6E
%LANCP-F-BADPARAM, bad parameter value
-LANCP-I-OTHERAPP, Another application may be using the device, device EWA0
%LANCP-E-CMDERROR, Error executing command

ITS $ help /mess OTHERAPP
%MSGHLP-F-NOTFOUND, message not found in Help Message database

   I know nothing, but I might guess that the OTHERAPP is DECnet, and
I'd need to do some NCL thing to let LANCP deal with the MOP stuff. 
(But what do I know?)

   On the bright side, I did register the address of a DECserver in the
LANCP database(s) on ITS:

ITS $ lancp show node an1

ITS Node Listing, volatile database (25-JUL-2023 15:45:57.25):
  AN1 (08-00-2B-A1-49-6E):
    DLL:  Load file:   (None specified)
          Load root:   (None specified)
          Boot type:   Other
          DECnet addr: 1.34

 So, I can now get the same failure using a (friendly) DECnet address
instead of a (less-friendly) MAC address:

ITS $ lancp connect node /device = ewa an1
%LANCP-F-BADPARAM, bad parameter value
-LANCP-I-OTHERAPP, Another application may be using the device, device EWA0
%LANCP-E-CMDERROR, Error executing command

instead of the "Who?" response I get when specifying an unregistered
node name:

ITS $ lancp connect node /device = ewa an2
%LANCP-E-UNRECADR, Unrecognized target LAN node/address
%LANCP-E-CMDERROR, Error executing command

   Another disappointment:

ITS $ help /mess UNRECADR
%MSGHLP-F-NOTFOUND, message not found in Help Message database

Are my expectations too high for "help /mess"?

   I'd guess that it'll be a week or so before I can get back to the LAN
with the DECservers, and straighten out the networking on the VMware
host Mac there.  If I get adventurous, I might look into switching the
MOP stuff on my main IA64 system from DECnet to LANCP.

[dgordon]

Are my expectations too high for "help /mess"?

Yes and no. If it's something that's in the message help database, you're golden.

Unfortunately, the message help database has been maintained only sporadically for way longer than VSI has been around. It used to be populated as a byproduct of generation of the VMS documentation set, but the tool chain changed multiple times and the number of actual tech writers diminished and some of the secret ways were forgotten, Within the last year I had to go hunt down how one added messages to the database. A project was proposed to bring the database up to a recent level, but carbon-based programming and writing resources are dedicated to other things at the moment.

Keep trying. Sometimes HELP/MESSAGE surprises me and teaches me something I don't know.

[sms]

Code: Select all

> Keep trying. [...]

   I expect to.  It's been handy ever since it first appeared.  But
closer-to-perfect would be better.

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

   Another day, another round of AUTOGEN-with-feedback.  Same behavior,
but the NPAGE parameters seem to have jumped again:

[...]
%NET$STARTUP-I-EXECUTESCRIPT, executing NCL script SYS$SYSROOT:[SYSMGR]NET$APPLI
CATION_STARTUP.NCL;
%NET-I-LOADED, executive image NET$LOOP_APPLICATION.EXE loaded
%NET$STARTUP-I-STARTPROCESS, starting process MOP
%RUN-S-PROC_ID, identification of created process is 0000041A
%NET$STARTUP-I-EXECUTESCRIPT, executing NCL script SYS$SYSROOT:[SYSMGR]NET$MOP_C
IRCUIT_STARTUP.NCL;
%SYSTEM-W-POOLEXPF, Pool expansion failed -- insufficient NPAGEVIR

%NET$STARTUP-I-OPERSTATUS, DECnet-Plus for OpenVMS operational status is RUNNING
-ALL
[...]


R86 $ sysgen show NPAGE
Parameter Name            Current    Default     Min.       Max.   Unit  Dynamic
--------------            -------    -------   -------    -------  ----  -------
NPAGEDYN                478150656    4194304    163840 1879048192 Bytes      
NPAGEVIR                956301312   16777216    163840 1879048192 Bytes      
NPAGECALC                       0          1         0          2 Coded-valu 
NPAGERAD                        0          0         0         -1 Bytes      
NPAGEDYN_S2                     7          2         2       1024 MBytes     
NPAGEXPVIR_S2                   9          4         4       2048 MBytes     

   Unless I'm more confused than usual, these are starting to look
unreasonable.  (Especially considering that R86 is doing approximately
no work.)

Parameter Name            Current
--------------            -------
NPAGEDYN                478150656  R86 x86_64
                         25149440  ITS IA64

NPAGEVIR                956301312  R86 x86_64
                        134152192  ITS IA64