Delay in Telnet (and over commands)

[brianreiter]

Hi Folks,

Has anyone come across and issue when using telnet to log onto a host results in a several second delay before login is allowed? The delay seems to be around 4 or 5 seconds and other command seem to fall foul of this (tcpip ifconfig -av, tcip sho comm)

Code: Select all

(MCOFS2)SYSTEM>@x
$ write sys$output f$time()
 2-AUG-2024 10:02:03.35
$ tcpip sho comm

Communication Parameters

Local host:      mcofs2_building        Domain:   cubic.com

                                 Maximum     Current        Peak
Proxies                               20


Remote Terminal
  Large buffers:           0
  UCBs:                    0
  Virtual term:     disabled
$ write sys$output f$time()
 2-AUG-2024 10:02:16.76

I'm pretty sure I've had the problem before but I can't remember what the solution was.

cheers

Brian

Added in 32 seconds:
This is 1 node of 2 node cluster. The other node doesn;t show this problem.

[m_detommaso]

Little information is provided here, however:

1- try to run telnet using ip address and not server name; If it works as expected, check your DNS server (DNS server to have a proper reverse zone for the IP address range involved).

2- run traceroute xx.xx.xx.xx. and eventually check your routing table configuration (and/or interface netmask)

/Maurizio

[brianreiter]

Hi there,

It looks like all IP addresses on that node are subject to the delay, this includes access via xxx.xxx.xxx.xxx or via host names defined in the host table.

tracert returns pretty much instantaneously, the machines are in the same rack and network connections are to local switches.

The DNS server doesn't show any sign of delays in processing responses (its hosted on a linux VM on a machine in the next rack).

[sms]

Code: Select all

> 1- try to run telnet using ip address and not server name; [...]

   The usual cause for such delay is a DNS problem on the _server_, not
anything on the client.  In such a case, specifying the server
differently (on the client) changes nothing.

   That usual problem is seen when the server tries to get the name
associated with the client IP address -- a reverse DNS look-up -- and
that fails.  (Typically after 30 seconds or so, in my experience.)

   To test that hypothesis, on the  (Telnet) _server_, try:
      nslookup <client_ip_address>
If that command gets the same delay (typically followed by a failure),
then you need to look at the name resolver (on the _server_).


> The DNS server doesn't show any sign of delays in processing responses
> [...]

   Too vague.  Whose DNS server?  "responses" to what, where?


> I'm pretty sure I've had the problem before [...]

   Many people have.
   

> [...]  (and over commands)

   What are "over commands"?

[debbee.west]

To start with, please check your bind resolver parameters

$ tcpip sho name

Thanks!

[brianreiter]

OK, TCPIP SHOW NAME returns:

Code: Select all

(MCOFS2)SYSTEM>tcpip sho name

BIND Resolver Parameters

 Local domain: cubic.com

 System

  State:     Started, Enabled

  Transport: UDP
  Domain:    cubic.com
  Retry:     2
  Timeout:   5
  Servers:    DNS_SERVER
  Path:       No values defined

 Process

  State:     Enabled

  Transport:
  Domain:
  Retry:
  Timeout:
  Servers:
  Path:

nslookup comes back with:

Code: Select all

(MCOFS2)SYSTEM>nslookup
*** Can't find server name for address 172.16.1.11: Non-existent host/domain
*** Default servers are not available

However the usual show host command works:

Code: Select all

(MCOFS2)SYSTEM>tcpip sho host bbc.co.uk

     BIND database

Server:   172.16.1.11      DNS_SERVER

Host address    Host name

151.101.128.81  BBC.CO.UK
151.101.0.81    BBC.CO.UK
151.101.64.81   BBC.CO.UK
151.101.192.81  BBC.CO.UK

The applications which require DNS seem to be working as expected.

On one of out test systems nslookup comes back with:

Code: Select all

(CATB1)SYSTEM>nslookup google.com
Server:  ME-DC2.ctsl.cubic.cub
Address:  10.26.3.81

Non-authoritative answer:
Name:    google.com
Addresses:  172.253.62.113, 172.253.62.139, 172.253.62.101, 172.253.62.102
          172.253.62.138, 172.253.62.100

Added in 3 minutes 10 seconds:
OpenVMS version is:

Code: Select all

(MCOFS2)SYSTEM>sho sys
OpenVMS V8.4-2L1  on node MCOFS2    5-AUG-2024 12:33:52.38   Uptime  72 21:59:36

tcp version is:

Code: Select all

(MCOFS2)SYSTEM>tcpip sho ver

  HP TCP/IP Services for OpenVMS Industry Standard 64 Version V5.7 - ECO 5
  on an HP rx2800 i6  (2.13GHz/24.0MB) running OpenVMS V8.4-2L1

[m_detommaso]

nslookup and telnet protocols require DNS server to have a proper reverse zone for the IP address range involved to run error free (as reported in my previous post). Check with your network team that the DNS servers (in your case 172.16.1.11) have configured correctly the reverse zone.

"A reverse lookup zone is an authoritative DNS zone that is used to resolve IP addresses to network resource names. It is created by inverting the IP address and using a special RR called a PTR record to map the IP address to the fully qualified domain name (FQDN)."

This should solve your problem.

/Maurizio

[brianreiter]

I'll get one of the network guys to looks into the reverse lookup zone on the DNS server.

I'm still not sure why one node is not showing any delays on the login and the other is. Both nodes show the same problem with nslookup i,.e. the "*** Can't find server name for address 172.16.1.11: Non-existent host/domain" message.

The "tcpip sho host" works on both nodes without any delays.

Running tcptrace to the DNS server on the problem node while logging in to via via telnet from another source doesn't show any traffic on port 53. The traffic from "nslookup" and "tcpip show host" is shown.

Running nslookup from a PC on the network (the PC I'm using for telnet) works OK with the same server:

Code: Select all

C:\Users\cubic\Desktop>nslookup google.com
Server:  UnKnown
Address:  172.16.1.11

Non-authoritative answer:
Name:    google.com
Addresses:  2a00:1450:4009:818::200e
          172.217.169.46

[lastovica@sciinc.com]

You may want to consider capturing a TCPDUMP trace and then analyze it with Wireshark. That will be the fastest way to know where the time is being spent. My guess is that it is related to a back translation.

something like this template example that you can modify for your needs. Pass a command as P1. Execute and then copy the .ZIP file to a (presumably windows) machine where wireshark is installed and begin analysis there.

FWIW

Code: Select all

$       PID := 'F$GETJPI("","PID")
$       XSPN := "SPAWN/NOWAIT/NOTIFY/NOKEYPAD/INPUT=NL:/PROCESS=TCPDMP_''PID'/OUTPUT=SYS$SCRATCH:TCPDMP_''PID'.LOG"
$       TCPDUMP := $TCPIP$TCPDUMP
$       NODE := 'F$GETSYI("NODENAME")
$       TAD = F$CVTIME() - ":" - "." - ":" - " " - "-" - "-"
$       OF := TCPDUMP_'NODE'_'TAD'.PCAP
$       ZF := TCPDUMP_'NODE'_'TAD'.ZIP
$       FF = ""
$       IF P5 .NES. "" THEN FF = """-F"" ''P5'"
$       SAVPRV = F$GETJPI("","CURPRIV")
$       SET PROCESS/PRIVILEGE = (OPER, PSWAPM, CMKRNL)
$       XSPN TCPDUMP "-b" 1000 "-s" 4096 "-w" 'OF' 'FF'
$       WAIT 0:0:1
$       SET NOON
$       'P1'
$       'P2'
$       'P3'
$       'P4'
$       SET ON
$       WAIT 0:0:1
$       STOP/IMAGE TCPDMP_'PID
$       SET PROCESS/PRIVILEGE = (NOALL, 'SAVPRV')
$       WAIT 0:0:1
$       ZIP "-9Vjm" 'ZF 'OF
$       WRITE SYS$OUTPUT "Output file ''ZF'"
$       EXIT

[babydr]

@ "lastovica@sciinc.com" , Sorry to snag the topic .
Hmmm , Nice to know that tcpdump is available on VSI's OpenVMS .
Something I was totally unaware of .

Thank You VSI Team .
Twyl , JimL