HAProxy and file access / unix paths

Talk about commercial or opensource products that already exist for OpenVMS or may be available in the future.
Post Reply

Topic author
raymii
Contributor
Posts: 11
Joined: Fri Dec 04, 2020 2:32 am
Reputation: 0
Status: Offline

HAProxy and file access / unix paths

Post by raymii » Sun Apr 16, 2023 4:33 pm

I've installed HAProxy on 9.2-1 x86 hobbyist and am trying to load a custom error file. I think I have some trouble with the correct path specification for a unix style file. The release notes do not indicate any help for file access.

This is in "sys$startup:haproxy.cfg"

Code: Select all

listen site1
        bind :80
        mode http
        errorfile 404 /sys$disk/haproxy/404.http
        server http1 10.29.95.110:80 maxconn 32
Running haproxy however shows an error accessing that file:

Code: Select all

$ haproxy :== $sys$system:haproxy.exe
$ haproxy "-f" "/sys$startup/haproxy.cfg"
[ALERT] 105/222650 (1069) : parsing [/sys$startup/haproxy.cfg:35] : error reading file </sys$disk/haproxy/404.http> for custom error message <404>.
[ALERT] 105/222650 (1069) : Error(s) found in configuration file : /sys$startup/haproxy.cfg
[ALERT] 105/222650 (1069) : Fatal errors found in configuration.

I've tried defining a logical name, but that gives the same error:

Code: Select all

 define haproxy SYS$DISK:[HAPROXY]
 
 $ dir haproxy

Directory DKA0:[HAPROXY]

404.HTTP;1          MAINTENANCE.HTML;1

Total of 2 files.

The file is there:

Code: Select all

$ type haproxy:404.http
HTTP/1.1 404 Not Found
Cache-Control: no-cache
Connection: close
Content-Type: text/html

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="utf-8" />
        <title>404 Not Found</title>
    </head>
    <body>
        <main>
            <h1>404 Not Found</h1>
            This is my custom 404 Not Found page, for HAProxy on OpenVMS! <br>
        </main>
    </body

I can also see that the file is being looked up:

Code: Select all


$ set watch file/class=(all,nodump)
$ haproxy "-f" "/sys$startup/haproxy.cfg"
%XQP, Thread #0, Volume protection: Access requested: 00000001, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, File protection (13,1,0): Access requested: 00000004, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, Read only directory access (13,1,0)
%XQP, Thread #0, Directory scan for: HAPROXY.EXE;0, Status: 00000000
[...]


%XQP, Thread #0, Control function  (11744,2,0) Status: 00000001
%XQP, Thread #0, Final status: 1C000870
%XQP, Thread #0, Volume protection: Access requested: 00000001, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, File protection (11566,4,0): Access requested: 00000004, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, Read only directory access (11566,4,0)
%XQP, Thread #0, Directory scan for: 404.HTTP;0, Status: 00000001
%XQP, Thread #0, File protection (11570,2,0): Access requested: 00000001, Status: 00000001, PrvUsd: 00000000
%XQP, Thread #0, Read attributes: Access mode 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Creation date 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Expiration date 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Backup date 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Last access date/time 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Last attribute update date/time 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Data modification date/time 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Revision date 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: ASCII dates 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Access mode 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Journal flags 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: RU active 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Statistics block 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Find ACE by type 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Record attributes 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: User file characteristics 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: File length hint field 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Read attributes: Symlink meta-data 404.HTTP;1 (11570,2,0)
%XQP, Thread #0, Access 404.HTTP;1 (11570,2,0) Status: 00000001
However, I can't find out which directory is being searched. How do I figure that out?

I'm now trying to serve a custom error page, but the next step is SSL certificates and I do need the path correctly for those as well.

HAProxy does work just fine when I'm not trying to access local files:

Image


sms
Master
Posts: 317
Joined: Fri Aug 21, 2020 5:18 pm
Reputation: 0
Status: Offline

Re: HAProxy and file access / unix paths

Post by sms » Sun Apr 16, 2023 6:23 pm

Code: Select all

> I've installed HAProxy [...]

   I know nothing about it, but...

>  This is in "sys$startup:haproxy.cfg"

   Ok.  _Where_ in "sys$startup:haproxy.cfg"?  Is line 35 the one I'd
guess?  (Why should I need to guess?)

   Why are you specifying "sys$disk" in that file?  It's variable,
depending on the user's default directory specification at run time.

      SHOW LOGICAL SYS$DISK  ! As user SYSTEM, and as some other user.

> $ haproxy "-f" "/sys$startup/haproxy.cfg"

   Did you mean "sys$startup" in the file instead of "sys$disk"?

   Where, exactly, did you put "haproxy.cfg"?  I might put mine in
SYS$MANAGER, but I wouldn't put it in SYS$SYSROOT:[SYS$STARTUP].  So I
also wouldn't use SYS$STARTUP to specify its location.

> Directory DKA0:[HAPROXY]
> 
> 404.HTTP;1          MAINTENANCE.HTML;1

   That part looks harmless.  And that might be a better place to put
all this stuff than SYS$<anything>.  So, apparently _your_ SYS$DISK when
you did this:

      define haproxy SYS$DISK:[HAPROXY]
      
was DKA0:.

> I've tried defining a logical name, but that gives the same error:

   SYS$DISK is already a logical name.  I didn't think that you could
stack multiple logical names that way.

>  The file is there:

   Where is "there"?  If you don't know what SYS$DISK is for the haproxy
job, then you don't know where "there" is for it.

> However, I can't find out which directory is being searched. How do I
> figure that out?

   Don't bother?  Use some reliable device name instead of "SYS$DISK"? 
Either a physical disk (like, say, DKA0:), or a logical name which has a
fixed equivalence name.  SYS$DISK does not qualify.  For example:

      errorfile 404 /dka0/haproxy/404.http

Or, you might do something like:

      DEFINE HAPROXY_DIR DKA0:[HAPROXY]

with (untested):

      errorfile 404 /haproxy_dir/404.http

   You might even use a more reliable logical name, like, say:

      errorfile 404 /sys$sysdevice/haproxy/404.http
      
or:
      DEFINE HAPROXY_DIR SYS$SYSDEVICE:[HAPROXY]
with:
      errorfile 404 /haproxy_dir/404.http

   Naming everything "haproxy" does not make things clearer.
      DEASSIGN haproxy

   Logical names are a powerful tool.  With great power, ...


Topic author
raymii
Contributor
Posts: 11
Joined: Fri Dec 04, 2020 2:32 am
Reputation: 0
Status: Offline

Re: HAProxy and file access / unix paths

Post by raymii » Wed Apr 19, 2023 1:53 pm

Thanks for your help so far!
sms wrote:
Sun Apr 16, 2023 6:23 pm
> I've installed HAProxy [...]

I know nothing about it, but...

> This is in "sys$startup:haproxy.cfg"

Ok. _Where_ in "sys$startup:haproxy.cfg"? Is line 35 the one I'd
guess? (Why should I need to guess?)
It's a part of the configuration I added myself.

sms wrote:
Sun Apr 16, 2023 6:23 pm
Why are you specifying "sys$disk" in that file? It's variable,
depending on the user's default directory specification at run time.

SHOW LOGICAL SYS$DISK ! As user SYSTEM, and as some other user.
That clears up a lot. I was unaware that this is a user-specific logical name.
sms wrote:
Sun Apr 16, 2023 6:23 pm

> $ haproxy "-f" "/sys$startup/haproxy.cfg"

Did you mean "sys$startup" in the file instead of "sys$disk"?

Where, exactly, did you put "haproxy.cfg"? I might put mine in
SYS$MANAGER, but I wouldn't put it in SYS$SYSROOT:[SYS$STARTUP]. So I
also wouldn't use SYS$STARTUP to specify its location.
I did not change the default path. Here's how I found the location of the file:

Code: Select all

$ type SYS$STARTUP:HAPROXY$Startup.com
$ ! HAPROXY$STARTUP.COM
$ !+
$ ! 24-Mar-2022
$ !-
$
$ set noon
$
$ run/detach -
/input=sys$startup:haproxy$run.com/output=sys$manager:haproxy.log -
/process_name="HAProxy" -
/authorize sys$system:loginout.exe
The next file:

Code: Select all

$ type sys$manager:haproxy.log
$ Set NoOn
$ VERIFY = F$VERIFY(F$TRNLNM("SYLOGIN_VERIFY"))
$
$ haproxy :== $sys$system:haproxy.exe
$ haproxy "-f" "/sys$startup/haproxy.cfg"
...

sms wrote:
Sun Apr 16, 2023 6:23 pm

> Directory DKA0:[HAPROXY]
>
> 404.HTTP;1 MAINTENANCE.HTML;1

That part looks harmless. And that might be a better place to put
all this stuff than SYS$<anything>. So, apparently _your_ SYS$DISK when
you did this:

define haproxy SYS$DISK:[HAPROXY]

was DKA0:.
Correct.
sms wrote:
Sun Apr 16, 2023 6:23 pm

> I've tried defining a logical name, but that gives the same error:

SYS$DISK is already a logical name. I didn't think that you could
stack multiple logical names that way.

> The file is there:

Where is "there"? If you don't know what SYS$DISK is for the haproxy
job, then you don't know where "there" is for it.

> However, I can't find out which directory is being searched. How do I
> figure that out?

Don't bother? Use some reliable device name instead of "SYS$DISK"?
Either a physical disk (like, say, DKA0:), or a logical name which has a
fixed equivalence name. SYS$DISK does not qualify. For example:

errorfile 404 /dka0/haproxy/404.http
That specific path fails, that is what I had when I started. I thought that there might be a special syntax for programs expecting a unix style file path, which is when I tried to do stuff with logical names and sys$disk. That line gives the same error (

Code: Select all

error reading file </sys$sysdevice/haproxy/404.http> for custom error message<404>.
)
sms wrote:
Sun Apr 16, 2023 6:23 pm

Or, you might do something like:

DEFINE HAPROXY_DIR DKA0:[HAPROXY]

with (untested):

errorfile 404 /haproxy_dir/404.http

You might even use a more reliable logical name, like, say:

errorfile 404 /sys$sysdevice/haproxy/404.http

or:
DEFINE HAPROXY_DIR SYS$SYSDEVICE:[HAPROXY]
with:
errorfile 404 /haproxy_dir/404.http

Naming everything "haproxy" does not make things clearer.
DEASSIGN haproxy

Logical names are a powerful tool. With great power, ...

Neither of those options work sadly. It might just be a bug in the haproxy port to OpenVMS. Could it be an issue with file permissions or ACL's? I'm running all this as the SYSTEM user.

Added in 17 minutes 33 seconds:
Okay this is strange. I generated a self signed certificate (the only other thing I know to let HAProxy read filesystem files) and it has no trouble reading that:

Code: Select all

listen site1
        bind :443 ssl crt /haproxy_dir/mydomain.pem
        mode http
#       errorfile 404 /haproxy_dir/404.http


The file is a standard ssl certificate + key as haproxy expects it.

Code: Select all

$ dir haproxy_dir

Directory DKA0:[HAPROXY]

404.HTTP;1          MAINTENANCE.HTML;1  mydomain.pem;1
Added in 9 minutes 7 seconds:
Okay I solved it. It's a HAProxy specific quirk regarding line endings, which only applies to ERROR files. It was able to access the files just fine, the error message it said was 'error READING', it couldn't parse the files.

After changing the line endings, it runs just fine:

Code: Select all

SET FILE/ATTRIBUTE=(RFM=STMLF) 404.http
Last edited by raymii on Wed Apr 19, 2023 1:53 pm, edited 1 time in total.


sms
Master
Posts: 317
Joined: Fri Aug 21, 2020 5:18 pm
Reputation: 0
Status: Offline

Re: HAProxy and file access / unix paths

Post by sms » Thu Apr 20, 2023 1:29 am

Code: Select all

>  That clears up a lot. I was unaware that this is a user-specific
> logical name.

   It's worse than user-specific; it's process-specific.

> [...] it said was 'error READING', [...]

   Ah.  That makes sense, _if_ the error messages are documented
someplace.  (Especially if there's also a "can't open" message.)  I
might very well have interpreted it as you did.

> SET FILE/ATTRIBUTE=(RFM=STMLF) 404.http

   What was it before?  "DIRECTORY /FULL"?

   Generally, when dealing with programs from a Unix(-like) environment,
Stream-LF is safest for text files.  Sadly, I know of no way to get the
standard VMS editors to default to that record format for a new file.

   Various file-transfer schemes can also leave you with Fixed-512
(typical for "binary") for what should have been Stream-LF.  Welcome to
files with attributes.

   Thanks for the update.  Glad to see you got it working.

User avatar

arne_v
Master
Posts: 308
Joined: Fri Apr 17, 2020 7:31 pm
Reputation: 0
Location: Rhode Island, USA
Status: Offline
Contact:

Re: HAProxy and file access / unix paths

Post by arne_v » Thu Apr 20, 2023 8:45 pm

raymii wrote:
Wed Apr 19, 2023 2:19 pm
After changing the line endings, it runs just fine:

Code: Select all

SET FILE/ATTRIBUTE=(RFM=STMLF) 404.http
Just to be sure that everybody understand what various commands does.

$ set file/attr=rfm:stmlf <filename>

changes the file header to say that the file is a text file with lines delimited by LF.

It does not change the content of the file (the bytes on disk).

This is the correct command to use if the raw bytes on disk are correct but the file header description of the content is wrong - aka inconsistent.

One example could be a text file by mistake transferred binary from *nix to VMS. The file has the LF's, but the record format is fixed 512 not stream_lf.

$ convert/fdl=sys$input <input-filename> <output-filename>
record
format stream_lf
$

converts a file to a stream_lf file. It creates a new file in stream_lf format and copies all lines. And unless the old file was already in stream_lf format, then the on disk bytes in the new file are different than in the old file.

This is the correct command to use if the file is consistent but in a wrong format like being a var file where a stream_lf file is needed (due to some *nix utility making assumptions).
Arne
arne@vajhoej.dk
VMS user since 1986

Post Reply