Python virtual environments - solved

[jeremybegg]

Hi,

Today I installed PYTHON A3.10-0RELEASE005 and PYTHWHLS A1.1-6FIX06.
I want to experiment with using Python to interface to a MySQL database (actually mysqld 5.5.63-MariaDB on my OpenVMS server).

It looks like the mysql connector is not included in the Python Wheels so I'd like to install it, which should work because it's supposedly a pure-Python implementation.

I'd like to use a Python virtual environment so I tried using the same command I use when on Linux:

Code: Select all

$ python3 -m venv --system-site-packages mysql_for_python
Error: Command '['/D2/JEREMY/mysql_for_python/bin/python.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero
exit status 1.
$ 

Should "venv" work?

Before it failed it created the following directory structure:

Code: Select all

$ direx/col=1 [.mysql_for_python...] /date /wid=fil:50

Directory D2:[JEREMY.mysql_for_python]

bin.DIR;1                                            2-OCT-2023 18:56
include.DIR;1                                        2-OCT-2023 18:56
lib.DIR;1                                            2-OCT-2023 18:56
pyvenv.cfg;1                                         2-OCT-2023 18:56

Total of 4 files.

Directory D2:[JEREMY.mysql_for_python.bin]

python.;1 -> python.exe                              2-OCT-2023 18:56
python.exe;1 -> /$10$DKD0/SYS0/SYSCOMMON/python/bin/python.exe
                                                     2-OCT-2023 18:56
python3.;1 -> python.exe                             2-OCT-2023 18:56
python3.10;1 -> python.exe                           2-OCT-2023 18:56

Total of 4 files.

Directory D2:[JEREMY.mysql_for_python.lib]

python3^.10.DIR;1                                    2-OCT-2023 18:56

Total of 1 file.

Directory D2:[JEREMY.mysql_for_python.lib.python3^.10]

site-packages.DIR;1                                  2-OCT-2023 18:56

Total of 1 file.

Grand total of 4 directories, 10 files.

Thanks,

Jeremy Begg

[sergey_vorfolomeev]

Hi.
It must work.
May be it fails because of filepath must be correctly present as a full path or as a relative path?

Code: Select all

$ python3 -m venv --system-site-packages ./mysql_for_python

[jeremybegg]

Hi Sergey,

Using the command you suggested makes no difference:

Code: Select all

$ python3 -m venv --system-site-packages ./mysql_for_python
Error: Command '['/D2/JEREMY/mysql_for_python/bin/python.exe', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero
exit status 1.
$

I tried running the command shown in the error message just to see what happens, with interesting results:

Code: Select all

$ mcr d2:[jeremy.mysql_for_python.bin]python.exe -Im ensurepip --upgrade --default-pip
%DCL-W-ACTIMAGE, error activating image d2:[jeremy.mysql_for_python.bin]python.exe
-CLI-E-IMGNAME, image file DSA2:[JEREMY.mysql_for_python.bin]python.exe;1
-IMGACT-F-NOT_I64, image is not an HP OpenVMS Industry Standard 64 image

Apparently the MCR command is not smart enough to follow a symbolic link.....

So I tried this instead:

Code: Select all

$ python -Im ensurepip --upgrade --default-pip
WARNING: Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/10151>
distutils: /python$root/include/python3.10/UNKNOWN
sysconfig: /python$root/include/UNKNOWN
WARNING: Value for scheme.scripts does not match. Please report this to <https://github.com/pypa/pip/issues/10151>
distutils: /python$root/bin
sysconfig: /python$root/scripts
WARNING: Additional context:
user = False
home = None
root = None
prefix = None
Looking in links: /python_wheels$root, /D2/JEREMY/tmpggkv1v1e
Requirement already satisfied: setuptools in /python$root/lib/python3.10/site-packages (57.4.0)
Requirement already satisfied: pip in /python$root/lib/python3.10/site-packages (21.2.3)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager.
It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
$

Despite the additional output, it didn't change the content of the virtual environment directory; there's plenty missing that should be there. (For example, no "activate" script.)

Thanks,
Jeremy Begg

[hb]

Code: Select all

$ mcr d2:[jeremy.mysql_for_python.bin]python.exe -Im ensurepip --upgrade --default-pip
%DCL-W-ACTIMAGE, error activating image d2:[jeremy.mysql_for_python.bin]python.exe
-CLI-E-IMGNAME, image file DSA2:[JEREMY.mysql_for_python.bin]python.exe;1
-IMGACT-F-NOT_I64, image is not an HP OpenVMS Industry Standard 64 image

Apparently the MCR command is not smart enough to follow a symbolic link.....

The error message is from the image activator. MCR for a symbolic link works, at least on x86, where ...

Code: Select all

$ dir /date sys$system:sysman 

Directory SYS$COMMON:[SYSEXE]

SYSMAN.EXE;1 -> /SYS$MD23230201308A/VMS$COMMON/SYSEXE/SYSMAN.EXE
                     18-AUG-2023 20:13:33.77

Total of 1 file.
$ mcr sysman
SYSMAN>  Exit 
$ mcr sys$system:sysman
SYSMAN>  Exit 

The image activator just opens the file and $OPEN should get to the target of the symbolic link. Which VMS version is this?

[jeremybegg]

Interesting!

MCR against a symbolic link appears not to work on IA64:

Code: Select all

$ d d2:[jeremy.mysql_for_python.bin]python.exe

Directory D2:[JEREMY.mysql_for_python.bin]

python.exe;1 -> /$10$DKD0/SYS0/SYSCOMMON/python/bin/python.exe
                                                   1/8            2-OCT-2023 20:40  [VSM,JEREMY]          (RWED,RWED,RE,)

Total of 1 file, 1/8 blocks.
HAVEN> d $10$DKD0:[SYS0.SYSCOMMON.python.bin]python.exe

Directory $10$DKD0:[SYS0.SYSCOMMON.python.bin]

python.exe;1                                      22/32          27-OCT-2022 12:51  [SYSTEM]              (RWED,RWED,RE,RE)

Total of 1 file, 22/32 blocks.
$ mcr $10$DKD0:[SYS0.SYSCOMMON.python.bin]python.exe
Python 3.10.0 (default, Oct 17 2022, 09:12:23) [C] on OpenVMS
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()
$ mcr d2:[jeremy.mysql_for_python.bin]python.exe
%DCL-W-ACTIMAGE, error activating image d2:[jeremy.mysql_for_python.bin]python.exe
-CLI-E-IMGNAME, image file DSA2:[JEREMY.mysql_for_python.bin]python.exe;1
-IMGACT-F-NOT_I64, image is not an HP OpenVMS Industry Standard 64 image
$

[hb]

Interesting!

MCR against a symbolic link appears not to work on IA64:

Which version of VMS?

Code: Select all

$ sh sys/noproc/full
OpenVMS V8.4-1H1  on node NODE      3-OCT-2023 04:35:19.05   Uptime  79 14:50:49
        HP rx2660  (1.67GHz/9.0MB)
$ 
$ dir/date hello.exe

Directory WORK20:[USER.mcr]

hello.exe;1 -> /work20/user/mcr/main.exe
                      3-OCT-2023 04:30:23.50
                      
Total of 1 file.
$ mc []hello world
argv[0] |_dsa78:[user.mcr]hello.exe;1|
argv[1] |world|
$
$ mc WORK20:[USER.mcr]hello world
argv[0] |_dsa78:[user.mcr]hello.exe;1|
argv[1] |world|
$ 

You (or a moderator) may want to move this discussion to a different topic. It seems unrelated to Python.

[jeremybegg]

Hi,

Thanks, that's even more interesting. I'll do some experiments here on various machines and then follow it up with VSI directly.

Jeremy Begg

[jeremybegg]

Hi again,

I found the cause of my problem: the particular disk volume on which I was trying to build the virtual environment was not enabled for "special files". Running

Code: Select all

$ SET VOLUME/VOLUME=SPECIAL_FILES D2:

fixed it, and I was able to create the virtual environment.

Regards,
Jeremy Begg

[sms]

Code: Select all

> $ SET VOLUME/VOLUME=SPECIAL_FILES D2:

   Hmmm.  I see it in a SHOW DEVI /FULL report, but not in HELP SET
VOLU /VOLU.

   Around here, it seems to be enabled everyplace (with ODS5).  What did
you do to get it disabled?

[jeremybegg]

Hi,

Turning "special files" processing off is very simple:

Code: Select all

$ SET VOLUME/VOLUME=NOSPECIAL device:

As you observed, the default is to have it enabled. I think we disabled it on that volume because of issues with AUDIT security alarms. (There's a comment about that in the HELP INIT/VOLUME)

Regards
Jeremy Begg