python performance on OpenVMS I64

User avatar

arne_v
Senior Member
Posts: 506
Joined: Fri Apr 17, 2020 7:31 pm
Reputation: 0
Location: Rhode Island, USA
Status: Offline
Contact:

Re: python performance on OpenVMS I64

Post by arne_v » Fri Aug 18, 2023 8:19 am

neilrieck wrote:
Thu Aug 17, 2023 12:16 pm
The tiny speedup is only seen with my loader example since it is the "import" statement which invokes the JIT compile with file save.
Translating from Python source code (.py) to Python byte code (.pyc) at runtime match all criteria for being called JIT compilation.

But I believe that the common terminology choice in the Python world is to reserve the JIT compilation term for when it is translated to native instructions at runtime. The standard implementation CPython does not do that. But some implementations like PyPy does do that.
Arne
arne@vajhoej.dk
VMS user since 1986

User avatar

neilrieck
Active Contributor
Posts: 40
Joined: Tue Jan 10, 2023 10:41 am
Reputation: 0
Location: Waterloo, Ontario, Canada
Status: Offline
Contact:

Re: python performance on OpenVMS I64

Post by neilrieck » Wed Aug 14, 2024 5:07 pm

This little DCL stub will demo ways to properly start a python script on OpenVMS :mrgreen:

Code: Select all

$! ==========================================================================
$! title  : python_starter_hack.com
$! author : Neil Rieck
$! created: 2024-08-14
$! notes  :
$! 1) attempt to invoke a python script so it is compiled once, then cached
$! 2) for this to work properly, the file format must be stream_lf
$!    a) this is the default format if initially created with the vi editor
$!    b) must be converted to stream_lf (via $convert/create) if first
$!       created via either the $EDT or $EVE editors
$! 3) while all three methods work, method 2 appears to be the best because
$!    method 3 is unable to determine the python program name via:
$!        script = sys.argv[0]
$! 4) CAVEAT: if the python file is not in the stream_lf format, then choices
$!    2+3 will always create a new pyc file which will eventually overflow
$!    folder [.__pycache__]
$! ==========================================================================
$ say :== write sys$output
$ ask :== inquire/nopunct
$ say "menu:
$ say "1) invoke as a single line  (compiles every time - no cache save)"
$ say "2) load-run via 'python -m' (compiles-saves only if required)"
$ say "3) load-run the Linux way   (compiles-saves only if required)"
$ ask method "method? "
$ if (method .eq. 1)
$ then
$     python trodb_from_maximo_100.py
$ endif
$ if (method .eq. 2)
$ then
$   python -m trodb_from_maximo_100
$ endif
$ if (method .eq. 3)
$ then
$!------------- block start
$ python
$DECK
import trodb_from_maximo_100
trodb_from_maximo_100.main()
$EOD
$!------------- block stop
$ endif
Last edited by neilrieck on Thu Aug 15, 2024 7:14 am, edited 1 time in total.

User avatar

arne_v
Senior Member
Posts: 506
Joined: Fri Apr 17, 2020 7:31 pm
Reputation: 0
Location: Rhode Island, USA
Status: Offline
Contact:

Re: python performance on OpenVMS I64

Post by arne_v » Wed Aug 14, 2024 7:22 pm

New meets old.

Python being new.

$deck and $eod being old.

:-)
Arne
arne@vajhoej.dk
VMS user since 1986

User avatar

neilrieck
Active Contributor
Posts: 40
Joined: Tue Jan 10, 2023 10:41 am
Reputation: 0
Location: Waterloo, Ontario, Canada
Status: Offline
Contact:

Re: python performance on OpenVMS I64

Post by neilrieck » Thu Aug 15, 2024 7:45 am

Yep.

FYI, some of our python scripts (on OpenVMS) run more than 1,000 times per day, so it is really important to us to have the JIT compiling to __pycache__ working properly. :mrgreen:

On our Linux platforms,
1) a BASH script first "sources a file with database account info" (think logical name tables on OpenVMS) then calls the python loader script
2) the python loader script starts python, then loads the desired program via (import) which invokes all the JIT checks
3) the last line calls "main()" in the destination code

This Linux description is the closest to my demo #3 which employs $DECK + $EOD. But I'm going to return to our Linux system today to explore the possibility of having the BASH script activate the destination python script via the "python -m file.py". On this system we have hundreds of python programs, so moving from 3 files to 2 would help a lot.

###################

Off topic comment: According to PEP-333 and PEP-3333, all web programming is supposed to shift from mod_cgi (and the python cgi module) to mod_wsdi. Last month I did a significant exploration of wsgi programming on Linux, and IMHO it does not pass my risk-reward sniff test. We will either install the replacement module "legacy-cgi" (available from pypi.org), or write our own. :mrgreen:

Post Reply