Null process / idle loop

[martin]

Curiosity only - low priority!

Years ago I did the VAX/VMS internals courses and I still have the documentation for the course, dated around 1987/8. The other day I was running OpenVMS 9.2 as a VM when I noticed that with nothing to do the host saw <1% CPU usage. I have 4 cores, and had 2 allocated to VMS at the time. I remembered back over the decades and thought that the VMS scheduler never really stopped, it just sat in a tight loop (not quite a spinlock) waiting for something to be available. Sure enough:

Code: Select all

60$:	TSTL	G^SCH$GL_COMQS		; POSSIBLY READY TO DO SOMETHING?
	BNEQ	20$			; YES, TRY AGAIN
	BEQL	60$			; NO, KEEP LOOKING

Now clearly internals have changed radically over 5 major versions and 4 architectures! So just to satisfy my idle curiosity, how does the VM release unused CPU back to the host whilst still being responsive to interrupts from scheduling events and device drivers?

[arne_v]

My understanding (but I am not an expert) is that it depends a little on the OS and the hypervisor.

My understanding is that there are 3 cases:
* the OS execute an instruction that releases the CPU
* the hypervisor detects the idle loop and releases the CPU
* 100% CPU usage

I cannot find any good documentation.

But https://docs.vmware.com/en/VMware-Cloud ... 26A57.html says:

Most guest operating systems run an idle loop during periods of inactivity. Within this loop, most of these guest operating systems halt by running the HLT or MWAIT instructions. Some very old guest operating systems, however, use busy-waiting within their idle loops. This results in the consumption of resources that might otherwise be available for other uses (other virtual machines, the VMkernel, and so on).

VMware Cloud on AWS automatically detects these loops and de-schedules the idle vCPU. Though this reduces the CPU overhead, it can also reduce the performance of some I/O-heavy workloads. For additional information see VMware KB articles 1077 and 2231.

which I think sort of confirms my understanding.

[martin]

Thanks for that. The MONITOR/MWAIT instructions appear to have been introduced to assist with thread handling and have been hijacked by virtualisation. I'm currently working my way through "Modern Operating Systems"¹, perhaps that will shed more light. I run QEMU/KVM under AlmaLinux (a RHEL clone) so it's not quite the same as VMware, but the basics are bound to be similar.

Anyhow, many thanks and please don't waste too much time to satisfy the curiosity of a retired old man.


¹ Tanenbaum, Andrew, and Herbert T Boschung. Modern Operating Systems, 2018. 005.4.

[dmjb]

Here is a writeup about power management on IA64 VMS: http://h41379.www4.hpe.com/openvms/jour ... r_mgmt.pdf

I would assume the idle handling on x86 is probably similar to the above, but with x86 specific mechanisms.

[martin]

Hi dmjb. Thanks for that, I've skimmed through it (more detailed study later) and the same mechanisms do indeed seem to be implemented:

Code: Select all

SYSGEN>  SHOW CPU_POWER_MGMT
Parameter Name            Current    Default     Min.       Max.   Unit  Dynamic
--------------            -------    -------   -------    -------  ----  -------
CPU_POWER_MGMT                  2          2         0         -1 Coded-valu D
SYSGEN>  SHOW CPU_POWER_THRSH
Parameter Name            Current    Default     Min.       Max.   Unit  Dynamic
--------------            -------    -------   -------    -------  ----  -------
CPU_POWER_THRSH                50         50         0        100 Percent    D
SYSGEN>

I might have a play with it tomorrow and see how much the sysgen parameters affect VM performance.

[thilo.lauer]

We use the HLT instruction.

[jonesd]

Curiosity only - low priority!

Years ago I did the VAX/VMS internals courses and I still have the documentation for the course, dated around 1987/8. The other day I was running OpenVMS 9.2 as a VM when I noticed that with nothing to do the host saw <1% CPU usage. I have 4 cores, and had 2 allocated to VMS at the time. I remembered back over the decades and thought that the VMS scheduler never really stopped, it just sat in a tight loop (not quite a spinlock) waiting for something to be available.

For Alpha emulators, there was a patch you could apply to the VMS scheduler that made the idle loop release the CPU back to the host. Without it, the host did see the emulator process running continually at 100%.

[martinv]

For the SimH VAX emulator, you can tell the program to look for the VAX/VMS idle loop and act accordingly.
I think the commercial Alpha emulators employ something similar.

[thilo.lauer]

Hi Martin,

while I don't know if and how Alpha emulators actively test for the idle loop executing, I do know that we explicitly cared for being executed within an emulator in the alpha (and lateron Integrity and x86 for sure) scheduler idle processing. So, similarly as with x86, we execute either an HLT instruction or something similar that could be easily handled accordingly by an emulator/hypervisor.

[martinv]

Hi Thilo!

while I don't know if and how Alpha emulators actively test for the idle loop executing, ...

Camiel could know that - he wrote an Alpha emulator (ES40, forked as AXPbox, and AFAIK the basis for Avanti/FreeAXP).

cu
Martin