Some random thoughts on disk structure (almost certainly not new ones)

Management of storage subsystems: SAN, volume shadowing, logical disks, file systems, and more.
Post Reply

Topic author
mberryman
Active Contributor
Posts: 40
Joined: Sat Sep 02, 2023 1:31 pm
Reputation: 0
Location: Colorado Springs, CO, USA
Status: Offline

Some random thoughts on disk structure (almost certainly not new ones)

Post by mberryman » Mon Jun 24, 2024 5:19 pm

As is well known, the current disk structure used by VMS is limited to disk sizes of roughly 2 TiB. This is largely due to the fact that the fields in indexf.sys that describe where everything is, such as highest block, end-of-file block, and map area LBNs are all 32-bits in size. What I am wondering is: what if there were a variant of ODS-5 that, rather than having these fields refer to block numbers, they instead referred to cluster numbers? Obviously, this would have ramifications elsewhere - such as changing all references of UCB$L_MAXBLOCK to UCB$Q_MAXBLOCK_64 as well as MSCP$, DIB$, and anywhere else that tracks the size of a volume. It would also mean that the I/O system services and disk drivers would need to fully support 64-bit block numbers.

The "first free byte" field in the file header is a 16-bit number, which is capable of dealing with a cluster size of up to 128 blocks. Assuming you limited cluster sizes to powers of 2 in this variant (and assuming my math is correct) this would allow support for disk sizes as follows:
cluster size --- disk size
16 --- 32 TiB
32 --- 64 TiB
64 --- 128 TiB
128 --- 256 TiB

Of course, a VBN would still refer to an individual 512-byte block but that is just math. It might also be necessary to initially continue to limit individual file size to 2 TiB until programs (and RMS) are updated to support 64-bit VBNs.

So my question is this: would this be easier, faster, etc. to implement and test compared to a whole new file system?
Thoughts, anyone?

User avatar

arne_v
Master
Posts: 448
Joined: Fri Apr 17, 2020 7:31 pm
Reputation: 0
Location: Rhode Island, USA
Status: Offline
Contact:

Re: Some random thoughts on disk structure (almost certainly not new ones)

Post by arne_v » Mon Jun 24, 2024 10:54 pm

There is certainly a need for support or larger disks.

But it is not obvious to me what the benefits of:

level=5
map area with 32 bit block index

level=5X
map area with 32 bit cluster index

compared to:

level=5
map area with 32 bit block index

level=8
map area with 64 bit block index

Any code will need to check level and treat map area differently depending on the level.
Arne
arne@vajhoej.dk
VMS user since 1986


sms
Master
Posts: 409
Joined: Fri Aug 21, 2020 5:18 pm
Reputation: 0
Status: Offline

Re: Some random thoughts on disk structure (almost certainly not new ones)

Post by sms » Mon Jun 24, 2024 11:24 pm

Code: Select all

>  But it is not obvious to me what the benefits of: [...]

   Same here.  I don't see how occupying the same amount of disk space,
but with different data, provides any better compatibility.



jonesd
Valued Contributor
Posts: 90
Joined: Mon Aug 09, 2021 7:59 pm
Reputation: 0
Status: Offline

Re: Some random thoughts on disk structure (almost certainly not new ones)

Post by jonesd » Tue Jun 25, 2024 3:15 am

Make a new device class (e.g. DC$_XDISK) where the driver uses a new API that supports 64-bit block addresses and transfer sizes. Addressing doesn't have to be tied to 512-byte disk blocks, either.

The on-disk structure can then start afresh and be designed to accommodate the scale of today's disks (or just import an existing design). LDDRIVER could be enhanced to allow it to connect to container files on the new structure and build traditional ODS-5 volumes (with a 2TB limit). Allow symbolic links to point to files on XDISK volumes.
Last edited by jonesd on Tue Jun 25, 2024 3:57 am, edited 1 time in total.


Topic author
mberryman
Active Contributor
Posts: 40
Joined: Sat Sep 02, 2023 1:31 pm
Reputation: 0
Location: Colorado Springs, CO, USA
Status: Offline

Re: Some random thoughts on disk structure (almost certainly not new ones)

Post by mberryman » Tue Jun 25, 2024 5:06 pm

Certainly a whole new file structure with more modern limits is the eventual goal. But projects to implement this have been started and stopped for various reasons. The musings that lead to the above concept were started by the following:

On my system, I have quite a few large disks, including a 24 TB volume set. I was wondering one day what would be the minimum change to VMS required to support that volume set as a single disk. What I came up with was an idea that required absolutely no changes to the on-disk structure, merely a change in how a few of the fields were interpreted. That change in interpretation would allow VMS to support disks up to 256 TB in size. I also thought this would involve the least amount of changes to VMS itself in order to support it.

So I guess part of my question is this: do we want something that could be implemented "quickly" in order to be able to use larger disks? Or, do we want to wait until the end goal can be achieved and we finally get a new file system? I guess the answer depends on how important getting support for larger disks is to one.

User avatar

cct
Master
Posts: 164
Joined: Sat Aug 15, 2020 9:00 am
Reputation: 0
Location: Cambridge, UK
Status: Offline

Re: Some random thoughts on disk structure (almost certainly not new ones)

Post by cct » Tue Jun 25, 2024 6:47 pm

Surely nobody needs more than 640k

:(
--
Chris


hein
Valued Contributor
Posts: 52
Joined: Fri Dec 25, 2020 5:20 pm
Reputation: 0
Status: Offline

Re: Some random thoughts on disk structure (almost certainly not new ones)

Post by hein » Wed Jun 26, 2024 11:57 am

I like the notion of keeping the various 32-bit size/allocation/position fields the same to minimize impact.
Multiplying by cluster size is interesting.
And indeed, sticking to the 16 bit access to byte-within-storage-unit, the limit would be 128 block storage units (clusters).
That would allow the usual FFB / EOB notions to work, and even allow RMS's internally heavily used RFA's to work unaltered.

What about only changing from 512 byte blocks (pagelettes) to 8192 byte blocks (pages)?
I'm just throwing that remark out there, because someone will and it might as well be me :-).
Obviously that's only a 16x increase in range to 32 TiB, not 128x but then what is enough? Not 128x either, so be happy with 8x?

The answer is that simply using 16 block units would only work for cluster sizes being a multiple of 16, because we don't want to deal with partial cluster allocation.
So if there is a dependency on the cluster size anyway, then one might as well just use the volume cluster size.

Hein.
Last edited by hein on Wed Jun 26, 2024 12:09 pm, edited 1 time in total.

User avatar

arne_v
Master
Posts: 448
Joined: Fri Apr 17, 2020 7:31 pm
Reputation: 0
Location: Rhode Island, USA
Status: Offline
Contact:

Re: Some random thoughts on disk structure (almost certainly not new ones)

Post by arne_v » Fri Jun 28, 2024 10:46 am

Being able to continue to use fat$w_ffbyte sounds nice, but I suspect a lot of such code will also have some 512 literals-named constants in the code meaning that code change will still be needed.
Arne
arne@vajhoej.dk
VMS user since 1986


greg@tssolutions.com.au
Contributor
Posts: 12
Joined: Wed May 29, 2024 10:29 am
Reputation: 0
Location: Australia
Status: Offline
Contact:

Re: Some random thoughts on disk structure (almost certainly not new ones)

Post by greg@tssolutions.com.au » Thu Jul 04, 2024 11:01 pm

All an interesting discussion that has been going on for a very long time.

My preference is to FIX VMS and make it 64bit... and allow IO of 128 bit, i.e. bring it into the 2020's rather than leaving it in the 1980's.
gt
VMS Ambassador
Downunder

Post Reply