Page 1 of 2

HP Basic Reverse file read

Posted: Tue Feb 28, 2023 6:32 pm
by kwfeese
I'm trying to read a file in reverse using HP Basic and I came across this paragraph in Guide to OpenVMS File Applications:

8.4.3 Processing Indexed Files
Indexed files provide the most record-processing flexibility. Your program can read existing records from the file in sequential, random access by RFA mode or random access by key mode. RMS also allows you to write any number of new records into an indexed file if you do not violate a specified key constraint, such as not allowing duplicate key values.

In random access by key mode, RMS provides two forward search key options for use with one of four match options (see Section 8.4.3.2). A reverse search key option permits reverse random access when used in combination with either of the two forward search key options.

There was an example but it was written in C and, unfortunately, my current company doesn't have a C license.

Has anyone ever accomplished this and if so can you provide examples?

Re: HP Basic Reverse file read

Posted: Tue Feb 28, 2023 8:32 pm
by sms

Code: Select all

> I'm trying to read a file in reverse [...]

   Is this an existing file (with some kind of structure), or some new
file which you'll be creating in some way or other, or what?

   It might be helpful to describe in more detail the actual problem
which you are trying to solve, rather than asking how to implement some
particular "solution" or technique ("read a file in reverse").

> [...] in Guide to OpenVMS File Applications: [...]
> There was an example but it was written in C [...]

   Have you looked at the BASIC documentation?

      https://docs.vmssoftware.com/   (Look for "COMPILERS"?)

   The "VSI BASIC for OpenVMS User Manual" includes "Chapter 13. File
Input and Output", which includes section 13.2.4 on "Indexed Files".

Re: HP Basic Reverse file read

Posted: Tue Feb 28, 2023 8:32 pm
by arne_v
You have seen some C code that are working with an index-sequential file using RMS API specifically SYS$FIND with rab.rab$l_rop set to something | RAB$M_REV and you want to do the same in Basic?

Added in 10 minutes 42 seconds:
In Basic you would most likely not use RMS directly but instead use Basic builtin statements. Instead of SYS$FIND and SYS$GET just use FIND and GET statements.

I am not a Basic person, but I believe that the way to read reverse is to specify PRIMARY KEY ... DESCENDING in the OPEN statement.

https://docs.vmssoftware.com/vsi-basic- ... nce-manual
https://docs.vmssoftware.com/vsi-basic- ... ser-manual

has all the details.

If you get stuck then I could try create a working example.

Re: HP Basic Reverse file read

Posted: Wed Mar 01, 2023 10:57 am
by fim
From what I can see in Basic manuals you can read indexed file only forward. Then there is an option, read the entire file from start to finish in key order, store the keys in an array in memory. Then you retrieve the keys in your array in backward order, with each key you use the basic command FIND with EQ and then the command GET.
/Fim W.

Re: HP Basic Reverse file read

Posted: Wed Mar 01, 2023 12:59 pm
by kwfeese
sms wrote:
Tue Feb 28, 2023 8:32 pm

Code: Select all

> I'm trying to read a file in reverse [...]

   Is this an existing file (with some kind of structure), or some new
file which you'll be creating in some way or other, or what?

   It might be helpful to describe in more detail the actual problem
which you are trying to solve, rather than asking how to implement some
particular "solution" or technique ("read a file in reverse").

> [...] in Guide to OpenVMS File Applications: [...]
> There was an example but it was written in C [...]

   Have you looked at the BASIC documentation?

      https://docs.vmssoftware.com/   (Look for "COMPILERS"?)

   The "VSI BASIC for OpenVMS User Manual" includes "Chapter 13. File
Input and Output", which includes section 13.2.4 on "Indexed Files".
It's an indexed History file currently at 67m blocks and the file is corrupt so I'm trying to extract as much data from the file as possible by sequentially reading the file forwards and backwards. I've tried to use the convert utility to no avail.

Re: HP Basic Reverse file read

Posted: Wed Mar 01, 2023 2:07 pm
by sms

Code: Select all

> [...] I've tried to use the convert utility to no avail.

   "tried" is not a very detailed description of what you did, and "to
no avail" is no better as a description of what happened when you did
it.

> [...] read the entire file from start to finish in key order, store
> the keys in an array in memory [...]

   I know nothing about that array stuff, but it did leap out at me as I
was skimming the BASIC docs.

Re: HP Basic Reverse file read

Posted: Wed Mar 01, 2023 8:11 pm
by arne_v
Unfortunately primary key descending is for the actual index in the file and can not be changed as one want.

So options are:
- use RMS API (yuck)
- read all keys into array and process in reverse order (sounds like that will not work based on the description)
- read chunks where each chunk is read forward but the order of which chunks in the file is read is reversed (hack)

Illustration of the last:

Code: Select all

$ type gen.pas
program gen(input,output);

type
   fixstring = packed array [1..8] of char;
   mydata = record
               nam : [KEY(0)] fixstring;
               v   : integer;
            end;

var
   f : file of mydata;
   d : mydata;

begin
   open(f, 'fwdrev.isq', new, organization := indexed, access_method := keyed);
   rewrite(f);
   d.nam := 'A A';
   d.v   := 1;
   f^ := d;
   put(f);
   d.nam := 'B B';
   d.v   := 2;
   f^ := d;
   put(f);
   d.nam := 'C C';
   d.v   := 3;
   f^ := d;
   put(f);
   d.nam := 'D D';
   d.v   := 4;
   f^ := d;
   put(f);
   close(f);
end.
$ pas gen
$ link gen
$ run gen
$ type fwd.bas
program fwd

record mydata
    string nam = 8
    integer v
end record
map (buf) mydata d

open "fwdrev.isq" as file #1, indexed fixed, recordtype none, map buf, primary key d::nam
handler eof_handler
end handler
when error use eof_handler
    while 1 = 1
        get #1
        print d::nam,d::v
    next
end when
close #1

end program
$ bas fwd
$ link fwd
$ run fwd
A A            1
B B            2
C C            3
D D            4
$ type rev.bas
program rev

record mydata
    string nam = 8
    integer v
end record
map (buf) mydata d
declare string target
declare integer i

handler eof_handler
end handler

open "fwdrev.isq" as file #1, indexed fixed, recordtype none, map buf, primary key d::nam
for i = 1 to 26
    target = chr$(91 - i)
    when error use eof_handler
        find #1, key #0 ge target
        get #1
        while mid(d::nam, 1, 1) = target
            print d::nam,d::v
            get #1
        next
    end when
next i
close #1

end program
$ bas rev
$ link rev
$ run rev
D D            4
C C            3
B B            2
A A            1

Re: HP Basic Reverse file read

Posted: Sun Apr 16, 2023 10:47 am
by hein
OP @kwfeese wrote in a reply>>
"It's an indexed History file currently at 67m blocks and the file is corrupt so I'm trying to extract as much data from the file as possible by sequentially reading the file forwards and backwards. I've tried to use the convert utility to no avail."

Hmmm... Why didn't you say so in the first place? That makes any suggestion to read (and store key values) useless.
Why do yo think reading backward would offer significant help (it might - if there is only 1 bad 'spot') ?

I only saw this submission just now. I suspect/hope you solved the issue by now possibly using a backup?
Or maybe, just maybe CONVERT works when reading by ALTERNATE key, which will take a long time and requires sorting afterwards.

Anyway to answer the initial question: To read an indexed file backwards in BASIC one supposedly needs to use a USEROPEN to snarf the RAB address. Next while processing, set the RAB$V_REV bit in the rab::rab$l_rop. This is probably best set by ORring with the RAB$M_REV mask as Arne indicated. Then go about with Basic's native GETs. There is a small chance BASIC fill nuke the whole ROP with FIND/GET by key. We (the group, me) could verify that easily, but by now the whole original problem probably went away already and it would likely not be the best solution for the underlying problem anyway. But please do ask if an example/test is still deemed valuable.

For the benefit of others, How 'bad' is/was the corruption? When (in the processing), Where (in the file) does it occur?
As Arne already suggested, then best way is probably to 'chunk' your way through the file in a binary search fashion, possibly guided by a (stale) backup of the file.
Supposedly there is a 'structure' and predictability to the Primary key. Perhaps there is a date-time component, or a slowly increasing value, if none (the proverbial 'Social Security Number') there is still an order. So is key 12345678 fails the you could try 123456000 or 123460000 until you find a spot that does work and step or binary search backwards (with forward searches :-). Or read a backup starting with 12345678 and start trying the next values from the backup on the target file.
Repeat until the end is reach.

OR, you could try fixing the file (or hire someone to do so).
For that you may want to take a peek at a presentation I wrote years ago (nothing changed)

https://docs.google.com/presentation/d/ ... ue&sd=true

Good luck!
Hein.

Re: HP Basic Reverse file read

Posted: Tue May 02, 2023 4:53 pm
by kwfeese
Thanks Hein,

You're presention shed light on my objective. Now I'm struggling with sorting out how to read by RFA, it's out of realm of expertise.

Basically, I want to start with REC 1 and read every subsequent record until end of the file. With any luck that will resolve the issue my client has.

Re: HP Basic Reverse file read

Posted: Wed May 03, 2023 10:00 am
by hein
Reading a file 'by RFA' is relatively easy in Basic - easier than any other language - with the GETRFA function and GET/FIND RFA <rfavariable> statements.
I don't think it will offer much though, and RFA's are 'opaque' and while you'll recognize some structure (VBN/ID) in them and may think you'll be able to 'predict' (VBN last VBN + bucket size) the next RFA or brute-force your way though the file you'll find it jumps around and skews (on file extends)

You are better off focusing on primary key values, aided as I indicated by values from backups.

It may well be still better to try and fully understand the actual corruption as such (chunk of file zeroed out? missing update causing a missing link? Random data in some block?) and try to 'patch' around that. Again a more or less recent back may help a lot with that process. I appreciate it is a pretty tall order to learn the indexed file internals needed for this, but on the other hand it is quite doable once you set your mind to it. Maybe someone can help you with this (for money)? It will be much better for your customer.

Hein.