UTC time handling using strptime() - problem


Topic author
garyrevell
Active Contributor
Posts: 38
Joined: Thu Nov 19, 2020 7:15 am
Reputation: 0
Location: Basingstoke, UK
Status: Offline
Contact:

UTC time handling using strptime() - problem

Post by garyrevell » Fri Apr 08, 2022 3:52 am

Morning all,

We've got a requirement to process UTC format dates from an external system e.g. 2022-04-02T11:23:05Z and want to use the strptime() function using the format mask "%FT%T%z".

I found a small example program on the net here https://ioncannon.net/programming/33/us ... imestamps/ and modified it for my own testing.

However, I can get it to compile & run OK on Windows WSL2 Debian but NOT on OpenVMS V8.4-2L1. I've raised a ticket with VSI but wanted to see if anyone else could see the problem, or have suggestions as to how to do what we want to do?

The example program is here:

Code: Select all

#include <string.h>
#include <stdio.h>
#include <time.h>

static const int date_buff_size = 64;

void convert_iso8601(const char *time_string, int ts_len, struct tm *tm_data)
{
  char* ptr = NULL;
  int iptr = 0;
  char temp[64];
  struct tm ctime;

  memset(temp, 0, sizeof(temp));
  tzset();

  strncpy(temp, time_string, ts_len);

  memset(&ctime, 0, sizeof(struct tm));
  iptr = strptime(temp, "%FT%T%z", &ctime);

  const unsigned long ts = mktime(&ctime) - timezone;
  localtime_r(&ts, tm_data);
}

int main()
{
/*  char date[] = "2006-03-28T16:49:29.000Z";   */
  char date[date_buff_size];
  struct tm tm;

  while (1)
  {
    printf("Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): ");
    scanf("%s",date );

    memset(&tm, 0, sizeof(struct tm));
    convert_iso8601(date, sizeof(date), &tm);

    char buf[date_buff_size];
    strftime(buf, sizeof(buf), "Date: %a, %d %b %Y %H:%M:%S %Z", &tm);
    printf("Convert from %s\n", date );
    printf("Result is    %s\n", buf);

  }
}
Output on Debian is as follows:

Code: Select all

garyr@BD10:~/dev/C$ ./utcTest
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2006-03-26T12:34:56.000Z
Convert from 2006-03-26T12:34:56.000Z
Result is    Date: Sun, 26 Mar 2006 13:34:56 BST
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2006-03-26T12:34:56.000Z
Convert from 2006-03-26T12:34:56.000Z
Result is    Date: Sun, 26 Mar 2006 13:34:56 BST
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2010-11-25T00:02:33.000Z
Convert from 2010-11-25T00:02:33.000Z
Result is    Date: Thu, 25 Nov 2010 00:02:33 GMT
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2022-03-27T00:30:00.000Z
Convert from 2022-03-27T00:30:00.000Z
Result is    Date: Sun, 27 Mar 2022 00:30:00 GMT
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2022-03-27T01:01:00.000Z
Convert from 2022-03-27T01:01:00.000Z
Result is    Date: Sun, 27 Mar 2022 02:01:00 BST
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2021-05-03T12:00:20Z
Convert from 2021-05-03T12:00:20Z
Result is    Date: Mon, 03 May 2021 13:00:20 BST
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2019-12-25T10:11:44Z
Convert from 2019-12-25T10:11:44Z
Result is    Date: Wed, 25 Dec 2019 10:11:44 GMT
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2019-09-30T15:00:27Z
Convert from 2019-09-30T15:00:27Z
Result is    Date: Mon, 30 Sep 2019 16:00:27 BST
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): ^C
garyr@BD10:~/dev/C$
Output on OpenVMS is as follows:

Code: Select all

(LAB6) $ run UTCTEST/nodeb
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2006-03-26T12:34:56.000Z
Convert from 2006-03-26T12:34:56.000Z
Result is    Date: Sun, 07 Feb 2106 06:28:15 GMT
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2021-12-23T11:00:07.003Z
Convert from 2021-12-23T11:00:07.003Z
Result is    Date: Sun, 07 Feb 2106 06:28:15 GMT
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2022-04-08T08:57:43Z
Convert from 2022-04-08T08:57:43Z
Result is    Date: Sun, 07 Feb 2106 06:28:15 GMT
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2015-10-31T22:17:09.000Z
Convert from 2015-10-31T22:17:09.000Z
Result is    Date: Sun, 07 Feb 2106 06:28:15 GMT
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z):
 Interrupt

(LAB6) $

At least the result coming back is consistent..... Sun, 07 Feb 2106 06:28:15 GMT ?!

Thanks in advance

Gary

User avatar

martinv
Master
Posts: 101
Joined: Fri Jun 14, 2019 11:05 pm
Reputation: 0
Location: Goslar, Germany
Status: Offline
Contact:

Re: UTC time handling using strptime() - problem

Post by martinv » Fri Apr 08, 2022 5:02 am

Gary,

it looks like the %z in the format string is not supported by the VMS' CRTL. If you omit it, the program works.

Interestingly, %F does work though it's not documented in $ help crtl strptime description

I have no explanation for the results you're getting other than, if strptime's return value is NULL, the contents of the tm structure is undefined, per the documentation. In my tests, it has been Thu, 01 Jan 1970 01:59:59 CET.

HTH,
Martin
Working hard for something we don't care about is called stress;
working hard for something we love is called passion.
(Simon Sinek)


Topic author
garyrevell
Active Contributor
Posts: 38
Joined: Thu Nov 19, 2020 7:15 am
Reputation: 0
Location: Basingstoke, UK
Status: Offline
Contact:

Re: UTC time handling using strptime() - problem

Post by garyrevell » Fri Apr 08, 2022 5:42 am

Martin,

Thanks for your reply, I tried what you suggested with both removing the %z and also trying %Z too, but it doesn't work as expected - though it does return valid date/times now.

Code: Select all

(LAB6) $ run UTCTEST/nodeb
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2006-03-26T12:34:56.000Z
Convert from 2006-03-26T12:34:56.000Z
Result is    Date: Sun, 26 Mar 2006 12:34:56 BST
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2021-12-25T09:40:23Z
Convert from 2021-12-25T09:40:23Z
Result is    Date: Sat, 25 Dec 2021 09:40:23 GMT
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z): 2022-05-01T10:00:00.000Z
Convert from 2022-05-01T10:00:00.000Z
Result is    Date: Sun, 01 May 2022 10:00:00 BST
Enter UTC date (e.g. 2006-03-26T12:34:56.000Z):
The dates I enter are UTC and for the BST dates it hasn't added on the hour to account for British Summer Time, whereas the GMT date is valid as no hour needs to be added on.

Regards

Gary

User avatar

arne_v
Master
Posts: 299
Joined: Fri Apr 17, 2020 7:31 pm
Reputation: 0
Location: Rhode Island, USA
Status: Offline
Contact:

Re: UTC time handling using strptime() - problem

Post by arne_v » Sat Apr 09, 2022 2:43 pm

A quick googling gave me the impression that standard strptime does not suppor %z and that %z is a glibc specific extension.

If that is correct then no surprise that VMS does not support it.
Arne
arne@vajhoej.dk
VMS user since 1986


Topic author
garyrevell
Active Contributor
Posts: 38
Joined: Thu Nov 19, 2020 7:15 am
Reputation: 0
Location: Basingstoke, UK
Status: Offline
Contact:

Re: UTC time handling using strptime() - problem

Post by garyrevell » Mon Apr 11, 2022 2:49 am

Yes, I have also Googled far and wide looking for a solution.

If this is the case for OpenVMS then it's not helpful, especially when it works fine on Debian and presumably that supports standard strptime.

https://www.gnu.org/software/libc/manua ... rsing.html
shows that %z supports "...The offset from GMT in ISO 8601/RFC822 format..."

I shall see what VSI come back with; may have to resort to coding my own routine or variation.
Last edited by garyrevell on Mon Apr 11, 2022 3:14 am, edited 1 time in total.

User avatar

martinv
Master
Posts: 101
Joined: Fri Jun 14, 2019 11:05 pm
Reputation: 0
Location: Goslar, Germany
Status: Offline
Contact:

Re: UTC time handling using strptime() - problem

Post by martinv » Mon Apr 11, 2022 4:07 am

Gary,

I now took some more time to look deeper into it, and experiment further.

Looking into the documentation of strptime at the VSI site instead of the online help, indeed %z (and %Z) are documented, as is %F.
First problem: %z indicates the timezone offset in hours and minutes; for the timezone name you need to use %Z.
Second problem: the time strings you gave as an example have decimal seconds that are not part of the format string, so it can't match.
Third problem: strptime() does not touch the tm_isdst field of struct tm. To indicate to mktime() that it should attempt to figure out if DST is in effect, it has to be negative, i.e. set it to -1 (see here).

The following snippet does correctly convert UTC time to local time including DST on my VMS 8.4-2L1 system:

Code: Select all

  memset(&ctime, 0, sizeof(struct tm));
  ptr = strptime(temp, "%FT%T.000%Z", &ctime);
  if (ptr == NULL) {
    fprintf(stderr, "strptime failure");
    exit(2);
  }

  /* strptime never sets the tm_isdst member of the tm structure.*/
  ctime.tm_isdst = -1; /* let mktime() sort out DST */

  const unsigned long ts = mktime(&ctime) - timezone;
  localtime_r(&ts, tm_data);
HTH,
Martin
Working hard for something we don't care about is called stress;
working hard for something we love is called passion.
(Simon Sinek)


Topic author
garyrevell
Active Contributor
Posts: 38
Joined: Thu Nov 19, 2020 7:15 am
Reputation: 0
Location: Basingstoke, UK
Status: Offline
Contact:

Re: UTC time handling using strptime() - problem

Post by garyrevell » Mon Apr 11, 2022 5:17 am

Thanks Martin for your work on this, I've been trying various combinations of format mask and addition of micros seconds over the last few days.

I'll try what you suggest soon too, VSI are also looking at this and will get back to me too later in the week.

Regards

Gary


sms
Master
Posts: 309
Joined: Fri Aug 21, 2020 5:18 pm
Reputation: 0
Status: Offline

Re: UTC time handling using strptime() - problem

Post by sms » Mon Apr 11, 2022 2:31 pm

Code: Select all

> If this is the case for OpenVMS then it's not helpful, especially when
> it works fine on Debian and presumably that supports standard strptime.

   _Whose_ "standard"?  If GNU strptime() did what I wanted, then I
might investigate building GNU strptime() on my VMS system, and, if that
succeeded, then using it.  It is open-source, isn't it?

   CC /PREFIX_LIBRARY_ENTRIES = EXCEPT = strptime

   The usual problem with that plan is the amount of other GNU stuff
which gets dragged along to make the one bit you care about work.

   What's the state of GNV these days?

User avatar

arne_v
Master
Posts: 299
Joined: Fri Apr 17, 2020 7:31 pm
Reputation: 0
Location: Rhode Island, USA
Status: Offline
Contact:

Re: UTC time handling using strptime() - problem

Post by arne_v » Mon Apr 11, 2022 8:01 pm

I still think the standard aspect is relevant.

If strptime %z is standard and the VMS version does not follow the standard then it is a bug.

But even though strptime is standard then %z is not standard.

Linux and glibc support it. But because it is not standard then VMS can omit it or support it but have different semantics.

But you could still ask for an enhancement - given how widely used glibc is then I think VMS C RTL would benefit from being as glibc compatible as possible.

Added in 2 minutes 2 seconds:
Regarding the idea of reusing the glibc strptime code, then that should be easy if that function is totally standalone. But if glibc is one big mess of dependent functions it may be more tricky.
Arne
arne@vajhoej.dk
VMS user since 1986


sms
Master
Posts: 309
Joined: Fri Aug 21, 2020 5:18 pm
Reputation: 0
Status: Offline

Re: UTC time handling using strptime() - problem

Post by sms » Mon Apr 11, 2022 11:55 pm

Code: Select all

   I discovered a GNU strptime.c in a reasonably recent Wget kit which I
had lying around.  It seems to have a 'Z' and a "z", after a fashion:

[...]
        case 'Z':
          /* XXX How to handle this?  */
          break;
        case 'z':
          /* We recognize two formats: if two digits are given, these
             specify hours.  If fours digits are used, minutes are
             also specified.  */
          {
[...]

   Faking a skimpy "config.h" got it through the compiler with only some
%CC-I-QUESTCOMPARE complaints.  Changing a "size_t" to an "ssize_t"
cleared those, if you care.  I did nothing about any funny symbol names.

   I didn't try to link it with anything, so I have no idea what else it
might want, so no bets on anything.  Be my guest:

      http://antinode.info/ftp/misc/GNU_strptime/config.h
      http://antinode.info/ftp/misc/GNU_strptime/strptime.c
      http://antinode.info/ftp/misc/GNU_strptime/strptime_ss.c

Post Reply