Missing handling of locale-dependent format for strftime/strflocaltime and strptime

[trantor]

Description
Using a "it_IT.UTF-8" locale, the output of the strftime/strflocaltime function in jq differs from the output of the date command in the same environment

$ ls -l /etc/localtime
lrwxrwxrwx 1 root root 31 ott  3 12:38 /etc/localtime -> /usr/share/zoneinfo/Europe/Rome
$ locale -a
C
C.UTF-8
en_US.utf8
it_IT.utf8
POSIX
$ locale
LANG=it_IT.UTF-8
LANGUAGE=
LC_CTYPE="it_IT.UTF-8"
LC_NUMERIC="it_IT.UTF-8"
LC_TIME="it_IT.UTF-8"
LC_COLLATE="it_IT.UTF-8"
LC_MONETARY="it_IT.UTF-8"
LC_MESSAGES="it_IT.UTF-8"
LC_PAPER="it_IT.UTF-8"
LC_NAME="it_IT.UTF-8"
LC_ADDRESS="it_IT.UTF-8"
LC_TELEPHONE="it_IT.UTF-8"
LC_MEASUREMENT="it_IT.UTF-8"
LC_IDENTIFICATION="it_IT.UTF-8"
LC_ALL=it_IT.UTF-8
$ date 
gio 11 ott 2018, 15.51.21, CEST
$ jq -n 'now | strflocaltime("%a %d %b %Y at %H:%M:%S")'
"Thu 11 Oct 2018 at 15:51:38"
$ jq -n 'now | strftime("%a %d %b %Y at %H:%M:%S")'
"Thu 11 Oct 2018 at 13:52:00"
$ date +"%a %d %b %Y at %H:%M:%S"
gio 11 ott 2018 at 16:02:37
$ date +"%a %d %b %Y at %H:%M:%S" | jq -R 'strptime("%a %d %b %Y at %H:%M:%S")'
jq: error (at <stdin>:1): date "gio 11 ott 2018 at 16:04:20" does not match format "%a %d %b %Y at %H:%M:%S"
$ export LC_ALL=en_US.UTF-8
$ date +"%a %d %b %Y at %H:%M:%S"
Thu 11 Oct 2018 at 16:04:51
$ date +"%a %d %b %Y at %H:%M:%S" | jq -R 'strptime("%a %d %b %Y at %H:%M:%S")'
[
  2018,
  9,
  11,
  16,
  4,
  55,
  4,
  283
]

To Reproduce
Generate a non-English locale, export the appropriate variables and run the commands above.
Expected behavior
For jq to output/parse dates using the correct representation of names according to the current locale

Environment (please complete the following information):

• OS: Ubuntu Linux 16.04
• jq version: jq-1.6rc1-16-g341a5fc-dirty

Additional context
jq compiled from the latest tree
Using the version shipped with the distro (only with strftime) gives the same output.

[trantor]

Looking at the source code I noticed the absence of calls to setlocale() or related functions.
It seems to me that any expectations of formats such as %a/%A/%b/%B/%x/%X being interpreted according to the current locale by strftime/strptime and variants will not be honoured.
I don't know about correcting that behaviour, but at least it might be made explicit in the documentation/FAQs.

[Alanscut]

hi @trantor
I personally think that this may be a problem with the user system settings, because after setting export LC_ALL = en_US.UTF-8, the following command can be executed normally.

jq-version: master

# date +"%a %d %b %Y at %H:%M:%S" | ../jq-master/jq/jq.exe -R 'strptime("%a %d %b %Y at %H:%M:%S")'
[
  2020,
  0,
  15,
  16,
  22,
  8,
  3,
  14
]

[trantor]

Well that's what locale-dependent format in the bug description means ;-) .
The point is setting a locale other than en_US ( or C as well I guess) and have it work with locale-dependent formats for the time to string and string to time functions.
According to strptime(3) %a stands for The name of the day of the week according to the current locale, in abbreviated form.
So if my locale is it_IT.UTF-8 %a should translate that gio to Thursday just as it would with Thu in the en_US.UTF-8 locale.
But it doesn't. And that's the problem. Allowing a locale-dependent format when you only decode English-specific names doesn't make sense.
I am sorry @Alanscut but your comment just repeated the commands already in my original bug report so I am not quite sure what you wished to point out.

[MiSawa]

Not familiar with strftime/strptime so unsure if this has the same cause, but %Z seems to have a problem as well.

$ date -u +"%Y-%m-%d %H:%M:%S %Z"
2022-02-22 03:28:57 UTC
$ jq -nr 'now | strftime("%Y-%m-%d %H:%M:%S %Z")'
2022-02-22 03:28:58 JST

[trantor]

@itchyny @nicowilliams I have not checked, but was this, by chance, handled by recent commits?

[nicowilliams]

@itchyny @nicowilliams I have not checked, but was this, by chance, handled by recent commits?

Can you check #2863?

[trantor]

$ LC_TIME="it_IT.UTF-8" date +"%a %d %b %Y at %H:%M:%S" | LC_TIME="it_IT.UTF-8" ./jq -R 'strptime("%a %d %b %Y at %H:%M:%S")'
jq: error (at <stdin>:1): date "mer 06 set 2023 at 12:20:26" does not match format "%a %d %b %Y at %H:%M:%S"

$  LC_TIME="en_US.UTF-8" date +"%a %d %b %Y at %H:%M:%S" | LC_TIME="en_US.UTF-8" ./jq -R 'strptime("%a %d %b %Y at %H:%M:%S")'
[
  2023,
  8,
  6,
  12,
  20,
  36,
  3,
  248,
  false,
  0
]

@nicowilliams I compiled your branch fix_1912 and there appears to be no change as far as this issue is concerned.

[nicowilliams]

@trantor ah, got it, #2874 fixes it for me.

[trantor]

Thanks a lot @nicowilliams ❤️