Skip to content

Commit

Permalink
parse-kp: do not convert timestamp to local timezone
Browse files Browse the repository at this point in the history
In summer 2022, all clinical parse functions were updated to convert
encountered date timestamps from UTC to local time
(see 3205e0b)

Now, in 2024, we are reingesting kp encounter metadata from 2018-2021.
These encounters were processed and uploaded to id3c with UTC encounter dates.
Since the encounter identifier depends on the encounter date,
and since encounter identifiers should be the same each time a sample
is uploaded to id3c, the encounter date should be in UTC for
these kp samples.
  • Loading branch information
sallybg committed Apr 22, 2024
1 parent 9b71b08 commit 681d6be
Showing 1 changed file with 8 additions and 1 deletion.
9 changes: 8 additions & 1 deletion lib/seattleflu/id3c/cli/command/clinical.py
Original file line number Diff line number Diff line change
Expand Up @@ -424,7 +424,14 @@ def parse_kp(kp_filename, kp_specimen_manifest_filename, manifest_format, output
clinical_records = clinical_records[column_map.values()]

# Convert dtypes
clinical_records["encountered"] = pd.to_datetime(clinical_records["encountered"]).dt.tz_localize('America/Los_Angeles')
#clinical_records["encountered"] = pd.to_datetime(clinical_records["encountered"]).dt.tz_localize('America/Los_Angeles')
# unlike other clinical parse functions, do not convert from UTC to local timezone
# this is because of a reingestion of kp 2018-2021 encounter metadata in 2024, in order to include ICD-10 codes
# timestamp conversion from UTC to local timezone only was added after kp 2018-2021 encounters were processed into id3c
# encounter identifiers are based on encounter date, so need to keep encounter date consistent with old
# records in order to avoid re-uploading the same encounter to id3c with a different encounter identifier than before

clinical_records["encountered"] = pd.to_datetime(clinical_records["encountered"])

# Insert static value columns
clinical_records["site"] = "KP"
Expand Down

0 comments on commit 681d6be

Please sign in to comment.