Convert date for cloud resource specifications to correct timezone from UTC #1391
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…convert-hypervisor-facts-local-tz
jtpalmer
reviewed
Jul 21, 2020
| "hostname": "raw.hostname", | ||
| "vcpus": "raw.vcpus", | ||
| "memory_mb": "raw.memory_mb", | ||
| "fact_date": "DATE(CONVERT_TZ(raw.fact_date, '+00:00', @@session.time_zone))" |
There was a problem hiding this comment.
Is it possible to use the time zone specified in resources.json? Does this handle resources in a different time zone than the database server?
There was a problem hiding this comment.
@jtpalmer Anywhere else in the cloud realm where we need to convert timezones it does it this way so I'd like to keep it consistent with that.
There was a problem hiding this comment.
Update the documentation to note that cloud resources ignore the timezone configuration option (https://github.com/ubccr/xdmod/blame/xdmod9.0/docs/configuration.md#L424).
…hub.com/eiffel777/xdmod into cloud-convert-hypervisor-facts-local-tz
jtpalmer
previously approved these changes
Jul 21, 2020
jtpalmer
approved these changes
Jul 21, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The specifications for a cloud system are ingested from a json file in a format that includes a timestamp. The format can be seen in
etl_schemas.d/cloud/resource_specs.schema.json. The timestamp in these files should be in UTC and when ingested for use with the cloud utilization statistic the timestamp should be changed to the time zone on the server. This is not currently being done. This PR uses a new etl action to create a staging table calledstaging_resource_specificationsthat converts the timestamp to the time zone on the server and then uses that table as the source for cloud_resource_specs table.While fixing the timezone issue I found that the MIN() function was not being used when getting the start time for a unique set of specifications and updated the
etl_action_defs.d/cloud_common/resource_specifications.jsonfile to fix this.In Asana, https://app.asana.com/0/342819846538629/1185459844560305
Tests performed
Tested manually in docker to make sure that time zone was converted correctly and that the resource specification were given the correct start and end times.
Types of changes
Checklist: