Timestamp in Pytest JUnit XML test report doesn't have time zone infomation

[jih-dk]

• a detailed description of the bug or suggestion
  Last year, pytest add attribute to the JUnit test report. In the test report I can see the timestamp like
  timestamp="2020-08-17T10:14:39.548365"

It looks like pytest just takes the system time from OS, and save it to test report.
Unfortunately, this timestamp doesn't have any timezone infomation, Our CI system will treat this timestamp as a UTC time, even the timestamp is actually from another time zone.

• pytest and operating system versions
  pytest 5.2.1, I have seen the issue in both Linux and Windows

Is it possible to add timezone information to the timestamp attribute, or always use UTC time in timestamp?

[Zac-HD]

I think it would be appropriate to always use UTC, and to include the timezone (i.e. utc) in the xml file.

https://blog.ganssle.io/articles/2019/11/utcnow.html has some good advice on the topic.

[graingert]

Local time with zone and UTC offset is probably the most useful. It's just whether or not JUnit supports it

…

On Thu, Aug 20, 2020, 04:47 Zac Hatfield-Dodds ***@***.***> wrote: I think it would be appropriate to always use UTC, and to include the timezone (i.e. utc) in the xml file. https://blog.ganssle.io/articles/2019/11/utcnow.html has some good advice on the topic. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#7662 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADFATBT6G6QX525WZJ4KJDSBSMD3ANCNFSM4QFCZVNA> .

[nicoddemus]

Currently the timestamp attribute is defined only as a xsd:string:

https://github.com/jenkinsci/xunit-plugin/blob/master/src/main/resources/org/jenkinsci/plugins/xunit/types/model/xsd/junit-10.xsd#L123

<xs:attribute name="timestamp" type="xs:string" />

So I think it is reasonable to write local time + UTC offset as @graingert mentioned.

[nicoddemus]

@jinxudb would you like to give that a try? Should be a simple patch (change that line + test + a changelog).

[graingert]

junit-team/junit5#373 looks like it's sort of implementation defined, so would be a good idea to test it on a real Jenkins box

[graingert]

@nicoddemus https://github.com/junit-team/junit5/blob/58bb81bfee73c5154b0a223ba68f7677bb98376d/junit-platform-reporting/src/main/java/org/junit/platform/reporting/legacy/xml/XmlReportWriter.java#L119

[nicoddemus]

Thanks @graingert. 👍

From the docs at https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html, that format does not include the time zone, but of course it doesn't mean that a Jenkins box cannot read time zone information if it is present.

Can anybody with access to a Jenkins box test this?

[graingert]

@nicoddemus we should also make sure that we store the timestamp of when the test started, not finished. This way a consumer can determine the end time of a test from testsuite.timestamp + dt.timedelta(seconds=testsuite.time)

[graingert]

@nicoddemus I think we should only land this if Junit5 also lands the same change

[graingert]

getTimestamp is an undocumented String getter: https://javadoc.jenkins.io/plugin/junit/hudson/tasks/junit/SuiteResult.html#getTimestamp--

[nicoddemus]

@nicoddemus we should also make sure that we store the timestamp of when the test started, not finished.

I'm not sure how it is currently, but I would not change the current behavior.

@nicoddemus I think we should only land this if Junit5 also lands the same change

Hmmm sorry, what's junit5?

getTimestamp is an undocumented String getter: javadoc.jenkins.io/plugin/junit/hudson/tasks/junit/SuiteResult.html#getTimestamp--

Bummer. I guess someone can try that in a Jenkins instance and see if it works.

@jinxudb could you check if adding the time zone information to that timestamp works on Jenkins?

You can do that by post-processing the generated XML file to include the time zone information using a custom script, just to check how Jenkins behaves. If it works we will feel safer to include that change into pytest.

[graingert]

@nicoddemus I think we should only land this if Junit5 also lands the same change

Hmmm sorry, what's junit5?

junit-team/junit5#2388

It appears that "junit" xml is implementation defined by whatever junit5 emits and Jenkins consumes

[graingert]

do we need to make the same change for xunit2?

[graingert]

further research I find: karma-runner/karma-junit-reporter#98

https://github.com/karma-runner/karma-junit-reporter/blob/master/junit-schema.xsd#L15

[graingert]

also Junit5 outputs timezone in the //testsuite/properties/property

<property name="user.timezone" value="Europe/London"/>

[graingert]

https://github.com/karma-runner/karma-junit-reporter/blob/2f43e5063e050b7853ebcc82c24ecc26310e0cff/junit-schema.xsd#L165-L169

		<xs:attribute name="timestamp" type="ISO8601_DATETIME_PATTERN" use="required">
			<xs:annotation>
				<xs:documentation xml:lang="en">when the test was executed. Timezone may not be specified.</xs:documentation>
			</xs:annotation>
		</xs:attribute>

You could read this as "Timezone may not be specified." or "Timezone may not be specified."

[joseph-sentry]

Any update on this issue? I think that having some timezone information in the timestamp makes sense.

It looks like the JUnit plugin for Jenkins does support datetimes with timezone information as shown in this sample file that's a part of their tests. This test here shows that the suites included in the sample junit xml file should be parsed correctly.

So I think as long as the timestamp generated by the junitxml plugin matches one of the valid ones shown in the sample file then it should be compatible with Jenkins.