Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Timeouts in Darwin seem to be capped at a max of 3 secs , some of the TC require upto 15 secs in some cases #14260

Closed
raju-apple opened this issue Jan 25, 2022 · 4 comments · Fixed by #14368
Closed

Timeouts in Darwin seem to be capped at a max of 3 secs , some of the TC require upto 15 secs in some cases #14260

raju-apple opened this issue Jan 25, 2022 · 4 comments · Fixed by #14368
Assignees
@bzbarsky-apple @raju-apple
Labels
bug Something isn't working cert darwin V1.0

Comments

@raju-apple
Copy link
Contributor

Is this by design ?

Probably the issue could be related to this :
https://github.com/project-chip/connectedhomeip/blob/master/src/darwin/Framework/CHIP/templates/clusters-tests.zapt#L22

Can this be fixed ? Blocking us from adding a few CC test cases.
@woody-apple
Copy link
Contributor

@raju-apple Care to update the zapt file? If not, someone can take a look!

@bzbarsky-apple
Copy link
Contributor

So it looks like the chip-tool version of this has a timeout for an entire .yaml file (currently set at 90s), while the Darwin version has a per-step timeout (3s).

We can certainly bump the Darwin timeout. What value would be useful there?

We can also add a way to specify a longer-than-default timeout for a given test step directly in the yaml, possibly in combination with also bumping the default to cover more cases.

@raju-apple Preferences?

@raju-apple
Copy link
Contributor Author

@bzbarsky-apple I like the second option , yeah if the yaml has a timeout specified in the step we should pick that up and if not it can fall back to the default. Maybe make the default as 5s or do you think its too much of a bump ?

@woody-apple woody-apple added cert Test Harness issues related to the CSG Test Harness bug Something isn't working and removed Test Harness issues related to the CSG Test Harness labels Jan 26, 2022
@bzbarsky-apple
Copy link
Contributor

5s default is easy. Let me do this. :)

@bzbarsky-apple bzbarsky-apple mentioned this issue Jan 26, 2022
Closed
bzbarsky-apple added a commit to bzbarsky-apple/connectedhomeip that referenced this issue Jan 26, 2022
@bzbarsky-apple
Make the Darwin per-step YAML timeouts larger and configurable.
4834ca4 
This makes the following changes:

1) Bumps kTimeoutInSeconds from 3 to 5.
2) For "WaitForMs" commands, sets the timeout to
   (waitTime/1000) + kTimeoutInSeconds, so longish waits work automatically.
3) Allows specifying a "stepTimeoutSecs" property in the YAML to override
   all of that and just set a value.

Fixes project-chip#14364
Fixes project-chip#14260
@bzbarsky-apple bzbarsky-apple mentioned this issue Jan 26, 2022
Merged
woody-apple pushed a commit that referenced this issue Jan 27, 2022
@bzbarsky-apple
Make the Darwin per-step YAML timeouts larger and configurable. (#14368)
a255464 
This makes the following changes:

1) Bumps kTimeoutInSeconds from 3 to 5.
2) For "WaitForMs" commands, sets the timeout to
   (waitTime/1000) + kTimeoutInSeconds, so longish waits work automatically.
3) Allows specifying a "stepTimeoutSecs" property in the YAML to override
   all of that and just set a value.

Fixes #14364
Fixes #14260
selissia pushed a commit to selissia/connectedhomeip that referenced this issue Jan 28, 2022
@bzbarsky-apple @selissia
Make the Darwin per-step YAML timeouts larger and configurable. (proj…
1dd4e77 
…ect-chip#14368)

This makes the following changes:

1) Bumps kTimeoutInSeconds from 3 to 5.
2) For "WaitForMs" commands, sets the timeout to
   (waitTime/1000) + kTimeoutInSeconds, so longish waits work automatically.
3) Allows specifying a "stepTimeoutSecs" property in the YAML to override
   all of that and just set a value.

Fixes project-chip#14364
Fixes project-chip#14260
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bzbarsky-apple bzbarsky-apple

@raju-apple raju-apple

Labels
bug Something isn't working cert darwin V1.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Make the Darwin per-step YAML timeouts larger and configurable. bzbarsky-apple/connectedhomeip
3 participants
@woody-apple @bzbarsky-apple @raju-apple