fix: retry reconcile on transient errors during reconcile #6299
fix: retry reconcile on transient errors during reconcile #6299arkodg merged 13 commits intoenvoyproxy:mainfrom
Conversation
c45000a to
a69693a
Compare
Codecov ReportAttention: Patch coverage is
❌ Your patch status has failed because the patch coverage (27.44%) is below the target coverage (60.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## main #6299 +/- ##
==========================================
- Coverage 70.92% 70.73% -0.19%
==========================================
Files 220 220
Lines 37260 37354 +94
==========================================
- Hits 26426 26423 -3
- Misses 9289 9383 +94
- Partials 1545 1548 +3 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
@arkodg, @patrostkowski |
a69693a to
ba30677
Compare
|
Notes from the community meeting:
|
|
can we also log the transient error |
|
@patrostkowski still working on this ? we'd like to get this into the patch release scheduled to be out next week |
|
It seems @patrostkowski is currently unavailable. I’ll take over from here since we need this in the upcoming patch. |
fcd7f69 to
40ec0c9
Compare
|
thanks @zhaohuabing and sorry, I am very busy time atm with other things 😞 |
No worries. I'll take care of it. |
5eeaad0 to
5313be2
Compare
74541e2 to
27b84b4
Compare
6f9a261 to
9b633af
Compare
a3b57a9 to
63c2eee
Compare
| service := new(corev1.Service) | ||
| err := r.client.Get(ctx, types.NamespacedName{Namespace: string(*backendRef.Namespace), Name: string(backendRef.Name)}, service) | ||
| if err != nil { | ||
| if isTransientError(err) { |
There was a problem hiding this comment.
is going to hard to remember to add this for every client call
There was a problem hiding this comment.
Yeah, we can figure out a better approach to handle this later.
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
9926890 to
8704fd8
Compare
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
…#6299) * fix: add isTransientError helper to classify retryable errors Introduces isTransientError to detect transient Kubernetes errors and enable proper reconciliation retries. Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com> handle errors from processing BackendRefs Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> handle errors from processing ConfigMap Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * skip invalid GatewayClass Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * address comment Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * handle all transient errors Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * don't skip failed GCs Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com> Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> (cherry picked from commit 71ce56f) Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
* fix(translator): ext-proc full duplex streamed trailers and validation (#6323) * fix ext proc validation and trailer management for full duplex streamed mode Signed-off-by: Guy Daich <guy.daich@sap.com> Signed-off-by: shawnh2 <shawnhxh@outlook.com> * feat: disable automountServiceAccountToken for proxy and ratelimit (#6364) Signed-off-by: Jeff Davis <mr.jefedavis@gmail.com> * bugfix: make EnvoyPatchPolicy able to replace telemetry cluster (#6367) Signed-off-by: zirain <zirain2009@gmail.com> Signed-off-by: shawnh2 <shawnhxh@outlook.com> * feat: add validation of section name for Gateway listener (#6343) * add validation of section name Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> * update error status reason Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> * refactor: define as function of validate section name for gateway listener Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> Signed-off-by: shawnh2 <shawnhxh@outlook.com> * fix: add configMap indexers for EEP reconciler (#6369) Signed-off-by: Rudrakh Panigrahi <rudrakh97@gmail.com> * fix: use buildEndpointType for access and tracing (#6370) Signed-off-by: zirain <zirain2009@gmail.com> * fix: default accesslog not working (#6441) * fix default accesslog Signed-off-by: zirain <zirain2009@gmail.com> * release notes Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> Signed-off-by: shawnh2 <shawnhxh@outlook.com> * chore: fix cve (#6446) * fix cve Signed-off-by: zirain <zirain2009@gmail.com> * lint Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> * fix: Do not set backendRequestTimeout when Retries are set (#6421) * fix: Do not set backendRequestTimeout when Retries are set Signed-off-by: sudipto baral <sudiptobaral.me@gmail.com> * fix: update comment Signed-off-by: sudipto baral <sudiptobaral.me@gmail.com> --------- Signed-off-by: sudipto baral <sudiptobaral.me@gmail.com> * gatewayapi: don't append gwcResource if there's invalid GatewayClass (#6379) * gatewayapi: don't process gloabal resources when acceptedGateways is 0 Signed-off-by: zirain <zirain2009@gmail.com> * update Signed-off-by: zirain <zirain2009@gmail.com> * fix test Signed-off-by: zirain <zirain2009@gmail.com> * don't skip gateways Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> Signed-off-by: shawnh2 <shawnhxh@outlook.com> * fix testdata Signed-off-by: shawnh2 <shawnhxh@outlook.com> * fix k8s provider controller Signed-off-by: shawnh2 <shawnhxh@outlook.com> * fix: retry reconcile on transient errors during reconcile (#6299) * fix: add isTransientError helper to classify retryable errors Introduces isTransientError to detect transient Kubernetes errors and enable proper reconciliation retries. Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com> handle errors from processing BackendRefs Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> handle errors from processing ConfigMap Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * skip invalid GatewayClass Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * address comment Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * handle all transient errors Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * don't skip failed GCs Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com> Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> (cherry picked from commit 71ce56f) Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * fix: fix bug in hostname overlap detection (#6332) fix bug in hostname overlap detection Signed-off-by: Rudrakh Panigrahi <rudrakh97@gmail.com> (cherry picked from commit e78e268) Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * fix telemetry with host port not working (#6460) Signed-off-by: zirain <zirain2009@gmail.com> (cherry picked from commit c0a2ce7) Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * bugfix: BackendTlsPolicy should not reference across namespace (#6309) * bugfix: BackendTlsPolicy should not reference across namespace Signed-off-by: zirain <zirain2009@gmail.com> (cherry picked from commit 9925189) Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Guy Daich <guy.daich@sap.com> Signed-off-by: shawnh2 <shawnhxh@outlook.com> Signed-off-by: Jeff Davis <mr.jefedavis@gmail.com> Signed-off-by: zirain <zirain2009@gmail.com> Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> Signed-off-by: Rudrakh Panigrahi <rudrakh97@gmail.com> Signed-off-by: sudipto baral <sudiptobaral.me@gmail.com> Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com> Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Co-authored-by: Guy Daich <guy.daich@sap.com> Co-authored-by: Jeff Davis <mr.jefedavis@gmail.com> Co-authored-by: zirain <zirain2009@gmail.com> Co-authored-by: Kota Kimura <86363983+kkk777-7@users.noreply.github.com> Co-authored-by: Rudrakh Panigrahi <rudrakh97@gmail.com> Co-authored-by: Sudipto Baral <sudiptobaral.me@gmail.com> Co-authored-by: Patryk Rostkowski <48490105+patrostkowski@users.noreply.github.com> Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com>
…#6299) * fix: add isTransientError helper to classify retryable errors Introduces isTransientError to detect transient Kubernetes errors and enable proper reconciliation retries. Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com> handle errors from processing BackendRefs Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> handle errors from processing ConfigMap Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * skip invalid GatewayClass Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * address comment Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * handle all transient errors Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * don't skip failed GCs Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com> Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: Tjeerd Jan van der Molen <34071+tjvdmolen@users.noreply.github.com>
* fix(translator): ext-proc full duplex streamed trailers and validation (envoyproxy#6323) * fix ext proc validation and trailer management for full duplex streamed mode Signed-off-by: Guy Daich <guy.daich@sap.com> Signed-off-by: shawnh2 <shawnhxh@outlook.com> * feat: disable automountServiceAccountToken for proxy and ratelimit (envoyproxy#6364) Signed-off-by: Jeff Davis <mr.jefedavis@gmail.com> * bugfix: make EnvoyPatchPolicy able to replace telemetry cluster (envoyproxy#6367) Signed-off-by: zirain <zirain2009@gmail.com> Signed-off-by: shawnh2 <shawnhxh@outlook.com> * feat: add validation of section name for Gateway listener (envoyproxy#6343) * add validation of section name Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> * update error status reason Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> * refactor: define as function of validate section name for gateway listener Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> Signed-off-by: shawnh2 <shawnhxh@outlook.com> * fix: add configMap indexers for EEP reconciler (envoyproxy#6369) Signed-off-by: Rudrakh Panigrahi <rudrakh97@gmail.com> * fix: use buildEndpointType for access and tracing (envoyproxy#6370) Signed-off-by: zirain <zirain2009@gmail.com> * fix: default accesslog not working (envoyproxy#6441) * fix default accesslog Signed-off-by: zirain <zirain2009@gmail.com> * release notes Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> Signed-off-by: shawnh2 <shawnhxh@outlook.com> * chore: fix cve (envoyproxy#6446) * fix cve Signed-off-by: zirain <zirain2009@gmail.com> * lint Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> * fix: Do not set backendRequestTimeout when Retries are set (envoyproxy#6421) * fix: Do not set backendRequestTimeout when Retries are set Signed-off-by: sudipto baral <sudiptobaral.me@gmail.com> * fix: update comment Signed-off-by: sudipto baral <sudiptobaral.me@gmail.com> --------- Signed-off-by: sudipto baral <sudiptobaral.me@gmail.com> * gatewayapi: don't append gwcResource if there's invalid GatewayClass (envoyproxy#6379) * gatewayapi: don't process gloabal resources when acceptedGateways is 0 Signed-off-by: zirain <zirain2009@gmail.com> * update Signed-off-by: zirain <zirain2009@gmail.com> * fix test Signed-off-by: zirain <zirain2009@gmail.com> * don't skip gateways Signed-off-by: zirain <zirain2009@gmail.com> --------- Signed-off-by: zirain <zirain2009@gmail.com> Signed-off-by: shawnh2 <shawnhxh@outlook.com> * fix testdata Signed-off-by: shawnh2 <shawnhxh@outlook.com> * fix k8s provider controller Signed-off-by: shawnh2 <shawnhxh@outlook.com> * fix: retry reconcile on transient errors during reconcile (envoyproxy#6299) * fix: add isTransientError helper to classify retryable errors Introduces isTransientError to detect transient Kubernetes errors and enable proper reconciliation retries. Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com> handle errors from processing BackendRefs Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> handle errors from processing ConfigMap Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * skip invalid GatewayClass Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * address comment Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * handle all transient errors Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * don't skip failed GCs Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com> Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> (cherry picked from commit 71ce56f) Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * fix: fix bug in hostname overlap detection (envoyproxy#6332) fix bug in hostname overlap detection Signed-off-by: Rudrakh Panigrahi <rudrakh97@gmail.com> (cherry picked from commit e78e268) Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * fix telemetry with host port not working (envoyproxy#6460) Signed-off-by: zirain <zirain2009@gmail.com> (cherry picked from commit c0a2ce7) Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> * bugfix: BackendTlsPolicy should not reference across namespace (envoyproxy#6309) * bugfix: BackendTlsPolicy should not reference across namespace Signed-off-by: zirain <zirain2009@gmail.com> (cherry picked from commit 9925189) Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> --------- Signed-off-by: Guy Daich <guy.daich@sap.com> Signed-off-by: shawnh2 <shawnhxh@outlook.com> Signed-off-by: Jeff Davis <mr.jefedavis@gmail.com> Signed-off-by: zirain <zirain2009@gmail.com> Signed-off-by: kkk777-7 <kota.kimura0725@gmail.com> Signed-off-by: Rudrakh Panigrahi <rudrakh97@gmail.com> Signed-off-by: sudipto baral <sudiptobaral.me@gmail.com> Signed-off-by: Patryk Rostkowski <patrostkowski@gmail.com> Signed-off-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Co-authored-by: Guy Daich <guy.daich@sap.com> Co-authored-by: Jeff Davis <mr.jefedavis@gmail.com> Co-authored-by: zirain <zirain2009@gmail.com> Co-authored-by: Kota Kimura <86363983+kkk777-7@users.noreply.github.com> Co-authored-by: Rudrakh Panigrahi <rudrakh97@gmail.com> Co-authored-by: Sudipto Baral <sudiptobaral.me@gmail.com> Co-authored-by: Patryk Rostkowski <48490105+patrostkowski@users.noreply.github.com> Co-authored-by: Huabing (Robin) Zhao <zhaohuabing@gmail.com> Signed-off-by: shawnh2 <shawnhxh@outlook.com>
What type of PR is this?
fix: #6284
What this PR does / why we need it:
This PR:
Description updated by @zhaohuabing on Jul 3 2025.
Which issue(s) this PR fixes:
Fixes #6284
Release Notes: Yes