Skip to content

deps: update abseil and grpc dependencies#16687

Merged
yanavlasov merged 9 commits intoenvoyproxy:mainfrom
antoniovicente:grpc-1.38
Jun 15, 2021
Merged

deps: update abseil and grpc dependencies#16687
yanavlasov merged 9 commits intoenvoyproxy:mainfrom
antoniovicente:grpc-1.38

Conversation

@antoniovicente
Copy link
Copy Markdown
Contributor

No description provided.

Signed-off-by: Antonio Vicente <avd@google.com>
@repokitteh-read-only repokitteh-read-only bot added the deps Approval required for changes to Envoy's external dependencies label May 27, 2021
@repokitteh-read-only
Copy link
Copy Markdown

CC @envoyproxy/dependency-shepherds: Your approval is needed for changes made to (bazel/.*repos.*\.bzl)|(bazel/dependency_imports\.bzl)|(api/bazel/.*\.bzl)|(.*/requirements\.txt)|(.*\.patch).

🐱

Caused by: #16687 was opened by antoniovicente.

see: more, trace.

@antoniovicente
Copy link
Copy Markdown
Contributor Author

antoniovicente commented May 27, 2021

Blocked by grpc/grpc#26359 and possibly grpc doing a release unless we chose to apply a local patch.

Or merging in #16029 before updating the grpc dependency.

Signed-off-by: Antonio Vicente <avd@google.com>
@antoniovicente
Copy link
Copy Markdown
Contributor Author

Windows failure looks like a clang compiler crash. Not sure how to get the input files that it wants attached to the bug report: https://dev.azure.com/cncf/4684fb3d-0389-4e0b-8251-221942316e06/_apis/build/builds/76968/logs/71

/retest

@repokitteh-read-only
Copy link
Copy Markdown

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #16687 (comment) was created by @antoniovicente.

see: more, trace.

@moderation
Copy link
Copy Markdown
Contributor

Preference from a dependency point of view is to not introduce a patch. Experience shows that patches will make future maintenance of the dependency more difficult / fragile /cc @htuch

@antoniovicente
Copy link
Copy Markdown
Contributor Author

Bother, the windows clang compiler crash seems to be deterministic.

@antoniovicente
Copy link
Copy Markdown
Contributor Author

Preference from a dependency point of view is to not introduce a patch. Experience shows that patches will make future maintenance of the dependency more difficult / fragile /cc @htuch

Upgrading the grpc dependency is important for testability of the ALPN extension. We can try to ask grpc of a release once they accept a patch to their build under C++ 17 and see how that goes. It would be good to track dependencies more closely, we fell behind on grpc by 6 months and multiple releases.

@asraa
Copy link
Copy Markdown
Contributor

asraa commented May 27, 2021

It would be good to track dependencies more closely, we fell behind on grpc by 6 months and multiple releases.

We have a log warning system that runs on dep check i think. Can we turn that into an issue filer on postsubmit?

@antoniovicente
Copy link
Copy Markdown
Contributor Author

It would be good to track dependencies more closely, we fell behind on grpc by 6 months and multiple releases.

We have a log warning system that runs on dep check i think. Can we turn that into an issue filer on postsubmit?

That would be great. Since you have more context on what is available, could you file an issue to track that improvement?

@moderation
Copy link
Copy Markdown
Contributor

@antoniovicente I'm working to get an updated release_dates.py but here is the shortened output of our deps.

*WARNING* antlr4_runtimes has a newer release than 4.7.2@<2018-12-18 19:46:46>: 4.9.2@<2021-03-11 22:18:16>
*WARNING* bazel-gazelle has a newer release than v0.22.2@<2020-10-02 21:22:18>: v0.22.3@<2020-12-23 18:58:38>
*WARNING* bazel-gazelle has a newer release than v0.22.2@<2020-10-02 21:22:18>: v0.23.0@<2021-03-08 20:19:35>
*WARNING* boringssl has 44 commits since c5ad6dcb65e532589e8acb9e9adbde62463af13d@<2021-04-22 18:26:10>
*WARNING* libcircllhist has 52 commits since 63a16dd6f2fc7bc841bb17ff92be8318df60e2e1@<2019-02-11 21:23:49>
*WARNING* com_github_cyan4973_xxhash has a newer release than v0.7.3@<2020-03-05 15:54:16>: v0.8.0@<2020-07-27 18:57:06>
*WARNING* com_github_datadog_dd_opentracing_cpp has a newer release than v1.2.1@<2021-01-27 03:07:27>: v1.3.0@<2021-03-25 00:41:46>
*WARNING* com_github_fmtlib_fmt has a newer release than 7.0.3@<2020-08-07 13:39:57>: 7.1.3@<2020-11-25 01:15:02>
*WARNING* com_github_gabime_spdlog has a newer release than v1.7.0@<2020-07-09 17:45:54>: v1.8.5@<2021-03-25 19:00:48>
*WARNING* com_github_google_benchmark has a newer release than v1.5.1@<2020-06-09 08:54:15>: v1.5.3@<2021-04-23 11:55:33>
*WARNING* flatbuffers has 379 commits since a83caf5910644ba1c421c002ef68e42f21c15f9f@<2020-04-02 19:55:30>
*WARNING* com_github_google_flatbuffers has a newer release than a83caf5910644ba1c421c002ef68e42f21c15f9f@<2020-04-02 19:55:30>: v2.0.0@<2021-05-10 18:45:16>
*WARNING* jwt_verify_lib has 8 commits since 28efec2e4df1072db0ed03597591360ec9f80aac@<2020-11-05 00:05:06>
*WARNING* tcmalloc has 141 commits since 9f385356c34d4fc11f76a000b609e2b446c20667@<2020-11-04 06:57:03>
*WARNING* com_github_gperftools_gperftools has a newer release than gperftools-2.8@<2020-07-06 10:00:10>: gperftools-2.9.1@<2021-03-03 03:32:31>
*WARNING* com_github_grpc_grpc has a newer release than v1.34.0@<2020-12-02 17:33:52>: v1.38.0@<2021-05-20 18:43:08>
*WARNING* yaml-cpp has 1 commits since 98acc5a8874faab28b82c28936f4b400b389f5d6@<2020-07-27 18:49:04>
*WARNING* libevent has 78 commits since 62c152d9a7cd264b993dad730c4163c6ede2e0a3@<2020-07-28 08:04:16>
*WARNING* LuaJIT has 53 commits since 1d8b747c161db457e032a023ebbff511f5de5ec2@<2020-10-12 14:18:02>
*WARNING* tclap has a newer release than tclap-1-2-1-release-final@<2011-04-16 22:31:18>: v1.2.2@<2017-12-26 14:40:35>
*WARNING* tclap has a newer release than tclap-1-2-1-release-final@<2011-04-16 22:31:18>: v1.2.3@<2021-01-05 23:05:02>
*WARNING* tclap has a newer release than tclap-1-2-1-release-final@<2011-04-16 22:31:18>: v1.2.4@<2021-03-21 12:43:00>
*WARNING* com_github_nghttp2_nghttp2 has a newer release than v1.42.0@<2020-11-23 15:06:32>: v1.43.0@<2021-02-02 10:45:20>
*WARNING* http-parser has 1 commits since 4f15b7d510dc7c6361a26a7c6d2f7c3a17f8d878@<2020-07-10 10:00:34>
*WARNING* rapidjson has 92 commits since dfbe1db9da455552f7a9ad5d2aea17dd9d832ac1@<2019-12-03 01:51:05>
*WARNING* com_github_wasmtime has a newer release than v0.26.0@<2021-04-05 20:39:37>: v0.26.1@<2021-05-21 21:19:07>
*WARNING* WAVM has 8 commits since 79c3aa29366615d9b1593cd527e5b4b94cc6072a@<2021-03-31 20:00:10>
*WARNING* zlib-ng has 126 commits since b802a303ce8b6c86fbe3f93d59e0a82333768c0c@<2020-10-18 13:39:11>
*WARNING* com_github_zlib_ng_zlib_ng has a newer release than b802a303ce8b6c86fbe3f93d59e0a82333768c0c@<2020-10-18 13:39:11>: 2.0.3@<2021-05-13 09:03:11>
*WARNING* abseil-cpp has 91 commits since 5d8fc9192245f0ea67094af57399d7931d6bd53f@<2020-11-24 07:43:25>
*WARNING* com_google_absl has a newer release than 5d8fc9192245f0ea67094af57399d7931d6bd53f@<2020-11-24 07:43:25>: 20210324.1@<2021-04-20 16:55:14>
*WARNING* cel-cpp has 31 commits since 9841e3ee251f3cc4cd5b6dd9deee6818bc9f2854@<2020-12-17 19:08:55>
*WARNING* com_google_cel_cpp has a newer release than 9841e3ee251f3cc4cd5b6dd9deee6818bc9f2854@<2020-12-17 19:08:55>: v0.5.0@<2021-03-31 20:11:06>
*WARNING* googletest has 242 commits since a4ab0abb93620ce26efad9de9296b73b16e88588@<2020-09-10 23:09:30>
*WARNING* com_google_protobuf has a newer release than v3.16.0@<2021-05-07 00:13:24>: v3.17.1@<2021-05-22 06:04:09>
*WARNING* com_googlesource_code_re2 has a newer release than 2020-07-06@<2020-07-06 05:35:12>: 2021-04-01@<2021-03-31 13:00:00>
*WARNING* lightstep-tracer-cpp has 1 commits since 1942b3f142e218ebc143a043f32e3278dafec9aa@<2020-08-25 05:02:09>
*WARNING* com_lightstep_tracer_cpp has a newer release than 1942b3f142e218ebc143a043f32e3278dafec9aa@<2020-08-25 05:02:09>: v0.14.0@<2020-09-06 22:13:39>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.8@<2020-10-24 17:42:23>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.9@<2020-11-16 22:58:38>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.10@<2020-12-04 16:53:37>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.11@<2020-12-17 22:47:38>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.12@<2021-01-09 15:25:52>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.13@<2021-01-30 02:25:41>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.14@<2021-02-14 19:46:10>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.15@<2021-03-06 07:18:22>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.16@<2021-03-26 02:00:07>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.17@<2021-04-11 00:41:55>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.18@<2021-04-24 01:35:03>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.19@<2021-05-04 22:25:01>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.20@<2021-05-06 21:31:10>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.21@<2021-05-18 15:39:21>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.22@<2021-05-25 14:21:40>
*WARNING* emsdk has a newer release than 2.0.7@<2020-10-13 13:51:20>: 2.0.23@<2021-05-27 01:10:10>
*WARNING* opencensus-cpp has 2 commits since ba631066779a534267fdb1321b19850eb2b0c000@<2020-10-08 04:36:50>
*WARNING* io_opentracing_cpp has a newer release than v1.5.1@<2019-01-16 20:32:57>: v1.6.0@<2019-09-18 21:59:25>
*WARNING* kafka has a newer release than 2.4.1@<2020-03-03 00:32:42>: 2.5.0@<2020-04-08 01:13:18>
*WARNING* kafka has a newer release than 2.4.1@<2020-03-03 00:32:42>: 2.5.1@<2020-07-24 00:05:07>
*WARNING* kafka has a newer release than 2.4.1@<2020-03-03 00:32:42>: 2.6.0@<2020-07-28 18:16:35>
*WARNING* kafka has a newer release than 2.4.1@<2020-03-03 00:32:42>: 2.6.1@<2020-12-10 19:31:07>
*WARNING* kafka has a newer release than 2.4.1@<2020-03-03 00:32:42>: 2.6.2@<2021-04-08 05:57:09>
*WARNING* kafka has a newer release than 2.4.1@<2020-03-03 00:32:42>: 2.7.0@<2020-12-16 13:58:29>
*WARNING* kafka has a newer release than 2.4.1@<2020-03-03 00:32:42>: 2.7.1@<2021-04-08 11:29:38>
*WARNING* kafka has a newer release than 2.4.1@<2020-03-03 00:32:42>: 2.8.0@<2021-04-14 14:28:09>
*WARNING* brotli has 14 commits since 0cd2e3926e95e7e2930f57ae3f4885508d462a25@<2020-09-08 14:48:31>
*WARNING* org_llvm_llvm has a newer release than llvmorg-10.0.0@<2020-03-24 13:46:06>: llvmorg-12.0.0@<2021-04-14 18:00:43>
*WARNING* org_llvm_releases_compiler_rt has a newer release than llvmorg-11.0.1@<2021-01-06 03:18:47>: llvmorg-12.0.0@<2021-04-14 18:00:43>
*WARNING* proxy-wasm-cpp-host has 6 commits since 31c75e0039f2f5c42dc6e12556cb151a38da6d8b@<2021-05-06 07:40:14>
*WARNING* proxy-wasm-rust-sdk has 14 commits since 28a94df25659b2107b67a11df0112f8f6833558b@<2021-02-09 21:48:04>
*WARNING* rules_antlr has 52 commits since 3cc2f9502a54ceb7b79b37383316b23c4da66f9a@<2019-06-21 20:13:52>
*WARNING* rules_antlr has a newer release than 3cc2f9502a54ceb7b79b37383316b23c4da66f9a@<2019-06-21 20:13:52>: 0.5.0@<2020-01-21 21:16:59>
*WARNING* rules_cc has 26 commits since b1c40e1de81913a3c40e5948f78719c28152486d@<2020-11-11 13:29:11>
*WARNING* rules_foreign_cc has 123 commits since d54c78ab86b40770ee19f0949db9d74a831ab9f0@<2020-10-26 14:26:01>
*WARNING* rules_foreign_cc has a newer release than d54c78ab86b40770ee19f0949db9d74a831ab9f0@<2020-10-26 14:26:01>: 0.2.0@<2021-03-18 15:41:36>
*WARNING* rules_python has a newer release than 0.1.0@<2020-10-15 22:29:27>: 0.2.0@<2021-03-31 04:32:44>
*WARNING* rules_rust has 59 commits since 1b648302edb64d3ddcc159655bf065bff40e6571@<2021-04-02 05:39:02>
*WARNING* upb has 77 commits since de76b31f9c56b28120580d53a6f8d7941fdb79eb@<2020-12-29 21:42:45>

Once I get the checker update merged I'll work on bumping the deps.

@htuch
Copy link
Copy Markdown
Member

htuch commented May 28, 2021

With @envoyproxy/dependency-shepherds hat on, if it's going to be fixed in a reasonable timely way upstream, let's just do that and skip patch; i.e. if it's going to happen in a week or so. We don't need to consume point release versions, we often use a SHA from main branch in this situation temporarily until the next point release.

If this is not going to happen upstream, then we can merge this as long as a tracking bug is filed to remove the patch.

Patches are expedient, but in the longer term they slow down upgrades and make it even harder to keep up with releases.

Signed-off-by: Antonio Vicente <avd@google.com>
Signed-off-by: Antonio Vicente <avd@google.com>
Signed-off-by: Antonio Vicente <avd@google.com>
Signed-off-by: Antonio Vicente <avd@google.com>
@antoniovicente
Copy link
Copy Markdown
Contributor Author

grpc/grpc#26359 was merged 4 days ago. No point release includes it yet. I guess that me being OOO the past week worked in the favor of skipping the patch.

Except that even newer grpc also needs a newer version of abseil, so I'm also updating that. Hopefully picking up 6 months of changes to grpc and abseil won't cause any problems.

@antoniovicente antoniovicente changed the title deps: update grpc dependency to 1.38.0 deps: update abseil and grpc dependencies Jun 8, 2021
@antoniovicente antoniovicente marked this pull request as ready for review June 8, 2021 16:15
urls = ["https://github.com/abseil/abseil-cpp/archive/{version}.tar.gz"],
use_category = ["dataplane_core", "controlplane"],
release_date = "2020-11-24",
release_date = "2021-06-07",
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UTC for this commit is 2021-06-03 and this is why CI has failed. As per prior conversation it would be preferable if we could use the latest release of abseil at https://github.com/abseil/abseil-cpp/releases/tag/20210324.2. There is a parallel PR to bump abseil at #16681

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@htuch @moderation

I see comments about #16681 having trouble compiling under Windows. Given all these difficulties, I would strongly prefer to go with a patch to grpc and decouple the abseil version bump from the grpc version bump.

Can I please go back to the version with the patch?

Signed-off-by: Antonio Vicente <avd@google.com>
@antoniovicente
Copy link
Copy Markdown
Contributor Author

/retest

#16907 for the test that failed in the most recent attempt.

@repokitteh-read-only
Copy link
Copy Markdown

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #16687 (comment) was created by @antoniovicente.

see: more, trace.

Copy link
Copy Markdown
Member

@htuch htuch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm deps

@repokitteh-read-only repokitteh-read-only bot removed the deps Approval required for changes to Envoy's external dependencies label Jun 9, 2021
@yanavlasov
Copy link
Copy Markdown
Contributor

So it looks like latest absl works fine with CL.

@yanavlasov
Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 1 pipeline(s).

@antoniovicente
Copy link
Copy Markdown
Contributor Author

/retest

@repokitteh-read-only
Copy link
Copy Markdown

Retrying Azure Pipelines:
Check envoy-presubmit isn't fully completed, but will still attempt retrying.
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #16687 (comment) was created by @antoniovicente.

see: more, trace.

Signed-off-by: Antonio Vicente <avd@google.com>
@repokitteh-read-only repokitteh-read-only bot added the deps Approval required for changes to Envoy's external dependencies label Jun 14, 2021
Signed-off-by: Antonio Vicente <avd@google.com>
@yanavlasov yanavlasov merged commit 460ac5f into envoyproxy:main Jun 15, 2021
mum4k pushed a commit to envoyproxy/nighthawk that referenced this pull request Jun 25, 2021
- Adding new enum to source/client/stream_decoder.cc to reflect changes in envoyproxy/envoy#17044
- Adding a binding for external dependency on gtest introduced in envoyproxy/envoy#16687
- Sync to latest envoy included changes to `.bazelrc` and `ci/run_envoy_docker.sh` copied directly.

Signed-off-by: William Juan <66322422+wjuan-AFK@users.noreply.github.com>
leyao-daily pushed a commit to leyao-daily/envoy that referenced this pull request Sep 30, 2021
Signed-off-by: Antonio Vicente <avd@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deps Approval required for changes to Envoy's external dependencies

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants