Skip to content

Export some more symbols for libgoogle-cloud #402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
h-vetinari wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Export some more symbols for libgoogle-cloud #402

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
4f21db4
simplify generate_def.py
h-vetinari Mar 14, 2025
7b0cfb3
fix `GRPC{,XX}_DLL_EXPORTS` for `grpc++_*` targets
h-vetinari Mar 14, 2025
2570b75
add some exports for classes needed by google-cloud
h-vetinari Mar 14, 2025
64dd760
move CallbackUnaryCall from header to library on win
h-vetinari Mar 14, 2025
a1a1d40
iterate
h-vetinari Mar 14, 2025
f848fd9
debug: skip non-windows
h-vetinari Mar 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions recipe/generate_def.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,14 +58,14 @@
# from another object, but since we do a union, it doesn't matter);
# Static symbols aren't visible to the linker, so even if we put
# them in the .def file, they would not be found; save the hassle.
if "UNDEF" in line or "Static" in line:
if any(x in line for x in ["UNDEF", "Static", "Label"]):
continue
# get pure symbol, i.e. what comes after "|", minus spaces, and removing potentially
# trailing demangled names (e.g. "(`upb_FieldType_CType'::`2'::c_type)" above);
# don't use [-1] because some demangled symbols contain `operator|`
symbol = line.split("|")[1].strip().split()[0]
# skip labels and metadata
if "Label" in line or any(symbol.startswith(x) for x in [".", "$", "@", "??", "?$", "__"]):
# skip internal symbols; prefix to #symbols as of v1.71: {"??": 57072, "?$": 102, "__": 84}
if any(symbol.startswith(x) for x in ["??", "?$", "__"]):
Comment on lines +67 to +68
Copy link
Member Author

@h-vetinari h-vetinari Mar 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had actually checked if the templated functions are somehow part of the symbols being skipped here, but CallbackUnaryCall is part of grpc++, not grpc.

continue
# skip opencensus for now; re-evaluate together with #220
if "opencensus" in symbol.lower():
Expand Down
5 changes: 4 additions & 1 deletion recipe/meta.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ source:
- patches/0013-add-linkage-to-openssl-to-grpc_unsecure.patch # [win]
- patches/0014-reduce-grpc-symbol-explosion-with-autogenerated-expo.patch
- patches/0015-update-GRPC_DLL-instances-where-necessary.patch
- patches/0016-add-some-GRPCXX_DLL-exports-for-symbols-needed-by-gc.patch # [win]

{% if build_platform == target_platform and not win %}
- folder: third_party/benchmark
Expand Down Expand Up @@ -74,7 +75,9 @@ source:
{% endif %}

build:
number: 0
number: 1
# debug
skip: true # [not win]

outputs:
- name: libgrpc
Expand Down
2 changes: 1 addition & 1 deletion recipe/patches/0001-windows-ssl-lib-names.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From a1f2bdb936cf0720b3e6a825276089e40a47499a Mon Sep 17 00:00:00 2001
From: Jonathan Helmus <jjhelmus@gmail.com>
Date: Mon, 17 Feb 2020 15:45:06 -0600
Subject: [PATCH 01/15] windows ssl lib names
Subject: [PATCH 01/16] windows ssl lib names

Co-Authored-By: Julien Schueller <schueller@phimeca.com>
Co-Authored-By: Nicholas Bollweg <nick.bollweg@gmail.com>
Expand Down
2 changes: 1 addition & 1 deletion recipe/patches/0002-fix-win-setup-cmds.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From a41de15b2b9cc889582212d65bc545e7c3b50cc5 Mon Sep 17 00:00:00 2001
From: Mike Sarahan <msarahan@gmail.com>
Date: Tue, 18 Feb 2020 13:53:05 -0600
Subject: [PATCH 02/15] fix win setup cmds
Subject: [PATCH 02/16] fix win setup cmds

Co-Authored-By: Julien Schueller <schueller@phimeca.com>
Co-Authored-By: Nicholas Bollweg <nick.bollweg@gmail.com>
Expand Down
2 changes: 1 addition & 1 deletion recipe/patches/0003-Link-against-grpc-and-abseil.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From f671ec44b769d0463dfae979d9e223400d49132c Mon Sep 17 00:00:00 2001
From: Marius van Niekerk <marius.v.niekerk@gmail.com>
Date: Mon, 13 Jun 2022 17:13:07 -0400
Subject: [PATCH 03/15] Link against grpc and abseil
Subject: [PATCH 03/16] Link against grpc and abseil

Co-Authored-By: H. Vetinari <h.vetinari@gmx.com>
---
Expand Down
2 changes: 1 addition & 1 deletion recipe/patches/0004-force-protoc-executable.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 50ba587ddae9dc56a66e8213d6ec1c132a247688 Mon Sep 17 00:00:00 2001
From: "Uwe L. Korn" <uwe.korn@quantco.com>
Date: Fri, 11 Sep 2020 14:20:04 +0200
Subject: [PATCH 04/15] force protoc executable
Subject: [PATCH 04/16] force protoc executable

---
cmake/protobuf.cmake | 17 ++---------------
Expand Down
2 changes: 1 addition & 1 deletion recipe/patches/0005-mark-linkages-of-grpc-_unsecure-as-private.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 61600719d36d394f9e02a23cf85926b9e31bfcfc Mon Sep 17 00:00:00 2001
From: "H. Vetinari" <h.vetinari@gmx.com>
Date: Tue, 23 Aug 2022 11:45:20 +0200
Subject: [PATCH 05/15] mark linkages of grpc{,_unsecure} as private
Subject: [PATCH 05/16] mark linkages of grpc{,_unsecure} as private

except openssl & gpr, which are needed by dependents

Expand Down
20 changes: 10 additions & 10 deletions recipe/patches/0006-for-main-targets-set-_DLL_EXPORTS-when-building-and-.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
From b52e1b28dcf98f6a9a1127c0f72bfc9caf38d903 Mon Sep 17 00:00:00 2001
From b6a184e18b663c5c655488cf9266f8923827356b Mon Sep 17 00:00:00 2001
From: "H. Vetinari" <h.vetinari@gmx.com>
Date: Thu, 29 Feb 2024 20:31:21 +1100
Subject: [PATCH 06/15] for main targets, set *_DLL_EXPORTS when building and
Subject: [PATCH 06/16] for main targets, set *_DLL_EXPORTS when building and
*_DLL_IMPORTS in interface

---
CMakeLists.txt | 36 ++++++++++--------------------------
1 file changed, 10 insertions(+), 26 deletions(-)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index aa48ccb04c..fb3f60850b 100644
index aa48ccb04c..4bd0b0cac2 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -2092,10 +2092,8 @@ if(WIN32 AND MSVC)
Expand All @@ -20,7 +20,7 @@ index aa48ccb04c..fb3f60850b 100644
- PRIVATE
- "GPR_DLL_EXPORTS"
- )
+ set_target_properties(gpr PROPERTIES DEFINE_SYMBOL "GPR_DLL_EXPORTS")
+ target_compile_definitions(gpr PRIVATE "GPR_DLL_EXPORTS")
+ target_compile_definitions(gpr INTERFACE "GPR_DLL_IMPORTS")
endif()
if(gRPC_INSTALL)
Expand All @@ -34,7 +34,7 @@ index aa48ccb04c..fb3f60850b 100644
- "GRPC_DLL_EXPORTS"
- "GPR_DLL_IMPORTS"
- )
+ set_target_properties(grpc PROPERTIES DEFINE_SYMBOL "GRPC_DLL_EXPORTS")
+ target_compile_definitions(grpc PRIVATE "GRPC_DLL_EXPORTS")
+ target_compile_definitions(grpc INTERFACE "GRPC_DLL_IMPORTS")
endif()
if(gRPC_INSTALL)
Expand All @@ -48,7 +48,7 @@ index aa48ccb04c..fb3f60850b 100644
- "GRPC_DLL_EXPORTS"
- "GPR_DLL_IMPORTS"
- )
+ set_target_properties(grpc_unsecure PROPERTIES DEFINE_SYMBOL "GRPC_DLL_EXPORTS")
+ target_compile_definitions(grpc_unsecure PRIVATE "GRPC_DLL_EXPORTS")
+ target_compile_definitions(grpc_unsecure INTERFACE "GRPC_DLL_IMPORTS")
endif()
if(gRPC_INSTALL)
Expand All @@ -63,8 +63,8 @@ index aa48ccb04c..fb3f60850b 100644
- "GPR_DLL_IMPORTS"
- "GRPC_DLL_IMPORTS"
- )
+ set_target_properties(grpc++ PROPERTIES DEFINE_SYMBOL "GRPCXX_DLL_EXPORTS")
+ target_compile_definitions(grpc++ INTERFACE "GRPCXX_DLL_IMPORTS")
+ target_compile_definitions(grpc++ PRIVATE "GRPC_DLL_IMPORTS" "GRPCXX_DLL_EXPORTS")
+ target_compile_definitions(grpc++ INTERFACE "GRPC_DLL_IMPORTS" "GRPCXX_DLL_IMPORTS")
endif()
if(gRPC_INSTALL)
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/grpc++.pdb
Expand All @@ -78,8 +78,8 @@ index aa48ccb04c..fb3f60850b 100644
- "GPR_DLL_IMPORTS"
- "GRPC_DLL_IMPORTS"
- )
+ set_target_properties(grpc++_unsecure PROPERTIES DEFINE_SYMBOL "GRPCXX_DLL_EXPORTS")
+ target_compile_definitions(grpc++_unsecure INTERFACE "GRPCXX_DLL_IMPORTS")
+ target_compile_definitions(grpc++_unsecure PRIVATE "GRPC_DLL_IMPORTS" "GRPCXX_DLL_EXPORTS")
+ target_compile_definitions(grpc++_unsecure INTERFACE "GRPC_DLL_IMPORTS" "GRPCXX_DLL_IMPORTS")
endif()
if(gRPC_INSTALL)
install(FILES ${CMAKE_CURRENT_BINARY_DIR}/grpc++_unsecure.pdb
6 changes: 3 additions & 3 deletions recipe/patches/0007-add-missing-linkages-for-gRPC_BUILD_TESTS-ON.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
From 1a8f40a423a7381590bbe6aef170c1cb247f2d8d Mon Sep 17 00:00:00 2001
From 2574106452e727e15f61b9537da2f38c128eb1b8 Mon Sep 17 00:00:00 2001
From: "H. Vetinari" <h.vetinari@gmx.com>
Date: Tue, 26 Sep 2023 18:25:54 +1100
Subject: [PATCH 07/15] add missing linkages for gRPC_BUILD_TESTS=ON
Subject: [PATCH 07/16] add missing linkages for gRPC_BUILD_TESTS=ON

---
CMakeLists.txt | 9 +++++++++
1 file changed, 9 insertions(+)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index fb3f60850b..dbc8ac939c 100644
index 4bd0b0cac2..0c782cde52 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -8089,6 +8089,7 @@ target_include_directories(authorization_matchers_test
Expand Down
6 changes: 3 additions & 3 deletions recipe/patches/0008-add-enable_testing-so-tests-get-run.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
From c80cd00b33125db5e62805554bcf46a9f9f89bb1 Mon Sep 17 00:00:00 2001
From 404ea6b43dda4f7ce3a118070cf9c7ef62716be2 Mon Sep 17 00:00:00 2001
From: "H. Vetinari" <h.vetinari@gmx.com>
Date: Fri, 28 Feb 2025 22:07:19 +1100
Subject: [PATCH 08/15] add enable_testing so tests get run
Subject: [PATCH 08/16] add enable_testing so tests get run

---
CMakeLists.txt | 4 ++++
1 file changed, 4 insertions(+)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index dbc8ac939c..72ee817a05 100644
index 0c782cde52..3a61b4d88b 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -51,6 +51,10 @@ option(gRPC_BUILD_TESTS "Build tests" OFF)
Expand Down
6 changes: 3 additions & 3 deletions recipe/patches/0009-Build-protobufs-for-tests-only-if-gRPC_BUILD_TESTS.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
From 5e660b8727fd0386388b7b13ac310fbe60786f7c Mon Sep 17 00:00:00 2001
From b653b4eac8fb6da5e72064cfaf501123c3a808c1 Mon Sep 17 00:00:00 2001
From: "Uwe L. Korn" <uwe.korn@quantco.com>
Date: Thu, 28 Nov 2024 20:40:54 +0100
Subject: [PATCH 09/15] Build protobufs for tests only if gRPC_BUILD_TESTS
Subject: [PATCH 09/16] Build protobufs for tests only if gRPC_BUILD_TESTS

---
CMakeLists.txt | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 72ee817a05..b69b216f3d 100644
index 3a61b4d88b..2e231c7491 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -720,6 +720,9 @@ protobuf_generate_grpc_cpp_with_import_path_correction(
Expand Down
6 changes: 3 additions & 3 deletions recipe/patches/0010-unvendor-upb-and-utf8_range-use-protobuf-s-version.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
From 1e2008eb800e4cc2aa9935fcd9c970a61d09544c Mon Sep 17 00:00:00 2001
From 825a9cbda464425f22fd6dc887dae18126bca5be Mon Sep 17 00:00:00 2001
From: "H. Vetinari" <h.vetinari@gmx.com>
Date: Fri, 28 Feb 2025 15:09:41 +1100
Subject: [PATCH 10/15] unvendor upb and utf8_range, use protobuf's version
Subject: [PATCH 10/16] unvendor upb and utf8_range, use protobuf's version

---
CMakeLists.txt | 522 ++----------------------------------------------
cmake/upb.cmake | 4 +-
2 files changed, 16 insertions(+), 510 deletions(-)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index b69b216f3d..f6b263e764 100644
index 2e231c7491..9654b4416d 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -346,11 +346,15 @@ else()
Expand Down
6 changes: 3 additions & 3 deletions recipe/patches/0011-reduce-buildtests_cxx-target-to-end2end-tests.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 41bcad7889bdb4f6ce7edf38594ad1cc3e1f64ad Mon Sep 17 00:00:00 2001
From 5778a21aa4f44beffd79002c3ac4a038ac97c61c Mon Sep 17 00:00:00 2001
From: "H. Vetinari" <h.vetinari@gmx.com>
Date: Sun, 2 Mar 2025 08:22:39 +1100
Subject: [PATCH 11/15] reduce buildtests_cxx target to end2end tests
Subject: [PATCH 11/16] reduce buildtests_cxx target to end2end tests

as of v1.69, the whole set of tests takes extremely long to build (>6h),
so reduce it to only "end2end" tests. This has the advantage of being
Expand All @@ -15,7 +15,7 @@ Plus four support libraries (`grpc{,++}_test{,_config,_util}`) and the vendored
1 file changed, 50 insertions(+), 688 deletions(-)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index f6b263e764..31d10c10a6 100644
index 9654b4416d..77f95ed57f 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -47,11 +47,12 @@ set(gRPC_INSTALL_SHAREDIR "share/grpc" CACHE STRING "Installation directory for
Expand Down
4 changes: 2 additions & 2 deletions recipe/patches/0012-build-grpcio-tools-from-source.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 6bdcff13fe6ef5c5fc18b50f0a12cf37343917c6 Mon Sep 17 00:00:00 2001
From 444b5a7225d2121cc5ffca19e8397f0652116beb Mon Sep 17 00:00:00 2001
From: "H. Vetinari" <h.vetinari@gmx.com>
Date: Tue, 4 Mar 2025 08:53:30 +1100
Subject: [PATCH 12/15] build grpcio-tools from source
Subject: [PATCH 12/16] build grpcio-tools from source

---
tools/distrib/python/grpcio_tools/setup.py | 43 ++++++++++++++++------
Expand Down
6 changes: 3 additions & 3 deletions recipe/patches/0013-add-linkage-to-openssl-to-grpc_unsecure.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 2aeef12d417bd880de8004aaf8ceaeaf26a82361 Mon Sep 17 00:00:00 2001
From 3cdda15b6bb09062417e229f2cc923cdccb86d9d Mon Sep 17 00:00:00 2001
From: "H. Vetinari" <h.vetinari@gmx.com>
Date: Sat, 1 Mar 2025 08:06:28 +1100
Subject: [PATCH 13/15] add linkage to openssl to grpc_unsecure
Subject: [PATCH 13/16] add linkage to openssl to grpc_unsecure

grpc_unsecure actually requires ParsePemCertificateChain / ParsePemPrivateKey
from that TU in ValidatePemKeyCertPair, see
Expand All @@ -12,7 +12,7 @@ https://github.com/grpc/grpc/blob/v1.69.0/src/core/tsi/ssl_transport_security_ut
1 file changed, 2 insertions(+)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 31d10c10a6..e533337cdf 100644
index 77f95ed57f..e8b8f0ed29 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -3029,6 +3029,7 @@ add_library(grpc_unsecure
Expand Down
6 changes: 3 additions & 3 deletions recipe/patches/0014-reduce-grpc-symbol-explosion-with-autogenerated-expo.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From 37e86c55123f8285c5db9ca3953bc236aad85882 Mon Sep 17 00:00:00 2001
From 51ce3916498bd730747f5a07e52075f4a3e11ad0 Mon Sep 17 00:00:00 2001
From: "H. Vetinari" <h.vetinari@gmx.com>
Date: Wed, 5 Mar 2025 13:21:01 +1100
Subject: [PATCH 14/15] reduce grpc symbol explosion with autogenerated export
Subject: [PATCH 14/16] reduce grpc symbol explosion with autogenerated export
definitions file

Basically, WINDOWS_EXPORT_ALL_SYMBOLS is a nuclear option that includes all symbols
Expand Down Expand Up @@ -38,7 +38,7 @@ other consumers end up linking to them.
1 file changed, 24 insertions(+)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index e533337cdf..86c06eb89b 100644
index e8b8f0ed29..d39f232f94 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -2396,6 +2396,30 @@ add_library(grpc
Expand Down
4 changes: 2 additions & 2 deletions recipe/patches/0015-update-GRPC_DLL-instances-where-necessary.patch
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
From a1b3b6828b80bc9b669aedc51c085992b05ed8f3 Mon Sep 17 00:00:00 2001
From 8d5565ecbd8abc79bed0f26fd09c19ab1c2b93a8 Mon Sep 17 00:00:00 2001
From: "H. Vetinari" <h.vetinari@gmx.com>
Date: Sun, 2 Mar 2025 07:16:19 +1100
Subject: [PATCH 15/15] update GRPC_DLL instances where necessary
Subject: [PATCH 15/16] update GRPC_DLL instances where necessary

---
include/grpc/event_engine/event_engine.h | 2 +-
Expand Down
Loading
Loading