Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Task][prism]: Be able to execute non-Go SDKs on Prism. #28187

Open
14 of 15 tasks
Tracked by #29650
lostluck opened this issue Aug 28, 2023 · 1 comment
Open
14 of 15 tasks
Tracked by #29650

[Task][prism]: Be able to execute non-Go SDKs on Prism. #28187

lostluck opened this issue Aug 28, 2023 · 1 comment

Comments

@lostluck
Copy link
Contributor

lostluck commented Aug 28, 2023

Backlink to global prism tracking issue: #29650

What needs to happen?

Umbrella task for tracking PRs to improve prism and get it to support non-go SDKs.

More targeted tasks should be filed for other work items, and linked here.

This includes

@lostluck lostluck self-assigned this Aug 28, 2023
lostluck added a commit that referenced this issue Aug 28, 2023
…ecv size (#28184)

* [prism] worker shutdown, cleanup, log fail

* Increase prism server receive size to max.
lostluck added a commit to lostluck/beam that referenced this issue Aug 30, 2023
…grpc recv size (apache#28184)

* [prism] worker shutdown, cleanup, log fail

* Increase prism server receive size to max.
lostluck added a commit that referenced this issue Sep 18, 2023
* Add container using standalone prism go precommit.

* Add GoPrism precommit action.

* Don't serve prism UI by default.

---------

Co-authored-by: lostluck <[email protected]>
lostluck added a commit that referenced this issue Sep 20, 2023
* Move wk.Stop() to context cancel.

* [prism] Basic Xlang support.

---------

Co-authored-by: lostluck <[email protected]>
@lostluck lostluck added the prism label Jul 2, 2024
acrites pushed a commit to acrites/beam that referenced this issue Jul 17, 2024
lostluck added a commit that referenced this issue Oct 7, 2024
* [#28187] Add gradle targets to execute python tests with prism.

* Add issues for non-expansion service skipped tests.

* Format diff.

---------

Co-authored-by: lostluck <[email protected]>
@lostluck
Copy link
Contributor Author

I think the final trick: Being able to have Java build a prism instance for you via a Go invocation is possible, but I don't know exactly how to test it presently, because Java development set ups are esoteric by nature.

Technically it would live in PrismLocator.java, and replicate the logic in the Python command here:

https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/prism_runner.py#L204

That is, roughly: "if the Beam version is a development version, then we use Go to try and and install build prism directly to the cache, and fail out if there's no Go, or there's a compile error".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant