Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BQ Storage Write API wrapper for Python SDK #25521

Closed
wants to merge 10,000 commits into from

Conversation

ahmedabu98
Copy link
Contributor

Implementing a wrapper for Python SDK that uses Java's Storage API SchemaTransform to write to BigQuery

Fixes #21961

pabloem and others added 30 commits January 18, 2023 12:53
Bumps [torch](https://github.com/pytorch/pytorch) from 1.12.1 to 1.13.1.
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/master/RELEASE.md)
- [Commits](pytorch/pytorch@v1.12.1...v1.13.1)

---
updated-dependencies:
- dependency-name: torch
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…#25072)

This was added as a temporary work-around when the XLang expansion service did not support returning additional dependencies. This removes that dependency prevent conflicts and allowing the user to control which Kafka dependency version they want to use.

For apache#25070
…ction. (apache#24871)

* timer fix rough draft (apache#24617)

* comments (apache#24617)

* comments (1) (apache#24617)

* cancel run on reset (apache#24617)

* regenerated mocks (apache#24617)

* runStopDate (apache#24617)

* OutputFilterTypeController (apache#24617)

* resetErrorMessageText (apache#24617)

Co-authored-by: alexeyinkin <[email protected]>

* resetErrorMessageText (apache#24617)

Co-authored-by: alexeyinkin <[email protected]>

* comments (apache#24617)

* missing license (apache#24617)

* a commit for re-running rat check on github

* deleted comment "a commit for re-running rat check on github"

* fixes after merge with master (apache#24617)

* merge fixes (apache#24617)

* regenerated yaml files (apache#24617)

* snippetEditingControllerGetter refinement (apache#24617)

Co-authored-by: darkhan.nausharipov <[email protected]>
Co-authored-by: alexeyinkin <[email protected]>
…5075)

Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.107.0 to 0.108.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](googleapis/google-api-go-client@v0.107.0...v0.108.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix input type

* use sys for size
…che#25080)

* Add support for templating in Tour of Beam learning materials

* Process learning materials through the template engine

* refactoring ci/cd for templated learning materials

* adding template processing

* refactoring learning content

* fixing incorrect sample path

* minor formatting comments

* missed complexity added add code review comments

* fix possible  err overwrite

Co-authored-by: Timur Sultanov <[email protected]>
* Add support for using builtins typing

* lint and formatting

* tests

* fix tests

* fix tests

* fix tests

* fix tests
…he#25090)

Bumps [cloud.google.com/go/spanner](https://github.com/googleapis/google-cloud-go) from 1.42.0 to 1.43.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](googleapis/google-cloud-go@spanner/v1.42.0...spanner/v1.43.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/spanner
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [torch](https://github.com/pytorch/pytorch) from 1.12.0 to 1.13.1.
- [Release notes](https://github.com/pytorch/pytorch/releases)
- [Changelog](https://github.com/pytorch/pytorch/blob/master/RELEASE.md)
- [Commits](pytorch/pytorch@v1.12.0...v1.13.1)

---
updated-dependencies:
- dependency-name: torch
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…he#25095)

Bumps [cloud.google.com/go/storage](https://github.com/googleapis/google-cloud-go) from 1.28.1 to 1.29.0.
- [Release notes](https://github.com/googleapis/google-cloud-go/releases)
- [Changelog](https://github.com/googleapis/google-cloud-go/blob/main/CHANGES.md)
- [Commits](googleapis/google-cloud-go@storage/v1.28.1...spanner/v1.29.0)

---
updated-dependencies:
- dependency-name: cloud.google.com/go/storage
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@ahmedabu98
Copy link
Contributor Author

Run Python_Xlang_Gcp_Dataflow PostCommit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment