Remove pyarrow from setup.py

[bnaul]

#414 added pyarrow to the setup.py as part of adding google-bigquery-storage to install_requires. I understand the motivation for adding google-bigquery-storage, but since that has its own allowed pyarrow version range, it seems simpler to just rely on that rather than specifying yet another pyarrow range here which can get (/ currently is) out of sync with that library.

@tswast what do you think?

[tswast]

If we were to remove it, I think we'd need to make sure we specify the pyarrow extras https://github.com/googleapis/python-bigquery-storage/blob/55d7c5656909b177fe198bd0dc61757f30f5b7a0/setup.py#L34 in the google-cloud-bigquery-storage dependency

python-bigquery-sqlalchemy/setup.py

Line 87 in 52ec808

"google-cloud-bigquery-storage>=2.0.0,<3.0.0dev",

Alternatively, there may be a google-cloud-bigquery "extra" that'll get the DB-API features we need.

[tswast]

Related: googleapis/python-bigquery#1196 is an open issue for a request to make pyarrow optional again in google-cloud-bigquery

[bnaul]

If we were to remove it, I think we'd need to make sure we specify the pyarrow extras https://github.com/googleapis/python-bigquery-storage/blob/55d7c5656909b177fe198bd0dc61757f30f5b7a0/setup.py#L34 in the google-cloud-bigquery-storage dependency

Why is that exactly? I don't see any explicit uses of pyarrow here, my thinking was that if other libraries that we depend on here need it then it'll get pulled in during the setup for those libraries. But for example this seems like another good motivation for removing:

Related: googleapis/python-bigquery#1196 is an open issue for a request to make pyarrow optional again in google-cloud-bigquery

If this were made optional, right now there'd need to be a separate PR (like this one) to remove that dependency here. But if you just defer to google-cloud-bigquery on what is required then you don't have to worry about it. Does that make sense or am I missing something about how this library relies on pyarrow?

[tswast]

Does that make sense or am I missing something about how this library relies on pyarrow?

The DB-API package that this depends on can optionally rely on the BigQuery Storage API, which requires pyarrow to deserialize the results. This can be much faster for downloading large results, but won't make much of a difference for smaller result sizes.

[tswast]

If we were to drop pyarrow, we should also drop bqstorage. Ideally, we'd include these as "extras" and make sure we have tests where these packages are not installed & installed.

[chalmerlowe]

Related: googleapis/python-bigquery#1196 is an open issue for a request to make pyarrow optional again in google-cloud-bigquery

1196 got closed via this PR:

https://github.com/googleapis/python-bigquery/pull/1282/files

That PR made pyarrow and google-cloud-bigquery-storage optional in python-bigquery

[chalmerlowe]

@bnaul closing this PR based on some work related to making both pyarrow and bqstorage optional in PR #847.
Thanks so much for issuing this PR and providing impetus to improving python-bigquery-sqlalchemy.