Conda gpu env

[jdye64]

Closes #627

[ayushdg]

cc: @charlesbluca Curious to know if you have opinions on this and if you have a preference for where these files should live and how many we should maintain

[codecov-commenter]

Codecov Report

❗ No coverage uploaded for pull request base (datafusion-sql-planner@f87aa36). Click here to learn what that means.
The diff coverage is n/a.

@@                    Coverage Diff                    @@
##             datafusion-sql-planner     #630   +/-   ##
=========================================================
  Coverage                          ?   62.07%           
=========================================================
  Files                             ?       73           
  Lines                             ?     3615           
  Branches                          ?      737           
=========================================================
  Hits                              ?     2244           
  Misses                            ?     1221           
  Partials                          ?      150           

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

[charlesbluca]

Thanks for the ping @ayushdg!

My preference here would be to have:

• a single environment file matching up with the gpuCI testing PYTHON_VER
• for this file to reside within continuous_integration/gpuci/
• potentially for this file to contain only the extra packages installed in this step of the Dockerfile

I'd ideally like this file to make it so we can easily recreate the gpuCI testing environment, which is done by creating the environment created by environment-{PYTHON_VER}-jdk11-dev.yaml and then installing the gpuCI requirements on top of this environment. In this case, we'd achieve this with something like:

conda env create -f continuous_integration/environment-$PYTHON_VER-jdk11-dev.yaml
conda env update -n dask-sql -f continuous_integration/gpuci/environment.yaml

I also think that keeping this file in the gpuCI folder would remove ambiguity on its intended purpose.

As an aside, I think that having the gpuCI requirements specified within the repo is a great way to make the contents of the gpuCI environment more transparent! It might be worthwhile to see if we can find a way to pull these requirements in from the repo during the image builds, which would make it a lot clearer how to bump packages in the gpuCI images (thinking of #614). But would probably like for something like that to be a Dask-wide change, so can hold off on that for now.

[charlesbluca]

We'll probably want to add the other GPU packages installed in the image to this list:

https://github.com/rapidsai/dask-build-environment/blob/8da1d032d855d487f760749f8bffeb1e532caea9/dask_sql.Dockerfile#L25-L35

[jdye64]

This has been handled with another PR