Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added instructions for using pip install #545

Merged
merged 8 commits into from
Aug 29, 2024
Merged

Added instructions for using pip install #545

merged 8 commits into from
Aug 29, 2024

Conversation

touma-I
Copy link
Collaborator

@touma-I touma-I commented Aug 26, 2024

Why are these changes needed?

Enhance quick-start documentation to explain how to use pip install for the data-prep-kit and transforms.

Related issue number (if any).

@touma-I touma-I marked this pull request as ready for review August 26, 2024 21:41
@Bytes-Explorer
Copy link
Collaborator

Bytes-Explorer commented Aug 27, 2024

Thanks for this PR. We should also add commands for user to create an environment and then use dpk in it. Something like below

conda create -n data-prep-kit-1 -y python=3.11

activate the new conda environment
conda activate data-prep-kit-1
make sure env is swithced to data-prep-kit-1

Check python version
python --version
should say : 3.11

install jupyter lab
pip3 install jupyterlab

In this environment dpk can be installed via pip.

This will make sure user is working on py3.11 and avoid py3.11 conflicts with any existing py version.

@Bytes-Explorer
Copy link
Collaborator

Also as part of this PR, can we remove the additional NB https://github.com/IBM/data-prep-kit/blob/dev/examples/notebooks/code/sample-notebook_llama.ipynb which was meant for addition to the llama repo

@@ -1,5 +1,39 @@
# Quick Start for Data Prep Kit
Here we provided short examples of various uses of the Data Prep Kit.
Here we provided short examples of various uses of the Data Prep Kit. Most users who want to jump right in can use standard pip install to deploy the data-prep-kit and the python or ray transforms to their virtual python environment.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have a create-venv.md that breaks this out and then has separate sections for conda and venv. It would also be good to have a link to install conda. I believe venv comes with the base python install so no link needed there.

python -m venv venv
source venv/bin/activate
pip install ...
...
deactivate

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@daw3rd Let's have that as a separate issue/PR to revisit the documentation. As a new comer, and please I don't want anyone to take this the wrong way as I know a lot of effort and thoughts went into it, but all those different MD we have are quite confusing to navigate and increase difficult for maintaining it all. I don't know if adding one more will help or hurt. Maybe neither but there is a point to be made about producing the right amount of documentation that meets most users need at this point and iterate as our users (emphasis on users) ask for it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just did this and tested on both conda and venv environments,

https://github.com/sujee/data-prep-kit/blob/rag-example1/examples/notebooks/rag/setup-python-dev-env.md

happy to submit a PR for dev-env setup, if needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@touma-I Yep this is a style thing. I personally do not like navigating through a long document to find the content I want and find it much easier to have a page full of links to focused content.

@daw3rd
Copy link
Member

daw3rd commented Aug 27, 2024

Also, we don't yet have the transforms/packaging directory so probably should not be telling folks how to install data-prep-toolkit-transforms until we have mainline support for that. @touma-I

daw3rd
daw3rd previously requested changes Aug 27, 2024
doc/quick-start/quick-start.md Show resolved Hide resolved
doc/quick-start/quick-start.md Show resolved Hide resolved
@daw3rd
Copy link
Member

daw3rd commented Aug 28, 2024

To fix the language failures I think you will need to merge from dev, which now includes #528.

Copy link
Member

@shahrokhDaijavad shahrokhDaijavad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went ahead and tested noop transform successfully after following the instructions in quick-start.md file.

@touma-I touma-I dismissed daw3rd’s stale review August 29, 2024 19:02

All valid comments and will be addressed in follow-up iteration.

@touma-I touma-I merged commit b811b95 into dev Aug 29, 2024
20 of 21 checks passed
@touma-I touma-I deleted the pip-instructions branch August 29, 2024 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants