Update requirements.txt

[jaketae]

This PR adds two more dependencies (transformers, deepspeed) to requirements.txt that are needed to install this repository.

[thomasw21]

Small question, can we specify version?

[jaketae]

@thomasw21 We could, and we probably should. I just wasn't sure which version people are using for their dev environment. Here's what pip list gives me on my local. Does this seem about right? If so, I'll go ahead and specify the versions.

Package            Version   Location
------------------ --------- ------------------------------------------------------
certifi            2021.5.30
charset-normalizer 2.0.4
click              8.0.1
deepspeed          0.4.4
filelock           3.0.12
huggingface-hub    0.0.12
idna               3.2
importlib-metadata 4.6.3
joblib             1.0.1
megatron-lm        1.1.5     /Users/jaketae/Documents/Dev/GitHub/Megatron-DeepSpeed
ninja              1.10.2
numpy              1.21.1
packaging          21.0
pip                21.2.2
protobuf           3.17.3
psutil             5.8.0
pybind11           2.7.1
pyparsing          2.4.7
PyYAML             5.4.1
regex              2021.8.3
requests           2.26.0
sacremoses         0.0.45
setuptools         40.8.0
six                1.16.0
tensorboardX       1.8
tokenizers         0.10.3
torch              1.9.0
tqdm               4.62.0
transformers       4.9.1
typing-extensions  3.10.0.0
urllib3            1.26.6
zipp               3.5.0

[stas00]

Thank you, @jaketae

With one correction

[stas00]

I don't think hardcoded versions are a great idea unless you know you really need a specific version. Otherwise you run into conflicts with other projects and necessitate a requirement for a dedicated conda env just for this repo. So my vote is no hardcoded versions unless it's needed. Specifying module>min_version is goodness, but again, only if you know need at least min_version of some module.

In general, when in doubt see how HF transformers does it as generally a lot of thought has been put into that project, so mimicking most things there in general should be a good path.

[jaketae]

So my vote is no hardcoded versions unless it's needed. Specifying module>min_version is goodness, but again, only if you know need at least min_version of some module.

I don't know on top of my head if there is a specific package version we need (other than the branch of deepspeed). If there is, maybe we can specify the minimum version, but unless that's the case I think we can go versionless for now!