Repository for the models used by the CodeClusters
We are using Jupyter to prototyping our models.
You should have Python >=3.5 installed, pyenv or mkvirtualenv for creating virtual environments. Also you need Java JDK >=11 to be able to run Checkstyle Java library to generate metrics. NOTE: as of 5.8.2022 the dev-requirements.txt
were not updated although requirements.txt
were. psycopg2
might work better now but the situation is still not ideal regarding Python's package management.
- Generate virtual environment eg:
mkvirtualenv cc
- Install requirements:
pip install -r dev-requirements.txt
- Due to some weird unimaginable things, installation of psycopg2 didn't work directly on macOS. So following this thread what I had to do was:
env LDFLAGS="-I/usr/local/opt/openssl/include -L/usr/local/opt/openssl/lib" pip install psycopg2
. But first try justpip install psycopg2
to see if it works - Install Checkstyle to
./tmp
:mkdir tmp && cd tmp && curl -OL https://github.com/checkstyle/checkstyle/releases/download/checkstyle-10.3.2/checkstyle-10.3.2-all.jar && cd ..
- Copy the example environment variables:
cp .example.env .env
- Launch the notebook:
jupyter notebook
- In http://localhost:8888 open the
notebooks
folder to find the models - You can also start the model server with:
./dev.sh
NOTE: there is some problems with Flask not triggering properly on code changes
You can install new dependencies without stopping the notebook, just open another terminal and remember to activate the virtualenv eg: workon cc
. Then just use pip: pip install tensorflow
. Remember to save the new library if you use it in some models: pip freeze > dev-requirements.txt
This is the setup used for production.
- Copy the example env:
cp .modeling.env.example .modeling.env
. Change variables if necessary - Build the image:
docker-compose -f prod-docker-compose.yml build
- Start it:
docker-compose -f prod-docker-compose.yml up
I have already added the required files to run ANTLR locally, but if you for some reason need to reinstall ANTLR this is how I did it (1.8.2022).
# Executed with macOS Big Sur
# needs Java 11
brew install openjdk@11
sudo ln -sfn /opt/homebrew/opt/openjdk@11/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk-11.jdk
# Download the JAR file
wget http://www.antlr.org/download/antlr-4.8-complete.jar
wget https://raw.githubusercontent.com/antlr/grammars-v4/master/java/java/JavaLexer.g4
wget https://raw.githubusercontent.com/antlr/grammars-v4/master/java/java/JavaParser.g4
java -jar ./antlr-4.8-complete.jar -Dlanguage=Python3 ./JavaLexer.g4
java -jar ./antlr-4.8-complete.jar -Dlanguage=Python3 ./JavaParser.g4