Helper functions to use within Colaboratory
If you want to use it, it's probably best to fork it since I just hack what I currently need and I don't make sure it's backwards compatible (unless I need it), so functions might change without notice.
On the plus side:
- Free software: MIT license
- Helper functions I use, e.g. save/load from GDrive, move files around when preparing data, plotting etc.
- View the source code in https://github.com/EdlinOrg/colaboratory_helpers/tree/master/colaboratory_helpers to see what is there
Note: if you work on the same drive in one notebook its easier to just mount it
from google.colab import drive drive.mount('/content/gdrive')
Save/load files to/from GDrive using PyDrive:
To get gdrive-file-id:
Click "Share" -> "Advanced" in GDrive, the "Link to share" looks like this https://drive.google.com/file/d/GDRIVE-ID/view?usp=sharing and there you have the id
colaboratory_helpers.drive_load("filename.ext", "gdrive-file-id", force=True) colaboratory_helpers.drive_save("filename.ext")
Predict using fastai model (resnet34)
Save the model you trained in fastai
torch.save(learn.model, modelfilept)
Load and use that model for prediction elsewhere
from colaboratory_helpers import fastai_pyt_gpu clss=fastai_pyt_gpu.FaiGPU(modelfilept) clss.predictscore("somefile.jpg")
Install dlib:
!pip install -q cmake !pip install -q dlib
Install fastai 0.7.0:
!pip install -q torchtext==0.2.3 !pip install -q fastai==0.7.0
Install fastText
!pip install -q cmake !pip install -q wrap-pybind11 !git clone https://github.com/facebookresearch/fastText.git !cd fastText; pip install -q .
Install fasttext
!pip install -q cython !pip install -q fasttext
Imagemagick
!apt-get install imagemagick
Add at the top of notebooks
%load_ext autoreload %autoreload 2
colabDev=False if not colabDev: #!pip uninstall colaboratory_helpers -y !pip --no-cache-dir -q install --upgrade --force-reinstall --ignore-installed git+https://github.com/EdlinOrg/colaboratory_helpers/ from colaboratory_helpers import colaboratory_helpers #from colaboratory_helpers import colaboratory_helpers, fastai_analyze #import colaboratory_helpers.ffsfastText as ffsfastText #import colaboratory_helpers.fastTextPrep as fastTextPrep else: !pip install -q PyDrive !rm -f colaboratory_helpers.py !wget "https://raw.githubusercontent.com/EdlinOrg/colaboratory_helpers/master/colaboratory_helpers/colaboratory_helpers.py" import colaboratory_helpers #!rm -f fastai_analyze.py #!wget "https://raw.githubusercontent.com/EdlinOrg/colaboratory_helpers/master/colaboratory_helpers/fastai_analyze.py" #import fastai_analyze %matplotlib inline %pylab inline pylab.rcParams['figure.figsize'] = (16, 6) !date gdrivepathmodules="/content/gdrive/My Drive/colab/"
Add this at end of the notebook to get a notification that it is finished
!date !test ! -e done.wav && wget https://github.com/EdlinOrg/colaboratory_helpers/blob/master/assets/benhill.wav?raw=true -O done.wav import IPython IPython.display.Audio("done.wav",autoplay=True)
Resize to 50% !convert -resize 50% a.jpg b.jpg
Convert to grayscale !convert <img_in> -set colorspace Gray -separate -average <img_out>