Anatomy Of a Conda Recipe

Reference material I used:
Conda: http://conda.pydata.org/docs/build.html
Luarocks: https://github.com/keplerproject/luarocks/wiki/Installation-instructions-for-Unix

There are four main files. I'll go over each using lua-gnuplot as an example of building a Lua package, with additional comments inline.

If you're interested in building binary packages, you can check out the files for e.g. fftw. The principles are the same, but with the (perhaps) more familiar make toolchain instead of the luarocks toolchain. Note in fftw's build.sh file that I'm spamming $PREFIX. A central difficulty/goal in conda is to get the package to install in local directory, not broadly on a user's system. Conda takes care of portability if you can really get a package to install in whatever conda sets $PREFIX to.

You can also peep the files in the repo for reference

meta.yaml
build.sh
post-link.sh
pre-unlink.sh

meta.yaml

This file specifies the overall architecture and providence of the package. Here is an example meta.yaml file from gnuplot.

# Pretty standard, just name and version. 
# NOTE: It seems like you CANNOT have non-numeric versions, e.g. scm-1. I got weird bugs then.
# I prefix all lua packages with lua-, to avoid future name collisions with non-lua packages in the conda ecosystem.
package:
  name: lua-gnuplot
  version: 1.0

# You can load code from github, or from a tarball at a URL.
# Consult http://conda.pydata.org/docs/build.html for all formats. 
# When possible, use git_tag, instead of git_rev. It's cleaner.
source:
  git_url: [email protected]:torch/gnuplot.git
  git_rev: 35f4250471217f66329781e2818c9ef8cbe7b5f3

# Leave this section as is.
build:
  number: 0
  detect_binary_files_with_prefix: true

# Two different requirements sections — 
# Those packages required for building, and those required for running. 
requirements:
  build:
    - luajit
    - luarocks
    - lua-torch7
    - lua-paths
  run:
    - luajit
    - lua-torch7

# Could do much more extensive testing, and should for packages with tests.
# For now, just trying to import the package is a big enough hurdle.
test:
  commands:
    - luajit -e "require 'gnuplot'"

# Usually gleaned from the rockspec. 
about:
  home: https://github.com/torch/gnuplot
  license: BSD

Nowhere in the meta.yaml file do I specify how to actually build the project (although you can). For that, i rely on a separate build.sh file.

build.sh

Here's an example build.sh file. It does not vary much for almost every lua package I've built so far.

# Make sure luarocks can see all dependencies
# e.g. if we've just installed a bunch of lua packages by copying, it won't know they've been installed,
# but we can make luarocks see those packages by running this command.
luarocks-admin make_manifest

# Install gnuplot
# (rely on the rockspec for doing the heavy lifting, if there is any)
luarocks install rocks/gnuplot-scm-1.rockspec

Last, there's two files that don't change much at all. If you're not installing a Lua package, they might not even be necessary.

post-link.sh

# This code gets called right after conda is done installing your package.
# Perhaps you need to do some extra set up, like registering a package with a user's 
# package manager (like we're going to do below), or symlinking some config files
# into $HOME, for example. Anything that build.sh doesn't cover, you can do here.
# However! Note that conda won't know to undo what you do here if the package gets removed!
# That's what the next file, "pre-unlink.sh" is for.

# Let luarocks know that we've installed a new project
luarocks-admin make_manifest

pre-unlink.sh

# This code gets called right before the package is removed from the user's conda setup.
# You don't have to remove files that are created during the build process. However,
# perhaps e.g. you symlinked some config files into $HOME, or registered a package
# with a package manager (like we do with luarocks). You'll have to undo that manually,
# because conda might not know you did that.

# Be a good citizen and update the luarocks manifest when we remove a package
luarocks remove gnuplot

Building

The files above are pretty much it. Place them in an appropriately named directory (recipes/gnuplot in this case).

> pwd
~/recipes
> conda build gnuplot
# ... all kindsa stuff.
# ... NOTE WARNING ALERT:
# ... If you see ANYTHING installing into $HOME or /usr/local or something other than 
# ... the build environment directory (should be like $HOME/anaconda/envs/_build/ or something of the like)
# ... then STOP immediately and make sure that all of your cmake $PREFIXs and what not are set to build and install
# ... only LOCALLY and not globally on the system. This is a key point for getting these packages to be relocatable
# ... and installable.
# ... 
# ... Also make sure that you see `TEST END` at the end of the build, and no errors crop up.
# ... if you want to try installing it and playing with the package, then do
> conda install gnuplot --use-local -y

Provide feedback

Saved searches