00-intro.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Mixing Compiled Code and Python\n",
    "\n",
    "## Henry Schreiner"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Motivation\n",
    "\n",
    "As a language, Python is great. (At least, Python 3.7+ is ;) ). And there are some *fantastic* libraries available for it. Numba, Pandas, and many more. It is flexible, powerful, and quite high-level, and doesn't require compiling; you can start getting results immediately.\n",
    "\n",
    "However, there is one thing that Python is not. And that's *Fast*.\n",
    "\n",
    "But, we can solve that very elegantly (see the CPU and GPU performance minicourse materials for ways to make plain Python supercharged Python!)\n",
    "\n",
    "But something is still missing.\n",
    "\n",
    "Compiled libraries. There are many compiled libraries out there that you'd like to be able to use *without* having to rewrite them. Maybe it's a huge or old library, or maybe it is supposed to \"work everywhere\" - whatever the reason, you want to access it from Python.\n",
    "\n",
    "You are in luck; Python historically has been a great \"glue\" language, and the multitude of libraries include some excellent adapter libraries!\n",
    "\n",
    "And, just to note; you *can* use this to speed up performance. It's probably not the best way (see above) unless you are using existing compiled libraries, however, in which case it falls into the category above."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What this course it is\n",
    "\n",
    "* A deep dive into compiled code + CPython.\n",
    "* A look at binding C and C++ libraries to CPython.\n",
    "* A *very* lightweight discussion of Python packaging"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What this course is not\n",
    "\n",
    "* An in-depth discussion of languages I'm not very familiar with (Fortran, Swift, Rust, Haskel, Go)\n",
    "* A discussion of how to speed up your code. That's a topic of past courses (see [CPU](https://github.com/henryiii/python-performance-minicourse), [GPU]()).\n",
    "* Python 2 compatible (ask me for old material if you are unlucky enough to be stuck on a dead language, and every library shown here will support Python 2, at least with an old version)\n",
    "* A course about *other* implementations of the Python language - we will focus on CPython. PyPy will be able to be used in some cases, and IronPython, Jython, etc. are out of the picture here.\n",
    "\n",
    "![Timeline](pybindingstimeline.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Topics covered\n",
    "\n",
    "* CTypes and shared object files\n",
    "    * CFFI on the side\n",
    "* CPython: The nitty-gritty of a hand written binding\n",
    "* SWIG: Multi language headaches\n",
    "* Cython: Two languages in one!\n",
    "* pybind11: Perfection, librarified\n",
    "* CPPYY: The new and wild kid on the block.\n",
    "\n",
    "Eventually I could add RPython, which is PyPy's preferred method and does support CPython.\n",
    "\n",
    "Note these are focused on C and C++. You can also access Rust, Fortran, and others using similar ideas."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ## Disclaimer\n",
    "> \n",
    "> The following notebooks are mostly designed for UNIX systems. If you are running directly on a Windows machine, you should probably use a remote system, the WSL (2), or cygwin. All the basic concepts work fine on Windows; it's just harder to run directly from the command line. About half of the notebooks now support Windows directly now by using setuptools or CMake instead of direct command line runs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ## Setup issues\n",
    ">\n",
    "> When we run the command line, including to run setuptools or CMake, you may not \"find\" the right python - the command line in the notebook may not have the conda environment active. On a bash-like shell (most UNIX), you can start the commands with `conda activate compiled-minicourse && `."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Reasons to Compile\n",
    "\n",
    "## Speed\n",
    "\n",
    "Try other methods first! See if there's a library that can help. Numba can give you full or beyond full performance of a compiled language without having to leave Python. You can access GPUs without compiling. You could try PyPy or Pyjion. Etc.\n",
    "\n",
    "## Access libraries\n",
    "\n",
    "This is the most common reason, and a very good reason to compile. We'll focus on this mostly today.\n",
    "\n",
    "## Single language\n",
    "\n",
    "Related to the last point, you can implement your code once, then access it from both a compiled language and from Python.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# About me\n",
    "\n",
    "I'm a Computation Physicist and Research Software Engineer at Princeton. I am an admin for Scikit-HEP, a collection of 30+ libraries for High Energy Physics, where I work on several libraries and on packaging. I also maintain Plumbum, a scripting library for Python. I am on the maintainer team for several projects, including pybind11 (which we will see today) and cibuildwheel, which is a tool to help you build all possible wheels for upload to PyPI. I am the author of CLI11, the world's most powerful C++ command line parser (used in the Windows Shell), and a few other things. My blog is https://iscinumpy.gitlab.io. I've also designed front-end and back-end software for a ventilation monitor in PyQT5 + Python 3.7 on Raspberry π's."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}