Skip to content
View microprediction's full-sized avatar

Block or report microprediction

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Please don't include any personal information such as legal names or email addresses. Maximum 100 characters, markdown supported. This note will be visible to only you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
microprediction/README.md

microprediction (Peter's Repos)

See also:

I'm a career quant, applied mathematician, open-source developer, entrepreneur and dad.

Interests

  • Portfolio and ensemble construction (e.g. paper and blog where I unified the two sides of portfolio theory).
  • OTC microstructure (my day job so not public .. but for older work see this or that or the other).
  • Thurstone models
  • Derivative-free optimization
  • Time-series
  • Sports analytics
  • Collective Intelligence
  • LLM uses for alpha, code gen, prediction and instrumentation (e.g. pi).
  • LLM samplers

Microprediction

I wrote a book on what we might call the Strong Indispensable Markets Hypothesis, and to help it along in a small way I run contests like mid-one which you might want to check out too as there is a lot of prize-money. I like to provoke people into using market-inspired collective mechanisms for prediction. I used the options market to effortlessly beat 97% of participants in the year-long M6 contest - see the post or article. My book is a meditation on the power of mini-markets and algorithmic micro-managers in a very specific yet ubiquitous domain: frequently repeated prediction. It predates and extends "Info Finance" coined by Vitalik Buterin. Read the awards and reviews.

Monteprediction

Allow me to mention a long-running game where you hurl a million 11-dimensional Monte Carlo samples at my server.

  1. Open this colab notebook or script (yes there's an R version),
  2. Change the email, at minimum,
  3. Run it. Every weekend.
  4. Check your scores at www.monteprediction.com

The notebook also describes the scoring mechanism. Ask questions in the slack (see bottom of leaderboard for slack invite).

Now, here's some attempt to introduce you to my open source work ...

Derivative-free optimizer comparisons

The humpDay package is intended to help you choose a derivative-free optimizer for your use case.

Schur Complementary Portfolios

My work on unifying Hierarchical Risk Parity with minimum variance portfolio optimization sits in the precise package although I've been meaning to put it somewhere else.

Incremental time-series and benchmarking

The timemachines package enumerates online methods and makes some effort to evaluate univariate methods against the corpus of time-series drawn from the microprediction platform. It is an attempt to reduce everything to relatively pure functions:

$$ f : (y_t, state; k) \mapsto ( [\hat{y}(t+1),\hat{y}(t+2),\dots,\hat{y}(t+k) ], [\sigma(t+1),\dots,\sigma(t+k)], posterior\ state)) $$

where $\sigma(t+l)$ estimates the standard error of the prediction $\hat{y}(t+l)$.

Benchmarking overview

Topic Package Elo ratings Methods Data sources
Univariate time-series timemachines Timeseries Elo ratings Most popular packages (list) microprediction streams
Global derivative-free optimization humpday Optimizer Elo ratings Most popular packages (list) A mix of classic and new objectives
Covariance, precision, correlation precise See notebooks cov and portfolio lists Stocks, electricity etc

These packages aspire to advance online autonomous prediction in a small way, but also help me notice if anyone else does.

Winning

The winning package includes my recently published fast algorithm for inferring relative ability from win probabilities, at any scale. As explained in the paper the uses extend well beyond the pricing of quinellas!

First Down

My firstdown repo contains analysis aspiring to ruin great game of football. See Wilmott paper and for heaven's sake, don't stretch out for the first down. That's obviously nuts.

Some other repos

  • randomcov - A set of quirky correlation and covariance matrix generators (I'd love your ideas).
  • embarrassingly - A speculative approach to robust optimization that sends impure objective functions to optimizers.
  • pandemic - Ornstein-Uhlenbeck epidemic simulation (related paper)
  • momentum - My most personally re-used mini package ... for incremental mean, var, skew, kurtosis.
  • muid - Memorable Unique Identifiers ... try to figure out how that can't be an oxymoron.
  • timeseries-notebooks - Lots of examples of using open source timeseries packages.
  • correlationbounds - Mini package for conf bounds
  • building_an_open_ai_network - Book related.
  • recalibrate - Utils related to Platt scaling etc.

The currently defunct real-time time-series platform

The real-time system previously maintained by yours truly has entered a trisoloran dehydrated state but will hopefully be revived at a future date, after one of my three hundred ChatGPT generated scientific grant proposals is successful. Here's how some of the open-source stuff used to propagate down into the "algo fight club".

  1. The "/skaters" provide canonical, single-line of code access to functionality drawn from packages like river, pydlm, tbats, pmdarima, statsmodels.tsa, neuralprophet, Facebook Prophet, Uber's orbit, Facebook's greykite and more.

  2. The StreamSkater makes it easy to use any "skater".

  3. Choices are sometimes advised by Elo ratings, but anyone can do what they want.

  4. It's not too hard to use my HumpDay package for offline meta-param tweaking, et cetera.

  5. It's not too hard to use my precise package for online ensembling.

A few repos that drove this:

  • The muid identifier package is explained in this video.
  • microconventions captures things common to client and server, and may answer many of your more specific questions about prediction horizons, et cetera.
  • rediz contains server side code. For the brave.
  • There are other rats and mice like getjson, runthis and momentum.

Pinned Loading

  1. timemachines timemachines Public

    Predict time-series with one line of code.

    Python 413 50

  2. microprediction microprediction Public

    If you can measure it, consider it predicted

    Jupyter Notebook 336 61

  3. humpday humpday Public

    Elo ratings for global black box derivative-free optimizers

    Python 133 18

  4. winning winning Public

    Inference of relative ability from winning probabilities

    Jupyter Notebook 42 13

  5. optimizer-elo-ratings optimizer-elo-ratings Public

    Assigns Elo Ratings to Python Global Optimizers

    22 6

  6. timeseries-elo-ratings timeseries-elo-ratings Public

    Elo ratings for time-series forecasting packages

    Python 23 5