Skip to content

pm.sample segfaults on Apple Silicon: fork default + Accelerate BLAS #8372

Description

@jessegrabowski

On Apple Silicon pm.sample defaults to mp_ctx="fork", but conda's numpy now links Apple Accelerate whose worker threads don't survive fork(), so any model large enough to hit Accelerate's threaded BLAS path segfaults every chain worker.

import numpy as np
import pymc as pm

N = 500_000  # large enough that X @ beta hits Accelerate's threaded BLAS path
X = np.random.default_rng(0).normal(size=(N, 10))
y = X @ np.arange(10.0) + np.random.default_rng(1).normal(size=N)

with pm.Model() as model:
    beta = pm.Normal("beta", shape=10)
    pm.Normal("y", mu=X @ beta, sigma=1.0, observed=y)
    pm.sample(draws=100, tune=100, chains=2, cores=2)  # workers die -> EOFError
    # workaround: pm.sample(..., mp_ctx="spawn")

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions