Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Imputations should also work with ndarray data #4437

Closed
michaelosthege opened this issue Jan 24, 2021 · 0 comments · Fixed by #4439
Closed

Imputations should also work with ndarray data #4437

michaelosthege opened this issue Jan 24, 2021 · 0 comments · Fixed by #4439
Assignees

Comments

@michaelosthege
Copy link
Member

michaelosthege commented Jan 24, 2021

There are many cases where observations are not just a pandas.Series but instead an ndim>=1 ndarray.
In such cases the automatic imputation of float("nan") values does not work, because pymc3.model.pandas_to_array only looks for NaN if data is a pandas object.

The point of pandas_to_array is to convert data to a numpy ndarray, using a numpy.ma.MaskedArray for the imputation case.
It would make sense to support ndarray input too.

Description of your problem

Please provide a minimal, self-contained, and reproducible example.

data = numpy.array([
    [1,2,3],
    [4,5,float("nan")],
    [7,8,9],
])
print(data)
with pymc3.Model():
    pymc3.Normal(
        "L",
        mu=pymc3.Normal("x", shape=data.shape),
        sd=10,
        observed=data,
        shape=data.shape
    )
    pymc3.sample()

Please provide the full traceback.

SamplingError: Initial evaluation of model at starting point failed!
Starting values:
{'x': array([[0., 0., 0.],
       [0., 0., 0.],
       [0., 0., 0.]])}

Initial evaluation results:
x   -8.27
L     NaN
Name: Log-probability of test_point, dtype: float64

Versions and main components

  • PyMC3 Version: 3.11.0
  • Theano Version: 1.1.0
  • Python Version: 3.7
@michaelosthege michaelosthege changed the title Sampling with imputations is broken Imputations only work with 1D pandas.Series Jan 24, 2021
@michaelosthege michaelosthege changed the title Imputations only work with 1D pandas.Series Imputations should also work with ndarray data Jan 24, 2021
michaelosthege added a commit to michaelosthege/pymc that referenced this issue Jan 24, 2021
michaelosthege added a commit to michaelosthege/pymc that referenced this issue Jan 24, 2021
@michaelosthege michaelosthege self-assigned this Jan 24, 2021
michaelosthege added a commit to michaelosthege/pymc that referenced this issue Jan 25, 2021
michaelosthege added a commit to michaelosthege/pymc that referenced this issue Jan 25, 2021
michaelosthege added a commit to michaelosthege/pymc that referenced this issue Jan 25, 2021
michaelosthege added a commit to michaelosthege/pymc that referenced this issue Jan 25, 2021
michaelosthege added a commit to michaelosthege/pymc that referenced this issue Jan 25, 2021
michaelosthege added a commit to michaelosthege/pymc that referenced this issue Jan 25, 2021
michaelosthege added a commit to michaelosthege/pymc that referenced this issue Jan 25, 2021
michaelosthege added a commit that referenced this issue Jan 28, 2021
* Support imputations with ndarray data (closes #4437)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant