Handle new pymc and pytensor releases

[jessegrabowski]

Closes #328

Lots of tests fail following pymc-devs/pymc#7047, pymc-devs/pymc#7211, and pymc-devs/pymc#7238

It also seems that 5.13 breaks RVs with scan in the logp if they have strict=True, because the rng isn't provided. MarginalModel, Statespace, and DiscreteMarkovChain fail now. Not sure what's up with that, but it doesn't seem right. Easy fix is to set strict=False, but curious why it's happening.

The new shape rules break the r2d2 tests, @ferrine

TestSkellam has an illegal test value that results in log(0), @wd60622

Also python 3.9 is no longer being supported following pymc-devs/pymc#7227, so we should update the CI to no longer test on it.

[ricardoV94]

There was already a PR open to fix the MarginalModel. Actually the breaking changes there provide a fix to the numpyro bug.

#323

[jessegrabowski]

Check the CI run for this PR, I'm pretty sure the MarginalModel failures are new and unrelated to #323

[jessegrabowski]

Also it seems like the windows and unbuntu runs are grabbing different versions of PyMC, because they can't agree on where dataset_to_point_list should be imported from

[ricardoV94]

Also it seems like the windows and unbuntu runs are grabbing different versions of PyMC, because they can't agree on where dataset_to_point_list should be imported from

probably the python 3.9 forcing an older version of pymc

[zaxtax]

Is that PR good to merge?

…

On Tue, 9 Apr 2024, 08:34 Ricardo Vieira, ***@***.***> wrote: There was already a PR open to fix the MarginalModel. Actually the breaking changes there are a fix to the numpyro bug. — Reply to this email directly, view it on GitHub <#329 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAACCUOSMH7ETCJZC5IM6WTY4ODWNAVCNFSM6AAAAABF5SEYNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBUGI2DKNBTHE> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[ricardoV94]

Is that PR good to merge?

No we still need to run on the CI

[zaxtax]

I meant is #323 ready to not be a draft?

…

On Tue, 9 Apr 2024, 09:12 Ricardo Vieira, ***@***.***> wrote: Is that PR good to merge? On Tue, 9 Apr 2024, 08:34 Ricardo Vieira, *@*.***> wrote: There was already a PR open to fix the MarginalModel. Actually the breaking changes there are a fix to the numpyro bug. — Reply to this email directly, view it on GitHub #329 (comment) <#329 (comment)> , or unsubscribe https://github.com/notifications/unsubscribe-auth/AAACCUOSMH7ETCJZC5IM6WTY4ODWNAVCNFSM6AAAAABF5SEYNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBUGI2DKNBTHE . You are receiving this because you are subscribed to this thread.Message ID: *@*.***> No we still need to run on the CI — Reply to this email directly, view it on GitHub <#329 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAACCUIBG4GBX4SQESS7HH3Y4OIGPAVCNFSM6AAAAABF5SEYNWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBUGI4TIMZVG4> . You are receiving this because you commented.Message ID: ***@***.***>

[ricardoV94]

I meant is #323 ready to not be a draft?

I'll probably merge it here

[ricardoV94]

@maresb any idea why the Windows environment is not getting solved? https://github.com/pymc-devs/pymc-experimental/actions/runs/8614534973/job/23608187593?pr=329#step:5:568

   C:\Windows\system32\cmd.exe /D /S /C "C:\Miniconda3\condabin\mamba.bat env update --name pymc-experimental-test --file D:\a\pymc-experimental\pymc-experimental\conda-envs\setup-miniconda-patched-windows-environment-test.yml"
  Channels:
   - conda-forge
   - defaults
  Platform: win-64
  Collecting package metadata (repodata.json): ...working... done
  Solving environment: ...working... failed
  Channels:
   - conda-forge
   - defaults
  Platform: win-64
  Collecting package metadata (repodata.json): ...working... done
  Solving environment: ...working... failed
  Warning: 
  LibMambaUnsatisfiableError: Encountered problems while solving:
    - package python-3.12.2-h1d929f7_0 is excluded by strict repo priority
  
  
  
  
  LibMambaUnsatisfiableError: Encountered problems while solving:
    - package python-3.12.2-h1d929f7_0 is excluded by strict repo priority

[ricardoV94]

@jessegrabowski I fixed the SymbolicRandomVariables. The major change is that we require rngs to be handled manually by default now. It's just a question of specifying it in the inputs of the respective OpFromGraph constructor.

[jessegrabowski]

I saw. I'm pushing a patch for the distributions used by statespace as well.

[ricardoV94]

I saw. I'm pushing a patch for the distributions used by statespace as well.

Already did that

[ricardoV94]

So we need to fix the windows CI (no idea what's going on), and the R2D2 thing. @ferrine can you look into the R2? There are no longer constant coords/dims so the code has to change

[ricardoV94]

For the nan mask thing, let's add a pytest.warns for now, and open an issue on the PyMC side. We should replace nan with a discrete value after masking so that this warning isn't triggered (or suppress it)

[ricardoV94]

It looks like it's set to ignore overflows, but we're getting a divide by zero and an invalid input warning. We'd have to set the errorstate to ignore both divide and invalid:

with np.errstate(divide='ignore', invalid='ignore'):
    stats.skellam(0.9, 0.99).pmf(-3)

Sounds good, we can pass a wrapper for the reference logp/logcdf that does that

[jessegrabowski]

You want me to pass a wrapper? I just set the context on the whole test

[ricardoV94]

You want me to pass a wrapper? I just set the context on the whole test

That's also fine, but if it's the same work, more specific is nicer

[ricardoV94]

You want me to pass a wrapper? I just set the context on the whole test

That's also fine, but if it's the same work, more specific is nicer

Ignore, I just added a comment. Looks fine

[ricardoV94]

The PyPI job is failing now, that sounds like fun :)

[ricardoV94]

Maybe related to my changes to the github actions

[jessegrabowski]

When I use a wrapper, the test fails completely:

        def skellam_pmf_with_errstate(value, mu1, mu2):
            with np.errstate(divide="ignore", invalid="ignore"):
               pmf_x = scipy.stats.skellam.pmf(value, mu1, mu2)
            return pmf_x

        check_logp(
            Skellam,
            I,
            {"mu1": Rplus_small, "mu2": Rplus_small},
            skellam_pmf_with_errstate,
        )

Can you reproduce? Makes me worried something else is going on inside the Op itself that is being hidden.

[ricardoV94]

Reverting the actions didn't help. @lucianopaz any idea what may be going wrong?

https://github.com/pymc-devs/pymc-experimental/actions/runs/8688042500/job/23822814914?pr=329#step:5:29

ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/home/runner/work/pymc-experimental/pymc-experimental/dist/pymc-experimental*.tar.gz'

[lucianopaz]

Reverting the actions didn't help. @lucianopaz any idea what may be going wrong?

https://github.com/pymc-devs/pymc-experimental/actions/runs/8688042500/job/23822814914?pr=329#step:5:29

ERROR: Could not install packages due to an OSError: [Errno 2] No such file or directory: '/home/runner/work/pymc-experimental/pymc-experimental/dist/pymc-experimental*.tar.gz'

Looking at what is different between the last successful run and the ones you're getting now, it seems like the source distribution is getting built into pymc_experimental-0.0.18.tar.gz instead of pymc-experimental-0.0.18.tar.gz. I'm not sure what commit from build made this change to the output file of build sdist, but it looks like we just have to change the dash with an underline to in the PYPI workflow to get this fixed.

[ricardoV94]

Seems to have worked @lucianopaz !