Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not cache numba decorated split_array #854

Merged
merged 1 commit into from
Jul 4, 2024

Conversation

dachengx
Copy link
Collaborator

@dachengx dachengx commented Jul 4, 2024

What is the problem / what does the code in this PR do

As title suggested. If still using cache=True, you will see errors like

Exception in thread load:event_se_density:
Traceback (most recent call last):
  File "/opt/XENONnT/anaconda/envs/XENONnT_2024.03.1/lib/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/opt/XENONnT/anaconda/envs/XENONnT_2024.03.1/lib/python3.9/threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
  File "/home/xudc/strax/strax/mailbox.py", line 290, in _send_from
    self.close()
  File "/home/xudc/strax/strax/mailbox.py", line 354, in close
    self.send(StopIteration)
  File "/home/xudc/strax/strax/mailbox.py", line 307, in send
    raise MailboxKilled(self.killed_because)
strax.mailbox.MailboxKilled: (<class 'ValueError'>, ValueError('no field of name cut_s1_single_scatter'), <traceback object at 0x7fee96cc5240>)
    inputs_merged = {
  File "/home/xudc/strax/strax/plugins/plugin.py", line 497, in <dictcomp>
    kind: strax.Chunk.merge([inputs[d] for d in deps_of_kind])
  File "/home/xudc/strax/strax/chunk.py", line 229, in merge
    data = strax.merge_arrs(
  File "/home/xudc/strax/strax/utils.py", line 215, in merge_arrs
    result[fn] = arr[fn]
ValueError: no field of name cut_s1_single_scatter

when multiple jobs run in parallel. The reason for this is deep. But the error log suggest that there is problem of numba function, so that split_array returns wrong results.

Can you briefly describe how it works?

Compile split_array inside each job, do not refer to the cached numba function.

Can you give a minimal working example (or illustrate with a figure)?

Please include the following if applicable:

  • Update the docstring(s)
  • Update the documentation
  • Tests to check the (new) code is working as desired.
  • Does it solve one of the open issues on github?

Please make sure that all automated tests have passed before asking for a review (you can save the PR as a draft otherwise).

@dachengx dachengx marked this pull request as ready for review July 4, 2024 17:19
@coveralls
Copy link

Coverage Status

coverage: 90.507%. remained the same
when pulling f6a1c74 on no_cache_numba_split_array
into 2d772f1 on master.

@dachengx dachengx merged commit 3440362 into master Jul 4, 2024
9 checks passed
@dachengx dachengx deleted the no_cache_numba_split_array branch July 4, 2024 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants