Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix for joblib not saturating CPU during multiprocessing #1188

Merged
merged 19 commits into from
Jul 22, 2024
Merged

Fix for joblib not saturating CPU during multiprocessing #1188

merged 19 commits into from
Jul 22, 2024

Commits on Jun 22, 2024

  1. refactoring simulate_for_sbi

    @janko-petkovic
    janko-petkovic committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    6a48eed View commit details
    Browse the repository at this point in the history

  2. refactored simulate_for_sbi, introduced new wrapper wrap_as_joblib_ef… 

    …ficient_simulator
    @janko-petkovic
    janko-petkovic committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    cb2dd49 View commit details
    Browse the repository at this point in the history

  3. Finished refactoring simulate_for_sbi and wrap_as_joblib_efficient_si… 

    …mulator. The wrapping/casting currently increases the runtime roughly three times, but the code cannot be breaking for now.
    @janko-petkovic
    janko-petkovic committed Jun 22, 2024
    Configuration menu
    Copy the full SHA
    a0eb104 View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. working on user_input_checks

    @janko-petkovic
    janko-petkovic committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    ba6cdbf View commit details
    Browse the repository at this point in the history

  2. added temporary benchmark folder

    @janko-petkovic
    janko-petkovic committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    8fe2334 View commit details
    Browse the repository at this point in the history

  3. adding process_simulator / process_prior to tests

    Janko Petkovic committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    cb9b25e View commit details
    Browse the repository at this point in the history

  4. Merge branch '1175-simulation-multiproc-fix' of github.com:janko-petk… 

    …ovic/sbi into 1175-simulation-multiproc-fix
    Janko Petkovic committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    2448fb2 View commit details
    Browse the repository at this point in the history

  5. added process_simulator and process_prior to some of the tests

    Janko Petkovic committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    825ae89 View commit details
    Browse the repository at this point in the history

  6. finished adding process_simulator and process_prior to\ndefault git t… 

    …ests (pytest -n auto -m "not slow\nand not gpu")
    Janko Petkovic committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    59159cc View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2024

  1. Merge branch 'main' into 1175-simulation-multiproc-fix

    Janko Petkovic committed Jul 1, 2024
    Configuration menu
    Copy the full SHA
    f64800b View commit details
    Browse the repository at this point in the history

Commits on Jul 12, 2024

  1. Merged main, solved conflicts in tests/inference_on_device (keept main)

    Janko Petkovic committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    5170fb8 View commit details
    Browse the repository at this point in the history

  2. Changes following PR 1188: removed if-else for show_progress_bar … 

    …in `simulate_for_sbi`, improved comments, removed `benchmark` folder
    Janko Petkovic committed Jul 12, 2024
    Configuration menu
    Copy the full SHA
    71ce9ab View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. restructured simulation_batch_size logic in simulate_for_sbi accordin… 

    …g to sbi-dev#1188 discussion
    Janko Petkovic committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    ad687ff View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into 1175-simulation-multiproc-fix

    Janko Petkovic committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    2f14147 View commit details
    Browse the repository at this point in the history

  3. Bypassed process_simulator and simulate_for_sbi in tests/inferenc… 

    …e_on_device_test.py


sbi-dev#1188

Co-authored-by: Jan <[email protected]>
    @janko-petkovic @janfb
    janko-petkovic and janfb authored Jul 19, 2024
    Configuration menu
    Copy the full SHA
    622ebb1 View commit details
    Browse the repository at this point in the history

  4. Merge branch '1175-simulation-multiproc-fix' of github.com:janko-petk… 

    …ovic/sbi into 1175-simulation-multiproc-fix
    Janko Petkovic committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    334c2de View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. adjusted imports and formatting

    Janko Petkovic committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    f6befc3 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into 1175-simulation-multiproc-fix

    Janko Petkovic committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    1a29910 View commit details
    Browse the repository at this point in the history

  3. changed | to Union in inference/base.py::simulate_for_sbi

    Janko Petkovic committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    eb1b222 View commit details
    Browse the repository at this point in the history