Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multiprocessing in eval command #418

Merged
merged 17 commits into from
Jun 26, 2020
Merged

Multiprocessing in eval command #418

merged 17 commits into from
Jun 26, 2020

Commits on Jun 5, 2020

  1. Refactor: pass subject_index as parameter to SuggestionResult.filter

    @osma
    osma committed Jun 5, 2020
    Configuration menu
    Copy the full SHA
    a6ec0f9 View commit details
    Browse the repository at this point in the history

  2. Refactor: replace SuggestionResult.hits with .as_list method that tak… 

    …es a SubjectIndex
    @osma
    osma committed Jun 5, 2020
    Configuration menu
    Copy the full SHA
    419e2c5 View commit details
    Browse the repository at this point in the history

  3. Refactor: replace SuggestionResult.vector with .as_vector method that… 

    … takes a SubjectIndex
    @osma
    osma committed Jun 5, 2020
    Configuration menu
    Copy the full SHA
    49e7d72 View commit details
    Browse the repository at this point in the history

  4. Refactor: don't store unnecessary subject_index in SuggestionResult c… 

    …lasses
    @osma
    osma committed Jun 5, 2020
    Configuration menu
    Copy the full SHA
    e31d0f6 View commit details
    Browse the repository at this point in the history

  5. first rough implementation of multiprocessing in eval command

    @osma
    osma committed Jun 5, 2020
    Configuration menu
    Copy the full SHA
    f255976 View commit details
    Browse the repository at this point in the history

  6. remove unused parameter

    @osma
    osma committed Jun 5, 2020
    Configuration menu
    Copy the full SHA
    dfc7f3d View commit details
    Browse the repository at this point in the history

  7. Implement --jobs parameter for eval CLI command

    @osma
    osma committed Jun 5, 2020
    Configuration menu
    Copy the full SHA
    743b152 View commit details
    Browse the repository at this point in the history

  8. Avoid creating subprocesses for eval command with jobs=1

    @osma
    osma committed Jun 5, 2020
    Configuration menu
    Copy the full SHA
    3ca4901 View commit details
    Browse the repository at this point in the history

  9. fix test failures caused by merging recent changes from master

    @osma
    osma committed Jun 5, 2020
    Configuration menu
    Copy the full SHA
    f44ab74 View commit details
    Browse the repository at this point in the history

  10. Add unit tests for different numbers of eval jobs

    @osma
    osma committed Jun 5, 2020
    Configuration menu
    Copy the full SHA
    b70191b View commit details
    Browse the repository at this point in the history

  11. Optimization: VectorSuggestionResult.filter now returns a 

    ListSuggestionResult, so less data to serialize/deserialize during
multiprocessing
    @osma
    osma committed Jun 5, 2020
    Configuration menu
    Copy the full SHA
    19d3707 View commit details
    Browse the repository at this point in the history

Commits on Jun 23, 2020

  1. Initialize the project (load model just once) before parallel eval

    @osma
    osma committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    7e80068 View commit details
    Browse the repository at this point in the history

  2. Initializing an ensemble backend initializes the source projects as well

    @osma
    osma committed Jun 23, 2020
    Configuration menu
    Copy the full SHA
    c545465 View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2020

  1. fix project creation/initialization sequence in AnnifRegistry

    @osma
    osma committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    0ff3388 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into issue65-eval-multiprocess

    @osma
    osma committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    7ea54ea View commit details
    Browse the repository at this point in the history

  3. Change default to jobs=1: run eval in parallel only if requested by user

    @osma
    osma committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    b8bf18b View commit details
    Browse the repository at this point in the history

  4. Refactor: split off AnnifRegistry to a separate annif.registry module

    @osma
    osma committed Jun 26, 2020
    Configuration menu
    Copy the full SHA
    495f4cc View commit details
    Browse the repository at this point in the history