Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Scheduler rate limiting #1413

Closed
wants to merge 53 commits into from
Closed

Scheduler rate limiting #1413

wants to merge 53 commits into from

Commits on Jul 12, 2023

  1. Start with rate limit implementation

    @jpbruinsslot
    jpbruinsslot committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    3248e35 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2023

  1. Start with rate limitting implementation

    @jpbruinsslot
    jpbruinsslot committed Jul 17, 2023
    Configuration menu
    Copy the full SHA
    7444ef2 View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2023

  1. Formatting

    @jpbruinsslot
    jpbruinsslot committed Jul 18, 2023
    Configuration menu
    Copy the full SHA
    a6f5d62 View commit details
    Browse the repository at this point in the history

  2. Start writing tests

    @jpbruinsslot
    jpbruinsslot committed Jul 18, 2023
    Configuration menu
    Copy the full SHA
    c6fa041 View commit details
    Browse the repository at this point in the history

  3. Rename to delayed tasks, update tests

    @jpbruinsslot
    jpbruinsslot committed Jul 18, 2023
    Configuration menu
    Copy the full SHA
    a9aac16 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2023

  1. Leverage tasks instead of in memory datastructure to store delayed tasks

    @jpbruinsslot
    jpbruinsslot committed Jul 19, 2023
    Configuration menu
    Copy the full SHA
    10a7f9f View commit details
    Browse the repository at this point in the history

  2. Fix consuming of the rate limiter

    @jpbruinsslot
    jpbruinsslot committed Jul 19, 2023
    Configuration menu
    Copy the full SHA
    0b552a0 View commit details
    Browse the repository at this point in the history

  3. Add optional consume parameter

    @jpbruinsslot
    jpbruinsslot committed Jul 19, 2023
    Configuration menu
    Copy the full SHA
    fda6991 View commit details
    Browse the repository at this point in the history

  4. Fix rate limit tests

    @jpbruinsslot
    jpbruinsslot committed Jul 19, 2023
    Configuration menu
    Copy the full SHA
    8ec4416 View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2023

  1. Fix formatting and type hint suggestions

    @jpbruinsslot
    jpbruinsslot committed Jul 20, 2023
    Configuration menu
    Copy the full SHA
    f5a71ea View commit details
    Browse the repository at this point in the history

  2. Debugging and writing tests for edge cases

    @jpbruinsslot
    jpbruinsslot committed Jul 20, 2023
    Configuration menu
    Copy the full SHA
    90786b9 View commit details
    Browse the repository at this point in the history

Commits on Jul 24, 2023

  1. Add order_by argument for task store

    @jpbruinsslot
    jpbruinsslot committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    2c4a1c2 View commit details
    Browse the repository at this point in the history

  2. Add migration

    @jpbruinsslot
    jpbruinsslot committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    46c1e6e View commit details
    Browse the repository at this point in the history

  3. Fix migration

    @jpbruinsslot
    jpbruinsslot committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    83e8f30 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into feature/mula/rate-limit 

    * main:
  Fix robot test (#1420)
  Use the correct clearance level variable in organization member list template (#1427)
  Fix translation in Debian package (#1432)
  Reschedule tasks when no results in bytes are found after grace period (#1410)
  Don't scan hostname nmap in nmap boefje (#1415)
  Add and use our own CVE API (#1383)
  Add `task_id` as a query parameter to the `GET /origins` endpoint (#1414)
  Remove member group checks and check for permission instead (#1275)
  Bump cryptography from 41.0.0 to 41.0.2 in /boefjes/boefjes/plugins/kat_ssl_certificates (#1396)
  Bump cryptography from 41.0.1 to 41.0.2 in /bytes (#1397)
  Build the Debian build image on the main branch (#1387)
  Add explicit `black` config to all modules (#1395)
  Fix <no title> in the user guide docs (#1391)
  Add configurable octpoes request timeout (#1382)
  Remove hardcoded clearance level in member list for superusers (#1390)
  Add Debian build depends for CVE API package (#1384)
  Add buttons to manual rerun tasks, both boefjes or normalizers (#1339)
  Use fix multiprocessing bug on macOS where `qsize()` is not implemented (#1374)
    @jpbruinsslot
    jpbruinsslot committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    f70c8fb View commit details
    Browse the repository at this point in the history

  5. Fix tests

    @jpbruinsslot
    jpbruinsslot committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    7f31f24 View commit details
    Browse the repository at this point in the history

  6. Update requirements.txt

    @jpbruinsslot
    jpbruinsslot committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    7b162ac View commit details
    Browse the repository at this point in the history

  7. Update tests timeout

    @jpbruinsslot
    jpbruinsslot committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    8485870 View commit details
    Browse the repository at this point in the history

  8. Finaly made the tests work

    @jpbruinsslot
    jpbruinsslot committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    fc17de7 View commit details
    Browse the repository at this point in the history

  9. Fixes for precommit

    @jpbruinsslot
    jpbruinsslot committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    d659a16 View commit details
    Browse the repository at this point in the history

  10. Update tests

    @jpbruinsslot
    jpbruinsslot committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    81223a8 View commit details
    Browse the repository at this point in the history

  11. Implement lock

    @jpbruinsslot
    jpbruinsslot committed Jul 24, 2023
    Configuration menu
    Copy the full SHA
    fe1197d View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2023

  1. Remove comment

    @jpbruinsslot
    jpbruinsslot committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    1362292 View commit details
    Browse the repository at this point in the history

  2. Adding valueerror tests

    @jpbruinsslot
    jpbruinsslot committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    f92e3d3 View commit details
    Browse the repository at this point in the history

  3. Fix pre-commit suggestions

    @jpbruinsslot
    jpbruinsslot committed Jul 25, 2023
    Configuration menu
    Copy the full SHA
    3077aa6 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into feature/mula/rate-limit

    @jpbruinsslot
    jpbruinsslot authored Jul 25, 2023
    Configuration menu
    Copy the full SHA
    880fca8 View commit details
    Browse the repository at this point in the history

Commits on Jul 27, 2023

  1. Merge branch 'main' into feature/mula/rate-limit 

    * main:
  Create new filters for findings  (#1293)
  Add Question OOI form rendering on the object detail page (#1408)
  Upgrade certifi (#1462)
  Default scan level filter to 0 (#1463)
  Remove some unused config options, and set better defaults for others (#1428)
  Remove unnecessary dependency on ipaddress package (#1448)
  Translations for release 1.11 - EN -> NL, PAP (#1439)
    @jpbruinsslot
    jpbruinsslot committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    56daa2e View commit details
    Browse the repository at this point in the history

Commits on Jul 31, 2023

  1. Merge branch 'main' into feature/mula/rate-limit

    @jpbruinsslot
    jpbruinsslot authored Jul 31, 2023
    Configuration menu
    Copy the full SHA
    3496344 View commit details
    Browse the repository at this point in the history

  2. Implement and update rate limiting attributes from katalogus

    @jpbruinsslot
    jpbruinsslot committed Jul 31, 2023
    Configuration menu
    Copy the full SHA
    2829e86 View commit details
    Browse the repository at this point in the history

Commits on Aug 1, 2023

  1. Merge branch 'main' into feature/mula/rate-limit

    @jpbruinsslot
    jpbruinsslot authored Aug 1, 2023
    Configuration menu
    Copy the full SHA
    443c31e View commit details
    Browse the repository at this point in the history

  2. Formatting

    @jpbruinsslot
    jpbruinsslot committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    cae249d View commit details
    Browse the repository at this point in the history

  3. Added rate limit field in boefje model 

    Signed-off-by: ammar <[email protected]>
    @ammar92
    ammar92 committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    42b6b12 View commit details
    Browse the repository at this point in the history

  4. Added rate limit in boefje scheduler model and identifier template re… 

    …ndering

Signed-off-by: ammar <[email protected]>
    @ammar92
    ammar92 committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    4194078 View commit details
    Browse the repository at this point in the history

  5. A bit of refactoring 

    Signed-off-by: ammar <[email protected]>
    @ammar92
    ammar92 committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    b6a7d31 View commit details
    Browse the repository at this point in the history

  6. Fixed errors 

    Signed-off-by: ammar <[email protected]>
    @ammar92
    ammar92 committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    0dd56d7 View commit details
    Browse the repository at this point in the history

  7. Making tests work

    @jpbruinsslot
    jpbruinsslot committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    ad72318 View commit details
    Browse the repository at this point in the history

  8. Make tests work

    @jpbruinsslot
    jpbruinsslot committed Aug 1, 2023
    Configuration menu
    Copy the full SHA
    cce498e View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2023

  1. Merge branch 'main' into feature/mula/rate-limit

    @jpbruinsslot
    jpbruinsslot authored Aug 2, 2023
    Configuration menu
    Copy the full SHA
    9bb95ac View commit details
    Browse the repository at this point in the history

  2. Mypy

    @jpbruinsslot
    jpbruinsslot committed Aug 2, 2023
    Configuration menu
    Copy the full SHA
    ab7d4d1 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2023

  1. Merge branch 'main' into feature/mula/rate-limit

    @jpbruinsslot
    jpbruinsslot authored Aug 7, 2023
    Configuration menu
    Copy the full SHA
    244ded1 View commit details
    Browse the repository at this point in the history

Commits on Aug 8, 2023

  1. Merge branch 'main' into feature/mula/rate-limit

    @jpbruinsslot
    jpbruinsslot authored Aug 8, 2023
    Configuration menu
    Copy the full SHA
    0639733 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into feature/mula/rate-limit

    @jpbruinsslot
    jpbruinsslot authored Aug 8, 2023
    Configuration menu
    Copy the full SHA
    4d39b20 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2023

  1. Fix rstcheck hook (#1584) 

    Signed-off-by: ammar <[email protected]>
    @ammar92 @jpbruinsslot
    ammar92 authored and jpbruinsslot committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    d055d80 View commit details
    Browse the repository at this point in the history

  2. OOI Detail page: Remember page position after clicking the "show inhe… 

    …ritance" link (#1590)

Co-authored-by: Patrick <[email protected]>
    @TwistMeister @Darwinkel @jpbruinsslot
    2 people authored and jpbruinsslot committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    ae268a3 View commit details
    Browse the repository at this point in the history

  3. Upgrade FastAPI (#1576) 

    Signed-off-by: ammar <[email protected]>
Co-authored-by: Patrick Darwinkel <[email protected]>
Co-authored-by: Patrick <[email protected]>
    @jpbruinsslot @Darwinkel
    3 people committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    5777629 View commit details
    Browse the repository at this point in the history

  4. Fix human-readable name for ImageMetadata (#1558) 

    Co-authored-by: Patrick <[email protected]>
Co-authored-by: Jeroen Dekkers <[email protected]>
    @praseodym @Darwinkel
    @dekkers @jpbruinsslot
    3 people authored and jpbruinsslot committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    93a8f1b View commit details
    Browse the repository at this point in the history

  5. fix many ports open normalizer (#1592) 

    Co-authored-by: Jeroen Dekkers <[email protected]>
    @noamblitz @dekkers @jpbruinsslot
    2 people authored and jpbruinsslot committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    1ea9154 View commit details
    Browse the repository at this point in the history

  6. Configure github actions in dependabot (#1594) 

    Co-authored-by: Mark Janssen <[email protected]>
    @dekkers @praseodym @jpbruinsslot
    2 people authored and jpbruinsslot committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    c31e6a8 View commit details
    Browse the repository at this point in the history

  7. Run docker-compose pull in make pull (#1585) 

    Co-authored-by: ammar92 <[email protected]>
    @dekkers @ammar92 @jpbruinsslot
    2 people authored and jpbruinsslot committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    7c97142 View commit details
    Browse the repository at this point in the history

  8. Bump actions/checkout from 1 to 3 (#1598) 

    Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jeroen Dekkers <[email protected]>
    @dependabot @dekkers @jpbruinsslot
    2 people authored and jpbruinsslot committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    0751d10 View commit details
    Browse the repository at this point in the history

  9. Add community install/update scripts (#1309) 

    Co-authored-by: Robjen <[email protected]>
Co-authored-by: Patrick Darwinkel <[email protected]>
Co-authored-by: Patrick <[email protected]>
Co-authored-by: Jan Klopper <[email protected]>
Co-authored-by: Jeroen Dekkers <[email protected]>
    @ring-ring-ring @RobboRob
    @Darwinkel @underdarknl @dekkers @jpbruinsslot
    6 people authored and jpbruinsslot committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    eb3779f View commit details
    Browse the repository at this point in the history

  10. Update scheduler documentation (#1476) 

    Co-authored-by: ring-ring-ring <[email protected]>
Co-authored-by: ammar92 <[email protected]>
Co-authored-by: Patrick <[email protected]>
Co-authored-by: Jan Klopper <[email protected]>
    @jpbruinsslot @ring-ring-ring
    @ammar92 @Darwinkel @underdarknl
    5 people committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    5fc6107 View commit details
    Browse the repository at this point in the history

  11. Translations update from Hosted Weblate (#1604) 

    Co-authored-by: Patrick <[email protected]>
Co-authored-by: Jan Klopper <[email protected]>
    @weblate @Darwinkel
    @underdarknl @jpbruinsslot
    3 people authored and jpbruinsslot committed Aug 14, 2023
    Configuration menu
    Copy the full SHA
    681d6de View commit details
    Browse the repository at this point in the history