chore(iast): optimize iast by migrating should_iast_patch function to c

[avara1986]

TL;DR: Before this PR, IAST startup time was x5,16 lower than an application without IAST startup time
After this PR, IAST startup time was x2,79

Description

In this PR, we aim to improve the startup times of an application when IAST (Interactive Application Security Testing) is enabled. The following improvements have been made:

Performance improvements

• These changes aim to streamline the codebase, improve the functionality of the IAST module, and remove outdated or redundant code.When reading the source code of a module for transformation with AST, it is now done in binary mode, which is 30% faste as shown in this gist: https://gist.github.com/avara1986/8e31a23978b5eecd9dc8f2f3135060b0. The execution times are:

  • open("r"): 0.9285 secs
  • open("rb"): 0.6362 secs

• The function should_iast_patch has been migrated to C. By comparing the original function with the new one using the following timeit command, is 40% faster.

  # refactor
  python -m timeit -n 5000000 -r 5 -u usec -s "from ddtrace.appsec._iast._ast import 
  iastpatch""iastpatch.should_iast_patch('mimodulo.submodulo')" 
  # main
  python -m timeit -n 5000000 -r 5 -u usec -s "from ddtrace.appsec._iast._ast.ast_patching import _should_iast_patch" 
  "_should_iast_patch('package.mypackage')"
  

• I wanted to migrate exec to PyEval_EvalCode but the results are very revealing, exec is approximately 17% faster. The results of this gist https://gist.github.com/avara1986/ae2a65dca1b90afa762af67db680bc2b

  • exec: mean time of 0.083023 seconds
  • PyEval_EvalCode: mean time of 0.099550 seconds

By combining all these changes, the microbenchmarks for startup times show that the baseline takes 2.46 seconds, while this branch takes 1.24 seconds, making it 1.99x faster.

Performance analysis

The _should_iast_patch logs are under asm_config._iast_debug because Debug mode has a significant performance impact, making the function between 1.14x and 3.08x slower

Benchmark Results
--------------------------------------------------------------------------------
Module Name                    Debug Time (s)       No Debug Time (s)    Ratio     
--------------------------------------------------------------------------------
os                                    0.017021            0.005522       3.08x
requests                              0.023237            0.020418       1.14x
ddtrace.appsec._iast                  0.024511            0.012280       2.00x
non.existent.module                   0.023403            0.011170       2.10x

Branch deployed in rel-env and no leaks found:

Checklist

• PR author has checked that all the criteria below are met
• The PR description includes an overview of the change
• The PR description articulates the motivation for the change
• The change includes tests OR the PR description describes a testing strategy
• The PR description notes risks associated with the change, if any
• Newly-added code is easy to change
• The change follows the library release note guidelines
• The change includes or references documentation updates if necessary
• Backport labels are set (if applicable)

Reviewer Checklist

• Reviewer has checked that all the criteria below are met
• Title is accurate
• All changes are related to the pull request's stated goal
• Avoids breaking API changes
• Testing strategy adequately addresses listed risks
• Newly-added code is easy to change
• Release note makes sense to a user of the library
• If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
• Backport labels are set in a manner that is consistent with the release branch maintenance policy

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 0% with 58 lines in your changes missing coverage. Please review.

Project coverage is 3.67%. Comparing base (3c49582) to head (120a5d3).

Files with missing lines	Patch %	Lines
ddtrace/appsec/_iast/_ast/ast_patching.py	0.00%	20 Missing ⚠️
tests/appsec/iast/_ast/test_ast_patching.py	0.00%	18 Missing ⚠️
tests/appsec/iast/test_env_var.py	0.00%	10 Missing ⚠️
ddtrace/appsec/_iast/_loader.py	0.00%	7 Missing ⚠️
ddtrace/appsec/_iast/_ast/visitor.py	0.00%	1 Missing ⚠️
...s/appsec/iast/_ast/test_ast_patching_type_hints.py	0.00%	1 Missing ⚠️
tests/appsec/iast/test_loader.py	0.00%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##             main   #12774    +/-   ##
========================================
  Coverage    3.67%    3.67%            
========================================
  Files        1378     1370     -8     
  Lines      135941   135817   -124     
========================================
+ Hits         4993     4995     +2     
+ Misses     130948   130822   -126     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

The backport to 2.21 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.21 2.21
# Navigate to the new working tree
cd .worktrees/backport-2.21
# Create a new branch
git switch --create backport-12774-to-2.21
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 9159b811c7c08f133e960b9cb17d7b72d9e4c8aa
# Push it to GitHub
git push --set-upstream origin backport-12774-to-2.21
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.21

Then, create a pull request where the base branch is 2.21 and the compare/head branch is backport-12774-to-2.21.