Skip to content

Fix unit tests #1462

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 27 commits into from
Dec 12, 2024
Merged

Fix unit tests #1462

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
9621aea
debug unit tests
YigitElma Dec 12, 2024
4f4c5ef
more debugging
YigitElma Dec 12, 2024
5db9c6c
update
YigitElma Dec 12, 2024
27e9550
update
YigitElma Dec 12, 2024
044afcc
add more debugging
YigitElma Dec 12, 2024
72e3824
check if new action version caused this
YigitElma Dec 12, 2024
005ee84
add more debugging. maybe the problem is 3.12.8
YigitElma Dec 12, 2024
a41e94e
more and more debugging
YigitElma Dec 12, 2024
27ec9c7
try renaming the cached venv
YigitElma Dec 12, 2024
caf8d33
rename format
YigitElma Dec 12, 2024
d261f5d
try renaming, revert some other files
YigitElma Dec 12, 2024
5e210bc
some minor changes
YigitElma Dec 12, 2024
553507e
Merge branch 'master' into yge/fix-unit
YigitElma Dec 12, 2024
d785c8b
test with regression to see if it works
YigitElma Dec 12, 2024
c168cdb
test with regression to see if it works
YigitElma Dec 12, 2024
e70e0af
some minor changes
YigitElma Dec 12, 2024
e0005dc
specify python patch too for actions
YigitElma Dec 12, 2024
21a1983
update script to see new cache work with other prs
YigitElma Dec 12, 2024
455a918
revert the change
YigitElma Dec 12, 2024
9770c24
try new method
YigitElma Dec 12, 2024
5fdf157
try new method
YigitElma Dec 12, 2024
1a54ac3
still trying
YigitElma Dec 12, 2024
487782e
?
YigitElma Dec 12, 2024
5768eef
apply
YigitElma Dec 12, 2024
cd6cf02
apply changes to all the actions
YigitElma Dec 12, 2024
bcc791b
Merge branch 'master' into yge/fix-unit
f0uriest Dec 12, 2024
4001b97
Merge branch 'master' into yge/fix-unit
f0uriest Dec 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions .github/workflows/benchmark.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,28 +58,35 @@ jobs:
with:
python-version: ${{ matrix.python-version }}

- name: Check full Python version
run: |
python --version
python_version=$(python --version 2>&1 | cut -d' ' -f2)
echo "Python version: $python_version"
echo "version=$python_version" >> $GITHUB_ENV

- name: Restore Python environment cache
if: env.has_changes == 'true'
id: restore-env
uses: actions/cache/restore@v4
with:
path: .venv-${{ matrix.python-version }}
key: ${{ runner.os }}-venv-${{ matrix.python-version }}-${{ hashFiles('devtools/dev-requirements.txt', 'requirements.txt') }}
path: .venv-${{ env.version }}
key: ${{ runner.os }}-venv-${{ env.version }}-${{ hashFiles('devtools/dev-requirements.txt', 'requirements.txt') }}

- name: Set up virtual environment if not restored from cache
if: steps.restore-env.outputs.cache-hit != 'true' && env.has_changes == 'true'
run: |
gh cache list
python -m venv .venv-${{ matrix.python-version }}
source .venv-${{ matrix.python-version }}/bin/activate
python -m venv .venv-${{ env.version }}
source .venv-${{ env.version }}/bin/activate
python -m pip install --upgrade pip
pip install -r devtools/dev-requirements.txt
pip install matplotlib==3.9.2

- name: Benchmark with pytest-benchmark (PR)
if: env.has_changes == 'true'
run: |
source .venv-${{ matrix.python-version }}/bin/activate
source .venv-${{ env.version }}/bin/activate
pwd
lscpu
pip list
Expand All @@ -106,7 +113,7 @@ jobs:
- name: Benchmark with pytest-benchmark (MASTER)
if: env.has_changes == 'true'
run: |
source .venv-${{ matrix.python-version }}/bin/activate
source .venv-${{ env.version }}/bin/activate
pwd
lscpu
pip list
Expand All @@ -122,7 +129,7 @@ jobs:
- name: Put benchmark results in same folder
if: env.has_changes == 'true'
run: |
source .venv-${{ matrix.python-version }}/bin/activate
source .venv-${{ env.version }}/bin/activate
pwd
cd tests/benchmarks
find .benchmarks/ -type f -printf "%T@ %p\n" | sort -n | cut -d' ' -f 2- | tail -n 1 > temp1
Expand All @@ -143,7 +150,7 @@ jobs:
- name: Compare latest commit results to the master branch results
if: env.has_changes == 'true'
run: |
source .venv-${{ matrix.python-version }}/bin/activate
source .venv-${{ env.version }}/bin/activate
cd tests/benchmarks
pwd
python compare_bench_results.py
Expand Down
17 changes: 12 additions & 5 deletions .github/workflows/black.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,25 +19,32 @@ jobs:
with:
python-version: ${{ matrix.python-version }}

- name: Check full Python version
run: |
python --version
python_version=$(python --version 2>&1 | cut -d' ' -f2)
echo "Python version: $python_version"
echo "version=$python_version" >> $GITHUB_ENV

- name: Restore Python environment cache
id: restore-env
uses: actions/cache/restore@v4
with:
path: .venv-${{ matrix.python-version }}
key: ${{ runner.os }}-venv-${{ matrix.python-version }}-${{ hashFiles('devtools/dev-requirements.txt', 'requirements.txt') }}
path: .venv-${{ env.version }}
key: ${{ runner.os }}-venv-${{ env.version }}-${{ hashFiles('devtools/dev-requirements.txt', 'requirements.txt') }}

- name: Set up virtual environment if not restored from cache
if: steps.restore-env.outputs.cache-hit != 'true'
run: |
gh cache list
python -m venv .venv-${{ matrix.python-version }}
source .venv-${{ matrix.python-version }}/bin/activate
python -m venv .venv-${{ env.version }}
source .venv-${{ env.version }}/bin/activate
python -m pip install --upgrade pip
pip install -r devtools/dev-requirements.txt

- name: Check files using the black formatter
run: |
source .venv-${{ matrix.python-version }}/bin/activate
source .venv-${{ env.version }}/bin/activate
black --version
black --check desc/ tests/ || black_return_code=$?
echo "BLACK_RETURN_CODE=$black_return_code" >> $GITHUB_ENV
Expand Down
22 changes: 15 additions & 7 deletions .github/workflows/cache_dependencies.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,21 +24,28 @@ jobs:
with:
python-version: ${{ matrix.python-version }}

- name: Check full Python version
run: |
python --version
python_version=$(python --version 2>&1 | cut -d' ' -f2)
echo "Python version: $python_version"
echo "version=$python_version" >> $GITHUB_ENV

- name: Delete old cached file with same python version
run: |
echo "Current Cached files list"
gh cache list
echo "Deleting cached files with pattern: ${{ runner.os }}-venv-${{ matrix.python-version }}-"
for cache_key in $(gh cache list --json key -q ".[] | select(.key | startswith(\"${{ runner.os }}-venv-${{ matrix.python-version }}-\")) | .key"); do
echo "Deleting cached files with pattern: ${{ runner.os }}-venv-${{ env.version }}-"
for cache_key in $(gh cache list --json key -q ".[] | select(.key | startswith(\"${{ runner.os }}-venv-${{ env.version }}-\")) | .key"); do
echo "Deleting cache with key: $cache_key"
gh cache delete "$cache_key"
done

# Update the matplotlib version if needed later
- name: Set up virtual environment
run: |
python -m venv .venv-${{ matrix.python-version }}
source .venv-${{ matrix.python-version }}/bin/activate
python -m venv .venv-${{ env.version }}
source .venv-${{ env.version }}/bin/activate
python -m pip install --upgrade pip
pip install -r devtools/dev-requirements.txt
pip install matplotlib==3.9.2
Expand All @@ -47,12 +54,13 @@ jobs:
id: cache-env
uses: actions/cache@v4
with:
path: .venv-${{ matrix.python-version }}
key: ${{ runner.os }}-venv-${{ matrix.python-version }}-${{ hashFiles('devtools/dev-requirements.txt', 'requirements.txt') }}
path: .venv-${{ env.version }}
key: ${{ runner.os }}-venv-${{ env.version }}-${{ hashFiles('devtools/dev-requirements.txt', 'requirements.txt') }}

- name: Verify virtual environment activation
run: |
source .venv-${{ matrix.python-version }}/bin/activate
source .venv-${{ env.version }}/bin/activate
which python
python --version
pip --version
pip list
17 changes: 12 additions & 5 deletions .github/workflows/notebook_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,28 +51,35 @@ jobs:
with:
python-version: ${{ matrix.python-version }}

- name: Check full Python version
run: |
python --version
python_version=$(python --version 2>&1 | cut -d' ' -f2)
echo "Python version: $python_version"
echo "version=$python_version" >> $GITHUB_ENV

- name: Restore Python environment cache
if: env.has_changes == 'true'
id: restore-env
uses: actions/cache/restore@v4
with:
path: .venv-${{ matrix.python-version }}
key: ${{ runner.os }}-venv-${{ matrix.python-version }}-${{ hashFiles('devtools/dev-requirements.txt', 'requirements.txt') }}
path: .venv-${{ env.version }}
key: ${{ runner.os }}-venv-${{ env.version }}-${{ hashFiles('devtools/dev-requirements.txt', 'requirements.txt') }}

- name: Set up virtual environment if not restored from cache
if: steps.restore-env.outputs.cache-hit != 'true' && env.has_changes == 'true'
run: |
gh cache list
python -m venv .venv-${{ matrix.python-version }}
source .venv-${{ matrix.python-version }}/bin/activate
python -m venv .venv-${{ env.version }}
source .venv-${{ env.version }}/bin/activate
python -m pip install --upgrade pip
pip install -r devtools/dev-requirements.txt
pip install matplotlib==3.9.2

- name: Test notebooks with pytest and nbmake
if: env.has_changes == 'true'
run: |
source .venv-${{ matrix.python-version }}/bin/activate
source .venv-${{ env.version }}/bin/activate
pwd
lscpu
pip list
Expand Down
17 changes: 12 additions & 5 deletions .github/workflows/regression_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,20 +50,27 @@ jobs:
with:
python-version: ${{ matrix.python-version }}

- name: Check full Python version
run: |
python --version
python_version=$(python --version 2>&1 | cut -d' ' -f2)
echo "Python version: $python_version"
echo "version=$python_version" >> $GITHUB_ENV

- name: Restore Python environment cache
if: env.has_changes == 'true'
id: restore-env
uses: actions/cache/restore@v4
with:
path: .venv-${{ matrix.python-version }}
key: ${{ runner.os }}-venv-${{ matrix.python-version }}-${{ hashFiles('devtools/dev-requirements.txt', 'requirements.txt') }}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of using matrix.python-version as the key why can't we use the actual version from python --version

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can. I don't know if the syntax will allow it, but that could resolve inconsistent Python version problem. However, we might need to clean some old cache created with older Python versions every 3-4 months.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw this problem will probably be solved when every Github runner starts using 3.12.8 and 3.11.11. For now, some of them use 3.12.7 and this causes the issue. They made some security changes regarding the venv activation with the latest patch

path: .venv-${{ env.version }}
key: ${{ runner.os }}-venv-${{ env.version }}-${{ hashFiles('devtools/dev-requirements.txt', 'requirements.txt') }}

- name: Set up virtual environment if not restored from cache
if: steps.restore-env.outputs.cache-hit != 'true' && env.has_changes == 'true'
run: |
gh cache list
python -m venv .venv-${{ matrix.python-version }}
source .venv-${{ matrix.python-version }}/bin/activate
python -m venv .venv-${{ env.version }}
source .venv-${{ env.version }}/bin/activate
python -m pip install --upgrade pip
pip install -r devtools/dev-requirements.txt
pip install matplotlib==3.9.2
Expand All @@ -77,7 +84,7 @@ jobs:
- name: Test with pytest
if: env.has_changes == 'true'
run: |
source .venv-${{ matrix.python-version }}/bin/activate
source .venv-${{ env.version }}/bin/activate
pwd
lscpu
pip list
Expand Down
26 changes: 20 additions & 6 deletions .github/workflows/unit_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,20 +56,27 @@ jobs:
with:
python-version: ${{ matrix.combos.python_version }}

- name: Check full Python version
run: |
python --version
python_version=$(python --version 2>&1 | cut -d' ' -f2)
echo "Python version: $python_version"
echo "version=$python_version" >> $GITHUB_ENV

- name: Restore Python environment cache
if: env.has_changes == 'true'
id: restore-env
uses: actions/cache/restore@v4
with:
path: .venv-${{ matrix.combos.python_version }}
key: ${{ runner.os }}-venv-${{ matrix.combos.python_version }}-${{ hashFiles('devtools/dev-requirements.txt', 'requirements.txt') }}
path: .venv-${{ env.version }}
key: ${{ runner.os }}-venv-${{ env.version }}-${{ hashFiles('devtools/dev-requirements.txt', 'requirements.txt') }}

- name: Set up virtual environment if not restored from cache
if: steps.restore-env.outputs.cache-hit != 'true' && env.has_changes == 'true'
run: |
gh cache list
python -m venv .venv-${{ matrix.combos.python_version }}
source .venv-${{ matrix.combos.python_version }}/bin/activate
python -m venv .venv-${{ env.version }}
source .venv-${{ env.version }}/bin/activate
python -m pip install --upgrade pip
pip install -r devtools/dev-requirements.txt
pip install matplotlib==3.9.2
Expand All @@ -80,13 +87,20 @@ jobs:
with:
swap-size-gb: 10

- name: Test with pytest
- name: Action Details
if: env.has_changes == 'true'
run: |
source .venv-${{ matrix.combos.python_version }}/bin/activate
source .venv-${{ env.version }}/bin/activate
which python
python --version
pwd
lscpu
pip list

- name: Test with pytest
if: env.has_changes == 'true'
run: |
source .venv-${{ env.version }}/bin/activate
python -m pytest -v -m unit \
--durations=0 \
--cov-report xml:cov.xml \
Expand Down
Loading