Repository navigation
fix(many): issues reported from deepsource #164
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 12 commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
9cc6ce3
save
YaoYinYing 7a2b4f6
save
YaoYinYing d10ab4b
Update README.md
YaoYinYing e148e73
Update pssm_gremlin_dashboard.html
YaoYinYing f497339
save
YaoYinYing 6351ecf
save
YaoYinYing 1104bf0
save
YaoYinYing 089a9bf
Update CHANGELOG.md
YaoYinYing 82d89fa
save
YaoYinYing f431039
Create codex_deepsource.md
YaoYinYing ce38545
save
YaoYinYing 535e7f2
save
YaoYinYing e0f7451
save
YaoYinYing 9351a20
save
YaoYinYing 62ac7de
save qlty
YaoYinYing 4f6ecd8
save
YaoYinYing b172e5e
save
YaoYinYing adca9ae
Update REvoDesign_PSSM_GREMLIN.sh
YaoYinYing adb589d
Update CHANGELOG.md
YaoYinYing e97d2bd
Update Makefile
YaoYinYing 108a335
Update Makefile
YaoYinYing 47962de
Update .deepsource.toml
YaoYinYing File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,220 @@ | ||
| # Codex DeepSource Fix Playbook | ||
|
|
||
| This document records the practical approach used to process DeepSource issues in this repository with minimal complexity and high safety. | ||
|
|
||
| ## Goal | ||
|
|
||
| Fix a large DeepSource backlog by prioritizing the most dangerous and complex findings first, while keeping behavior stable and tests green. | ||
|
|
||
| ## Principles | ||
|
|
||
| 1. Risk first, not count first. | ||
| 2. Root-cause patches, not one-off local silencing. | ||
| 3. Small safe diffs over broad refactors. | ||
| 4. Every risky fix must have verification. | ||
| 5. Keep changelog and docs updated in the same PR. | ||
|
|
||
| ## Inputs Required | ||
|
|
||
| 1. DeepSource issues URL. | ||
| 2. Browser-like user agent for issue fetching. | ||
| 3. Local runtime/test environment details (`conda` env, test commands, CI constraints). | ||
|
|
||
| ## Access Pattern For DeepSource | ||
|
|
||
| Use a regular browser user agent when fetching issue pages or APIs. | ||
|
|
||
| ```bash | ||
| curl -sS -L -A "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36" \ | ||
| "https://app.deepsource.com/gh/YaoYinYing/REvoDesign/issues" \ | ||
| -o /tmp/deepsource_issues.html | ||
| ``` | ||
|
|
||
| If static HTML is insufficient, use the same UA for authenticated GraphQL/API calls (with csrf/cookies) and save raw payloads for traceability. | ||
|
|
||
| ## Issue Classification Model | ||
|
|
||
| Classify each finding on two dimensions: danger and complexity. | ||
|
|
||
| ### Danger Levels | ||
|
|
||
| 1. `D0 Critical`: Security, data corruption, remote execution, credential/path leakage, unsafe deserialization. | ||
| 2. `D1 High`: Runtime crashes, concurrency hazards, deadlocks, silent wrong results. | ||
| 3. `D2 Medium`: Reliability/perf regressions with user-visible impact. | ||
| 4. `D3 Low`: Readability/style/maintainability only. | ||
|
|
||
| ### Complexity Levels | ||
|
|
||
| 1. `C0 Low`: Local one-file change, obvious behavior, easy regression test. | ||
| 2. `C1 Medium`: Multi-file behavior but stable boundaries. | ||
| 3. `C2 High`: Cross-module flow, threading/process/network interactions. | ||
| 4. `C3 Very High`: Architectural shifts, migration-level changes. | ||
|
|
||
| ## Prioritization Rule | ||
|
|
||
| Work in this order: | ||
|
|
||
| 1. `D0/C0-C2` | ||
| 2. `D1/C0-C2` | ||
| 3. `D0/D1` items with `C3` only when isolated safely | ||
| 4. `D2` batches | ||
| 5. `D3` cleanup | ||
|
|
||
| Within the same bucket, prioritize by blast radius: | ||
|
|
||
| 1. Core runtime modules (`src/REvoDesign/...` hot paths) | ||
| 2. Shared tools/utilities | ||
| 3. Optional paths and edge flows | ||
|
|
||
| ## Batch Strategy | ||
|
|
||
| Use small batches (5 to 10 issues) grouped by root cause family: | ||
|
|
||
| 1. Serialization and file safety | ||
| 2. Subprocess/network/IO robustness | ||
| 3. Qt/thread interaction safety | ||
| 4. Path and temp-file handling | ||
| 5. Test determinism and ordering | ||
|
|
||
| This avoids mixing unrelated risk and simplifies rollback. | ||
|
|
||
| ## Fix Workflow (Repeatable) | ||
|
|
||
| 1. Fetch and snapshot issue list. | ||
| 2. Normalize and deduplicate findings by root cause. | ||
| 3. Assign `danger` and `complexity`. | ||
| 4. Select next batch by priority. | ||
| 5. Patch with minimal code movement. | ||
| 6. Add or adjust targeted tests. | ||
| 7. Run `make clean` before test validation. | ||
| 8. Run focused `kw-test` first, then broader suite if needed. | ||
| 9. Update changelog. | ||
| 10. Record unresolved items and blockers. | ||
|
|
||
| ## Typical Fix Patterns Used In This Repo | ||
|
|
||
| ### 1) Unsafe Data Loading | ||
|
|
||
| Preferred approach: | ||
|
|
||
| 1. Replace unsafe loaders with constrained loaders. | ||
| 2. Validate input type and expected schema early. | ||
| 3. Fail closed with explicit, actionable errors. | ||
|
|
||
| Avoid: | ||
|
|
||
| 1. Broad `except Exception` without context. | ||
| 2. Silent fallback that hides corruption. | ||
|
|
||
| ### 2) Subprocess and External Tools | ||
|
|
||
| Preferred approach: | ||
|
|
||
| 1. Use explicit argument lists (no implicit shell parsing unless required). | ||
| 2. Set timeout and capture stderr/stdout. | ||
| 3. Return structured errors for UI/logging. | ||
| 4. Validate executable existence before launch. | ||
|
|
||
| Avoid: | ||
|
|
||
| 1. Fire-and-forget calls in critical paths. | ||
| 2. Weak error propagation. | ||
|
|
||
| ### 3) Paths, Files, and Temporary Artifacts | ||
|
|
||
| Preferred approach: | ||
|
|
||
| 1. Normalize and validate paths before use. | ||
| 2. Create parent dirs deterministically. | ||
| 3. Use atomic write patterns where practical. | ||
| 4. Guard cleanup operations carefully. | ||
|
|
||
| Avoid: | ||
|
|
||
| 1. Trusting user-provided file names blindly. | ||
| 2. Destructive deletes without explicit path checks. | ||
|
|
||
| ### 4) Thread/UI Safety (Qt + Worker Threads) | ||
|
|
||
| Preferred approach: | ||
|
|
||
| 1. Keep UI updates on main thread. | ||
| 2. Use centralized worker orchestration. | ||
| 3. Register and clean up worker state deterministically. | ||
|
|
||
| Avoid: | ||
|
|
||
| 1. Cross-thread UI updates. | ||
| 2. Orphan worker/process state. | ||
|
|
||
| ### 5) Test Determinism and Ordering | ||
|
|
||
| Preferred approach: | ||
|
|
||
| 1. Express logical prerequisites with explicit dependencies. | ||
| 2. Keep order assumptions declarative. | ||
| 3. Ensure required plugins are part of test dependencies. | ||
|
|
||
| Recent example: | ||
|
|
||
| 1. Refactored tabs dependency chain to `pytest-dependency`. | ||
| 2. Added `pytest-dependency` to test extras and `make prepare-test`. | ||
| 3. Enabled `--order-dependencies`. | ||
|
|
||
| ## Verification Protocol | ||
|
|
||
| Always verify in two layers: | ||
|
|
||
| 1. Fast targeted checks | ||
| 2. Broader regression checks | ||
|
|
||
| ### Commands | ||
|
|
||
| ```bash | ||
| make clean | ||
| conda run -n REvoDesignLatestDev make kw-test PYTEST_KW='single keywords' | ||
| conda run -n REvoDesignLatestDev make kw-test PYTEST_KW='"keywordA or keywordB"' | ||
| ``` | ||
|
|
||
| For dependency-chain checks, include anchor tests and downstream tests in the keyword expression. | ||
|
|
||
| ## Documentation Checklist For Each Batch | ||
|
|
||
| 1. What rule/finding was fixed. | ||
| 2. Why it was dangerous. | ||
| 3. What changed and why this is minimal. | ||
| 4. Which tests validate the change. | ||
| 5. Any known limitations or follow-up tasks. | ||
|
|
||
| ## Suggested Tracking Table | ||
|
|
||
| Use this table format in PR descriptions or internal notes: | ||
|
|
||
| | DeepSource ID | Rule | Danger | Complexity | Files | Fix summary | Validation | | ||
| |---|---|---|---|---|---|---| | ||
| | example-id | PY-XXXX | D1 | C1 | `src/...` | safer error path | `kw-test ...` | | ||
|
|
||
| ## Anti-Patterns To Avoid | ||
|
|
||
| 1. Batch-fixing unrelated modules in one commit. | ||
| 2. Silencing warnings without code-level mitigation. | ||
| 3. Skipping `make clean` before validating flaky areas. | ||
| 4. Reordering tests to hide dependency problems. | ||
| 5. Editing test expectations without validating runtime behavior. | ||
|
|
||
| ## Definition Of Done (DeepSource Batch) | ||
|
|
||
| A batch is done only if all are true: | ||
|
|
||
| 1. Selected findings are fixed or explicitly deferred with reason. | ||
| 2. No new regressions in targeted tests. | ||
| 3. Changelog updated. | ||
| 4. Rationale documented for non-obvious changes. | ||
| 5. Diff remains minimal and reviewable. | ||
|
|
||
| ## Notes For Future Codex Runs | ||
|
|
||
| 1. Start from highest danger findings, not oldest findings. | ||
| 2. Keep a local snapshot of issue payloads to prevent paging/context loss. | ||
| 3. Prefer deterministic tests and explicit dependency wiring for UI case chains. | ||
| 4. If network is restricted, document the blocker and still finish code/test/docs work that can run locally. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,23 +1,48 @@ | ||
| # Copy this file to server/.env.production for production use. | ||
|
|
||
| ## How the images will be named | ||
| SERVER_IMAGE=revodesign-pssm-gremlin-server-non-root | ||
| RUNNER_IMAGE=revodesign-pssm-gremlin-non-root | ||
|
|
||
| ## In which directory the server will run | ||
| SERVER_DIR=/Users/yyy/Documents/protein_design/REvoDesign/playground/server_test | ||
| LOG_DIR=/Users/yyy/Documents/protein_design/REvoDesign/playground/server_test/revodesign/server/logs | ||
|
|
||
| ## Databases for MSA searching | ||
| DB_UNIREF30=/Users/yyy/Documents/protein_design/REvoDesign/playground/miniuc/uc30/miniuc30 | ||
| DB_UNIREF90=/Users/yyy/Documents/protein_design/REvoDesign/playground/miniuc/uc90/uniref90 | ||
|
|
||
| ## Users for accessing the server via basic authentication | ||
| USERS_FILE=/Users/yyy/Documents/protein_design/REvoDesign/server/pssm_gremlin/users.template.txt | ||
|
|
||
| ## redis settings | ||
| REDIS_URL=redis://redis:6379/0 | ||
| BROKER_URL=redis://redis:6379/0 | ||
| RESULT_BACKEND=redis://redis:6379/0 | ||
|
|
||
| # Runner settings | ||
| ## Maximum number to run the server: NPROC * WORKER_CONCURRENCY | ||
| NPROC=4 | ||
|
|
||
| ## Gunicorn settings | ||
| GUNICORN_WORKERS=2 | ||
|
|
||
| ## Celery settings | ||
| WORKER_CONCURRENCY=2 | ||
|
|
||
| ## Runner user at local machine | ||
| RUNNER_UID=1000 | ||
| RUNNER_GID=1000 | ||
| DOCKER_GID=998 | ||
| RUNNER_USERNAME=revodesign | ||
| RUNNER_GROUP=revodesign_appgroup | ||
|
|
||
| ## Server settings | ||
| DOMAIN=0.0.0.0 | ||
| PORT=8080 | ||
|
|
||
| ## Whether to enable the dashboard user isolation | ||
| PUBLIC_DASHBOARD=false | ||
|
|
||
| ## Admin users, comma separated | ||
| ADMIN_USERS=admin | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Developer-specific absolute paths committed in the example file leak local filesystem structure (including username) and render the template unusable for any other user.
Replace every concrete path with a clearly labelled placeholder:
📝 Proposed fix
📝 Committable suggestion
🧰 Tools
🪛 dotenv-linter (4.0.0)
[warning] 9-9: [UnorderedKey] The LOG_DIR key should go before the SERVER_DIR key
(UnorderedKey)
🤖 Prompt for AI Agents