fix: harden NAR hash validation and improve URL parsing

[kalbasit]

This change strengthens the validation of NAR hashes by enforcing a
strict Nix32 format (52 characters) and optional narinfo hash prefix.
The ParseURL function has been rewritten to be more robust, separating
the hash and compression components more reliably and validating the
hash before further processing.

Key improvements:

• Defined NormalizedHashPattern and HashPattern for precise hash validation.
• Enhanced ParseURL to correctly handle various NAR URL formats including those with query parameters.
• Improved the Normalize method in the URL struct to better handle
  narinfo hash prefixes and sanitize the hash against path traversal.
• Updated all relevant tests to use valid Nix32 hashes and removed redundant test cases.
• Ensured consistent URL construction in JoinURL by correctly handling paths and query parameters.

This hardening prevents potential security issues related to invalid or
malicious NAR URLs and ensures consistent behavior across the
application.

[kalbasit]

• fix: harden NAR hash validation and improve URL parsing #842 👈 (View in Graphite)
• fix: harden the narinfo route pattern #841
• fix: harden the narinfo hash validation #840
• fix: Fix test data generation and skip signature verification #839
• fix: Improve NAR file serving and remove old URL prefix workaround #838
• fix: Apply NAR URL normalization in storage layer #837
• feat: Add NAR URL normalization function and support dashes/underscores in hashes #836
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[gemini-code-assist]

Summary of Changes

Hello @kalbasit, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the security and reliability of NAR hash validation and URL parsing. By introducing stricter validation rules for NAR hashes, enforcing the Nix32 format, and completely overhauling the URL parsing logic, the system is now more resilient to invalid or malicious inputs. These changes ensure consistent and secure handling of NAR-related URLs across the application.

Highlights

• NAR Hash Validation: Enforced a strict Nix32 format (52 characters) for NAR hashes and introduced optional narinfo hash prefix validation to strengthen security.
• URL Parsing Improvement: The ParseURL function has been rewritten to be more robust, reliably separating hash and compression components and validating the hash before further processing.
• New Hash Patterns: Defined NormalizedHashPattern and HashPattern constants for precise and consistent hash validation across the codebase.
• Test Updates: Updated all relevant tests to utilize valid Nix32 hashes and removed a specific, redundant deadlock test case.

Changelog

• pkg/cache/cache_test.go
  • Removed unused errors import.
  • Removed the testDeadlockNarInfoTriggersNarRefetch test function.
  • Removed the call to testDeadlockNarInfoTriggersNarRefetch from the test suite.
• pkg/cache/cdc_test.go
  • Updated nar.URL hash values in testCDCMixedMode to valid Nix32 hashes.
  • Updated nar.URL hash values in testCDCGetNarInfo to valid Nix32 hashes.
  • Modified niText to use a valid Nix32 store path and hash in testCDCGetNarInfo.
  • Updated PutNarInfo and GetNarInfo calls with valid Nix32 hashes.
• pkg/nar/filepath_test.go
  • Replaced example hashes with valid Nix32 hashes in TestNarFilePath test cases.
• pkg/nar/hash.go
  • Imported github.com/kalbasit/ncps/pkg/narinfo.
  • Defined NormalizedHashPattern constant for Nix32 hash format.
  • Defined HashPattern constant, incorporating narinfo.HashPattern and NormalizedHashPattern.
  • Removed the old narHashPattern variable.
  • Updated narHashRegexp to use the new HashPattern.
• pkg/nar/url.go
  • Replaced narRegexp with hashValidationRegexp using the new HashPattern.
  • Rewrote the ParseURL function to robustly extract hash, compression, and query parameters from NAR URLs, including validation of the hash against HashPattern.
• pkg/nar/url_test.go
  • Corrected the argument order for assert.ErrorIs in TestParseURL.
• pkg/storage/s3/s3_test.go
  • Updated hardcoded hash strings to valid Nix32 hashes in TestHasNarInfo_ErrorPaths, TestHasNar_ErrorPaths, TestPutNar_ErrorPaths, and TestNar_ErrorPaths functions.

Activity

• No specific activity (comments, reviews, progress) has been recorded for this pull request yet.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[dosubot]

Related Documentation

Checked 7 published document(s) in 1 knowledge base(s). No updates required.

^{How did I do? Any feedback?}  

[gemini-code-assist]

Code Review

This pull request significantly hardens NAR hash validation and improves URL parsing, which is a great step towards improving security and robustness. The new ParseURL implementation is much clearer and safer than the previous regex-based approach. The introduction of a strict HashPattern for NAR hashes prevents invalid or malicious inputs. The updates to tests to use valid Nix32 hashes are also a welcome change, ensuring consistency. Overall, this is a high-quality contribution that strengthens the application's core logic.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 3.96%. Comparing base (a6df951) to head (9d82f3a).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@          Coverage Diff          @@
##            main    #842   +/-   ##
=====================================
  Coverage   3.96%   3.96%           
=====================================
  Files          6       6           
  Lines        429     429           
=====================================
  Hits          17      17           
  Misses       409     409           
  Partials       3       3           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[kalbasit]

Backport failed for release-0.8, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release-0.8
git worktree add -d .worktree/backport-842-to-release-0.8 origin/release-0.8
cd .worktree/backport-842-to-release-0.8
git switch --create backport-842-to-release-0.8
git cherry-pick -x 2bb8649e9ede4161b25b801fcffe5b795728aa2d

[kalbasit]

Backport failed for release-0.8, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release-0.8
git worktree add -d .worktree/backport-842-to-release-0.8 origin/release-0.8
cd .worktree/backport-842-to-release-0.8
git switch --create backport-842-to-release-0.8
git cherry-pick -x 2bb8649e9ede4161b25b801fcffe5b795728aa2d