Repository navigation
Conversation
Add an additive fast-path parser for the constrained YAML dialect pnpm emits (2-space block indent, no anchors/aliases/multiline scalars, flow style only for small inline maps/seqs). It walks the bytes directly with memchr for line boundaries and the structural ':' separator, producing the same RawPnpmLockfile the serde/yaml_serde path produces. The parser is a strict subset parser: on any construct outside the recognized shape (multi-document streams, unmodeled top-level keys, unexpected indent, quoting it cannot normalize) it returns None and parse_raw_lockfile transparently falls back to the existing yaml_serde parser. Fiddly inline flow values (resolution/variants/string_or_seq) and package bodies are handed to yaml_serde so those shapes stay in lockstep with serde. Standalone behavior is preserved by construction. Adds a criterion bench (benches/pnpm_lock_parse.rs) comparing both paths; set AUBE_BENCH_LOCKFILE to point at a real lockfile. On a generated 1.1MB / 4000-package lockfile the subset path parses in ~28ms vs ~45ms for the serde path. Existing aube-lockfile tests (345 unit + 5 proptest, including the pnpm roundtrip proptest) stay green.
…npm subset parser
Add a differential test harness for the pnpm-lock.yaml subset parser:
parse a fixture both ways (subset fast path and the yaml_serde path)
and assert the resulting RawPnpmLockfile is structurally identical, or
that the subset path declined (an acceptable fallback). Equality is
compared via the {:#?} rendering since the raw types are
Deserialize-only. Covers committed fixtures and an opt-in sweep over an
external corpus directory (AUBE_PNPM_CORPUS), which fails on any
divergence.
Add fixtures exercising catalog:/catalogs:, overrides,
patchedDependencies, npm-alias deps, scoped packages,
peerDependencies(+Meta), os/cpu/libc, deprecated, bundledDependencies,
remote-tarball resolutions, empty importer {}, optional snapshots,
transitivePeerDependencies, the v6 layout, and git/ssh resolutions with
peer-paren and subpath keys. Add byte-scanner edge-case unit tests:
CRLF, tab indentation (declines), comments/blank lines, colon-bearing
values, quote styles and escapes, trailing whitespace.
Fix: the subset parser declined any importer written as an inline `{}`
(a workspace package with no dependencies), falling the whole lockfile
back to serde. Accept `{}` as an empty importer body, mirroring the
existing package/snapshot handling, so the fast path fires on
multi-package workspaces. Verified default-preserving against a 38-file
real-world corpus (pnpm lockfileVersion 9): 36 match, 2 decline
(multi-document streams), 0 divergences; the fix raised matches from 13
to 36 with no silent wrong parse.
Add real pnpm-lock.yaml fixtures harvested from pinned pnpm releases (pnpm 6 -> lv5.3, 7 -> lv5.4, 8 -> lv6.0, 9/10 -> lv9.0), covering plain, workspace, patched/overrides, and git/tarball/npm-alias deps. Differential coverage added: - cross-version match-or-decline check over the committed fixtures - pre-v9 top-level-dependency formats decline rather than misparse - v9 git/tarball/alias package keys (embedded #, @https://) fire and match serde - adversarial unit tests: snapshot dep values with peer parens, scoped peer keys, numeric/bool-looking string values, quoted bool for optional (declines), multiline block scalars (decline) Corpus sweep (differential_real_corpus) print per-file outcome. No divergences found across any pnpm/lockfile version.
…pnpm subset parser
The subset parser called the block-map/serde-delegate helpers
unconditionally for map-typed fields, ignoring an inline `{...}` value on
the header line. For `overrides`/`time`/`settings`/`catalogs`/
`patchedDependencies`/`importers`/`packages`/`snapshots` written in inline
flow form, it scanned an empty block and produced `Some({})` while serde
parsed the real entries — a silent divergence. An empty block map (`key:`
with a null body, or an empty snapshot `dependencies:`) also produced
`Some({})` where serde reads `None`.
parse_string_map now takes the inline value and declines on an inline map
or an empty result; the block-bodied top-level arms decline on any inline
value. Adds differential tests covering the inline-flow and empty-block
forms for every field touched.
Also:
- remove the unused `header` parameter from parse_seq_value
- broaden has_document_separator to catch `---\r`/`---\t`/`---#`; note it
is a fast-path hint with the structural-colon decline as the guarantee
- gate the `__bench_parse_*`/`__bench_counts` shims behind a `bench`
feature so they leave the crate's default public surface; the
pnpm_lock_parse bench requires it
- replace default_package_info's per-call `from_str("{}").expect(...)`
with `RawPackageInfo::default()` (derive Default), pinned equal to
serde's `{}` parse by a test
- comment the intentionally-orphaned bundled-pkg kitchen-sink fixture
entry
- credit jzon-rs for the byte-cursor design in the module header
…it-rate guards for pnpm subset parser Hardens the c766fc4 decline fix against two regressions: - inline_vs_block_never_diverges: a proptest (256 cases, deterministic ChaCha seed) generates a logical lockfile over the map-typed fields (overrides/time) as Null/Empty/Populated, renders it both block and inline-flow style, and asserts diff_subset_vs_serde returns Match or Declined for BOTH renderings. Fuzzes the inline-vs-block boundary so a re-introduced silent inline misparse is caught. - populated_block_forms_still_fire_per_field: pins the legitimate populated BLOCK form of overrides/time/settings/patchedDependencies/ catalogs/importers/snapshot.optionalDependencies to the fast path (fires + matches), evidencing the inline-decline fix did not regress the common-case hit rate. - map_field_null_empty_populated_track_serde: asserts the null/empty/populated decision matches serde's None/empty/populated reading (populated block fires; null body and inline {} decline; no divergence either way).
…een package and snapshot
Author
|
Closing — pursuing this upstream in jdx/aube instead (aubepkg#912). nub will pick it up via the normal upstream merge-sync rather than carrying it as a fork customization. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a byte-cursor subset parser for
pnpm-lock.yamlthat extracts the fields the resolver needs without materializing the full YAML document, cutting parse time by ~1.6x on representative lockfiles.What it does
crates/aube-lockfile/src/pnpm/subset.rs: a hand-written byte-cursor scanner that reads only the importer/package fields the engine consumes, with a fast path for the common block-mapping layout.Default-preserving
The subset parser is an internal fast path; any input it cannot handle defers to the existing parser, so output is byte-for-byte identical to the prior behavior across the full fixture corpus. No public surface or default changes.
Verification (on the nub-fork base)
cargo build --release -p aube-lockfile— greencargo test -p aube-lockfile— 442 passed, 0 failed, 1 ignoredcargo clippy -p aube-lockfile --release— clean on touched filesUpstream context: drafted against jdx/aube as aubepkg#912.