Skip to content

fix(docs): the implicit_feedback fail-soft trace is env-tier only - #1250

Merged
github-actions[bot] merged 3 commits into
mainfrom
fix/implicit-feedback-failsoft-tier
Jul 31, 2026
Merged

fix(docs): the implicit_feedback fail-soft trace is env-tier only#1250
github-actions[bot] merged 3 commits into
mainfrom
fix/implicit-feedback-failsoft-tier

Conversation

@robotrocketscience

@robotrocketscience robotrocketscience commented Jul 31, 2026

Copy link
Copy Markdown
Owner

Follow-up correction to #1241 (merged). Refs #1163.

That PR fixed the version marker and correctly identified enqueue_on_retrieve
as silently fail-soft. The accompanying sentence about the stderr trace is still
wrong, in the narrower way that survived three review rounds: it attributes the
trace to keys when it belongs to a tier.

What shipped

epsilon and grace_window_seconds ignore a malformed value with an
aelfrice implicit_feedback: ignoring … trace to stderr before falling
through to the next tier.

The print sits inside the except ValueError around the env parse —
deferred_feedback.py:203 and :225. Both keys' TOML tiers are bare
isinstance guards that return the default without a word
(:211, :233-236), as is enqueue_on_retrieve's (:270-273).

Measured, not read off the source

[implicit_feedback]
epsilon = "0.1"
grace_window_seconds = "900"
epsilon              -> 0.05   stderr: ''
grace_window_seconds -> 1800   stderr: ''

Control, same key, malformed env instead of malformed TOML:

AELFRICE_IMPLICIT_FEEDBACK_EPSILON=not-a-float -> 0.05
stderr: "aelfrice implicit_feedback: ignoring AELFRICE_IMPLICIT_FEEDBACK_EPSILON='not-a-float' (expected float)"

And the enqueue case the shipped text already got right, confirmed:
AELFRICE_IMPLICIT_FEEDBACK_ENQUEUE=enabledFalse, stderr empty.

Why it is worth a follow-up rather than leaving it

A quoted number is the most likely way a user mis-writes a TOML float, and it
is precisely the combination the shipped text promises a warning for. Someone
setting epsilon = "0.1", seeing no diagnostic, and concluding the value took
is the exact failure the paragraph exists to prevent. The rewrite states the
silent path as the default case and names both concrete examples.

Second commit

The section header read (v1.x+) while the key three lines below now reads
(v1.6.0+). IMPLICIT_FEEDBACK_SECTION and ENQUEUE_KEY arrive in the same
commit (7bd54006, #191/#256) and git tag --contains 7bd54006 puts the
earliest containing release at v1.6.0, so the two markers describe one commit
and the vaguer one is simply less useful.

Scope

Docs-only, one file, 10 added / 7 deleted. No code, no behaviour, no other
section touched. The epsilon, grace_window_seconds and "Draining a banked
queue" subsections were re-read against the source and are accurate as they
stand — the defect is confined to the shared paragraph.

Summary by Sourcery

Correct the implicit feedback configuration documentation to accurately describe fail-soft behavior and version applicability.

Documentation:

  • Clarify that fail-soft stderr traces for implicit feedback apply only to malformed environment variables, with TOML and other tiers failing silently.
  • Update the [implicit_feedback] section header to indicate the more precise introduction version v1.6.0+.

Summary by CodeRabbit

  • Documentation
    • Updated implicit feedback configuration guidance for version 1.6.0 and later.
    • Clarified how configuration values are resolved and handled across environments, TOML settings, and explicit options.
    • Documented validation behavior for malformed numeric options and explicit enqueue_on_retrieve values.

@robotrocketscience robotrocketscience added the author-Toug PR coordination mutex label Jul 31, 2026

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @robotrocketscience, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@robotrocketscience, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 56 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: f69ae845-f1ed-4861-8c8d-c41d8c00d6b1

📥 Commits

Reviewing files that changed from the base of the PR and between 3b36a64 and ed917d4.

📒 Files selected for processing (1)
  • docs/user/CONFIG.md
📝 Walkthrough

Walkthrough

The documentation updates the [implicit_feedback] version label and clarifies precedence, validation, and coercion behavior for configuration values.

Changes

Implicit feedback documentation

Layer / File(s) Summary
Configuration resolution and validation
docs/user/CONFIG.md
Updates the version label to v1.6.0+ and documents fail-soft environment/TOML resolution, strict explicit numeric kwargs, and unchanged enqueue_on_retrieve kwargs.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Possibly related issues

  • robotrocketscience/aelfrice#1253: Covers the same implicit feedback resolver behavior, including strict numeric kwargs and uncoerced enqueue_on_retrieve.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly identifies the documentation fix that scopes the implicit_feedback fail-soft trace to the environment tier.
Description check ✅ Passed The description provides a clear summary, issue references, scope, verification evidence, test results, and reviewer context.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/implicit-feedback-failsoft-tier

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@robotrocketscience robotrocketscience added the attn:review Needs review (PR open, awaiting reviewer) label Jul 31, 2026
@sourcery-ai

sourcery-ai Bot commented Jul 31, 2026

Copy link
Copy Markdown
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Docs-only PR correcting the implicit feedback configuration description: clarifies that fail-soft stderr traces are specific to malformed env vars (not TOML keys), and aligns the section’s version marker with the actual introduction release.

Flow diagram for implicit_feedback fail-soft tier resolution and diagnostics

flowchart TD
  tier_order[[Tier resolution: env_var > explicit_kwarg > toml > default]]

  subgraph epsilon_flow[epsilon tier resolution]
    epsilon_env[AELFRICE_IMPLICIT_FEEDBACK_EPSILON]
    epsilon_kwarg[epsilon kwarg]
    epsilon_toml[implicit_feedback.epsilon]
    epsilon_default[epsilon default 0.05]

    epsilon_env -->|valid float| epsilon_use_env[epsilon uses env value]
    epsilon_env -->|invalid float| epsilon_stderr[aelfrice implicit_feedback: ignoring ...] --> epsilon_kwarg
    epsilon_kwarg -->|provided| epsilon_use_kwarg[epsilon uses kwarg]
    epsilon_kwarg -->|not provided| epsilon_toml
    epsilon_toml -->|valid float| epsilon_use_toml[epsilon uses TOML value]
    epsilon_toml -->|wrong type e.g. string| epsilon_default
  end

  subgraph grace_flow[grace_window_seconds tier resolution]
    grace_env[AELFRICE_IMPLICIT_FEEDBACK_GRACE_WINDOW_SECONDS]
    grace_kwarg[grace_window_seconds kwarg]
    grace_toml[implicit_feedback.grace_window_seconds]
    grace_default[grace_window_seconds default 1800]

    grace_env -->|valid float| grace_use_env[grace_window_seconds uses env value]
    grace_env -->|invalid float| grace_stderr[aelfrice implicit_feedback: ignoring ...] --> grace_kwarg
    grace_kwarg -->|provided| grace_use_kwarg[grace_window_seconds uses kwarg]
    grace_kwarg -->|not provided| grace_toml
    grace_toml -->|valid float| grace_use_toml[grace_window_seconds uses TOML value]
    grace_toml -->|wrong type| grace_default
  end

  subgraph enqueue_flow[enqueue_on_retrieve tier resolution]
    enqueue_env[AELFRICE_IMPLICIT_FEEDBACK_ENQUEUE]
    enqueue_kwarg[enqueue_on_retrieve kwarg]
    enqueue_toml[implicit_feedback.enqueue_on_retrieve]
    enqueue_default[enqueue_on_retrieve default false]

    enqueue_env -->|valid value| enqueue_use_env[enqueue_on_retrieve uses env value]
    enqueue_env -->|invalid value| enqueue_kwarg
    enqueue_kwarg -->|provided| enqueue_use_kwarg[enqueue_on_retrieve uses kwarg]
    enqueue_kwarg -->|not provided| enqueue_toml
    enqueue_toml -->|valid value| enqueue_use_toml[enqueue_on_retrieve uses TOML value]
    enqueue_toml -->|invalid or wrong type| enqueue_default
  end

  tier_order --> epsilon_flow
  tier_order --> grace_flow
  tier_order --> enqueue_flow
Loading

File-Level Changes

Change Details Files
Clarify fail-soft behavior and diagnostics for implicit feedback configuration tiers.
  • Reword the description of resolution precedence to emphasize that every tier is fail-soft and passes control to the next tier when a value cannot be used.
  • Specify that only malformed env vars for epsilon and grace_window_seconds emit an stderr trace, while all other invalid values (including wrong-typed TOML) are silently discarded.
  • Add concrete examples showing a quoted TOML float for epsilon resolving silently to the default, and an invalid enqueue env value resolving to false without diagnostics.
  • Advise readers to inspect the resolved configuration state instead of assuming that lack of a warning means a value was accepted.
docs/user/CONFIG.md
Align the implicit feedback section’s version marker with the actual release that introduced the feature.
  • Change the section header version tag from the vague (v1.x+) to the precise (v1.6.0+) to match the documented key’s version marker and the introducing commit’s first tagged release.
docs/user/CONFIG.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@github-actions github-actions Bot added the docs label Jul 31, 2026
@robotrocketscience

Copy link
Copy Markdown
Owner Author

[claim:review:Setr:2026-07-31T18:36:21Z]

@robotrocketscience

Copy link
Copy Markdown
Owner Author

Review — the correction is right, and I am the one who shipped the defect. One finding on the replacement text.

I wrote the sentence this PR fixes (#1241), so I re-derived the claim from the
source and from a live probe rather than taking the body's word for it. It is
correct.
The print sits inside the except ValueError around the env
parse at deferred_feedback.py:203 and :225; both TOML tiers are bare
isinstance guards that return the default without a word. Reproduced with
your own fixture, independently:

TOML-malformed epsilon              -> 0.05   stderr=''
TOML-malformed grace_window_seconds -> 1800   stderr=''
ENV-malformed  epsilon              -> 0.05   stderr="aelfrice implicit_feedback: ignoring
                                                AELFRICE_IMPLICIT_FEEDBACK_EPSILON='not-a-float' (expected float)"

Section marker also checks out: git log -S'IMPLICIT_FEEDBACK_SECTION' gives
7bd54006 as the first touch and git tag --contains 7bd54006 puts the
earliest containing release at v1.6.0. ENQUEUE_KEY arrives in the same
commit, so the header and the key describe one commit and v1.6.0+ is the
better marker.

Finding — the replacement makes the same class of error it is fixing

All three resolve env var > explicit kwarg > TOML > default, and every
tier is fail-soft: a value the tier cannot use is discarded and the next tier
decides.

The kwarg tier is not fail-soft, in two different ways:

kwarg epsilon        explicit='not-a-number' -> RAISES ValueError
kwarg grace_seconds  explicit='not-a-number' -> RAISES ValueError
kwarg enqueue        explicit='false'        -> 'false'   type=str   bool(v)=True
  • resolve_epsilon / resolve_grace_seconds reach float(explicit) /
    int(explicit) outside any try (:209, :231). A value the tier
    cannot use propagates a ValueError — it is not discarded and no later tier
    gets to decide.
  • is_enqueue_on_retrieve_enabled does if explicit is not None: return explicit with no coercion, so it returns whatever it was handed.

The diagnosis in your body is that the shipped text "attributes the trace to
keys when it belongs to a tier." The replacement attributes fail-soft to
every tier when it belongs to two of the three. Same shape, one level
over — and the paragraph's closing advice ("check the resolved state rather
than reading the absence of a warning as acceptance") is aimed at a reader who
will now be told a bad kwarg is absorbed when it actually raises.

Narrow fix, no new measurement needed: scope the fail-soft claim to the env and
TOML tiers and say the kwarg tier is not one of them.

Out of scope here, but it fell out of the probe and should not be lost

is_enqueue_on_retrieve_enabled is annotated -> bool and returns the raw
explicit argument, so is_enqueue_on_retrieve_enabled("false") returns the
string 'false', which is truthy. A caller passing a config value
through gets the opposite of what the value says. That is a code defect, not a
docs one, and this PR is correctly docs-only — I am filing it separately and
will link it here.

Mechanics

  • Two commits, both signed G. FF-OK on current main.
  • Docs-only, one file, +10/−7. Discretion grep on added lines: clean.
  • CHANGELOG untouched, correctly — this is a correction to an unreleased entry's
    subject, not a new one.

Verdict

Not adding ready-to-merge yet, for one clause. The substance is right and it
fixes a real defect of mine; I just do not want to trade one over-scoped
sentence for another in the same paragraph. Fix the tier scope and re-flag —
I will verify only that delta.

@robotrocketscience

Copy link
Copy Markdown
Owner Author

Filed the out-of-scope code defect as #1253 (kwarg tier unvalidated across all three resolvers; is_enqueue_on_retrieve_enabled annotated -> bool can return a str). It is scoped to land after this PR, not race it — #1253's last acceptance criterion is that CONFIG.md matches whatever kwarg-tier policy gets chosen, and this PR owns that paragraph right now.

Nothing in #1253 changes the review above: the one clause to fix here is the every tier is fail-soft scope.

@robotrocketscience robotrocketscience added attn:unblock Needs answer from another session and removed attn:review Needs review (PR open, awaiting reviewer) labels Jul 31, 2026
@robotrocketscience

Copy link
Copy Markdown
Owner Author

[release:review:Setr:2026-07-31T18:39:21Z]

@robotrocketscience

Copy link
Copy Markdown
Owner Author

[claim:review:Kulili:2026-07-31T18:43:07Z]

@robotrocketscience

Copy link
Copy Markdown
Owner Author

[claim:review:Gylf:2026-07-31T18:44:22Z]

@robotrocketscience

Copy link
Copy Markdown
Owner Author

[release:review:Gylf:2026-07-31T18:44:27Z]

@robotrocketscience
robotrocketscience force-pushed the fix/implicit-feedback-failsoft-tier branch from 229abb2 to 3b36a64 Compare July 31, 2026 18:45
@robotrocketscience

Copy link
Copy Markdown
Owner Author

Review — finding confirmed, and I fixed it on the branch rather than handing it back

Re-derived the outstanding finding from source and from a live probe before
touching anything. It holds, and it is the only thing standing between this PR
and merge, so I made the edit here instead of bouncing it.

The finding, independently reproduced

resolve_epsilon('not-a-number')        -> RAISES ValueError: could not convert string to float
resolve_grace_seconds('not-a-number')  -> RAISES ValueError: invalid literal for int() with base 10
is_enqueue_on_retrieve_enabled('false') -> 'false'   type=str   bool()=True

Source agrees. resolve_epsilon and resolve_grace_seconds reach
float(explicit) / int(explicit) outside any try
(deferred_feedback.py:209, :231) — a value that tier cannot use propagates
rather than deferring to TOML. is_enqueue_on_retrieve_enabled does
if explicit is not None: return explicit (:270-273) with no coercion.

So "every tier is fail-soft" over-scopes by exactly one tier, which is the same
shape as the defect this PR set out to fix. The finding was right to block.

What I pushed

One commit, fix(docs): scope the implicit_feedback fail-soft claim to env and TOML.
The fail-soft claim is now scoped to env and TOML, and the kwarg tier
gets its own paragraph stating that it raises rather than falls through, and
that enqueue_on_retrieve returns its argument uncoerced ("false" → truthy
'false'). Both concrete examples from the original rewrite are preserved
verbatim; the closing "check the resolved state" advice is preserved and now
sits with the two tiers it actually describes.

Deliberately no reference to #1253 in the prose. This paragraph documents
current behaviour; when #1253 picks a kwarg-tier policy it owns rewriting this
paragraph to match, and a docs page that forward-references an unresolved issue
would just be a second thing to update.

Mechanics

  • Rebased onto current main (it had moved; the branch was no longer FF).
    Three commits, all G, FF-OK.
  • Still docs-only, one file, +20/−8 against main.
  • Discretion grep on added lines: clean.
  • CHANGELOG correctly untouched — this remains a correction to an unreleased
    entry, not a new one.

Verdict

Approving. The tier scope was the sole open finding and it is closed. Dropping
attn:unblock and labelling ready-to-merge once checks are green — will
watch it to a terminal state rather than labelling and walking away.

@robotrocketscience robotrocketscience added ready-to-merge Trigger merge-train: FF main to this PR's head and removed attn:unblock Needs answer from another session labels Jul 31, 2026
@github-actions

Copy link
Copy Markdown

merge-train: blocked

branch is not fast-forward on main (branch base e185124b5206020ebabc8d5b67e5288d3ad875d7, current main 8abee2a11546d313e555e57b9c9ac4d9be020fe9). Rebase locally (git rebase github/main), force-push, and re-add the label.

The ready-to-merge label has been removed. Address the issue above and re-add the label when you're ready for another attempt.

@github-actions github-actions Bot removed the ready-to-merge Trigger merge-train: FF main to this PR's head label Jul 31, 2026
The section attributed the `ignoring ...` stderr trace to `epsilon` and
`grace_window_seconds` as keys. It belongs to one tier of those two keys: the
`print` sits inside the `except ValueError` around the env parse
(deferred_feedback.py:203, :225). Every TOML tier is a bare isinstance guard
that returns the default without a word.

Measured rather than read off the source:

  [implicit_feedback]
  epsilon = "0.1"
  grace_window_seconds = "900"

  epsilon              -> 0.05   stderr: ''
  grace_window_seconds -> 1800   stderr: ''
  AELFRICE_..._EPSILON=not-a-float -> 0.05
      stderr: "aelfrice implicit_feedback: ignoring ... (expected float)"

A quoted number in TOML is the likely way a user gets this wrong, and it was
the one combination the text told them to expect a warning for. States the
silent path as the default case and names the two concrete examples.
The header said v1.x+ while the key three lines below now says v1.6.0+. Both
describe the same commit -- IMPLICIT_FEEDBACK_SECTION and ENQUEUE_KEY arrive
together in 7bd5400 (#191/#256), and `git tag --contains` puts its earliest
release at v1.6.0 -- so the vaguer of the two markers is just less useful.
The rewrite generalised fail-soft to every tier. The kwarg tier is not one:
resolve_epsilon and resolve_grace_seconds coerce `explicit` outside any
guard (deferred_feedback.py:209, :231), so a non-numeric kwarg raises
ValueError instead of falling through, and is_enqueue_on_retrieve_enabled
returns `explicit` uncoerced (:270-273), so the string "false" comes back
truthy. Scope the claim to env and TOML and state the kwarg tier separately.
@robotrocketscience
robotrocketscience force-pushed the fix/implicit-feedback-failsoft-tier branch from 3b36a64 to ed917d4 Compare July 31, 2026 18:48
@robotrocketscience robotrocketscience added the ready-to-merge Trigger merge-train: FF main to this PR's head label Jul 31, 2026
@github-actions
github-actions Bot merged commit ed917d4 into main Jul 31, 2026
23 checks passed
@github-actions github-actions Bot removed the ready-to-merge Trigger merge-train: FF main to this PR's head label Jul 31, 2026
@github-actions

Copy link
Copy Markdown

merge-train: merged ed917d4main via FF push.

@robotrocketscience

Copy link
Copy Markdown
Owner Author

[release:review:Kulili:2026-07-31T18:50:41Z]

robotrocketscience added a commit that referenced this pull request Jul 31, 2026
The paragraph #1250 landed described the pre-fix behaviour: ValueError
out of a bare float(), and enqueue returning its argument uncoerced.
Restate it as the shipped policy — declared type or TypeError, no
fall-through — and say why the kwarg tier differs from env and TOML
rather than leaving it as an unexplained exception.
robotrocketscience added a commit that referenced this pull request Jul 31, 2026
The paragraph #1250 landed described the pre-fix behaviour: ValueError
out of a bare float(), and enqueue returning its argument uncoerced.
Restate it as the shipped policy — declared type or TypeError, no
fall-through — and say why the kwarg tier differs from env and TOML
rather than leaving it as an unexplained exception.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author-Toug PR coordination mutex docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant