Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][broker] Fix shadow topics cannot be consumed when the entry is not cached #23147

Merged

Commits on Aug 9, 2024

  1. [fix][broker] Fix shadow topics cannot be consumed when the entry is …

    …not cached
    
    ### Motivation
    
    For shadow topics, a `ReadOnlyLedgerHandle` is created to read messages
    from the source topic when the entry is not cached. However, it
    leverages the `readAsync` API that validates the `lastAddConfirmed`
    field (LAC). In `ReadOnlyLedgerHandle`, this field could never be updated,
    so `readAsync` could fail immediately. See `LedgerHandle#readAsync`:
    
    ```java
    if (lastEntry > lastAddConfirmed) {
        LOG.error("ReadAsync exception on ledgerId:{} firstEntry:{} lastEntry:{} lastAddConfirmed:{}",
                ledgerId, firstEntry, lastEntry, lastAddConfirmed);
        return FutureUtils.exception(new BKReadException());
    }
    ```
    
    This bug is not exposed because:
    1. `PulsarMockReadHandle` does not maintain a LAC field.
    2. The case for cache miss is never tested.
    
    ### Modifications
    
    Replace `readAsync` with `readUnconfirmedAsync`. The managed ledger
    already maintains a `lastConfirmedEntry` to limit the last entry. See
    `ManagedLedgerImpl#internalReadFromLedger`:
    
    ```java
    Position lastPosition = lastConfirmedEntry;
    
    if (ledger.getId() == lastPosition.getLedgerId()) {
        lastEntryInLedger = lastPosition.getEntryId();
    ```
    
    Add `ShadowTopicRealBkTest` to cover two code changes
    `RangeEntryCacheImpl#readFromStorage` and `EntryCache#asyncReadEntry`.
    BewareMyPower committed Aug 9, 2024
    Configuration menu
    Copy the full SHA
    c14b699 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    cb36d00 View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2024

  1. Configuration menu
    Copy the full SHA
    38a4c84 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0d143a9 View commit details
    Browse the repository at this point in the history