Skip to content

MINOR: simplify OpenIterator #20060

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mjsax merged 1 commit into from
Jul 1, 2025
Merged

MINOR: simplify OpenIterator #20060

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 3 additions & 7 deletions streams/src/main/java/org/apache/kafka/streams/internals/metrics/OpenIterators.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,13 @@
import java.util.Comparator;
import java.util.NavigableSet;
import java.util.concurrent.ConcurrentSkipListSet;
import java.util.concurrent.atomic.LongAdder;

public class OpenIterators {
private final TaskId taskId;
private final String metricsScope;
private final String name;
private final StreamsMetricsImpl streamsMetrics;

private final LongAdder numOpenIterators = new LongAdder();
private final NavigableSet<MeteredIterator> openIterators = new ConcurrentSkipListSet<>(Comparator.comparingLong(MeteredIterator::startTimestamp));

private MetricName metricName;
Expand All @@ -50,24 +48,22 @@ public OpenIterators(final TaskId taskId,

public void add(final MeteredIterator iterator) {
openIterators.add(iterator);
numOpenIterators.increment();

if (numOpenIterators.intValue() == 1) {
if (openIterators.size() == 1) {

@chia7712 chia7712 Jun 28, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was worried for adding the same iterator, which could create unexpected metrics, but it seems StateStoreMetrics.addOldestOpenIteratorGauge already handle this case.

@mjsax mjsax Jun 30, 2025

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sue if I can follow. Can you elaborate?

@chia7712 chia7712 Jul 1, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the unclear comment. Please see the following example

        var openIterators = new OpenIterators();
        var iterator = new MeteredIterator(1000L);
        openIterators.add(iterator);
        openIterators.add(iterator);

The second openIterators.add(iterator) did not invoke StateStoreMetrics.addOldestOpenIteratorGauge before, but it does now. However, this should be fine, since StateStoreMetrics.addOldestOpenIteratorGauge does not create new metrics in this scenario.

@mjsax mjsax Jul 1, 2025 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, would be a bug anyway, to call openIterators.add(iterator); twice on the same iterator object. Each iterator should be registered exactly one time.

But yes, we would not add a new metric.

metricName = StateStoreMetrics.addOldestOpenIteratorGauge(taskId.toString(), metricsScope, name, streamsMetrics,
(config, now) -> openIterators.first().startTimestamp()
);
}
}

public void remove(final MeteredIterator iterator) {
if (numOpenIterators.intValue() == 1) {
if (openIterators.size() == 1) {
streamsMetrics.removeMetric(metricName);
}
numOpenIterators.decrement();
openIterators.remove(iterator);
}

public long sum() {
return numOpenIterators.sum();
return openIterators.size();
}
}