Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: skip closed streams in flush tick. Log when flushing due to time wind… #48605

Merged
merged 1 commit into from
Nov 21, 2024

Conversation

tryangul
Copy link
Contributor

What

Checks if queue (channel) is closed before sending tick tasks
Adds log for visibility on time window trigger

How

  • Adds a check in the loop for closed channels
  • Handles exceptions due to any race
  • Creates MessageQueue#isClosedForPublish for checking underlying channel
  • Log line

@tryangul tryangul requested a review from a team as a code owner November 21, 2024 19:51
Copy link

vercel bot commented Nov 21, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Nov 21, 2024 7:51pm

@octavia-squidington-iii octavia-squidington-iii added the CDK Connector Development Kit label Nov 21, 2024
@@ -29,6 +31,8 @@ abstract class ChannelMessageQueue<T> : MessageQueue<T> {
override suspend fun close() {
channel.close()
}
@OptIn(DelicateCoroutinesApi::class)
override fun isClosedForPublish(): Boolean = channel.isClosedForSend
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option besides opting in is to add an explicit closed flag to the interface

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. I went with the don't add more state approach, but a flag could be a thing. I think I'm gonna go with this for now to get some test cycles in.

Copy link
Contributor

@johnny-schmidt johnny-schmidt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that this works. I'm not sure about the DelicateCoroutinesApi, I've been avoiding delicate/experimental b/c coroutines are complex enough. Up to you

@tryangul tryangul merged commit eb25860 into master Nov 21, 2024
32 checks passed
@tryangul tryangul deleted the rbroughan/cadence-flush-tweaks branch November 21, 2024 21:38
matteogp pushed a commit that referenced this pull request Nov 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CDK Connector Development Kit
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants