Skip to content

improvement(event-streaming): strong type streamer IDs#2441

Merged
onur-ozkan merged 10 commits intoGLEECBTC:devfrom
BigFish2086:2207-strong-type-streamer-id
Jun 9, 2025
Merged

improvement(event-streaming): strong type streamer IDs#2441
onur-ozkan merged 10 commits intoGLEECBTC:devfrom
BigFish2086:2207-strong-type-streamer-id

Conversation

@BigFish2086
Copy link
Copy Markdown

@BigFish2086 BigFish2086 commented May 6, 2025

(#2207): Refactored the Streamer Id to its own enumeration type.

@BigFish2086 BigFish2086 changed the title Strong Type Streamer Ids #2207 Strong Type Streamer Ids May 6, 2025
@mariocynicys mariocynicys changed the title Strong Type Streamer Ids improvement(event-streaming): strong type streamer IDs May 6, 2025
@mariocynicys mariocynicys added status: pending review priority: high Important tasks that need attention soon. improvement: API labels May 6, 2025
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the event streaming system to use a strongly typed StreamerId enum rather than plain strings for streamer identifiers. Key changes include updating various event streamer implementations, responses, and manager functions across the codebase to adopt the new StreamerId type, and adding a new StreamerId enum with dedicated variants.

Reviewed Changes

Copilot reviewed 21 out of 21 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
mm2src/mm2_p2p/src/application/network_event.rs Changed streamer_id return type to StreamerId for Network events.
mm2src/mm2_main/src/rpc/streaming_activations/mod.rs/disable.rs Updated request/response types from String to StreamerId.
mm2src/mm2_main/src/* (lp_swap, orderbook_events, etc.) Replaced String-based streamer ids with strongly typed StreamerId.
mm2src/mm2_event_stream/src/streamer.rs Introduced the new StreamerId enum and updated Display/Debug impls.
mm2src/mm2_event_stream/src/event.rs Changed event handling to use StreamerId instead of raw strings.
mm2src/coins/* Updated coin-related event streamers to use StreamerId for balance, fee, tx history, etc.

…BTC#2207

used enumeration to represent the IDs of the different types of
streamers that are currently supported.
@BigFish2086 BigFish2086 force-pushed the 2207-strong-type-streamer-id branch from 52a34a5 to 80b19c2 Compare May 6, 2025 18:14
Copy link
Copy Markdown

@borngraced borngraced left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work! two notes from me


fn streamer_id(&self) -> String { Self::derive_streamer_id().to_string() }
fn streamer_id(&self) -> StreamerId { Self::derive_streamer_id().clone() }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can tell this that this method will be called frequently. Can you refactor StreamerId enum to something like this

#[derive(Debug)]
pub enum StreamerIdInner {
    Network,
    Heartbeat,
    SwapStatus,
    OrderStatus,
    Task(u64),
    Balance(String),
    DataNeeded(String),
    TxHistory(String),
    FeeEstimation(String),
    OrderbookUpdate(String),
    #[cfg(test)]
    ForTesting(String),
}

pub type StreamerId = Arc<StreamerIdInner>;

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems good, I have tried this idea and pushed the changes.


#[inline(always)]
pub const fn derive_streamer_id() -> &'static str { "ORDER_STATUS" }
pub const fn derive_streamer_id() -> &'static StreamerId { &StreamerId::OrderStatus }
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

derive_streamer_id returns Owned value in some places while static value in another place. Let's stick to one

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it to return StreamerId in all places.

@smk762
Copy link
Copy Markdown

smk762 commented May 7, 2025

Please add comments to related docs PR GLEECBTC/komodo-docs-mdx#457 if this includes any changes to streamer_id format etc. so I can update examples.

@BigFish2086 BigFish2086 force-pushed the 2207-strong-type-streamer-id branch from e3c3f64 to 702befe Compare May 8, 2025 17:10
borngraced
borngraced previously approved these changes May 9, 2025
Copy link
Copy Markdown

@borngraced borngraced left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Copy link
Copy Markdown
Collaborator

@mariocynicys mariocynicys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

done a quick review over the last commit. will review thoroughly next week.

BigFish2086 added 2 commits May 12, 2025 18:43
`derive_streamer_id` was returning `StreamerId` in some places and
`&'static StreamerId` in others, so it's now fixed to return
`StreamerId` in every place.
@BigFish2086 BigFish2086 force-pushed the 2207-strong-type-streamer-id branch from 9cb39a2 to fd72859 Compare May 12, 2025 15:51
BigFish2086 pushed a commit to BigFish2086/komodo-defi-framework that referenced this pull request May 12, 2025
…ers GLEECBTC#2441

- Added DeriveStreamerId trait with InitParam for new and DeriveParam for derive_streamer_id, including lifetime 'a for flexible references.
- Refactored streamer structs to use &str for DeriveParam where applicable.
Copy link
Copy Markdown
Collaborator

@mariocynicys mariocynicys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! ❤️

Done reviewing everything. Will just follow up with my comments and approve when they are done.

Comment on lines +21 to +45
#[derive(Clone, Deserialize, Eq, Hash, PartialEq, Serialize)]
pub enum StreamerId {
Network,
Heartbeat,
SwapStatus,
OrderStatus,
Task(u64), // XXX: should be TaskId (from rpc_task)
Balance(String),
DataNeeded(String),
TxHistory(String),
FeeEstimation(String),
OrderbookUpdate(String),
#[cfg(test)]
ForTesting(String),
}

impl fmt::Display for StreamerId {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
StreamerId::Network => write!(f, "NETWORK"),
StreamerId::Heartbeat => write!(f, "HEARTBEAT"),
StreamerId::SwapStatus => write!(f, "SWAP_STATUS"),
StreamerId::OrderStatus => write!(f, "ORDER_STATUS"),
StreamerId::Task(task_id) => write!(f, "TASK:{task_id}"),
StreamerId::Balance(coin) => write!(f, "BALANCE:{coin}"),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so the main reason i didn't wanna do this in the original PR was that this introduces mm2 specific lingo/logic in this crate (mm2_event_stream) which is trying to be abstract.

there looks to be no way around it though - if we wanna strong type streamer ids and avoid possible dev errors.

could you please move this logic to a different isolated/new file. as it doens't really fit in streamer.rs. maybe call it streamer_ids.rs.

Copy link
Copy Markdown
Author

@BigFish2086 BigFish2086 May 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved it to its own file, streamer_ids.rs as suggested in 044d932. I have thought about the how to make the Event more generic like this:

pub struct Event<'m, 's, M, S>
where
    M: Clone + Debug + Deserialize<'m> + Serialize,
    S: Clone + Debug + Deserialize<'s> + Eq + Hash + PartialEq + Serialize,
{
    pub streamer_id: S,
    pub message: M,
    pub error: bool,
    _phantom_streamer_id: PhantomData<&'s ()>,
    _phantom_message: PhantomData<&'m ()>,
}

pub type Event01<'m, 's> = Event<'m, 's, StreamerId, serde_json::Value>;

But that would require us using the lifetimes all over the code and I thought it would be better to discuss it more before implementing it (in another PR maybe.)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q about this suggestion, using S and M as generics in the event type essentially creates multiple event types for each S & M pair.

we could ofc then dyn things up and box them to be able to work with them from the streaming manager side. but just saying.

…er_ids.rs

- move StreamerId from streamer.rs to streamer_ids.rs
- use the default Debug implementation for StreamerId
- use custom serialization and deserialization for StreamerId
  e.g. StreamerId::Balance(String::from("ETH")) will look like BALANCE:ETH instead of {"Balance":"ETH"}
use std::fmt;

#[derive(Clone, Debug, Eq, Hash, PartialEq)]
pub enum StreamerId {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not make this streamer list dynamic and agnostic about existing event sources?
I think, modules, which want to send events, may register themselves as event sources in the streamer. In register call they provide event prefixes. In response the streamer would return generated streamer_id which is used as a handle to streamer functions.
With that, we may add new modules as event sources, without touching the streamer code.
@mariocynicys

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so let's say we enable the orderstatus streamer. it registers itself in such list and gets an ID back. 1- what's the type of this ID? 2- how other parts of the code would know this ID so to send data to the streamer (e.g.)

… fix OrderbookUpdate format

- Moved all StreamerId string constants to module-level scope for reuse across Display and Deserialize implementations.
- Fixed inconsistency in OrderbookUpdate variant string format (was using "/" instead of ":").
- Improved test variant handling using FOR_TESTING_PREFIX under #[cfg(test)].
@mariocynicys mariocynicys requested review from dimxy and laruh May 27, 2025 14:51
BigFish2086 added 2 commits May 28, 2025 13:51
Replaces `#[cfg(test)]` with `#[cfg(any(test, target_arch = "wasm32"))]`
so that the ForTesting variant is compiled when targeting WebAssembly.
Copy link
Copy Markdown

@laruh laruh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@BigFish2086 @mariocynicys please leave a todo in the issue #2207 or create a new one about optimisations were discussed in the thread #2441 (comment)

@BigFish2086 good start!

@mariocynicys mariocynicys mentioned this pull request Apr 30, 2025
42 tasks
@onur-ozkan onur-ozkan merged commit 37bf06a into GLEECBTC:dev Jun 9, 2025
22 of 23 checks passed
BigFish2086 pushed a commit to BigFish2086/komodo-defi-framework that referenced this pull request Jun 12, 2025
…ers GLEECBTC#2441

- Added DeriveStreamerId trait with InitParam for new and DeriveParam for derive_streamer_id, including lifetime 'a for flexible references.
- Refactored streamer structs to use &str for DeriveParam where applicable.
BigFish2086 pushed a commit to BigFish2086/komodo-defi-framework that referenced this pull request Jun 12, 2025
…ers GLEECBTC#2441

- Added DeriveStreamerId trait with InitParam for new and DeriveParam for derive_streamer_id, including lifetime 'a for flexible references.
- Refactored streamer structs to use &str for DeriveParam where applicable.
dimxy pushed a commit to dimxy/komodo-defi-framework that referenced this pull request Jun 27, 2025
* dev: (30 commits)
  chore(core): replace hash_raw_entry with stable entry() API (GLEECBTC#2473)
  chore(core): adapt `MmError` and usages for compatibility with new rustc versions (GLEECBTC#2443)
  feat(wallet): add `delete_wallet` RPC (GLEECBTC#2497)
  chore(release): add changelog entries for v2.5.0-beta (GLEECBTC#2494)
  chore(release): bump kdf version to 2.5.0-beta (GLEECBTC#2492)
  feat(tests): zcoin unit test to validate dex fee (GLEECBTC#2460)
  fix(zcoin): correctly track unconfirmed z-coin notes (GLEECBTC#2331)
  improvement(orders): remove BTC specific volume from min_trading_vol logic (GLEECBTC#2483)
  feat(ibc-routing-part-2): supporting entire Cosmos network for swaps (GLEECBTC#2476)
  fix(startup): don't initialize WalletConnect during startup (GLEECBTC#2485)
  fix(dns): better ip resolution (GLEECBTC#2487)
  improvement(event-streaming): strong type streamer IDs (GLEECBTC#2441)
  bump timed-map to `1.4.1` (GLEECBTC#2481)
  improvement(RPC): unified interface for legacy and current RPC interfaces (GLEECBTC#2450)
  improvement(tendermint): `tendermint_tx_internal_id` helper (GLEECBTC#2438)
  feat(walletconnect): add WalletConnect v2 support for EVM and Cosmos (GLEECBTC#2223)
  feat(ibc-routing-part-1): supporting entire Cosmos network for swaps (GLEECBTC#2459)
  fix(test): fix HD Wallet message signing tests (GLEECBTC#2474)
  improvement(builds): enable static CRT linking for MSVC builds (GLEECBTC#2464)
  feat(wallet): implement HD multi-address support for message signing (GLEECBTC#2432)
  ...

# Conflicts:
#	mm2src/coins/qrc20.rs
#	mm2src/mm2_main/src/lp_swap/maker_swap.rs
BigFish2086 added a commit to BigFish2086/komodo-defi-framework that referenced this pull request Jul 16, 2025
…eParam GLEECBTC#2441

Replaces the raw tuple (&str, &str) with a named type alias for clarity
BigFish2086 added a commit to BigFish2086/komodo-defi-framework that referenced this pull request Jul 16, 2025
onur-ozkan pushed a commit that referenced this pull request Jul 16, 2025
…eamers (#2489)

* refactor(event-streaming): impl DeriveStreamerId trait for all streamers #2441

- Added DeriveStreamerId trait with InitParam for new and DeriveParam for derive_streamer_id, including lifetime 'a for flexible references.
- Refactored streamer structs to use &str for DeriveParam where applicable.

* refactor(orderbook_events): introduce BaseAndRel type alias for DeriveParam #2441

Replaces the raw tuple (&str, &str) with a named type alias for clarity

* docs(event-streaming): add documentation for DeriveStreamerId trait and its associated types #2441

---------

Co-authored-by: BigFish2086 <a8686.ibrahim@gmail.com>
dimxy pushed a commit that referenced this pull request Oct 15, 2025
…eamers (#2489)

* refactor(event-streaming): impl DeriveStreamerId trait for all streamers #2441

- Added DeriveStreamerId trait with InitParam for new and DeriveParam for derive_streamer_id, including lifetime 'a for flexible references.
- Refactored streamer structs to use &str for DeriveParam where applicable.

* refactor(orderbook_events): introduce BaseAndRel type alias for DeriveParam #2441

Replaces the raw tuple (&str, &str) with a named type alias for clarity

* docs(event-streaming): add documentation for DeriveStreamerId trait and its associated types #2441

---------

Co-authored-by: BigFish2086 <a8686.ibrahim@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement: API priority: high Important tasks that need attention soon. status: pending review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants