Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kucoin: Add subscription templating and various fixes #1579

Merged
merged 18 commits into from
Aug 9, 2024

Conversation

gbjk
Copy link
Collaborator

@gbjk gbjk commented Jul 12, 2024

Kucoin

  • Add subscription templating
  • Support (undocumented) Batching for most websocket endpoints
  • Remove support for deprecated margin funding book endpoint
  • Remove unimplemented market snapshot by currency endpoint
  • Add subscription documentation, replacing in-situ boilerplate docs
  • Use ticker:all for more than 10 pairs, otherwise use specific pairs
  • Renames channel variables for consistency and clarity
  • Add Assets to existing sub configs
  • Consolidates Futures ticker and orderbook sub config into asset.All

Subscription Templates

  • Add BatchSize support
  • Fix subscription Pairs not respected
  • $.AssetPairs only receives relevant assets for the subscription
  • Fixes and tests for separator trimming

Common

  • Add Batch and Sort to common
  • Remove error from pairs.Remove
  • Add common.SortStrings
  • Variadic pairs.Add and pairs.Remove

Stacked Dependencies

Type of change

  • Feature

How has this been tested

  • go test ./... -race
  • golangci-lint run

@gbjk gbjk added the review me This pull request is ready for review label Jul 12, 2024
@gbjk gbjk self-assigned this Jul 12, 2024
Copy link

codecov bot commented Jul 12, 2024

Codecov Report

Attention: Patch coverage is 90.06024% with 33 lines in your changes missing coverage. Please review.

Project coverage is 36.32%. Comparing base (d1e3669) to head (7c3d7fb).
Report is 6 commits behind head on master.

Files Patch % Lines
exchanges/kucoin/kucoin_websocket.go 87.86% 14 Missing and 7 partials ⚠️
exchanges/btcmarkets/btcmarkets_wrapper.go 0.00% 9 Missing ⚠️
common/common.go 87.50% 2 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1579      +/-   ##
==========================================
- Coverage   38.20%   36.32%   -1.89%     
==========================================
  Files         421      422       +1     
  Lines      152345   183014   +30669     
==========================================
+ Hits        58204    66480    +8276     
- Misses      86072   108483   +22411     
+ Partials     8069     8051      -18     
Files Coverage Δ
currency/manager.go 94.01% <100.00%> (-0.96%) ⬇️
currency/pairs.go 96.28% <100.00%> (-0.80%) ⬇️
exchanges/asset/asset.go 93.70% <ø> (+0.25%) ⬆️
exchanges/exchange.go 76.64% <100.00%> (-3.08%) ⬇️
exchanges/kucoin/kucoin.go 14.58% <100.00%> (-1.06%) ⬇️
exchanges/kucoin/kucoin_types.go 60.86% <ø> (-3.95%) ⬇️
exchanges/kucoin/kucoin_wrapper.go 34.51% <100.00%> (-3.20%) ⬇️
exchanges/orderbook/orderbook.go 90.43% <100.00%> (-1.84%) ⬇️
exchanges/subscription/list.go 100.00% <100.00%> (ø)
exchanges/subscription/template.go 100.00% <100.00%> (+2.81%) ⬆️
... and 3 more

... and 386 files with indirect coverage changes

@gbjk gbjk force-pushed the feature/kucoin_sub_templating branch 3 times, most recently from 7754e57 to c969696 Compare July 13, 2024 09:27
@gloriousCode gloriousCode added the tests require fixing This is to denote that the PR has some tests that require fixing before the review can be completed label Jul 14, 2024
@gbjk gbjk force-pushed the feature/kucoin_sub_templating branch 3 times, most recently from f45a386 to 7718f2b Compare July 16, 2024 10:18
Copy link
Collaborator

@shazbert shazbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, nothing really jumps out, just some things I noticed and there are some ci errors that need to be resolved. 🔥

common/common.go Outdated Show resolved Hide resolved
common/common_test.go Show resolved Hide resolved
currency/manager.go Outdated Show resolved Hide resolved
currency/pairs.go Show resolved Hide resolved
currency/pairs.go Outdated Show resolved Hide resolved
exchanges/exchange.go Show resolved Hide resolved
exchanges/kucoin/kucoin_test.go Outdated Show resolved Hide resolved
exchanges/kucoin/kucoin_test.go Outdated Show resolved Hide resolved
@shazbert shazbert added reconstructing Based on PR feedback, this is currently being reworked and is not to be merged and removed review me This pull request is ready for review labels Jul 16, 2024
@gbjk gbjk force-pushed the feature/kucoin_sub_templating branch 3 times, most recently from 52f9f16 to bd970c0 Compare July 18, 2024 05:24
@gbjk
Copy link
Collaborator Author

gbjk commented Jul 19, 2024

Tests fixed. Can't and shouldn't cache a compiled template closuring anything on Kucoin itself.
Not a hot enough path to justify caching at all really though.

@gbjk gbjk requested a review from shazbert July 19, 2024 12:06
@gbjk gbjk added review me This pull request is ready for review and removed reconstructing Based on PR feedback, this is currently being reworked and is not to be merged tests require fixing This is to denote that the PR has some tests that require fixing before the review can be completed labels Jul 19, 2024
@gbjk gbjk mentioned this pull request Jul 21, 2024
2 tasks
Copy link
Collaborator

@shazbert shazbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great just some minor niggles, tested websocket.

common/common.go Outdated Show resolved Hide resolved
common/common.go Outdated Show resolved Hide resolved
common/common.go Outdated Show resolved Hide resolved
exchanges/kucoin/kucoin_types.go Show resolved Hide resolved
@gbjk gbjk requested a review from shazbert July 22, 2024 01:10
@gbjk gbjk force-pushed the feature/kucoin_sub_templating branch from 9c56aab to d3d2fa5 Compare July 22, 2024 01:33
Copy link
Collaborator

@shazbert shazbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tACK! Nice work.

@shazbert shazbert added the szrc shazbert review complete label Jul 23, 2024
@gbjk gbjk requested a review from gloriousCode August 7, 2024 05:49
Copy link
Collaborator

@gloriousCode gloriousCode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tACK!

Copy link
Collaborator

@shazbert shazbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ch ack

gbjk added 16 commits August 8, 2024 08:41
Now only the assetPairs relevant to the subscription are in the context
We want to trim before checking for "AssetSeparator vs All" because a
template should be allowed to reuse a range template and generate just one trailing AssetSeparator
whilst using a specific Asset
Turns out that contary to the documentation, kucoin supports batching of
all symbols and currencies
This reduces the complexity of error checking to just be "do we get the
correct numbers".

Fixes Asset.All with only one asset erroring on xpandPairs, because we
trimmed the only asset separator, and then errored that we're not
xpanding Assets and the asset on the sub is asset.All

This use-case conflicted with commit 6bbd546, which required:
```
Subscriptions: Trim AssetSeparator early

We want to trim before checking for "AssetSeparator vs All" because a
template should be allowed to reuse a range template and generate just one trailing AssetSeparator
whilst using a specific Asset
```

Now we set up the assets earlier, and we remove the check for xpandAssets, since the number of asset lines matching is all that matters.

I've removed the asset tests for this, but they were correctly erroring
on the number of asset lines instead.

Everything hits coverage, as well.
@gbjk gbjk force-pushed the feature/kucoin_sub_templating branch from 01a9f9f to 82c5a54 Compare August 8, 2024 01:49
@gbjk
Copy link
Collaborator Author

gbjk commented Aug 8, 2024

Squashed down for final review 😄
Zero changes in the squash though.

@gbjk gbjk mentioned this pull request Aug 8, 2024
2 tasks
@thrasher- thrasher- changed the title Kucoin: Add Sub Templating and various fixes Kucoin: Add subscription templating and various fixes Aug 9, 2024
Copy link
Collaborator

@thrasher- thrasher- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one minor nit on the introduction of the new Batch helper. It makes a pre-existing helper func obsolete.

Tests, functionality for Kucoin and config uprade all worked well. Nice stuff! Appreciate the strings.HasPrefix improvement as well on the websocket handling topic info!

common/common.go Show resolved Hide resolved
@gbjk gbjk requested a review from thrasher- August 9, 2024 01:59
Copy link
Collaborator

@thrasher- thrasher- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making those changes

@thrasher- thrasher- merged commit b41fe27 into thrasher-corp:master Aug 9, 2024
8 of 13 checks passed
@gbjk gbjk deleted the feature/kucoin_sub_templating branch August 9, 2024 04:08
@gbjk gbjk mentioned this pull request Sep 3, 2024
15 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
high priority review me This pull request is ready for review szrc shazbert review complete
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants