-
Notifications
You must be signed in to change notification settings - Fork 17
feat: add StreamedListObjects support #156
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
30 commits
Select commit
Hold shift + click to select a range
84992d8
feat: add StreamedListObjects support
daniel-jonathan bead1e7
feat: add StreamedListObjects support
daniel-jonathan 5ce96b4
refactor: use using statement for cancellation registration
daniel-jonathan 35fb401
fix: improve streaming cancellation and test reliability
daniel-jonathan 71df2e8
Merge branch 'main' into feat/streamed_list_objects
daniel-jonathan eb03120
test: use FgaConstants for test values instead of hardcoded strings
daniel-jonathan 1064700
docs: simplify StreamedListObjects example and enhance README
daniel-jonathan de2ffb9
docs: add StreamedListObjects section to main README
daniel-jonathan a66c34b
test: add custom headers and rate limit tests for StreamedListObjects
daniel-jonathan 4a2ec7a
refactor: use using statement for CancellationTokenSource
daniel-jonathan 0b61657
fix: address CodeQL and resource disposal issues
daniel-jonathan d5aa28b
fix: address CodeQL and resource ContainsKey issue Addresses CodeQL, …
daniel-jonathan b28428a
refactor: add using statements for all IDisposable resources in tests
daniel-jonathan 2e4e35e
refactor: remove useless assignments in tests
daniel-jonathan bd0aa89
refactor: Updated commenting and Assertion messaging based on CodeRab…
daniel-jonathan 520e78e
refactor: Fixed dispose issue based on CodeQL feedback
daniel-jonathan 2f4d426
refactor: improve example to demonstrate computed relations
daniel-jonathan 1becd72
fix: batch tuple writes to respect 100-tuple limit
daniel-jonathan aa40ebd
fix: prevent logging sensitive data in error handler
daniel-jonathan 826d8e5
refactor: Updated the example README based on review feedback
daniel-jonathan db885db
test: add comprehensive tests for partial NDJSON streaming handling
daniel-jonathan bb2bea8
Merge branch 'main' into feat/streamed_list_objects
daniel-jonathan 99cca29
feat: add StreamedListObjects API support
daniel-jonathan 9ca91cd
perf: optimize dictionary lookups in model Equals methods
daniel-jonathan 4c98c13
perf: optimize model Equals methods for CodeQL compliance
daniel-jonathan 67d019e
Merge branch 'main' into feat/streamed_list_objects
SoulPancake 1c8b560
Merge branch 'main' into feat/streamed_list_objects
SoulPancake a50dce3
Merge branch 'main' into feat/streamed_list_objects
SoulPancake c01c35f
Merge branch 'main' into feat/streamed_list_objects
SoulPancake 677c4d5
fix: remove dupl package reference
SoulPancake File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| # OpenFga.Sdk.Model.StreamResultOfStreamedListObjectsResponse | ||
|
|
||
| ## Properties | ||
|
|
||
| Name | Type | Description | Notes | ||
| ------------ | ------------- | ------------- | ------------- | ||
| **Result** | [**StreamedListObjectsResponse**](StreamedListObjectsResponse.md) | | [optional] | ||
| **Error** | [**Status**](Status.md) | | [optional] | ||
|
|
||
| [[Back to Model list]](../README.md#models) [[Back to API list]](../README.md#api-endpoints) [[Back to README]](../README.md) | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| # OpenFga.Sdk.Model.StreamedListObjectsResponse | ||
| The response for a StreamedListObjects RPC. | ||
|
|
||
| ## Properties | ||
|
|
||
| Name | Type | Description | Notes | ||
| ------------ | ------------- | ------------- | ------------- | ||
| **Object** | **string** | | | ||
|
|
||
| [[Back to Model list]](../README.md#models) [[Back to API list]](../README.md#api-endpoints) [[Back to README]](../README.md) | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,119 @@ | ||
| # Streamed List Objects Example | ||
|
|
||
| Demonstrates using `StreamedListObjects` to retrieve objects via the streaming API in the .NET SDK. | ||
|
|
||
| ## What is StreamedListObjects? | ||
|
|
||
| The Streamed ListObjects API is very similar to the ListObjects API, with two key differences: | ||
|
|
||
| 1. **Streaming Results**: Instead of collecting all objects before returning a response, it streams them to the client as they are collected. | ||
| 2. **No Pagination Limit**: Returns all results without the 1000-object limit of the standard ListObjects API. | ||
|
|
||
| This makes it ideal for scenarios where you need to retrieve large numbers of objects, especially when querying computed relations. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - .NET 6.0 or higher | ||
| - OpenFGA server running on `http://localhost:8080` (or set `FGA_API_URL`) | ||
|
daniel-jonathan marked this conversation as resolved.
|
||
|
|
||
| ## Running | ||
|
|
||
| ```bash | ||
| # From the SDK root directory, build the SDK first | ||
| dotnet build src/OpenFga.Sdk/OpenFga.Sdk.csproj | ||
|
|
||
| # Then run the example | ||
| cd example/StreamedListObjectsExample | ||
| dotnet run | ||
| ``` | ||
|
|
||
| ## What it does | ||
|
|
||
| - Creates a temporary store | ||
| - Writes an authorization model with **computed relations** | ||
| - Adds 2000 tuples (1000 owners + 1000 viewers) | ||
| - Queries the **computed `can_read` relation** via `StreamedListObjects` | ||
| - Shows all 2000 results (demonstrating computed relations) | ||
| - Shows progress (first 3 objects and every 500th) | ||
| - Cleans up the store | ||
|
|
||
| ## Authorization Model | ||
|
|
||
| The example demonstrates OpenFGA's **computed relations**: | ||
|
|
||
| ``` | ||
| type user | ||
|
|
||
| type document | ||
| relations | ||
| define owner: [user] | ||
| define viewer: [user] | ||
| define can_read: owner or viewer | ||
| ``` | ||
|
|
||
| **Why this matters:** | ||
| - We write tuples to `owner` and `viewer` (base permissions) | ||
| - We query `can_read` (computed from owner OR viewer) | ||
|
|
||
| **Example flow:** | ||
| 1. Write: `user:anne owner document:1-1000` | ||
| 2. Write: `user:anne viewer document:1001-2000` | ||
| 3. Query: `StreamedListObjects(user:anne, relation:can_read, type:document)` | ||
| 4. Result: All 2000 documents (because `can_read = owner OR viewer`) | ||
|
|
||
| ## Key Features Demonstrated | ||
|
|
||
| ### IAsyncEnumerable Pattern | ||
|
|
||
| The `StreamedListObjects` method returns `IAsyncEnumerable<StreamedListObjectsResponse>`, which is the idiomatic .NET way to handle streaming data: | ||
|
|
||
| ```csharp | ||
| await foreach (var response in fgaClient.StreamedListObjects(request)) { | ||
| Console.WriteLine($"Received: {response.Object}"); | ||
| } | ||
| ``` | ||
|
|
||
| ### Early Break and Cleanup | ||
|
|
||
| The streaming implementation properly handles early termination: | ||
|
|
||
| ```csharp | ||
| await foreach (var response in fgaClient.StreamedListObjects(request)) { | ||
| Console.WriteLine(response.Object); | ||
| if (someCondition) { | ||
| break; // Stream is automatically cleaned up | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### Cancellation Support | ||
|
|
||
| Full support for `CancellationToken`: | ||
|
|
||
| ```csharp | ||
| using var cts = new CancellationTokenSource(); | ||
| cts.CancelAfter(TimeSpan.FromSeconds(5)); | ||
|
|
||
| try { | ||
| await foreach (var response in fgaClient.StreamedListObjects(request, options, cts.Token)) { | ||
| Console.WriteLine(response.Object); | ||
| } | ||
| } | ||
| catch (OperationCanceledException) { | ||
| Console.WriteLine("Operation cancelled"); | ||
| } | ||
| ``` | ||
|
|
||
| ## Benefits Over ListObjects | ||
|
|
||
| - **No Pagination**: Retrieve all objects in a single streaming request | ||
| - **Lower Memory**: Objects are processed as they arrive, not held in memory | ||
| - **Early Termination**: Can stop streaming at any point without wasting resources | ||
| - **Better for Large Results**: Ideal when expecting hundreds or thousands of objects | ||
|
|
||
| ## Performance Considerations | ||
|
|
||
| - Streaming starts immediately - no need to wait for all results | ||
| - HTTP connection remains open during streaming | ||
| - Properly handles cleanup if consumer stops early | ||
| - Supports all the same options as `ListObjects` (consistency, contextual tuples, etc.) | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.