Add custom application model for EventHubs emulator #7005

sebastienros · 2024-12-31T01:06:47Z

Description

This PR introduces a custom model to configure the emulator and the cloud service.
Counterpart of #6737 for Event Hubs emulator.

Checklist

Is this feature complete?
- Yes. Ready to ship.
- No. Follow-up changes expected.
Are you including unit tests for the changes and scenario tests if relevant?
- Yes
- No
Did you add public API?
- Yes
  - If yes, did you have an API Review for it?
    - Yes
    - No
  - Did you add <remarks /> and <code /> elements on your triple slash comments?
    - Yes
    - No
- No
Does the change make any security assumptions or guarantees?
- Yes
  - If yes, have you done a threat model and had a security review?
    - Yes
    - No
- No
Does the change require an update in our Aspire docs?
- Yes
  - Is this introducing a breaking change?
    - Yes
      - Link to aspire-docs issue (please use this breaking-change template):
    - No
      - Link to aspire-docs issue (please use this doc-idea template):
- No

Microsoft Reviewers: Open in CodeFlow

davidfowl · 2025-01-04T00:13:21Z

src/Aspire.Hosting.Azure.EventHubs/AzureEventHubsExtensions.cs

@@ -148,8 +148,7 @@ public static IResourceBuilder<AzureEventHubsResource> RunAsEmulator(this IResou
            return builder;
        }

-        var configHostFile = Path.Combine(Path.GetTempPath(), "AspireEventHubsEmulator", Path.GetRandomFileName() + ".json");
-        Directory.CreateDirectory(Path.GetDirectoryName(configHostFile)!);
+        var configHostFile = Path.Combine(Directory.CreateTempSubdirectory("AspireEventHubsEmulator").FullName, "Config.json");


@eerhardt Longer term we should have a service for doing this that gets cleaned up on exit. @karolz-ms maybe something DCP can provide since it controls the clean up.

Definitely. For example, I can imagine us having the capability for Containers where you can say "here is a blob of JSON that I need to appear as file inside the container, under this path". Sort of like Kubernetes ConfigMaps.

Opened #7029 to track.

eerhardt · 2025-01-06T16:30:56Z

This will resolve #5561.

eerhardt · 2025-01-06T16:16:21Z

src/Aspire.Hosting.Azure.EventHubs/ConfigJsonAnnotation.cs

+namespace Aspire.Hosting.Azure.EventHubs;
+
+/// <summary>
+/// Represents an annotation for updating the JSON content of a mounted document.


What does "a mounted document" mean?

The Config.json file (json document) that is mounted to the container using ContainerMountType.BindMount.

src/Aspire.Hosting.Azure.EventHubs/EventHubConsumerGroup.cs

src/Aspire.Hosting.Azure.EventHubs/EventHub.cs

src/Aspire.Hosting.Azure.EventHubs/AzureEventHubsExtensions.cs

eerhardt · 2025-01-06T17:00:29Z

src/Aspire.Hosting.Azure.EventHubs/AzureEventHubsExtensions.cs

@@ -148,8 +148,7 @@ public static IResourceBuilder<AzureEventHubsResource> RunAsEmulator(this IResou
            return builder;
        }

-        var configHostFile = Path.Combine(Path.GetTempPath(), "AspireEventHubsEmulator", Path.GetRandomFileName() + ".json");
-        Directory.CreateDirectory(Path.GetDirectoryName(configHostFile)!);
+        var configHostFile = Path.Combine(Directory.CreateTempSubdirectory("AspireEventHubsEmulator").FullName, "Config.json");


Opened #7029 to track.

src/Aspire.Hosting.Azure.EventHubs/AzureEventHubsExtensions.cs

eerhardt · 2025-01-06T17:03:36Z

src/Aspire.Hosting.Azure.EventHubs/AzureEventHubsExtensions.cs

-
-                    writer.WriteStartObject();                  //           {
-                    writer.WriteString("Name", hub);            //             "Name": "hub",
-                    writer.WriteString("PartitionCount", "2");  //             "PartitionCount": "2",


We used to have "2" as the default, but hub.WriteJsonObjectProperties now writes "1" by default.

Is this intentional?

AFAIR it was 2 because of the sample they provided. But since the schema states [1-32] I decided to use 1 when the value is not set explicitly.

eerhardt · 2025-01-06T17:07:24Z

src/Aspire.Hosting.Azure.EventHubs/AzureEventHubsExtensions.cs

+    /// <param name="builder">The builder for the <see cref="AzureEventHubsEmulatorResource"/>.</param>
+    /// <param name="path">Path to the file on the AppHost where the emulator configuration is located.</param>
+    /// <returns>A reference to the <see cref="IResourceBuilder{T}"/>.</returns>
+    public static IResourceBuilder<AzureEventHubsEmulatorResource> WithConfigurationFile(this IResourceBuilder<AzureEventHubsEmulatorResource> builder, string path)


Do we have a test for this method?

It was also missing in SB, so I added it. At the same time I tried to simplify the implementation. However I intentionally didn't add support for a custom file provided for config, AND configuration hooks. I would assume if you provide a file, you could have updated it yourself. Hooks are useful when the emulator is generating a file from the resource model and you can't update some section from this model.

eerhardt · 2025-01-09T18:03:22Z

tests/Aspire.Hosting.Azure.Tests/AzureServiceBusExtensionsTests.cs


        using var builder = TestDistributedApplicationBuilder.Create().WithTestAndResourceLogging(output);
        var serviceBus = builder.AddAzureServiceBus("servicebusns")
            .RunAsEmulator()
-            .WithQueue("queue1");
+            .WithQueue("queue123");


Why this name change?

tests/Aspire.Hosting.Azure.Tests/AzureServiceBusExtensionsTests.cs

tests/Aspire.Hosting.Azure.Tests/AzureEventHubsExtensionsTests.cs

src/Aspire.Hosting.Azure.EventHubs/AzureEventHubsExtensions.cs

eerhardt

LGTM once the comments are addressed.

src/Aspire.Hosting.Azure.ServiceBus/AzureServiceBusExtensions.cs

sebastienros added 2 commits December 30, 2024 16:55

Add custom application model for EventHubs emulator

ebb6645

Update doc

c2c3424

sebastienros requested review from radical and eerhardt as code owners December 31, 2024 01:06

sebastienros added 2 commits January 2, 2025 16:57

Merge branch 'main' into sebros/eventhubsmodel

f3aded3

Use CreateTempSubdirectory

fd36a41

davidfowl reviewed Jan 4, 2025

View reviewed changes

eerhardt reviewed Jan 6, 2025

View reviewed changes

sebastienros added 2 commits January 6, 2025 12:31

Merge branch 'main' into sebros/eventhubsmodel

77fbc9c

Apply suggestions

00a5fd1

build-analysis bot mentioned this pull request Jan 6, 2025

Some tests timing out on helix #5247

Open

sebastienros added 4 commits January 6, 2025 18:15

Increase SB timeouts

a667a7a

Improve config.json implementation

dbe2c26

Update docs

1ff9b22

Increase timeout for flaky test

283b511

This was referenced Jan 8, 2025

The active test run was aborted. Reason: Test host process crashed dotnet/dnceng#451

Open

The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008

Open

sebastienros added 2 commits January 8, 2025 13:25

Fix configuration

71238be

Fix build

454fb7c

eerhardt reviewed Jan 9, 2025

View reviewed changes

tests/Aspire.Hosting.Azure.Tests/AzureServiceBusExtensionsTests.cs Outdated Show resolved Hide resolved

eerhardt reviewed Jan 9, 2025

View reviewed changes

tests/Aspire.Hosting.Azure.Tests/AzureServiceBusExtensionsTests.cs Outdated Show resolved Hide resolved

eerhardt reviewed Jan 9, 2025

View reviewed changes

tests/Aspire.Hosting.Azure.Tests/AzureEventHubsExtensionsTests.cs Outdated Show resolved Hide resolved

eerhardt reviewed Jan 9, 2025

View reviewed changes

src/Aspire.Hosting.Azure.EventHubs/AzureEventHubsExtensions.cs Outdated Show resolved Hide resolved

eerhardt approved these changes Jan 9, 2025

View reviewed changes

Improve tests

86ad563

eerhardt reviewed Jan 9, 2025

View reviewed changes

src/Aspire.Hosting.Azure.ServiceBus/AzureServiceBusExtensions.cs Outdated Show resolved Hide resolved

Filter events

c8edd15

eerhardt approved these changes Jan 9, 2025

View reviewed changes

sebastienros merged commit cbf34f0 into main Jan 10, 2025
9 checks passed

sebastienros deleted the sebros/eventhubsmodel branch January 10, 2025 00:32

This was referenced Jan 13, 2025

Aspire.Hosting.Azure.EventHubs should support configuring Consumer Groups #5561

Closed

Add Azure storage blob containers #5167

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add custom application model for EventHubs emulator #7005

Add custom application model for EventHubs emulator #7005

sebastienros commented Dec 31, 2024 •

edited

Loading

davidfowl Jan 4, 2025

karolz-ms Jan 4, 2025

eerhardt Jan 6, 2025

eerhardt commented Jan 6, 2025

eerhardt Jan 6, 2025

sebastienros Jan 6, 2025

eerhardt Jan 6, 2025

eerhardt Jan 6, 2025

sebastienros Jan 6, 2025

eerhardt Jan 6, 2025

sebastienros Jan 7, 2025

eerhardt Jan 9, 2025

eerhardt left a comment

Add custom application model for EventHubs emulator #7005

Add custom application model for EventHubs emulator #7005

Conversation

sebastienros commented Dec 31, 2024 • edited Loading

Description

Checklist

Microsoft Reviewers: Open in CodeFlow

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eerhardt commented Jan 6, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

eerhardt left a comment

Choose a reason for hiding this comment

sebastienros commented Dec 31, 2024 •

edited

Loading