-
Notifications
You must be signed in to change notification settings - Fork 1.4k
[Schema Registry Avro] Post release #20796
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
deyaaeldeen
merged 1 commit into
Azure:main
from
deyaaeldeen:schemaregistryavro/post-release
Mar 11, 2022
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
80 changes: 80 additions & 0 deletions
80
...chemaregistry/schema-registry-avro/samples/v1-beta/javascript/schemaRegistryAvroSample.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,80 @@ | ||
| // Copyright (c) Microsoft Corporation. | ||
| // Licensed under the MIT License. | ||
|
|
||
| /** | ||
| * @summary Demonstrates the use of AvroSerializer to create messages with avro-serialized payload using schema from Schema Registry. | ||
| */ | ||
|
|
||
| const { DefaultAzureCredential } = require("@azure/identity"); | ||
| const { SchemaRegistryClient } = require("@azure/schema-registry"); | ||
| const { AvroSerializer } = require("@azure/schema-registry-avro"); | ||
|
|
||
| // Load the .env file if it exists | ||
| require("dotenv").config(); | ||
|
|
||
| // The fully qualified namespace for schema registry | ||
| const schemaRegistryFullyQualifiedNamespace = | ||
| process.env["SCHEMA_REGISTRY_ENDPOINT"] || "<endpoint>"; | ||
|
|
||
| // The schema group to use for schema registeration or lookup | ||
| const groupName = process.env["SCHEMA_REGISTRY_GROUP"] || "AzureSdkSampleGroup"; | ||
|
|
||
| // Sample Avro Schema for user with first and last names | ||
| const schemaObject = { | ||
| type: "record", | ||
| name: "User", | ||
| namespace: "com.azure.schemaregistry.samples", | ||
| fields: [ | ||
| { | ||
| name: "firstName", | ||
| type: "string", | ||
| }, | ||
| { | ||
| name: "lastName", | ||
| type: "string", | ||
| }, | ||
| ], | ||
| }; | ||
|
|
||
| const schema = JSON.stringify(schemaObject); | ||
|
|
||
| // Description of the schema for registration | ||
| const schemaDescription = { | ||
| name: `${schemaObject.namespace}.${schemaObject.name}`, | ||
| groupName, | ||
| format: "Avro", | ||
| definition: schema, | ||
| }; | ||
|
|
||
| async function main() { | ||
| // Create a new client | ||
| const client = new SchemaRegistryClient( | ||
| schemaRegistryFullyQualifiedNamespace, | ||
| new DefaultAzureCredential() | ||
| ); | ||
|
|
||
| // Register the schema. This would generally have been done somewhere else. | ||
| // You can also skip this step and let `serializeMessageData` automatically register | ||
| // schemas using autoRegisterSchemas=true, but that is NOT recommended in production. | ||
| await client.registerSchema(schemaDescription); | ||
|
|
||
| // Create a new serializer backed by the client | ||
| const serializer = new AvroSerializer(client, { groupName }); | ||
|
|
||
| // serialize an object that matches the schema and put it in a message | ||
| const value = { firstName: "Jane", lastName: "Doe" }; | ||
| const message = await serializer.serializeMessageData(value, schema); | ||
| console.log("Created message:"); | ||
| console.log(JSON.stringify(message)); | ||
|
|
||
| // deserialize the message back to an object | ||
| const deserializedObject = await serializer.deserializeMessageData(message); | ||
| console.log("Deserialized object:"); | ||
| console.log(JSON.stringify(deserializedObject)); | ||
| } | ||
|
|
||
| main().catch((err) => { | ||
| console.error("The sample encountered an error:", err); | ||
| }); | ||
|
|
||
| module.exports = { main }; | ||
102 changes: 102 additions & 0 deletions
102
...ry/schema-registry-avro/samples/v1-beta/javascript/withEventHubsBufferedProducerClient.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,102 @@ | ||
| // Copyright (c) Microsoft Corporation. | ||
| // Licensed under the MIT License. | ||
|
|
||
| /** | ||
| * @summary Demonstrates the use of AvroSerializer to create messages with avro-serialized payload using schema from Schema Registry and send them to an Event Hub using the EventHub Buffered Producer Client. | ||
| */ | ||
|
|
||
| const { DefaultAzureCredential } = require("@azure/identity"); | ||
| const { SchemaRegistryClient } = require("@azure/schema-registry"); | ||
| const { AvroSerializer } = require("@azure/schema-registry-avro"); | ||
| const { EventHubBufferedProducerClient, createEventDataAdapter } = require("@azure/event-hubs"); | ||
|
|
||
| // Load the .env file if it exists | ||
| require("dotenv").config(); | ||
|
|
||
| // The fully qualified namespace for schema registry | ||
| const schemaRegistryFullyQualifiedNamespace = | ||
| process.env["SCHEMA_REGISTRY_ENDPOINT"] || "<endpoint>"; | ||
|
|
||
| // The schema group to use for schema registeration or lookup | ||
| const groupName = process.env["SCHEMA_REGISTRY_GROUP"] || "AzureSdkSampleGroup"; | ||
|
|
||
| // The connection string for Event Hubs | ||
| const eventHubsConnectionString = process.env["EVENTHUB_CONNECTION_STRING"] || ""; | ||
|
|
||
| // Sample Avro Schema for user with first and last names | ||
| const schemaObject = { | ||
| type: "record", | ||
| name: "User", | ||
| namespace: "com.azure.schemaregistry.samples", | ||
| fields: [ | ||
| { | ||
| name: "firstName", | ||
| type: "string", | ||
| }, | ||
| { | ||
| name: "lastName", | ||
| type: "string", | ||
| }, | ||
| ], | ||
| }; | ||
|
|
||
| const schema = JSON.stringify(schemaObject); | ||
|
|
||
| // Description of the schema for registration | ||
| const schemaDescription = { | ||
| name: `${schemaObject.namespace}.${schemaObject.name}`, | ||
| groupName, | ||
| format: "Avro", | ||
| definition: schema, | ||
| }; | ||
|
|
||
| async function handleError() { | ||
| console.log("An error occured when sending a message"); | ||
| } | ||
|
|
||
| async function main() { | ||
| // Create a new client | ||
| const schemaRegistryClient = new SchemaRegistryClient( | ||
| schemaRegistryFullyQualifiedNamespace, | ||
| new DefaultAzureCredential() | ||
| ); | ||
|
|
||
| // Register the schema. This would generally have been done somewhere else. | ||
| // You can also skip this step and let `serializeMessageData` automatically register | ||
| // schemas using autoRegisterSchemas=true, but that is NOT recommended in production. | ||
| await schemaRegistryClient.registerSchema(schemaDescription); | ||
|
|
||
| // Create a new serializer backed by the client | ||
| const serializer = new AvroSerializer(schemaRegistryClient, { | ||
| groupName, | ||
| messageAdapter: createEventDataAdapter(), | ||
| }); | ||
|
|
||
| const eventHubsBufferedProducerClient = new EventHubBufferedProducerClient( | ||
| eventHubsConnectionString, | ||
| { | ||
| onSendEventsErrorHandler: handleError, | ||
| } | ||
| ); | ||
|
|
||
| // serialize an object that matches the schema | ||
| const value = { firstName: "Jane", lastName: "Doe" }; | ||
| const message = await serializer.serializeMessageData(value, schema); | ||
| console.log("Created message:"); | ||
| console.log(message); | ||
|
|
||
| await eventHubsBufferedProducerClient.enqueueEvent(message); | ||
| console.log(`Message was added to the queue and is about to be sent`); | ||
|
|
||
| // Wait for a bit before cleaning up the sample | ||
| setTimeout(async () => { | ||
| await eventHubsBufferedProducerClient.close({ flush: true }); | ||
| console.log(`Exiting sample`); | ||
| }, 30 * 1000); | ||
| } | ||
|
|
||
| main().catch((err) => { | ||
| console.error("The sample encountered an error:", err); | ||
| }); | ||
|
|
||
| module.exports = { main }; |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I must have missed this in the PR to switch to using Serializer, but we actually renamed this to just serialize in .NET. We also renamed MessageWithMetadata to BinaryContent. Is there a reason not to do this same change in JS?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok I will talk to @bterlson about those renames.