Add ?pretty URL extension. Add /schemas/ids/:id/schema #876

OneCricketeer · 2018-08-26T07:42:26Z

Enhancements over

issue#629: endpoint to return schema literal #686
Fixes ==> Provide API endpoint that returns just the schema #629
Closes ==> Endpoint with schema’s raw JSON #381

Utilize the SchemaRegistryResourceExtension class to inject Jackson's ObjectMapper & PrettyPrinter.

No longer do we have to use CLI pipes into jq / python -m json.tool to format outputs from the CLI!!

Example

Not only do GET's work, but also POSTs. I've went over the using document to update the entire page.

Without ?pretty

$ curl -X GET http://localhost:8081/subjects/LogLine-value/versions/1/schema
{"type":"record","name":"LogLine","namespace":"JavaSessionize.avro","fields":[{"name":"ip","type":"string"},{"name":"timestamp","type":"long"},{"name":"url","type":"string"},{"name":"referrer","type":"string"},{"name":"useragent","type":"string"},{"name":"sessionid","type":["null","int"],"default":null}]}

And with

$ curl -X GET http://localhost:8081/subjects/LogLine-value/versions/1/schema?pretty
{
  "type" : "record",
  "name" : "LogLine",
  "namespace" : "JavaSessionize.avro",
  "fields" : [ {
    "name" : "ip",
    "type" : "string"
  }, {
    "name" : "timestamp",
    "type" : "long"
  }, {
    "name" : "url",
    "type" : "string"
  }, {
    "name" : "referrer",
    "type" : "string"
  }, {
    "name" : "useragent",
    "type" : "string"
  }, {
    "name" : "sessionid",
    "type" : [ "null", "int" ],
    "default" : null
  } ]
}

Details

Added new URL for /schemas/ids/:id/schema, which was missing from issue#629: endpoint to return schema literal #686.
Updated the RestClient methods that returned String type because the other object types are handled by the PrettyQueryExtension class.
Added a UrlParser that for now is used to parse out query parameters.
Updated documentation ==> Closes Document the schema's JSON in a pretty way #847

ghost · 2018-08-26T07:42:27Z

@confluentinc It looks like @Cricket007 just signed our Contributor License Agreement. 👍

Always at your service,

clabot

OneCricketeer · 2018-08-26T07:46:53Z

docs/using.rst

-Checking if a Schema Is Registered Under Subject "Kafka-key"
------------------------------------------------------------
+--------------------------------------------------------------
+Checking if a Schema Is Registered Under Subject "Kafka-value"


The key still had version 1 and all the earlier examples update the value, so I think this was a typo.

OneCricketeer · 2018-08-26T07:47:22Z

docs/api.rst

@@ -439,7 +477,7 @@ The compatibility resource allows the user to test schemas for compatibility aga

 .. http:post:: /compatibility/subjects/(string: subject)/versions/(versionId: version)

-   Test input schema against a particular version of a subject's schema for compatibility. Note that the compatibility level applied for the check is the configured compatibility level for the subject (``http:get:: /config/(string: subject)``). If this subject's compatibility level was never changed, then the global compatibility level applies (``http:get:: /config``).
+   Test input schema against a particular version of a subject's schema for compatibility. Note that the compatibility level applied for the check is the configured compatibility level for the subject (``GET /config/(string: subject)``). If this subject's compatibility level was never changed, then the global compatibility level applies (``GET /config``).


Minor confusing doc fixing.

OneCricketeer · 2018-08-26T07:47:35Z

docs/api.rst


 .. http:post:: /subjects/(string: subject)/versions

   Register a new schema under the specified subject. If successfully registered, this returns the unique identifier of this schema in the registry. The returned identifier should be used to retrieve this schema from the schemas resource and is different from the schema's version which is associated with the subject.
   If the same schema is registered under a different subject, the same identifier will be returned. However, the version of the schema may be different under different subjects.

-   A schema should be compatible with the previously registered schema or schemas (if there are any) as per the configured compatibility level. The configured compatibility level can be obtained by issuing a ``GET http:get:: /config/(string: subject)``. If that returns null, then ``GET http:get:: /config``
+   A schema should be compatible with the previously registered schema or schemas (if there are any) as per the configured compatibility level. The configured compatibility level can be obtained by issuing a ``GET /config/(string: subject)``. If that returns null, then ``GET /config``


Minor confusing doc fixing.

OneCricketeer · 2018-08-26T07:49:34Z

client/src/main/java/io/confluent/kafka/schemaregistry/client/rest/utils/UrlParser.java

+              .map(s -> Arrays.copyOf(s.split("="), 2))
+              .collect(groupingBy(s -> s[0],
+                      mapping(s -> s[1] == null ? "" : s[1], toList())));
+    }


Building over Kafka 2.0+, so Stream API support 👍

OneCricketeer · 2018-08-26T07:50:44Z

client/src/test/java/io/confluent/kafka/schemaregistry/client/rest/utils/UrlParserTest.java

+        // no-op
+      }
+      return null;
+    }).collect(toList());


Building over Kafka 2.0+, so Stream API support 👍

OneCricketeer · 2018-08-29T19:41:36Z

Looking for feedback

Would adding the extension to the other resource list be cleaner?

-    schemaRegistryResourceExtensions =
-        schemaRegistryConfig.getConfiguredInstances(
-            schemaRegistryConfig.definedResourceExtensionConfigName(),
-            SchemaRegistryResourceExtension.class);
-
     config.register(RootResource.class);
     config.register(new ConfigResource(schemaRegistry));
     config.register(new SubjectsResource(schemaRegistry));
@@ -82,16 +76,17 @@
     config.register(new SubjectVersionsResource(schemaRegistry));
     config.register(new CompatibilityResource(schemaRegistry));
 
-    try {
-      new PrettyQueryExtension()
-              .register(config, schemaRegistryConfig, schemaRegistry);
-    } catch (SchemaRegistryException e) {
-      log.error(String.format(
-              "Error registering ?%s query resource extension. Starting up without it",
-              QUERY_PARAM_PRETTY), e);
+    schemaRegistryResourceExtensions = new ArrayList<>();
+    schemaRegistryResourceExtensions.add(new PrettyQueryExtension());
+
+    List<SchemaRegistryResourceExtension> extensions =
+            schemaRegistryConfig.getConfiguredInstances(
+                    schemaRegistryConfig.definedResourceExtensionConfigName(),
+                    SchemaRegistryResourceExtension.class);
+    if (extensions != null) {
+      schemaRegistryResourceExtensions.addAll(extensions);
     }
 
-    if (schemaRegistryResourceExtensions != null) {
     try {
       for (SchemaRegistryResourceExtension
           schemaRegistryResourceExtension : schemaRegistryResourceExtensions) {
@@ -101,7 +96,7 @@
       log.error("Error starting the schema registry", e);
       System.exit(1);
     }
-    }

OneCricketeer · 2018-10-08T21:27:14Z

@mageshn Could I get a review once I get the tests passing again?

OneCricketeer · 2018-10-22T04:09:21Z

client/src/main/java/io/confluent/kafka/schemaregistry/client/rest/entities/SchemaString.java

@@ -47,7 +49,15 @@ public void setSchemaString(String schemaString) {
    this.schemaString = schemaString;
  }

-  public String toJson() throws IOException {
+  public String toJson(boolean pretty) throws IOException {


I don't think I need this

Suggested change

public String toJson(boolean pretty) throws IOException {

public String toJson() throws IOException {

rayokota

Hi @Cricket007 , we're in the process of making Schema Registry pluggable, so that schemas are not necessarily JSON format (such as Protobuf). Since these changes assume JSON schemas, we're going to hold off on merging this PR. We can return to whether a pretty query param makes sense after SR is pluggable. Until then you can try using a custom REST extension to get the behavior that you want. Thanks!

rayokota · 2018-11-09T20:03:31Z

I'm closing this PR for now but feel free to reopen if you have further changes or suggestions. Thanks!

OneCricketeer · 2018-11-09T21:22:18Z

@rayokota It might be possible for me to extend to detect the response header to see if it's actually JSON that needs formatted

rayokota · 2018-11-09T23:55:31Z

@Cricket007 , you can see the initial PR for making SR pluggable here: #927. Probably once this gets merged, we can consider adding a toString(boolean pretty) to ParsedSchema, if we want to continue to explore prettifying the result in a general fashion.

OneCricketeer · 2020-04-11T20:02:57Z

#927. Probably once this gets merged, we can consider adding a toString(boolean pretty) to ParsedSchema

@rayokota - Worth reopening / rebasing this ?

rayokota · 2020-04-13T22:55:41Z

Hi @Cricket007 , there is now a ?format={format} query param for /ids/{id}. The format gets passed to ParsedSchema.formattedString(String format).

If you still want a /ids/{id}/schema?format={format} endpoint, PRs are welcome :)

OneCricketeer commented Aug 26, 2018

View reviewed changes

Add ?pretty URL extension

326670a

OneCricketeer mentioned this pull request Aug 26, 2018

DOCS-709 - Docs fixup #867

Merged

OneCricketeer changed the title ~~Add ?pretty URL extension~~ Add ?pretty URL extension. Add /schemas/ids/:id/schema Sep 11, 2018

mageshn requested review from mageshn and a team October 15, 2018 22:59

OneCricketeer commented Oct 22, 2018

View reviewed changes

mageshn requested review from rayokota and removed request for mageshn November 8, 2018 21:51

rayokota reviewed Nov 9, 2018

View reviewed changes

rayokota closed this Nov 9, 2018

rayokota mentioned this pull request Apr 6, 2020

Endpoint with schema’s raw JSON #381

Closed

OneCricketeer mentioned this pull request Apr 14, 2020

Implement GET /ids/:id/schema #1419

Open

ImFlog mentioned this pull request Jan 3, 2024

Ability to toggle pretty-print of outputted schema files for Avro/JSON formats ImFlog/schema-registry-plugin#158

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ?pretty URL extension. Add /schemas/ids/:id/schema #876

Add ?pretty URL extension. Add /schemas/ids/:id/schema #876

OneCricketeer commented Aug 26, 2018 •

edited

Loading

ghost commented Aug 26, 2018

OneCricketeer Aug 26, 2018

OneCricketeer Aug 26, 2018

OneCricketeer Aug 26, 2018

OneCricketeer Aug 26, 2018

OneCricketeer Aug 26, 2018

OneCricketeer commented Aug 29, 2018 •

edited

Loading

OneCricketeer commented Oct 8, 2018

OneCricketeer Oct 22, 2018

rayokota left a comment

rayokota commented Nov 9, 2018

OneCricketeer commented Nov 9, 2018

rayokota commented Nov 9, 2018

OneCricketeer commented Apr 11, 2020

rayokota commented Apr 13, 2020

	public String toJson(boolean pretty) throws IOException {
	public String toJson() throws IOException {

Add ?pretty URL extension. Add /schemas/ids/:id/schema #876

Add ?pretty URL extension. Add /schemas/ids/:id/schema #876

Conversation

OneCricketeer commented Aug 26, 2018 • edited Loading

Example

Details

ghost commented Aug 26, 2018

OneCricketeer Aug 26, 2018

Choose a reason for hiding this comment

OneCricketeer Aug 26, 2018

Choose a reason for hiding this comment

OneCricketeer Aug 26, 2018

Choose a reason for hiding this comment

OneCricketeer Aug 26, 2018

Choose a reason for hiding this comment

OneCricketeer Aug 26, 2018

Choose a reason for hiding this comment

OneCricketeer commented Aug 29, 2018 • edited Loading

OneCricketeer commented Oct 8, 2018

OneCricketeer Oct 22, 2018

Choose a reason for hiding this comment

rayokota left a comment

Choose a reason for hiding this comment

rayokota commented Nov 9, 2018

OneCricketeer commented Nov 9, 2018

rayokota commented Nov 9, 2018

OneCricketeer commented Apr 11, 2020

rayokota commented Apr 13, 2020

OneCricketeer commented Aug 26, 2018 •

edited

Loading

OneCricketeer commented Aug 29, 2018 •

edited

Loading