Skip to content

feat: add JSON schema generator tool#655

Merged
jem-davies merged 5 commits intowarpstreamlabs:mainfrom
mueckinger:main
Jan 19, 2026
Merged

feat: add JSON schema generator tool#655
jem-davies merged 5 commits intowarpstreamlabs:mainfrom
mueckinger:main

Conversation

@mueckinger
Copy link
Copy Markdown
Contributor

Add a new command-line tool json_schema_gen that generates JSON schemas from Bento configuration specifications. This tool converts FieldSpec definitions into proper JSON Schema format, handling various field types, nested objects, arrays, and special component configurations like processors, caches, and rate limits.

The implementation includes:

  • Core JSON Schema structs (JSONSchema, Property)
  • Conversion logic from Bento field types to JSON Schema types
  • Special handling for component arrays (processors, caches, etc.)
  • Default value inclusion in property descriptions
  • Example values preservation
  • Proper type mapping for all supported Bento field types

This enables auto completion and validation capabilities for Bento configurations in almost every IDE.

This resolves #647.

I decided not to include it in the main bento CLI, but create a separate tool as schema creation is usually not required locally. But if you like, we can include this in the CLI later. It's more important to serve the schema via the bento repo.

If you accept this PR please run go run cmd/tools/json_schema_gen/main.go > bento.json and put the resulting JSON Schema in a place of your choice in the bento repo. It is VERY IMPORTANT that this location and file name MUST NOT change!
Once the schema file is in place I will register the Schema for bento.yaml as it is used in the Dockerfile at schemastore.org, so the schema will automatically be imported by the IDEs, once a user creates a bento.yaml file.

Add a new command-line tool `json_schema_gen` that generates JSON schemas from Bento configuration specifications. This tool converts `FieldSpec` definitions into proper JSON Schema format, handling various field types, nested objects, arrays, and special component configurations like processors, caches, and rate limits.

The implementation includes:
- Core JSON Schema structs (`JSONSchema`, `Property`)
- Conversion logic from Bento field types to JSON Schema types
- Special handling for component arrays (processors, caches, etc.)
- Default value inclusion in property descriptions
- Example values preservation
- Proper type mapping for all supported Bento field types

This enables auto completion and validation capabilities for Bento configurations.
The JSON schema generator now correctly identifies and handles array fields by checking the field's Kind property. Previously, array fields with string types were incorrectly mapped to "string" type in the JSON schema. This change ensures that array fields are properly mapped to "array" type in the generated JSON schema.
@aronchick
Copy link
Copy Markdown
Contributor

oooo very very cool

os.Exit(1)
}

fmt.Println(string(output))
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wondering if it would be better to write to a file?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can take a look at this as part of: #658

Signed-off-by: Jem Davies <jemsot@gmail.com>
Signed-off-by: Jem Davies <jemsot@gmail.com>
Signed-off-by: Jem Davies <jemsot@gmail.com>
@jem-davies jem-davies merged commit c2ab7d2 into warpstreamlabs:main Jan 19, 2026
3 checks passed
@jem-davies
Copy link
Copy Markdown
Collaborator

jem-davies commented Jan 19, 2026

I have added the output of the binary to ./resources/schemastore and have opened the PR: SchemaStore/schemastore#5293

🙏 Thanks for this contribution !

I will open up an issue to see about adding this to the makefile & Release process.

Issue #658

jem-davies added a commit to jem-davies/bento that referenced this pull request Feb 2, 2026
* feat: add JSON schema generator tool

Add a new command-line tool `json_schema_gen` that generates JSON schemas from Bento configuration specifications. This tool converts `FieldSpec` definitions into proper JSON Schema format, handling various field types, nested objects, arrays, and special component configurations like processors, caches, and rate limits.

The implementation includes:
- Core JSON Schema structs (`JSONSchema`, `Property`)
- Conversion logic from Bento field types to JSON Schema types
- Special handling for component arrays (processors, caches, etc.)
- Default value inclusion in property descriptions
- Example values preservation
- Proper type mapping for all supported Bento field types

This enables auto completion and validation capabilities for Bento configurations.

* fix: handle array fields in JSON schema generation

The JSON schema generator now correctly identifies and handles array fields by checking the field's Kind property. Previously, array fields with string types were incorrectly mapped to "string" type in the JSON schema. This change ensures that array fields are properly mapped to "array" type in the generated JSON schema.

* fix lint

Signed-off-by: Jem Davies <jemsot@gmail.com>

* fix lints

Signed-off-by: Jem Davies <jemsot@gmail.com>

* add bento.json to ./resources/schemastore

Signed-off-by: Jem Davies <jemsot@gmail.com>

---------

Signed-off-by: Jem Davies <jemsot@gmail.com>
Co-authored-by: Jem Davies <jemsot@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Provide JSON Schema to enable auto completion in IDEs

3 participants