Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC-511 Schema CRD #848

Merged
merged 24 commits into from
Nov 22, 2024
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
2caf0c6
Update config for new beta
Deflaimun Oct 2, 2024
c3ffdd4
update with rpk 24.3.1-rc1 (#819)
Deflaimun Oct 21, 2024
58e0bff
DOC-502 License enforcement updates (#813)
JakeSCahill Oct 22, 2024
121d5f4
Properties 24 3 (#822)
Deflaimun Oct 22, 2024
19c0ec1
Add User resource docs (#773)
JakeSCahill Oct 23, 2024
fce336b
Leader pinning (#809)
kbatuigas Oct 28, 2024
3e93983
DOC-287 Mountable TS topics (#725)
kbatuigas Oct 30, 2024
03d3e60
Update with latest rpk commands from v0.0.0-20241104git4a0f859 (#835)
Deflaimun Nov 4, 2024
41bfe40
what's new in 24.3 beta (#811)
micheleRP Nov 4, 2024
afba214
Force-update fallback 24.3-rc2
Deflaimun Nov 4, 2024
488c9f6
Michele rp patch 1 (#837)
micheleRP Nov 6, 2024
67c90f5
DOC-470 Debug bundle in Redpanda Console (#825)
JakeSCahill Nov 12, 2024
41cece9
First draft
JakeSCahill Nov 11, 2024
64e0eb8
Add schema examples
JakeSCahill Nov 12, 2024
3581c99
Fix syntax
JakeSCahill Nov 12, 2024
35f107a
Fix indentation
JakeSCahill Nov 12, 2024
16e9cda
add Tombstone property (#847)
Deflaimun Nov 12, 2024
58ded83
Tombstone retention (#829)
kbatuigas Nov 12, 2024
621f2ec
Apply suggestions from code review
JakeSCahill Nov 15, 2024
e6dd9c9
Apply suggestions from code review
JakeSCahill Nov 15, 2024
c39dc82
Merge branch 'v-WIP/24.3' into DOC-511
JakeSCahill Nov 15, 2024
8c26d11
Apply suggestions from code review
JakeSCahill Nov 15, 2024
7fa424b
Merge branch 'v-WIP/24.3' into DOC-511
JakeSCahill Nov 19, 2024
99c62c6
Apply suggestions from code review
JakeSCahill Nov 22, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -167,8 +167,11 @@
*** xref:manage:topic-recovery.adoc[Topic Recovery]
*** xref:manage:whole-cluster-restore.adoc[Whole Cluster Restore]
** xref:manage:schema-reg/index.adoc[Schema Registry]
*** xref:manage:schema-reg/schema-reg-overview.adoc[]
*** xref:manage:schema-reg/schema-reg-api.adoc[]
*** xref:manage:schema-reg/schema-reg-overview.adoc[Overview]
*** xref:manage:schema-reg/manage-schema-reg.adoc[]
**** xref:manage:schema-reg/schema-reg-api.adoc[API]
**** xref:console:ui/schema-reg.adoc[Redpanda Console]
**** xref:manage:kubernetes/k-schema-controller.adoc[Kubernetes]
*** xref:manage:schema-reg/schema-id-validation.adoc[]
*** xref:console:ui/schema-reg.adoc[Manage in Redpanda Console]
** xref:manage:console/index.adoc[Redpanda Console]
Expand Down
106 changes: 106 additions & 0 deletions modules/manage/examples/kubernetes/schema-crds.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
@cluster:basic
Feature: Schema CRDs
Background: Cluster available
Given cluster "basic" is available

@skip:gke @skip:aks @skip:eks
Scenario: Manage customer profile schema (Avro)
Given there is no schema "customer-profile" in cluster "basic"
When I apply Kubernetes manifest:
"""
# tag::customer-profile-avro-schema-manifest[]
# This manifest creates an Avro schema named "customer-profile" in the "basic" cluster.
# The schema defines a record with fields for customer ID, name, and age.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Schema
metadata:
name: customer-profile
spec:
cluster:
clusterRef:
name: basic
schemaType: avro
compatibilityLevel: Backward
text: |
{
"type": "record",
"name": "CustomerProfile",
"fields": [
{ "type": "string", "name": "customer_id" },
{ "type": "string", "name": "name" },
{ "type": "int", "name": "age" }
]
}
# end::customer-profile-avro-schema-manifest[]
"""
And schema "customer-profile" is successfully synced
Then I should be able to check compatibility against "customer-profile" in cluster "basic"

@skip:gke @skip:aks @skip:eks
Scenario: Manage product catalog schema (Protobuf)
Given there is no schema "product-catalog" in cluster "basic"
When I apply Kubernetes manifest:
"""
# tag::product-catalog-protobuf-schema-manifest[]
# This manifest creates a Protobuf schema named "product-catalog" in the "basic" cluster.
# The schema defines a message "Product" with fields for product ID, name, price, and category.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Schema
metadata:
name: product-catalog
spec:
cluster:
clusterRef:
name: basic
schemaType: protobuf
compatibilityLevel: Backward
text: |
syntax = "proto3";

message Product {
int32 product_id = 1;
string product_name = 2;
double price = 3;
string category = 4;
}
# end::product-catalog-protobuf-schema-manifest[]
"""
And schema "product-catalog" is successfully synced
Then I should be able to check compatibility against "product-catalog" in cluster "basic"

@skip:gke @skip:aks @skip:eks
Scenario: Manage order event schema (JSON)
Given there is no schema "order-event" in cluster "basic"
When I apply Kubernetes manifest:
"""
# tag::order-event-json-schema-manifest[]
# This manifest creates a JSON schema named "order-event" in the "basic" cluster.
# The schema requires an "order_id" (string) and a "total" (number) field, with no additional properties allowed.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Schema
metadata:
name: order-event
spec:
cluster:
clusterRef:
name: basic
schemaType: json
compatibilityLevel: None
text: |
{
"$schema": "http://json-schema.org/draft-07/schema#",
"type": "object",
"properties": {
"order_id": { "type": "string" },
"total": { "type": "number" }
},
"required": ["order_id", "total"],
"additionalProperties": false
}
# end::order-event-json-schema-manifest[]
"""
And schema "order-event" is successfully synced
Then I should be able to check compatibility against "order-event" in cluster "basic"
27 changes: 27 additions & 0 deletions modules/manage/examples/kubernetes/topic-crds.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
@cluster:basic
Feature: Topic CRDs
Background: Cluster available
Given cluster "basic" is available

@skip:gke @skip:aks @skip:eks
Scenario: Manage topics
Given there is no topic "topic1" in cluster "basic"
When I apply Kubernetes manifest:
"""
# tag::basic-topic-example[]
# In this example manifest, a topic called "topic1" is created in a cluster called "basic". It has a replication factor of 1 and is distributed across a single partition.
---
apiVersion: cluster.redpanda.com/v1alpha2
kind: Topic
metadata:
name: topic1
spec:
cluster:
clusterRef:
name: basic
partitions: 1
replicationFactor: 1
# end::basic-topic-example[]
"""
And topic "topic1" is successfully synced
Then I should be able to produce and consume from "topic1" in cluster "basic"
10 changes: 5 additions & 5 deletions modules/manage/examples/kubernetes/user-crds.feature
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Feature: User CRDs
| jason | [{"type":"allow","resource":{"type":"cluster"},"operations":["Read"]}] |
When I apply Kubernetes manifest:
"""
# tag::manage-authn-only-manifest[]
# tag::manage-authn-only-manifest[]
# In this example manifest, a user called "jason" is created in a cluster called "sasl".
# The user's password is defined in a Secret called "jason-password".
# This example assumes that you will create ACLs for this user separately.
Expand All @@ -45,7 +45,7 @@ Feature: User CRDs
secretKeyRef:
name: jason-password
key: password
# end::manage-authn-only-manifest[]
# end::manage-authn-only-manifest[]
"""
And user "jason" is successfully synced
And I delete the CRD user "jason"
Expand All @@ -58,7 +58,7 @@ Feature: User CRDs
| travis | password | SCRAM-SHA-256 |
When I apply Kubernetes manifest:
"""
# tag::manage-authz-only-manifest[]
# tag::manage-authz-only-manifest[]
# In this example manifest, an ACL called "travis" is created in a cluster called "sasl".
# The ACL give an existing user called "travis" permissions to read from all topics whose names start with some-topic.
# This example assumes that you already have a user called "travis" in your cluster.
Expand All @@ -79,8 +79,8 @@ Feature: User CRDs
name: some-topic
patternType: prefixed
operations: [Read]
# end::manage-authz-only-manifest[]
# end::manage-authz-only-manifest[]
"""
And user "travis" is successfully synced
And I delete the CRD user "travis"
Then "travis" should be able to authenticate to the "sasl" cluster with password "password" and mechanism "SCRAM-SHA-256"
Then "travis" should be able to authenticate to the "sasl" cluster with password "password" and mechanism "SCRAM-SHA-256"
Loading