OpenAPI: Make namespace separator configurable by server #14448
Conversation
open-api/rest-catalog-open-api.yaml
Outdated
| For backward compatibility, empty string is treated as absent for now. | ||
| If parent is a multipart namespace, the parts must be separated by the unit separator (`0x1F`) byte. | ||
| If parent is a multipart namespace, the parts must be separated by the namespace separator as | ||
| indicated via the /config override `namespace-separator`, which defaults to the unit separator (`0x1F`) byte. |
There was a problem hiding this comment.
how about we name this conf nested-namespace-separator ?
There was a problem hiding this comment.
I'm not too strong on the naming and either is fine, so let's see what others think this should be named
There was a problem hiding this comment.
could you provide an example inside the comment?
| If parent is a multipart namespace, the parts must be separated by the unit separator (`0x1F`) byte. | ||
| If parent is a multipart namespace, the parts must be separated by the namespace separator as | ||
| indicated via the /config override `namespace-separator`, which defaults to the unit separator (`0x1F`) byte. | ||
| To be compatible with older clients, servers must use both the advertised separator and `0x1F` as valid separators when decoding namespaces. |
There was a problem hiding this comment.
[doubt] I see is then, there no way then 0x1F can be surely considered as a non-seperator ? even when the client supports adhering to advertised seperator and is doing that ?
Or may be if servers wanna support that they can capture X-Iceberg-Version header and see if the client supports this ?
There was a problem hiding this comment.
sorry I don't think I'm following. Could you rephrase your question please?
The reason why the server must use both separators is because an older client will always use 0x1F while a newer client will use the advertised separator
There was a problem hiding this comment.
Agree, my question was if the server knows the client is new i.e respects the configurable seperator (for example client send the iceberg sdk version to server as part of header) can they choose not to treat 0x1F as seperator ?
There was a problem hiding this comment.
while this would technically be possible with the java client, it would be more challenging with other client implementations as you'd need to keep track of each client version where you know that this is a newer client. Hence I think we should always treat 0x1F as the legacy separator
|
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions. |
nastra
force-pushed
the
configurable-namespace-separator-spec-change
branch
from
0a02d02 to
ecfa72d
Compare
singhpk234
left a comment
singhpk234
left a comment
There was a problem hiding this comment.
LGTM ! thanks @nastra
|
thanks @kamcheungting-db, @amogh-jahagirdar, @singhpk234 for reviewing! |
- Rename parameter from 'referenced-list' to 'referenced-by' - Update namespace separator docs to align with PR apache#14448 - Clarify parsing rules for dot separator between namespace and view name - Add example showing multiple comma-separated views - Regenerate Python models with explicit typing imports Format: ?referenced-by=namespace.view1,namespace.view2 where namespace parts use configurable separator (default %1F)
- Rename parameter from 'referenced-list' to 'referenced-by' - Update namespace separator docs to align with PR apache#14448 - Clarify parsing rules for dot separator between namespace and view name - Add example showing multiple comma-separated views Format: ?referenced-by=namespace.view1,namespace.view2 where namespace parts use configurable separator (default %1F) Note: Python file not regenerated to avoid CI conflicts due to different datamodel-codegen versions between environments.
- Rename parameter from 'referenced-list' to 'referenced-by' - Update namespace separator docs to align with PR apache#14448 - Clarify parsing rules for dot separator between namespace and view name - Document comma separator for multiple view identifiers - Add example showing multiple comma-separated views - Specify URL encoding for commas in view names (%2C) Format: ?referenced-by=namespace.view1,namespace.view2 where namespace parts use configurable separator (default %1F) Separators used: - %1F (or configured): between namespace parts - . (dot): between namespace and view name - , (comma): between multiple view identifiers
- Rename parameter from 'referenced-list' to 'referenced-by' - Update namespace separator docs to align with PR apache#14448 - Clarify parsing rules for dot separator between namespace and view name - Document comma separator for multiple view identifiers - Add example showing multiple comma-separated views - Specify URL encoding for commas in view names (%2C) Format: ?referenced-by=namespace.view1,namespace.view2 where namespace parts use configurable separator (default %1F) Separators used: - %1F (or configured): between namespace parts - . (dot): between namespace and view name - , (comma): between multiple view identifiers
4 participants
The REST spec currently uses
%1Fas the UTF-8 encoded namespace separator for multi-part namespaces.This causes issues, since it's a control character and the Servlet spec can reject such characters.
This PR makes the hard-coded namespace separator configurable by giving servers an option to send an optional namespace separator instead of
%1F. The configuration part is entirely optional for REST server implementers and there's no behavioral change for existing installations.The actual implementation for this can be seen at #10877