Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL-like Expression Language for patterns #79

Open
embano1 opened this issue Jul 1, 2022 · 5 comments
Open

SQL-like Expression Language for patterns #79

embano1 opened this issue Jul 1, 2022 · 5 comments

Comments

@embano1
Copy link
Collaborator

embano1 commented Jul 1, 2022

In a discussion with a colleague, were I was showcasing quamina, he asked whether the quamina authors considered SQL(-like) expressions instead of JSON patterns, e.g. using CloudEvents SQL Expression Language.

Example:

// input event
{
    "source": "commerce.mysource.example.com",
    "type": "order.created",
    "id": 123
}
// quamina pattern to match on source prefix and type set
{
    "source": [{"shellstyle": "commerce*"}],
    "type": ["order.created", "order.updated", "order.canceled"]
}
// equivalent cesql expression
cesql: "source LIKE 'commerce*' AND type IN ('order.created', 'order.updated', 'order.canceled')"

Without reading the quamina pattern match specification, SQL-like expressions generally are more readable and explicit, e.g. AND. Also, SQL is a widely used and accepted standard, lowering the bar for adopters.

Note that a SQL-like dialect does not have to replace the existing JSON pattern match implementation, but could augment it.

@timbray
Copy link
Owner

timbray commented Jul 1, 2022

Doesn't sound crazy. Empirically, the groups using Quamina's big sister inside AWS seemed to like the pattern language, basically 100% of our complaints were about wanting new matching features and smaller memory size. To be honest, I have no opinion, we thought the pattern language up in a hurry.

The easiest way to do such a thing would be translate the SQL-style language into the existing pattern language? But it doesn't sound too horribly difficult to go directly from SQL-like language to the underlying Quamina finite automata.

@evankanderson
Copy link

Hi, I'm that colleague. I don't have a strong feeling about whether there's an IR (intermediate representation) which looks like the current language, or whether the direct interface is the SQL pattern builder. If SQL were not quite so common, I might suggest the IR as a better choice so that people could build simple UIs exposing the match patterns, but it's probably equally easy to take that UI and build a SQL pattern at this point.

One interesting question between SQL and IR is whether or not the Quamina implementation stores and can return the SQL representation, or whether it's an input convenience which is discarded on storage.

Note that a SQL representation may also lend support for more complex "OR" queries, like source = 'billing.invoice.po' OR (source LIKE 'billing.*' AND value > 10000.0 ) (to find interesting audit items). I don't know if that's desired or not.

@jsmorph
Copy link
Collaborator

jsmorph commented Jul 1, 2022

Incidentally I have a side project which compiles a SQL SELECT statement from a Quamina pattern. The target database should have an inverted-index-style table (with branch+values). Re shellstyle: this compiler doesn't enjoy anything other than a trailing wildcard in order to end up with a clause than can be reasonably supported with a simple SQL index.

@timbray
Copy link
Owner

timbray commented Jul 1, 2022

Hmm. At runtime, nothing exists but the compiled DFA structure starting at the coreMatcher type. The current pattern language and a hypothetical SQL-like expression are probably about equally removed from it. @jsmorph has been working on a pattern store which remembers the patterns that have been compiled in - there's no architectural reason why such a store couldn't remember a query-language expression.

Quamina's big sister, the implementation inside EventBridge, SNS, etc, recently grew a way to express "OR". My initial reaction is to dislike it but there's plenty of room to discuss there. There's nothing architectural about the compiled automaton that gets in the way of OR.

@timbray
Copy link
Owner

timbray commented Jul 2, 2022

Just to be clear, my priorities going forward are going to be adding to the set of supported Patterns for Quamina, and improving its efficiency. But if anyone else wanted to design and propose an SQL-like query language I'd be supportive and be inclined to include it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants