Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add overload that allows a custom parameter placeholder. #3622

Merged
merged 5 commits into from
Jan 24, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions docs/documents/execute-custom-sql.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@

Use `QueueSqlCommand(string sql, params object[] parameterValues)` method to register and execute any custom/arbitrary SQL commands with the underlying unit of work, as part of the batched commands within `IDocumentSession`.

`?` placeholders can be used to denote parameter values. Postgres [type casts `::`](https://www.postgresql.org/docs/15/sql-expressions.html#SQL-SYNTAX-TYPE-CASTS) can be applied to the parameter if needed.
`?` placeholders can be used to denote parameter values. Postgres [type casts `::`](https://www.postgresql.org/docs/15/sql-expressions.html#SQL-SYNTAX-TYPE-CASTS) can be applied to the parameter if needed. If the `?` character is not suitable as a placeholder because you need to use `?` in your sql query, you can change the placeholder by providing an alternative. Pass this in before the sql argument.

<!-- snippet: sample_QueueSqlCommand -->
<a id='snippet-sample_queuesqlcommand'></a>
<a id='snippet-sample_QueueSqlCommand'></a>
```cs
theSession.QueueSqlCommand("insert into names (name) values ('Jeremy')");
theSession.QueueSqlCommand("insert into names (name) values ('Babu')");
Expand All @@ -14,6 +14,8 @@ theSession.QueueSqlCommand("insert into names (name) values ('Oskar')");
theSession.Store(Target.Random());
var json = "{ \"answer\": 42 }";
theSession.QueueSqlCommand("insert into data (raw_value) values (?::jsonb)", json);
// Use ^ as the parameter placeholder
theSession.QueueSqlCommand('^', "insert into data (raw_value) values (^::jsonb)", json);
```
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/CoreTests/executing_arbitrary_sql_as_part_of_transaction.cs#L39-L47' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_queuesqlcommand' title='Start of snippet'>anchor</a></sup>
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/CoreTests/executing_arbitrary_sql_as_part_of_transaction.cs#L39-L49' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_QueueSqlCommand' title='Start of snippet'>anchor</a></sup>
<!-- endSnippet -->
15 changes: 14 additions & 1 deletion docs/documents/querying/advanced-sql.md
Original file line number Diff line number Diff line change
Expand Up @@ -139,6 +139,19 @@ results[1].detail.Detail.ShouldBe("Likes to cook");
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/DocumentDbTests/Reading/advanced_sql_query.cs#L101-L138' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_advanced_sql_query_related_documents_and_scalar' title='Start of snippet'>anchor</a></sup>
<!-- endSnippet -->

All `AdvancedSql` methods also support parameters:

<!-- snippet: sample_document_schema_resolver_resolve_schemas -->
<a id='snippet-sample_document_schema_resolver_resolve_schemas'></a>
```cs
var schema = theSession.DocumentStore.Options.Schema;

schema.DatabaseSchemaName.ShouldBe("public");
schema.EventsSchemaName.ShouldBe("public");
```
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/CoreTests/DocumentSchemaResolverTests.cs#L24-L29' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_document_schema_resolver_resolve_schemas' title='Start of snippet'>anchor</a></sup>
<!-- endSnippet -->

For sync queries you can use the `AdvancedSql.Query<T>(...)` overloads.

When you need to query for large datasets, the `AdvancedSql.StreamAsync<>(...)` methods can be used. They will return
Expand Down Expand Up @@ -182,7 +195,7 @@ await foreach (var result in asyncEnumerable)
collectedResults.Add(result);
}
```
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/DocumentDbTests/Reading/advanced_sql_query.cs#L145-L180' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_advanced_sql_stream_related_documents_and_scalar' title='Start of snippet'>anchor</a></sup>
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/DocumentDbTests/Reading/advanced_sql_query.cs#L173-L208' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_advanced_sql_stream_related_documents_and_scalar' title='Start of snippet'>anchor</a></sup>
<!-- endSnippet -->

Using this you can resolve schemas:
Expand Down
17 changes: 11 additions & 6 deletions docs/documents/querying/linq/sql.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,19 +18,24 @@ public async Task query_with_matches_sql()
user.Id.ShouldBe(u.Id);
}
```
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/DocumentDbTests/Reading/query_by_sql.cs#L267-L282' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_query_with_matches_sql' title='Start of snippet'>anchor</a></sup>
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/DocumentDbTests/Reading/query_by_sql.cs#L336-L351' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_query_with_matches_sql' title='Start of snippet'>anchor</a></sup>
<!-- endSnippet -->

**But**, if you want to take advantage of the more recent and very powerful JSONPath style querying, use this flavor of
the same functionality that behaves exactly the same, but uses the '^' character for parameter placeholders to disambiguate
from the '?' character that is widely used in JSONPath expressions:
**But**, if you want to take advantage of the more recent and very powerful JSONPath style querying, you will find that using `?` as a placeholder is not suitable, as that character is widely used in JSONPath expressions. If you encounter this issue or write another query where the `?` character is not suitable, you can change the placeholder by providing an alternative. Pass this in before the sql argument.

Older version of Marten also offer the `MatchesJsonPath()` method which uses the `^` character as a placeholder. This will continue to be supported.

<!-- snippet: sample_using_MatchesJsonPath -->
<a id='snippet-sample_using_matchesjsonpath'></a>
<a id='snippet-sample_using_MatchesJsonPath'></a>
```cs
var results2 = await theSession
.Query<Target>().Where(x => x.MatchesSql('^', "d.data @? '$ ? (@.Children[*] == null || @.Children[*].size() == 0)'"))
.ToListAsync();

// older approach that only supports the ^ placeholder
var results3 = await theSession
.Query<Target>().Where(x => x.MatchesJsonPath("d.data @? '$ ? (@.Children[*] == null || @.Children[*].size() == 0)'"))
.ToListAsync();
```
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/LinqTests/Bugs/Bug_3087_using_JsonPath_with_MatchesSql.cs#L28-L34' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_using_matchesjsonpath' title='Start of snippet'>anchor</a></sup>
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/LinqTests/Bugs/Bug_3087_using_JsonPath_with_MatchesSql.cs#L28-L39' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_using_MatchesJsonPath' title='Start of snippet'>anchor</a></sup>
<!-- endSnippet -->
12 changes: 8 additions & 4 deletions docs/documents/querying/sql.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,12 @@ Or with parameterized SQL:
```cs
var millers = session
.Query<User>("where data ->> 'LastName' = ?", "Miller");

// custom placeholder parameter
var millers2 = await session
.QueryAsync<User>('$', "where data ->> 'LastName' = $", "Miller");
```
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/Marten.Testing/Examples/QueryBySql.cs#L20-L25' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_query_with_sql_and_parameters' title='Start of snippet'>anchor</a></sup>
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/Marten.Testing/Examples/QueryBySql.cs#L20-L29' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_query_with_sql_and_parameters' title='Start of snippet'>anchor</a></sup>
<!-- endSnippet -->

And finally asynchronously:
Expand All @@ -37,7 +41,7 @@ And finally asynchronously:
var millers = await session
.QueryAsync<User>("where data ->> 'LastName' = ?", "Miller");
```
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/Marten.Testing/Examples/QueryBySql.cs#L30-L35' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_query_with_sql_async' title='Start of snippet'>anchor</a></sup>
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/Marten.Testing/Examples/QueryBySql.cs#L34-L39' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_query_with_sql_async' title='Start of snippet'>anchor</a></sup>
<!-- endSnippet -->

All of the samples so far are selecting the whole `User` document and merely supplying
Expand All @@ -50,7 +54,7 @@ a document body, but in that case you will need to supply the full SQL statement
var sumResults = await session
.QueryAsync<int>("select count(*) from mt_doc_target");
```
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/DocumentDbTests/Reading/query_by_sql.cs#L376-L381' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_query_by_full_sql' title='Start of snippet'>anchor</a></sup>
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/DocumentDbTests/Reading/query_by_sql.cs#L458-L463' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_query_by_full_sql' title='Start of snippet'>anchor</a></sup>
<!-- endSnippet -->

When querying single JSONB properties into a primitive/value type, you'll need to cast the value to the respective postgres type:
Expand All @@ -61,7 +65,7 @@ When querying single JSONB properties into a primitive/value type, you'll need t
var times = await session.QueryAsync<DateTimeOffset>(
"SELECT (data ->> 'ModifiedAt')::timestamptz from mt_doc_user");
```
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/DocumentDbTests/Reading/query_by_sql.cs#L330-L335' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_using-queryasync-casting' title='Start of snippet'>anchor</a></sup>
<sup><a href='https://github.com/JasperFx/marten/blob/master/src/DocumentDbTests/Reading/query_by_sql.cs#L412-L417' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_using-queryasync-casting' title='Start of snippet'>anchor</a></sup>
<!-- endSnippet -->

The basic rules for how Marten handles user-supplied queries are:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ public async Task can_run_extra_sql()
theSession.Store(Target.Random());
var json = "{ \"answer\": 42 }";
theSession.QueueSqlCommand("insert into data (raw_value) values (?::jsonb)", json);
// Use ^ as the parameter placeholder
theSession.QueueSqlCommand('^', "insert into data (raw_value) values (^::jsonb)", json);
#endregion

await theSession.SaveChangesAsync();
Expand Down
28 changes: 28 additions & 0 deletions src/DocumentDbTests/Reading/advanced_sql_query.cs
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,34 @@ limit 2
#endregion
}

[Fact]
public async Task can_query_with_parameters()
{
await using var session = theStore.LightweightSession();
session.Store(new DocWithMeta { Id = 1, Name = "Max" });
await session.SaveChangesAsync();

#region sample_advanced_sql_query_parameters
var schema = session.DocumentStore.Options.Schema;

var name = (await session.AdvancedSql.QueryAsync<string>(
$"select data ->> ? from {schema.For<DocWithMeta>()} limit 1",
CancellationToken.None,
"Name")).First();

// Use ^ as the parameter placeholder
var name2 = (await session.AdvancedSql.QueryAsync<string>(
'^',
$"select data ->> ^ from {schema.For<DocWithMeta>()} limit 1",
CancellationToken.None,
"Name")).First();

#endregion

name.ShouldBe("Max");
name2.ShouldBe("Max");
}

[Fact]
public async Task can_async_stream_multiple_documents_and_scalar()
{
Expand Down
84 changes: 83 additions & 1 deletion src/DocumentDbTests/Reading/query_by_sql.cs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
using System;
using System;
using System.IO;
using System.Linq;
using System.Threading;
Expand Down Expand Up @@ -70,6 +70,31 @@ public async Task stream_query_by_one_parameter()
firstnames[2].ShouldBe("Max");
}

[Fact]
public async Task stream_query_by_one_parameter_custom_placeholder()
{
await using var session = theStore.LightweightSession();
session.Store(new User { FirstName = "Jeremy", LastName = "Miller" });
session.Store(new User { FirstName = "Lindsey", LastName = "Miller" });
session.Store(new User { FirstName = "Max", LastName = "Miller" });
session.Store(new User { FirstName = "Frank", LastName = "Zombo" });
await session.SaveChangesAsync();

var stream = new MemoryStream();
await session.StreamJson<User>(stream, '$', "where data ->> 'LastName' = $", "Miller");

stream.Position = 0;
var results = theStore.Options.Serializer().FromJson<User[]>(stream);
var firstnames = results
.OrderBy(x => x.FirstName)
.Select(x => x.FirstName).ToArray();

firstnames.Length.ShouldBe(3);
firstnames[0].ShouldBe("Jeremy");
firstnames[1].ShouldBe("Lindsey");
firstnames[2].ShouldBe("Max");
}

[Fact]
public async Task query_by_one_parameter()
{
Expand All @@ -90,6 +115,50 @@ public async Task query_by_one_parameter()
firstnames[2].ShouldBe("Max");
}

[Fact]
public async Task query_by_one_parameter_async()
{
await using var session = theStore.LightweightSession();
session.Store(new User { FirstName = "Jeremy", LastName = "Miller" });
session.Store(new User { FirstName = "Lindsey", LastName = "Miller" });
session.Store(new User { FirstName = "Max", LastName = "Miller" });
session.Store(new User { FirstName = "Frank", LastName = "Zombo" });
await session.SaveChangesAsync();

var firstnames =
(await session.QueryAsync<User>("where data ->> 'LastName' = ?", "Miller"))
.OrderBy(x => x.FirstName)
.Select(x => x.FirstName)
.ToArray();

firstnames.Length.ShouldBe(3);
firstnames[0].ShouldBe("Jeremy");
firstnames[1].ShouldBe("Lindsey");
firstnames[2].ShouldBe("Max");
}

[Fact]
public async Task query_by_one_parameter_async_custom_placeholder()
{
await using var session = theStore.LightweightSession();
session.Store(new User { FirstName = "Jeremy", LastName = "Miller" });
session.Store(new User { FirstName = "Lindsey", LastName = "Miller" });
session.Store(new User { FirstName = "Max", LastName = "Miller" });
session.Store(new User { FirstName = "Frank", LastName = "Zombo" });
await session.SaveChangesAsync();

var firstnames =
(await session.QueryAsync<User>('$', "where data ->> 'LastName' = $", "Miller"))
.OrderBy(x => x.FirstName)
.Select(x => x.FirstName)
.ToArray();

firstnames.Length.ShouldBe(3);
firstnames[0].ShouldBe("Jeremy");
firstnames[1].ShouldBe("Lindsey");
firstnames[2].ShouldBe("Max");
}

[Fact]
public async Task query_ignores_case_of_where_keyword()
{
Expand Down Expand Up @@ -280,6 +349,19 @@ public async Task query_with_matches_sql()
}

#endregion

[Fact]
public async Task query_with_matches_sql_custom_placeholder()
{
await using var session = theStore.LightweightSession();
var u = new User { FirstName = "Eric", LastName = "Smith" };
session.Store(u);
await session.SaveChangesAsync();

var user = await session.Query<User>().Where(x => x.MatchesSql('$', "data->> 'FirstName' = $", "Eric")).SingleAsync();
user.LastName.ShouldBe("Smith");
user.Id.ShouldBe(u.Id);
}

[Fact]
public async Task query_with_select_in_query()
Expand Down
5 changes: 5 additions & 0 deletions src/LinqTests/Bugs/Bug_3087_using_JsonPath_with_MatchesSql.cs
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,11 @@ public async Task can_use_json_path_operations()
#region sample_using_MatchesJsonPath

var results2 = await theSession
.Query<Target>().Where(x => x.MatchesSql('^', "d.data @? '$ ? (@.Children[*] == null || @.Children[*].size() == 0)'"))
.ToListAsync();

// older approach that only supports the ^ placeholder
var results3 = await theSession
.Query<Target>().Where(x => x.MatchesJsonPath("d.data @? '$ ? (@.Children[*] == null || @.Children[*].size() == 0)'"))
.ToListAsync();

Expand Down
8 changes: 6 additions & 2 deletions src/Marten.Testing/Examples/QueryBySql.cs
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,17 @@ public void QueryForWholeDocumentByWhereClause(IQuerySession session)
#endregion
}

public void QueryWithParameters(IQuerySession session)
public async Task QueryWithParameters(IQuerySession session)
{
#region sample_query_with_sql_and_parameters

var millers = session
.Query<User>("where data ->> 'LastName' = ?", "Miller");

// custom placeholder parameter
var millers2 = await session
.QueryAsync<User>('$', "where data ->> 'LastName' = $", "Miller");

#endregion
}

Expand All @@ -35,4 +39,4 @@ public async Task QueryAsynchronously(IQuerySession session)
#endregion
}

}
}
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,12 @@ internal class CallUpsertFunctionFrame: MethodCall, IEventHandlingFrame
private readonly DbObjectName _functionIdentifier;
private readonly MemberInfo[] _members;

private static readonly MethodInfo QueueSqlMethod =
typeof(IDocumentOperations).GetMethod(nameof(IDocumentOperations.QueueSqlCommand),
[typeof(string), typeof(object[])])!;

public CallUpsertFunctionFrame(Type eventType, DbObjectName functionIdentifier, List<IColumnMap> columnMaps,
MemberInfo[] members): base(typeof(IDocumentOperations), nameof(IDocumentOperations.QueueSqlCommand))
MemberInfo[] members): base(typeof(IDocumentOperations), QueueSqlMethod)
{
_functionIdentifier = functionIdentifier ?? throw new ArgumentNullException(nameof(functionIdentifier));
_columnMaps = columnMaps;
Expand Down
6 changes: 5 additions & 1 deletion src/Marten/Events/Projections/Flattened/DeleteRowFrame.cs
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,12 @@ internal class DeleteRowFrame: MethodCall, IEventHandlingFrame
private readonly Table _table;
private Variable? _event;

private static readonly MethodInfo QueueSqlMethod =
typeof(IDocumentOperations).GetMethod(nameof(IDocumentOperations.QueueSqlCommand),
[typeof(string), typeof(object[])])!;

public DeleteRowFrame(Table table, Type eventType, MemberInfo[] members): base(typeof(IDocumentOperations),
nameof(IDocumentOperations.QueueSqlCommand))
QueueSqlMethod)
{
if (!members.Any())
{
Expand Down
Loading
Loading