Skip to content

feat(firestore): [PQ] Add all Aggregation and Timestamp functions#13194

Merged
bhshkh merged 1 commit intogoogleapis:feature/fs-pipeline-queriesfrom
bhshkh:feature/fs-pipeline-queries-all-bitwise-datetime
Oct 30, 2025
Merged

feat(firestore): [PQ] Add all Aggregation and Timestamp functions#13194
bhshkh merged 1 commit intogoogleapis:feature/fs-pipeline-queriesfrom
bhshkh:feature/fs-pipeline-queries-all-bitwise-datetime

Conversation

@bhshkh
Copy link
Copy Markdown
Contributor

@bhshkh bhshkh commented Oct 21, 2025

b/364927702

  1. add all the remaining private preview aggregate functions. Merging this PR completes the implementation of all the type: "Function" subType : "Accumulators (Aggregation)" private preview features.
    See "Firestore Features (Pipeline)" sheet in go/firestore-query-tracker for the list of features.

    Java reference:

  2. add all the remaining private preview timestamp functions. Merging this PR completes the implementation of all the type: "Function" subType: "Date / Timestamp" private preview features. (except timestamp_trunc function which is not yet inmplemented in any of the SDKs. Requires additonal approvals from Firestore team and will be added to separate PR).
    See "Firestore Features (Pipeline)" sheet in go/firestore-query-tracker for the list of functions.

    Java reference:

  3. Add integration tests for all functions.

  4. Remove Rand function since it is not targeted for private preview.

  5. Renamed numericExprOrField to numericExprOrFieldPath since field is a separate type/expression.

    // field represents a reference to a field in a Firestore document, or outputs of a [Pipeline] stage.
    // It implements the [Expr] and [Selectable] interfaces.
    //
    // Field references are used to access document field values in expressions and to specify fields
    // for sorting, filtering, and projecting data in Firestore pipelines.
    type field struct {
    *baseExpr
    fieldPath FieldPath
    }
    // FieldOf creates a new field [Expr] from a field path string.
    func FieldOf(path string) Expr {
    fieldPath, err := parseDotSeparatedString(path)
    if err != nil {
    return &field{baseExpr: &baseExpr{err: err}}
    }
    return FieldOfPath(fieldPath)
    }
    // FieldOfPath creates a new field [Expr] for the given [FieldPath].
    func FieldOfPath(fieldPath FieldPath) Expr {

Previous pull requests

@bhshkh bhshkh changed the base branch from main to feature/fs-pipeline-queries October 21, 2025 09:49
@bhshkh bhshkh marked this pull request as ready for review October 27, 2025 03:51
@bhshkh bhshkh requested review from a team October 27, 2025 03:51
@bhshkh bhshkh force-pushed the feature/fs-pipeline-queries-all-bitwise-datetime branch from d79c2d4 to f16ab71 Compare October 27, 2025 04:46
@bhshkh bhshkh changed the title Feature/fs pipeline queries all bitwise datetime feat(firestore): Add all Aggregation and Timestamp functions Oct 27, 2025
@bhshkh bhshkh changed the title feat(firestore): Add all Aggregation and Timestamp functions feat(firestore): [Pipeline Queries] Add all Aggregation and Timestamp functions Oct 27, 2025
@bhshkh bhshkh changed the title feat(firestore): [Pipeline Queries] Add all Aggregation and Timestamp functions feat(firestore): [PQ] Add all Aggregation and Timestamp functions Oct 27, 2025
@bhshkh
Copy link
Copy Markdown
Contributor Author

bhshkh commented Oct 27, 2025

/gemini review

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request completes the implementation of private preview aggregation and timestamp functions for Firestore pipelines. It adds several new functions like Sum, Average, Count, Maximum, Minimum, various timestamp manipulation functions, and their corresponding integration tests. The changes are well-structured, and the addition of tests is great. I've found a few minor issues, mostly related to documentation copy-paste errors in the new aggregate functions and some leftover code in tests. There is also one change that looks like a temporary debug modification that should probably be reverted before merging.

Comment thread firestore/integration_test.go Outdated
Comment thread firestore/integration_test.go Outdated
Comment thread firestore/pipeline_aggregate.go Outdated
Comment thread firestore/pipeline_aggregate.go Outdated
@bhshkh bhshkh force-pushed the feature/fs-pipeline-queries-all-bitwise-datetime branch from 024e4a7 to 9b67878 Compare October 27, 2025 05:25
Copy link
Copy Markdown
Contributor

@daniel-sanche daniel-sanche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but left a few comments

Comment thread firestore/pipeline_aggregate.go
Comment thread firestore/pipeline_function.go
Copy link
Copy Markdown
Member

@hongalex hongalex left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for Go, want approval for firestore logic

@bhshkh bhshkh merged commit f1df882 into googleapis:feature/fs-pipeline-queries Oct 30, 2025
7 of 8 checks passed
@bhshkh bhshkh deleted the feature/fs-pipeline-queries-all-bitwise-datetime branch October 30, 2025 18:56
bhshkh added a commit that referenced this pull request Oct 30, 2025
)

1. add all the private preview 'array' functions. 
Merging this PR completes the implementation of all the **type:
"Function" subType : "Array"** private preview features (except
'maximum' and 'minimum' which are not yet implemented in any of the
SDKs. Requires additonal approvals from Firestore team and will be added
to separate PR). See "Firestore Features (Pipeline)" sheet in
[go/firestore-query-tracker](http://go/firestore-query-tracker) for the
list of features.
    Java reference: 
-
https://github.com/googleapis/java-firestore/blob/wuandy/JavaPplPP/google-cloud-firestore/src/main/java/com/google/cloud/firestore/pipeline/expressions/Expression.java
2. add all the private preview 'string' functions. (except
'string_split' which is not yet implemented in any of the SDKs. Requires
additonal approvals from Firestore team and will be added to separate
PR)

3. add all the private preview 'vector' functions. 
4. add remaining types to ConstantOf to match Java's implementation.
    Java reference: 
-
https://github.com/googleapis/java-firestore/blob/ccaf9d4fac5bd87a4da3d37493ca66fdc7681bc3/google-cloud-firestore/src/main/java/com/google/cloud/firestore/pipeline/expressions/Expression.java#L70-L211

Previous pull requests

- #12217
- #12425
- #12538
- #13147
- #13199
- #13218
- #13194
bhshkh added a commit that referenced this pull request Oct 30, 2025
b/364927702

- toExprOrField was renamed to asFieldExpr in
https://github.com/googleapis/google-cloud-go/pull/13194/files#diff-4a55211f7d38a1f0599e2f4cc92795073f138b2c56b846c933bda19e26bc3a7a
. There were a few call locations were rename was missed while resolving
merge conflicts which caused build failures. Fixing those failures in
this PR.
- the function signature of Data was changed in
#13218. It no longer
returns err as second argument. Fixing this in this PR.
- remove duplicate asInt64Expr and asStringExpr
- Move pipeline tests to their own file


Previous pull requests

- #12217
- #12425
- #12538
- #13147
- #13199
- #13218
- #13194
- #13245
bhshkh added a commit that referenced this pull request Oct 30, 2025
b/364927702

- Combine FieldOf and FieldOfPath to avoid verbose name FieldOfPath


Previous pull requests

- #12217
- #12425
- #12538
- #13147
- #13199
- #13218
- #13194
- #13245
- #13270

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
bhshkh added a commit that referenced this pull request Oct 31, 2025
1. Move PipelineStages integration tests.
2. Remove IsNaN, IsNotNaN, IsNull, IsNotNull, Equivalent as they are no
longer supported by backend
3. Remove examples as commented here

#13245 (comment)

Previous pull requests

- #12217
- #12425
- #12538
- #13147
- #13199
- #13218
- #13194
- #13245
- #13270
- #13271
bhshkh added a commit that referenced this pull request Nov 3, 2025
Add raw stage similar to Java

https://github.com/googleapis/java-firestore/blob/742fab6583c9a6f9c47cf0496124c3c9b05fe0ee/google-cloud-firestore/src/main/java/com/google/cloud/firestore/Pipeline.java#L997-L1021


The raw stage is an escape hatch to allow customers to consume new
stages supported by the backend without having to update their SDK to a
version that adds the stage.



Previous pull requests

- #12217
- #12425
- #12538
- #13147
- #13199
- #13218
- #13194
- #13245
- #13270
- #13271
- #13279
bhshkh added a commit that referenced this pull request Nov 3, 2025
1. Move PipelineStages integration tests.
2. Remove IsNaN, IsNotNaN, IsNull, IsNotNull, Equivalent as they are no
longer supported by backend
3. Remove examples as commented here

#13245 (comment)

Previous pull requests

- #12217
- #12425
- #12538
- #13147
- #13199
- #13218
- #13194
- #13245
- #13270
- #13271
bhshkh added a commit that referenced this pull request Nov 3, 2025
Add raw stage similar to Java

https://github.com/googleapis/java-firestore/blob/742fab6583c9a6f9c47cf0496124c3c9b05fe0ee/google-cloud-firestore/src/main/java/com/google/cloud/firestore/Pipeline.java#L997-L1021


The raw stage is an escape hatch to allow customers to consume new
stages supported by the backend without having to update their SDK to a
version that adds the stage.



Previous pull requests

- #12217
- #12425
- #12538
- #13147
- #13199
- #13218
- #13194
- #13245
- #13270
- #13271
- #13279
bhshkh added a commit that referenced this pull request Nov 10, 2025
#13283)

Changes in this PR:

1. firestore_client.go : Updated generated client as per
googleapis/gapic-generator-go#1661 . Removed
retries from tests since the headers have now been fixed.
2. Remove Equivalent since it was removed from backend.
3. Add/update comments
4. Add timestamp truncate (pending from
#13194) and string
split (pending from
#13245) functions.
5. add all the private preview general, key, logical (except iferror),
type and object functions.
See "Firestore Features (Pipeline)" sheet in
[go/firestore-query-tracker](http://go/firestore-query-tracker) for the
list of functions.
    Java reference:
-
https://github.com/googleapis/java-firestore/blob/ccaf9d4fac5bd87a4da3d37493ca66fdc7681bc3/google-cloud-firestore/src/main/java/com/google/cloud/firestore/pipeline/expressions/Expression.java



Previous pull requests

- #12217
- #12425
- #12538
- #13147
- #13199
- #13218
- #13194
- #13245
- #13270
- #13271
- #13279
- #13280
- #13281
- #13282
- googleapis/gapic-generator-go#1661
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants