Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Firestore: Disable bloom filter integration tests when using Firestore emulator #11555

Merged
merged 1 commit into from
Jul 18, 2023

Conversation

dconeybe
Copy link
Contributor

The Firestore emulator does not (as of July 14, 2023) send an existence filter when a query is resumed by clients. As a result the client fails to delete documents that were deleted since the query was last run. The emulator will soon be fixed to send the existence filter (Googlers see b/270731363).

There are two bloom filter integration tests, the first of which tried to do as much as it could when running against the emulator and completing successfully if it got that far. That test, though, didn't really validate anything and by it passing it incorrectly implied that the bloom filter logic was validated. This test is now completely skipped when running against the Firestore emulator, which also makes the test simpler and easier to maintain.

These two bloom filter integration tests also mentioned the bug b/270731363 in their comments. These comments have now been updated to instead mention b/291365820, which is the new issue that tracks adding bloom filter support to the Firestore emulator.

Note to reviewer: set your diff tool to ignore whitespace to make reviewing easier; some indentation changes caused a large number of lines to be affected with whitespace-only changes.

The corresponding PRs in the other SDKs are firebase/firebase-js-sdk#7463 and firebase/firebase-android-sdk#5168.

#no-changelog

@google-oss-bot
Copy link

Coverage Report 1

Affected Products

  • FirebaseFirestore-iOS-FirebaseFirestore.framework

    Overall coverage changed from ? (9205183) to 88.04% (bca83f9) by ?.

    227 individual files with coverage change

    FilenameBase (9205183)Merge (bca83f9)Diff
    aggregate_alias.cc?100.00%?
    aggregate_field.cc?100.00%?
    aggregate_query.cc?100.00%?
    aggregation_result.nanopb.cc?0.00%?
    any.nanopb.cc?0.00%?
    array_contains_any_filter.cc?100.00%?
    array_contains_filter.cc?100.00%?
    async_queue.cc?100.00%?
    auth_token.cc?100.00%?
    autoid.cc?100.00%?
    background_queue.cc?100.00%?
    bits.cc?100.00%?
    bloom_filter.cc?90.12%?
    bloom_filter.nanopb.cc?0.00%?
    bound.cc?84.29%?
    bundle.nanopb.cc?0.00%?
    bundle_loader.cc?95.65%?
    bundle_reader.cc?91.26%?
    bundle_serializer.cc?91.49%?
    byte_stream_apple.mm?86.36%?
    byte_stream_cpp.cc?83.33%?
    byte_string.cc?80.30%?
    collection_reference.cc?100.00%?
    common.nanopb.cc?34.48%?
    comparison.cc?100.00%?
    composite_filter.cc?98.11%?
    connectivity_monitor.cc?100.00%?
    connectivity_monitor_apple.mm?51.92%?
    converters.mm?100.00%?
    database_id.cc?65.38%?
    database_info.cc?100.00%?
    datastore.cc?96.69%?
    delete_mutation.cc?84.00%?
    direction.cc?76.92%?
    document.cc?0.00%?
    document.nanopb.cc?96.67%?
    document_change.cc?62.50%?
    document_key.cc?93.06%?
    document_key_reference.cc?65.00%?
    document_overlay_cache.cc?100.00%?
    document_reference.cc?100.00%?
    document_set.cc?87.30%?
    document_snapshot.cc?100.00%?
    empty.nanopb.cc?0.00%?
    error_apple.mm?92.31%?
    event_manager.cc?98.08%?
    exception.cc?23.68%?
    exception_apple.mm?96.55%?
    executor_libdispatch.mm?96.06%?
    executor_std.cc?97.71%?
    exponential_backoff.cc?100.00%?
    field_filter.cc?95.12%?
    field_index.cc?92.68%?
    field_mask.cc?100.00%?
    field_path.cc?98.17%?
    field_transform.cc?41.67%?
    filesystem_apple.mm?83.33%?
    filesystem_common.cc?81.67%?
    filesystem_posix.cc?81.82%?
    filter.cc?62.50%?
    FIRAggregateField.mm?98.41%?
    FIRAggregateQuery.mm?100.00%?
    FIRAggregateQuerySnapshot.mm?100.00%?
    FIRCollectionReference.mm?95.08%?
    FIRDocumentChange.mm?76.32%?
    FIRDocumentReference.mm?97.10%?
    FIRDocumentSnapshot.mm?97.89%?
    firebase_app_check_credentials_provider_apple.mm?92.93%?
    firebase_auth_credentials_provider_apple.mm?76.64%?
    firebase_metadata_provider_apple.mm?86.96%?
    firebase_metadata_provider_noop.cc?100.00%?
    firestore.cc?92.13%?
    firestore.nanopb.cc?35.95%?
    firestore_client.cc?98.91%?
    firestore_index_value_writer.cc?90.00%?
    FIRFieldPath.mm?90.24%?
    FIRFieldValue.mm?89.41%?
    FIRFilter.mm?100.00%?
    FIRFirestore.mm?88.70%?
    FIRFirestoreSettings.mm?81.00%?
    FIRFirestoreSource.mm?90.91%?
    FIRGeoPoint.mm?82.22%?
    FIRListenerRegistration.mm?100.00%?
    FIRLoadBundleTask.mm?80.70%?
    FIRLocalCacheSettings.mm?48.97%?
    FIRQuery.mm?87.87%?
    FIRQuerySnapshot.mm?95.45%?
    FIRSnapshotMetadata.mm?100.00%?
    FIRTimestamp.m?79.35%?
    FIRTransaction.mm?97.64%?
    FIRTransactionOptions.mm?100.00%?
    FIRWriteBatch.mm?100.00%?
    FSTFirestoreComponent.mm?95.74%?
    FSTUserDataReader.mm?95.38%?
    FSTUserDataWriter.mm?87.06%?
    geo_point.cc?65.00%?
    grpc_completion.cc?100.00%?
    grpc_connection.cc?77.50%?
    grpc_nanopb.cc?94.87%?
    grpc_root_certificate_finder_generated.cc?100.00%?
    grpc_stream.cc?99.01%?
    grpc_streaming_reader.cc?100.00%?
    grpc_unary_call.cc?100.00%?
    grpc_util.cc?100.00%?
    hard_assert.cc?100.00%?
    http.nanopb.cc?0.00%?
    index.nanopb.cc?0.00%?
    index_backfiller.cc?100.00%?
    index_entry.cc?60.00%?
    in_filter.cc?100.00%?
    json_reader.cc?87.50%?
    key_field_filter.cc?100.00%?
    key_field_in_filter.cc?100.00%?
    key_field_not_in_filter.cc?100.00%?
    latlng.nanopb.cc?86.67%?
    leveldb_bundle_cache.cc?76.00%?
    leveldb_document_overlay_cache.cc?97.17%?
    leveldb_index_manager.cc?97.67%?
    leveldb_key.cc?98.82%?
    leveldb_lru_reference_delegate.cc?95.93%?
    leveldb_migrations.cc?92.64%?
    leveldb_mutation_queue.cc?92.42%?
    leveldb_opener.cc?76.81%?
    leveldb_overlay_migration_manager.cc?100.00%?
    leveldb_persistence.cc?90.82%?
    leveldb_remote_document_cache.cc?94.61%?
    leveldb_target_cache.cc?94.68%?
    leveldb_transaction.cc?98.79%?
    leveldb_util.cc?71.43%?
    load_bundle_task.cc?97.06%?
    local_documents_view.cc?96.86%?
    local_serializer.cc?87.78%?
    local_store.cc?100.00%?
    local_view_changes.cc?100.00%?
    logic_utils.cc?97.94%?
    log_apple.mm?93.33%?
    lru_garbage_collector.cc?91.34%?
    maybe_document.nanopb.cc?28.89%?
    md5.cc?91.88%?
    memory_bundle_cache.cc?100.00%?
    memory_document_overlay_cache.cc?100.00%?
    memory_eager_reference_delegate.cc?100.00%?
    memory_index_manager.cc?50.00%?
    memory_lru_reference_delegate.cc?95.41%?
    memory_mutation_queue.cc?100.00%?
    memory_persistence.cc?100.00%?
    memory_remote_document_cache.cc?93.51%?
    memory_target_cache.cc?100.00%?
    message.cc?100.00%?
    mutable_document.cc?68.52%?
    mutation.cc?86.15%?
    mutation.nanopb.cc?75.76%?
    mutation_batch.cc?88.30%?
    mutation_batch_result.cc?52.17%?
    nanopb_util.cc?100.00%?
    not_in_filter.cc?100.00%?
    object_value.cc?100.00%?
    online_state_tracker.cc?100.00%?
    ordered_code.cc?93.90%?
    order_by.cc?50.00%?
    overlay.cc?100.00%?
    patch_mutation.cc?100.00%?
    path.cc?100.00%?
    precondition.cc?86.49%?
    pretty_printing.cc?83.33%?
    proto_sizer.cc?100.00%?
    query.cc?98.45%?
    query.nanopb.cc?67.92%?
    query_core.cc?96.23%?
    query_engine.cc?98.28%?
    query_listener.cc?100.00%?
    query_listener_registration.cc?100.00%?
    query_snapshot.cc?84.21%?
    reader.cc?100.00%?
    reference_set.cc?88.89%?
    remote_event.cc?97.67%?
    remote_objc_bridge.cc?92.96%?
    remote_store.cc?91.16%?
    resource.nanopb.cc?0.00%?
    resource_path.cc?100.00%?
    schedule.cc?100.00%?
    secure_random_arc4random.cc?100.00%?
    serializer.cc?90.80%?
    server_timestamp_util.cc?95.77%?
    settings.cc?83.11%?
    set_mutation.cc?89.13%?
    snapshots_in_sync_listener_registration.cc?100.00%?
    snapshot_metadata.cc?100.00%?
    snapshot_version.cc?85.71%?
    status.cc?73.05%?
    status.nanopb.cc?72.22%?
    statusor.cc?80.00%?
    status_apple.mm?96.61%?
    status_errno.cc?37.82%?
    stream.cc?98.83%?
    strerror.cc?100.00%?
    string_apple.cc?100.00%?
    string_format.cc?93.94%?
    string_util.cc?100.00%?
    struct.nanopb.cc?7.32%?
    sync_engine.cc?95.26%?
    target.cc?96.23%?
    target.nanopb.cc?45.00%?
    target_data.cc?43.04%?
    target_id_generator.cc?100.00%?
    target_index_matcher.cc?97.80%?
    task.cc?94.78%?
    testing_hooks.cc?100.00%?
    timestamp.cc?94.00%?
    timestamp.nanopb.cc?100.00%?
    timestamp_internal.cc?53.33%?
    transaction.cc?90.06%?
    transaction_runner.cc?100.00%?
    transform_operation.cc?77.03%?
    user.cc?100.00%?
    user_data.cc?96.30%?
    value_util.cc?95.71%?
    verify_mutation.cc?12.50%?
    view.cc?98.39%?
    view_snapshot.cc?78.99%?
    watch_change.cc?90.00%?
    watch_stream.cc?90.70%?
    wrappers.nanopb.cc?11.11%?
    write.nanopb.cc?60.57%?
    writer.cc?96.97%?
    write_batch.cc?91.89%?
    write_stream.cc?91.55%?

  • FirebaseFirestore-iOS-FirebaseFirestoreSwift.framework

    Overall coverage changed from ? (9205183) to 40.79% (bca83f9) by ?.

    22 individual files with coverage change

    FilenameBase (9205183)Merge (bca83f9)Diff
    CodablePassThroughTypes.swift?100.00%?
    CollectionReference+AsyncAwait.swift?96.88%?
    CollectionReference+WriteEncodable.swift?100.00%?
    DocumentID.swift?86.44%?
    DocumentReference+Codable.swift?50.00%?
    DocumentReference+ReadDecodable.swift?0.00%?
    DocumentReference+WriteEncodable.swift?100.00%?
    DocumentSnapshot+ReadDecodable.swift?80.00%?
    EncoderDecoder.swift?100.00%?
    ExplicitNull.swift?90.48%?
    FieldValue+Encodable.swift?100.00%?
    Firestore+AsyncAwait.swift?97.75%?
    FirestoreQuery.swift?0.00%?
    FirestoreQueryObservable.swift?0.00%?
    GeoPoint+Codable.swift?100.00%?
    QueryPredicate.swift?0.00%?
    ServerTimestamp.swift?96.97%?
    Timestamp+Codable.swift?100.00%?
    TimestampDecodingStrategy.swift?100.00%?
    TimestampEncodingStrategy.swift?100.00%?
    Transaction+WriteEncodable.swift?100.00%?
    WriteBatch+WriteEncodable.swift?100.00%?

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/90QQpQ3B6D.html

@google-oss-bot
Copy link

Size Report 1

Affected Products

  • FirebaseFirestore

    TypeBase (9205183)Merge (bca83f9)Diff
    CocoaPods?-51.5 kB? (?)

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/TKCJ54RImd.html

Copy link
Contributor

@milaGGL milaGGL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dconeybe dconeybe merged commit 013d869 into master Jul 18, 2023
@dconeybe dconeybe deleted the dconeybe/BloomFilterBug270731363To291365820 branch July 18, 2023 13:50
@firebase firebase locked and limited conversation to collaborators Aug 18, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants