Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test FLX sync geospatial queries #7042

Merged
merged 10 commits into from
Nov 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
### Fixed
* <How do the end-user experience this issue? what was the impact?> ([#????](https://github.com/realm/realm-core/issues/????), since v?.?.?)
* A crash at a very specific time during a DiscardLocal client reset on a FLX Realm could leave subscriptions in an invalid state ([#7110](https://github.com/realm/realm-core/pull/7110), since v12.3.0).
* Fixed an error "Invalid schema change (UPLOAD): cannot process AddColumn instruction for non-existent table" when using automatic client reset with recovery in dev mode to recover schema changes made locally while offline. ([#7042](https://github.com/realm/realm-core/pull/7042) since the server introduced the feature that allows client to redefine the server's schema if the server is in dev mode - fall 2023)

### Breaking changes
* None.
Expand Down
2 changes: 1 addition & 1 deletion dependencies.list
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ PACKAGE_NAME=realm-core
VERSION=13.23.3
OPENSSL_VERSION=3.0.8
ZLIB_VERSION=1.2.13
MDBREALM_TEST_SERVER_TAG=2023-08-11
MDBREALM_TEST_SERVER_TAG=2023-10-20
2 changes: 1 addition & 1 deletion evergreen/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -887,7 +887,7 @@ tasks:
commands:
- func: "launch remote baas"
vars:
baas_branch: 3f31617aacfe5d31b9057fc298b735b60acd6424
baas_branch: 27f42f55a7944ed7d8ba9fad1854a4b22714cb8d
- func: "compile"
vars:
target_to_build: ObjectStoreTests
Expand Down
16 changes: 0 additions & 16 deletions evergreen/config_overrides.json

This file was deleted.

50 changes: 29 additions & 21 deletions evergreen/install_baas.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ case $(uname -s) in
Darwin)
if [[ "$(uname -m)" == "arm64" ]]; then
export GOARCH=arm64
STITCH_SUPPORT_LIB_URL="https://s3.amazonaws.com/static.realm.io/stitch-support/stitch-support-macos-arm64-6.1.0-rc3-8-gb6e0525.tgz"
STITCH_ASSISTED_AGG_URL="https://stitch-artifacts.s3.amazonaws.com/stitch-mongo-libs/stitch_mongo_libs_osx_patch_75b3f1896aaa2e344817795c8bfc5cb6b2f2c310_632211a5d1fe0757f8c416fa_22_09_14_17_38_46/assisted_agg"
GO_URL="https://s3.amazonaws.com/static.realm.io/evergreen-assets/go1.19.3.darwin-arm64.tar.gz"
STITCH_SUPPORT_LIB_URL="https://stitch-artifacts.s3.amazonaws.com/stitch-support/macos-arm64/stitch-support-6.1.0-alpha-527-g796351f.tgz"
STITCH_ASSISTED_AGG_URL="https://stitch-artifacts.s3.amazonaws.com/stitch-mongo-libs/stitch_mongo_libs_osx_patch_1e7861d9b7462f01ea220fad334f10e00f0f3cca_6513254ad6d80abfffa5fbdc_23_09_26_18_39_06/assisted_agg"
GO_URL="https://s3.amazonaws.com/static.realm.io/evergreen-assets/go1.21.1.darwin-arm64.tar.gz"
MONGODB_DOWNLOAD_URL="https://downloads.mongodb.com/osx/mongodb-macos-arm64-enterprise-6.0.0-rc13.tgz"
MONGOSH_DOWNLOAD_URL="https://downloads.mongodb.com/compass/mongosh-1.5.0-darwin-arm64.zip"

Expand All @@ -34,21 +34,28 @@ case $(uname -s) in
export GOMAXPROCS
else
export GOARCH=amd64
STITCH_SUPPORT_LIB_URL="https://s3.amazonaws.com/static.realm.io/stitch-support/stitch-support-macos-4.4.17-rc1-2-g85de0cc.tgz"
STITCH_ASSISTED_AGG_URL="https://stitch-artifacts.s3.amazonaws.com/stitch-mongo-libs/stitch_mongo_libs_osx_patch_75b3f1896aaa2e344817795c8bfc5cb6b2f2c310_632211a5d1fe0757f8c416fa_22_09_14_17_38_46/assisted_agg"
GO_URL="https://s3.amazonaws.com/static.realm.io/evergreen-assets/go1.19.1.darwin-amd64.tar.gz"
STITCH_SUPPORT_LIB_URL="https://stitch-artifacts.s3.amazonaws.com/stitch-support/macos-arm64/stitch-support-4.4.17-rc1-2-g85de0cc.tgz"
STITCH_ASSISTED_AGG_URL="https://stitch-artifacts.s3.amazonaws.com/stitch-mongo-libs/stitch_mongo_libs_osx_patch_1e7861d9b7462f01ea220fad334f10e00f0f3cca_6513254ad6d80abfffa5fbdc_23_09_26_18_39_06/assisted_agg"
GO_URL="https://s3.amazonaws.com/static.realm.io/evergreen-assets/go1.21.1.darwin-amd64.tar.gz"
MONGODB_DOWNLOAD_URL="https://downloads.mongodb.com/osx/mongodb-macos-x86_64-enterprise-5.0.3.tgz"
fi

NODE_URL="https://s3.amazonaws.com/static.realm.io/evergreen-assets/node-v14.17.0-darwin-x64.tar.gz"
JQ_DOWNLOAD_URL="https://s3.amazonaws.com/static.realm.io/evergreen-assets/jq-1.6-darwin-amd64"
;;
Linux)
GO_URL="https://s3.amazonaws.com/static.realm.io/evergreen-assets/go1.19.1.linux-amd64.tar.gz"
GO_URL="https://s3.amazonaws.com/static.realm.io/evergreen-assets/go1.21.1.linux-amd64.tar.gz"
JQ_DOWNLOAD_URL="https://s3.amazonaws.com/static.realm.io/evergreen-assets/jq-1.6-linux-amd64"
NODE_URL="https://s3.amazonaws.com/static.realm.io/evergreen-assets/node-v14.17.0-linux-x64.tar.gz"

# Detect what distro/versionf of Linux we are running on to download the right version of MongoDB to download
# Only x86_64 Linux machines are supported
linux_arch="$(uname -m)"
if [[ "${linux_arch}" != "x86_64" ]]; then
echo "Error: only x86_64 Linux machines are supported: ${linux_arch}"
exit 1
fi

# Detect what distro/version of Linux we are running on to determine the right version of MongoDB to download
# /etc/os-release covers debian/ubuntu/suse
if [[ -e /etc/os-release ]]; then
# Amazon Linux 2 comes back as 'amzn'
Expand All @@ -64,49 +71,49 @@ case $(uname -s) in
case $DISTRO_NAME in
ubuntu | linuxmint)
MONGODB_DOWNLOAD_URL="http://downloads.10gen.com/linux/mongodb-linux-$(uname -m)-enterprise-ubuntu${DISTRO_VERSION_MAJOR}04-5.0.3.tgz"
STITCH_ASSISTED_AGG_LIB_URL="https://stitch-artifacts.s3.amazonaws.com/stitch-mongo-libs/stitch_mongo_libs_ubuntu2004_x86_64_86b48e3cb2a8d5bbf3d18281c9f42c1835bbb83b_22_11_08_03_08_06/libmongo-ubuntu2004-x86_64.so"
STITCH_ASSISTED_AGG_LIB_URL="https://stitch-artifacts.s3.amazonaws.com/stitch-mongo-libs/stitch_mongo_libs_ubuntu2004_x86_64_patch_1e7861d9b7462f01ea220fad334f10e00f0f3cca_65135b432fbabe741bd24429_23_09_26_22_29_24/libmongo-ubuntu2004-x86_64.so"
STITCH_SUPPORT_LIB_URL="https://s3.amazonaws.com/static.realm.io/stitch-support/stitch-support-ubuntu2004-4.4.17-rc1-2-g85de0cc.tgz"
;;
rhel)
case ${DISTRO_VERSION_MAJOR} in
7)
MONGODB_DOWNLOAD_URL="https://downloads.mongodb.com/linux/mongodb-linux-x86_64-enterprise-rhel70-5.0.3.tgz"
STITCH_ASSISTED_AGG_LIB_URL="https://stitch-artifacts.s3.amazonaws.com/stitch-mongo-libs/stitch_mongo_libs_linux_64_86b48e3cb2a8d5bbf3d18281c9f42c1835bbb83b_22_11_08_03_08_06/libmongo.so"
STITCH_SUPPORT_LIB_URL="https://s3.amazonaws.com/static.realm.io/stitch-support/stitch-support-rhel70-4.4.17-rc1-2-g85de0cc.tgz"
STITCH_ASSISTED_AGG_LIB_URL="https://stitch-artifacts.s3.amazonaws.com/stitch-mongo-libs/stitch_mongo_libs_linux_64_patch_1e7861d9b7462f01ea220fad334f10e00f0f3cca_65135b432fbabe741bd24429_23_09_26_22_29_24/libmongo.so"
STITCH_SUPPORT_LIB_URL="https://stitch-artifacts.s3.amazonaws.com/stitch-support/linux-x64/stitch-support-4.4.17-rc1-2-g85de0cc.tgz"
;;
*)
echo "Unsupported version of RHEL ${DISTRO_VERSION}"
echo "Error: unsupported version of RHEL ${DISTRO_VERSION}"
exit 1
;;
esac
;;
*)
if [[ -z "${MONGODB_DOWNLOAD_URL}" ]]; then
echo "Missing MONGODB_DOWNLOAD_URL env variable to download mongodb from."
echo "Error: missing MONGODB_DOWNLOAD_URL env variable to download mongodb from."
exit 1
fi
if [[ -z "${STITCH_ASSISTED_AGG_LIB_PATH}" ]]; then
echo "Missing STITCH_ASSISTED_AGG_LIB_PATH env variable to find assisted agg libmongo.so"
echo "Error: missing STITCH_ASSISTED_AGG_LIB_PATH env variable to find assisted agg libmongo.so"
exit 1
fi
if [[ -z "${STITCH_SUPPORT_LIB_PATH}" ]]; then
echo "Missing STITCH_SUPPORT_LIB_PATH env variable to find the mongo stitch support library"
echo "Error: missing STITCH_SUPPORT_LIB_PATH env variable to find the mongo stitch support library"
exit 1
fi
;;
esac
;;
*)
if [[ -z "${MONGODB_DOWNLOAD_URL}" ]]; then
echo "Missing MONGODB_DOWNLOAD_URL env variable to download mongodb from."
echo "Error: missing MONGODB_DOWNLOAD_URL env variable to download mongodb from."
exit 1
fi
if [[ -z "${STITCH_ASSISTED_AGG_LIB_PATH}" ]]; then
echo "Missing STITCH_ASSISTED_AGG_LIB_PATH env variable to find assisted agg libmongo.so"
echo "Error: missing STITCH_ASSISTED_AGG_LIB_PATH env variable to find assisted agg libmongo.so"
exit 1
fi
if [[ -z "${STITCH_SUPPORT_LIB_PATH}" ]]; then
echo "Missing STITCH_SUPPORT_LIB_PATH env variable to find the mongo stitch support library"
echo "Error: missing STITCH_SUPPORT_LIB_PATH env variable to find the mongo stitch support library"
exit 1
fi
exit 1
Expand Down Expand Up @@ -453,8 +460,9 @@ echo "Adding fake appid to skip baas server drop optimization"
# Start the baas server on port *:9090 with the provided config JSON files
echo "Starting baas app server"

# see config overrides at https://github.com/10gen/baas/blob/master/etc/configs/test_rcore_config.json
"${WORK_PATH}/baas_server" \
--configFile=etc/configs/test_config.json --configFile="${BASE_PATH}/config_overrides.json" > "${BAAS_SERVER_LOG}" 2>&1 &
--configFile=etc/configs/test_config.json --configFile=etc/configs/test_rcore_config.json > "${BAAS_SERVER_LOG}" 2>&1 &
Comment on lines -457 to +465
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tyler requested that the BAAS team maintain the config overrides for our tests so that they can keep them up to date with server features. I've removed our local overrides file evergreen/config_overrides.json and now this points to a config in their repo https://github.com/10gen/baas/blob/master/etc/configs/test_rcore_config.json

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! I will eventually move our dependency definitions (e.g. STITCH_SUPPORT_LIB_URL) to use the baas definitions as well, but they need to add the definitions for Ubuntu before this happens.

echo $! > "${BAAS_PID_FILE}"

WAIT_BAAS_OPTS=()
Expand All @@ -472,7 +480,7 @@ ${CURL} 'http://localhost:9090/api/admin/v3.0/auth/providers/local-userpass/logi
--silent \
--fail \
--output /dev/null \
--data-raw '{"username":"[email protected]","password":"password"}'
--data '{"username":"[email protected]","password":"password"}'

"${MONGO_BINARIES_DIR}/bin/${MONGOSH}" --quiet mongodb://localhost:26000/auth "${BASE_PATH}/add_admin_roles.js"

Expand All @@ -483,4 +491,4 @@ echo "---------------------------------------------"
echo "Baas server ready"
echo "---------------------------------------------"
wait
popd > /dev/null # baas
popd > /dev/null # baas
3 changes: 3 additions & 0 deletions src/realm/sync/client.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1137,6 +1137,9 @@ SessionWrapper::SessionWrapper(ClientImpl& client, DBRef db, std::shared_ptr<Sub
REALM_ASSERT(m_db);
REALM_ASSERT(m_db->get_replication());
REALM_ASSERT(dynamic_cast<ClientReplication*>(m_db->get_replication()));
if (m_client_reset_config) {
m_session_reason = SessionReason::ClientReset;
}

update_subscription_version_info();
}
Expand Down
8 changes: 6 additions & 2 deletions test/object-store/sync/flx_migration.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -441,8 +441,12 @@ TEST_CASE("Test client migration and rollback with recovery", "[sync][flx][flx m
// Migrate back to FLX - and keep the realm session open
trigger_server_migration(session.app_session(), MigrateToFLX, logger_ptr);

REQUIRE(!wait_for_upload(*outer_realm));
REQUIRE(!wait_for_download(*outer_realm));
// wait for the subscription store to initialize after downloading
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of polling, one other way to fix this (I think) is to commit a change and keep the completion callbacks.

timed_wait_for(
[&outer_realm]() {
return outer_realm->sync_session() && outer_realm->sync_session()->get_flx_subscription_store();
},
std::chrono::seconds(180));

// Verify data has been sync'ed and there is only 1 subscription for the Object table
{
Expand Down
Loading