Add API to mount a snapshot as a searchable index#53084
Add API to mount a snapshot as a searchable index#53084DaveCTurner merged 27 commits intoelastic:feature/searchable-snapshotsfrom
Conversation
This commit adds an API for mounting a snapshot as a searchable index.
Internally, a request is made to the repository to fetch metadata information necessary from the
snapshot in question, which is then passed to the regular snapshot restore mechanism with some
specialized settings.
Assume you have an existing snapshot repository and snapshot, the API looks like:
```json
POST /my-new-index/_snapshot/mount?wait_for_completion=true
{
"repository": "my-s3-repo",
"snapshot": "production-2019-12-11-aoeufao",
"snapshot_index": "important-data",
"index_settings": {
"index.number_of_replicas": 2
},
"ignore_index_settings": ["index.refresh_interval"]
}
```
While the request may appear similar to a snapshot restore request, it has slight differences. The
`repository`, `snapshot`, and `snapshot_index` fields are required (all other fields are optional).
The response is identical to the response when restoring a snapshot.
Relates to elastic#50999
|
Pinging @elastic/es-distributed (:Distributed/Snapshot/Restore) |
|
Note for reviewers: this adds the API, but I tried to make as minimal changes as possible to the rest of the searchable snapshot infrastructure. This does not remove the "searchable" repository type, and only changes a few assertions made about the repository type where necessary to have mounting work with no issues. |
tlrx
left a comment
There was a problem hiding this comment.
Thanks@dakrone, and thanks for having minimized the changes. I made a first bunch of comments (I didn't look at the tests yet).
.../src/main/java/org/elasticsearch/xpack/searchablesnapshots/SearchableSnapshotRepository.java
Outdated
Show resolved
Hide resolved
...n/java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotAction.java
Show resolved
Hide resolved
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Outdated
Show resolved
Hide resolved
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Outdated
Show resolved
Hide resolved
...g/elasticsearch/xpack/searchablesnapshots/action/TransportMountSearchableSnapshotAction.java
Outdated
Show resolved
Hide resolved
...g/elasticsearch/xpack/searchablesnapshots/action/TransportMountSearchableSnapshotAction.java
Show resolved
Hide resolved
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Outdated
Show resolved
Hide resolved
...g/elasticsearch/xpack/searchablesnapshots/action/TransportMountSearchableSnapshotAction.java
Outdated
Show resolved
Hide resolved
DaveCTurner
left a comment
There was a problem hiding this comment.
Thanks @dakrone, I left a few small comments too.
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Outdated
Show resolved
Hide resolved
.../src/main/java/org/elasticsearch/xpack/searchablesnapshots/SearchableSnapshotRepository.java
Outdated
Show resolved
Hide resolved
...java/org/elasticsearch/xpack/searchablesnapshots/rest/RestMountSearchableSnapshotAction.java
Outdated
Show resolved
Hide resolved
|
@elasticmachine please run elasticsearch-ci/bwc |
tlrx
left a comment
There was a problem hiding this comment.
This looks good, thanks for picking this work David. I've left some comments, but nothing to worry about.
...n/java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotAction.java
Show resolved
Hide resolved
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Show resolved
Hide resolved
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Outdated
Show resolved
Hide resolved
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Outdated
Show resolved
Hide resolved
.../java/org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequest.java
Outdated
Show resolved
Hide resolved
...g/elasticsearch/xpack/searchablesnapshots/action/TransportMountSearchableSnapshotAction.java
Outdated
Show resolved
Hide resolved
...g/elasticsearch/xpack/searchablesnapshots/action/TransportMountSearchableSnapshotAction.java
Show resolved
Hide resolved
...g/elasticsearch/xpack/searchablesnapshots/action/TransportMountSearchableSnapshotAction.java
Outdated
Show resolved
Hide resolved
...g/elasticsearch/xpack/searchablesnapshots/action/TransportMountSearchableSnapshotAction.java
Outdated
Show resolved
Hide resolved
.../org/elasticsearch/xpack/searchablesnapshots/action/MountSearchableSnapshotRequestTests.java
Outdated
Show resolved
Hide resolved
|
Everything addressed @tlrx, I think, this is ready for another look. |
tlrx
left a comment
There was a problem hiding this comment.
LGTM - if CI is happy, I'll be happy
This commit adds an API for mounting a snapshot as a searchable index.
Internally, a request is made to the repository to fetch metadata information necessary from the
snapshot in question, which is then passed to the regular snapshot restore mechanism with some
specialized settings.
Assume you have an existing snapshot repository and snapshot, the API looks like:
While the request may appear similar to a snapshot restore request, it has slight differences. The
repository,snapshot, andsnapshot_indexfields are required (all other fields are optional).The response is identical to the response when restoring a snapshot. In the above example the newly
restored/mounted index will be called
my-new-index.Relates to #50999