Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Snapshot Create API #533

Merged
merged 2 commits into from
May 29, 2017
Merged

Add Snapshot Create API #533

merged 2 commits into from
May 29, 2017

Conversation

wedneyyuri
Copy link
Contributor

This PR adds support for Elasticsearch's snapshot create API.

Example program:

package main

import (
	"context"
	"log"
	"os"
	"time"

	elastic "gopkg.in/olivere/elastic.v5"
)

func main() {
	client, err := elastic.NewClient()
	if err != nil {
		panic(err)
	}

	// create snapshot repo
	repo := elastic.
		NewSnapshotCreateRepositoryService(client).
		Repository("repo1").
		BodyJson(map[string]interface{}{
			"type": "fs",
			"settings": map[string]interface{}{
				"location": "/tmp/elasticsearch",
			},
		})

	repoResponse, err := repo.Do(context.TODO())
	if err != nil {
		panic(err)
	}

	if !repoResponse.Acknowledged {
		panic("could not ack repository create")
	}

	// create snapshot
	service := elastic.
		NewSnapshotCreateService(client).
		Repository("repo1").
		Snapshot(time.Now().Format("2006-01-02-15-04-05")).
		WaitForCompletion(true).
		BodyJson(map[string]interface{}{
			"indices":              "website",
			"ignore_unavailable":   false,
			"include_global_state": true,
		})

	response, err := service.Do(context.TODO())
	if err != nil {
		panic(err)
	}

	if response.Snapshot.State != "SUCCESS" {
		panic("elasticsearch snapshot state != SUCCESS: " + response.Snapshot.State)
	}
}

"net/url"
"time"

"github.com/olivere/elastic/uritemplates"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This must be "gopkg.in/olivere/elastic.v5/uritemplates". The generator need some polish, I'm sorry.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No problem, the generator is pretty good 👍

Accepted *bool `json:"accepted"`

// Snapshot is available when waitForCompletion is true.
Snapshot *struct {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The *struct, although valid, doesn't look that good. Does ES always send back the snapshot? We have to look it up in the Java source. If it does, we should make it a normal struct, not a pointer to a struct.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Snapshot *struct {
Snapshot string `json:"snapshot"`
UUID string `json:"uuid"`
VersionID int `json:"version_id"`
Copy link
Owner

@olivere olivere May 20, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you compare these fields with what ES really sends, or did you make an educated guess? I typically consult the Java source to see what fields (and their types) ES really sends and, more often than not, I find fields that are undocumented but sent anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On my previous commit I compared these field with what ES really sends (turning logging on), but today I made a new commit consulting the java source code.

StartTimeInMillis int64 `json:"start_time_in_millis"`
EndTime time.Time `json:"end_time"`
EndTimeInMillis int64 `json:"end_time_in_millis"`
DurationInMillis int `json:"duration_in_millis"`
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This e.g. should be int64 if StartTimeInMillis is int64 as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

EndTimeInMillis int64 `json:"end_time_in_millis"`
DurationInMillis int `json:"duration_in_millis"`
Failures []interface{} `json:"failures"`
Shards struct {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a shardsInfo in errors.go that you can use here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@olivere
Copy link
Owner

olivere commented May 20, 2017

The PR is looking good. Can you please take a look at those comments, so I can merge in the next run. I'm off for vacation next week so it'll take a few more days, I'm afraid.

@wedneyyuri
Copy link
Contributor Author

Thanks @olivere, I looked at your comments and pushed some changes.

@olivere olivere merged commit fb33dee into olivere:release-branch.v5 May 29, 2017
@olivere
Copy link
Owner

olivere commented May 29, 2017

It'll roll with 5.0.39. Thanks.

@wedneyyuri wedneyyuri deleted the feature/snapshot_create_api branch May 29, 2017 12:38
@wedneyyuri
Copy link
Contributor Author

Thanks @olivere, can I backport it to v3?

@olivere
Copy link
Owner

olivere commented May 29, 2017

@wedneyyuri Sure. That seems to be really easy. All you need to do is change some import paths.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants