Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iterating in reverse Order - Empty Content #347

Closed
rebootcode opened this issue Dec 10, 2017 · 7 comments
Closed

Iterating in reverse Order - Empty Content #347

rebootcode opened this issue Dec 10, 2017 · 7 comments
Labels
kind/question Something requiring a response

Comments

@rebootcode
Copy link

rebootcode commented Dec 10, 2017

I am trying to Iterate in Reverse Order, but every time, I set Reverse to true. It gives empty value.

Below is code -

txn := db.NewTransaction(false)
defer txn.Discard()
err = db.View(func(txn *badger.Txn) error {
	DefaultIteratorOptions := badger.DefaultIteratorOptions
	DefaultIteratorOptions.Reverse = true
	it = txn.NewIterator(DefaultIteratorOptions)
	key := strconv.Itoa(t.Year()) + "-" + strconv.Itoa(int(t.Month())) + "-" + strconv.Itoa(int(t.Day()))
	prefix := []byte(key)
	for it.Seek(prefix); it.ValidForPrefix(prefix); it.Next() {
		item := it.Item()
		k := item.Key()
		v, err := item.ValueCopy([]byte(k))
		if err != nil {
			ctx.WriteString("Content not available")
			return err
		}
		blogList, err := decodeBlog(v)
		if err != nil {
			fmt.Println("Error: decode: " + err.Error())
		}
		buffer.WriteString("My Content First")
	}
})
@janardhan1993
Copy link

When you seek for a key in reverse iterator, we find first key which is less than or equal to the seek key.
Say you have a list of keys

  1. "a:key1"
  2. "b:key2"
  3. "b:key3"
  4. "b:key4"
    when you seek for "b:" in reverse mode and iterate you will see only key "a:key1" and since you are doing validForPrefix check you won't see any data.

If you want all values for that prefix in reverse mode you should seek for
prefixKey := append([]byte("prefix"), 0xFF)

@janardhan1993 janardhan1993 added the kind/question Something requiring a response label Dec 11, 2017
@peterstace
Copy link

Just be careful of the case where you have keys of the form "prefix", 0xff, ... some other bytes ....

When seeking to the initial iterator position, it would be more correct to seek to the prefix but with the last byte incremented. So if your prefix is "prefix", then you should seek to "prefiy". (Watch out for the case where the last byte in the prefix is 0xff... you'll have to do something a bit tricker to avoid edge cases then).

@peterstace
Copy link

This whole thing seems a bit tricky for users to get right. Maybe we need a SeekAfter method?

@rebootcode
Copy link
Author

rebootcode commented Dec 11, 2017

When you seek for a key in reverse iterator, we find first key which is less than or equal to the seek key.
Say you have a list of keys

"a:key1"
"b:key2"
"b:key3"
"b:key4"
when you seek for "b:" in reverse mode and iterate you will see only key "a:key1" and since you are doing validForPrefix check you won't see any data.
If you want all values for that prefix in reverse mode you should seek for
prefixKey := append([]byte("prefix"), 0xFF)

Wow, I just missed the whole context. Isn't reverse suppose to iterate the value for key b in reverse order like b:key4, b:key3, b:key2 ?

I tried, prefixing 0xFF too like below, yet i see empty results.

txn := db.NewTransaction(false)
		defer txn.Discard()
		err = db.View(func(txn *badger.Txn) error {
			DefaultIteratorOptions := badger.DefaultIteratorOptions
			DefaultIteratorOptions.Reverse = true
			it = txn.NewIterator(DefaultIteratorOptions)

			key := strconv.Itoa(t.Year()) + "-" + strconv.Itoa(int(t.Month())) + "-" + strconv.Itoa(int(t.Day()))

			prefix := append([]byte(key), 0xFF)
			for it.Seek(prefix); it.ValidForPrefix(prefix); it.Next() {
				item := it.Item()
				k := item.Key()
				v, err := item.ValueCopy([]byte(k))
				if err != nil {
					ctx.WriteString("Content not available")
					return err
				}
				blogList, err := decodeBlog(v)
				if err != nil {
					fmt.Println("Error: decode: " + err.Error())
				}
				buffer.WriteString("My Content First")
			}
                })

@manishrjain
Copy link
Contributor

Just right out of bat, I find this strange:

				k := item.Key()
				v, err := item.ValueCopy([]byte(k))

Reusing an internal byte slice from item would cause weird issues -- and the one you're seeing might be related. So, fix that, and then re-run the iterator. Also, have a look at the tests which do reverse iteration, and see how they achieve the same results.

@rebootcode
Copy link
Author

rebootcode commented Dec 11, 2017

@manishrjain, Ok, the only other option I see is:-

k := item.Key()
v, err := item.Value()

But I read in FAQ https://github.com/dgraph-io/badger#frequently-asked-questions, where it says, If writes are getting stuck, then it's better to use Use Item::ValueCopy instead of Item::Value when retrieving a value.

Is there any other hack around available where reuse can be fixed.

Update: Even after using item.Value() the results are empty.

@rebootcode
Copy link
Author

The new GetSequence method is more easy to adopt for my requirement, I am switching my option to nextSequence approach like an update here #346

Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/question Something requiring a response
Development

No branches or pull requests

4 participants