DeleteByQuery not working for same query used in Search

[PoOoZaQ]

Hey there,
Possibly i'm not writing the correct code, but I have a query I use to get docs using Search, and the same query doesn't work for DeleteByQuery.

Works:

var results = client.Search<JObject>(s => s.Query(q => q.Terms("string.ProgramId", ids))

Doesn't work:

var response = client.DeleteByQuery<string>(s => s.Query(q => q.Terms("string.ProgramId", ids)));

• ids is List

The error i'm getting is:

Could not dispatch IElasticClient.DeleteByQuery() into any of the following paths:

• /{index}/_query
• /{index}/{type}/_query

Any ideas on what I'm doing wrong?

Thanks!

[Mpdreamz]

When you are using destructive endpoints such as DELETE you will need to be explicit about your index selection. The NEST client follows the same semantics as elasticsearch here see:

http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/_indices_apis_2.html

i.e:

client.DeleteByQuery<>(s=>s.AllIndices().Query());

[PoOoZaQ]

So is adding s.AllIndices() like this:

var response = client.DeleteByQuery(s => s.AllIndices().Query(q => q.Terms("string.ProgramId", ids)));

Should do the trick, right?
Now it doesn't throw an exception, but returns 0 found documents.
Thats quiet odd since as I said the Search query does return results.

[Mpdreamz]

Your search uses JObjects while your delete uses strings by default nest uses the generic type you pass in to set up some defaults. Our documentation on inference: http://nest.azurewebsites.net/nest/index-type-inference.html fully explains this behavior.

In your search:

var results = client.Search<JObject>(s => s.Query(q => q.Terms("string.ProgramId", ids))

Searches /default_index_for_jobject/default_typename_for_jobject/_search

Your delete uses string for documents

var response = client.DeleteByQuery<string>(s => s.Query(q => q.Terms("string.ProgramId", ids)));

Deletes on this url:

/default_index_for_string/default_typename_for_string/_query

By specifying AllIndices() you change the url to:

/_all/default_typename_for_string/_query

You can use AllTypes() in conjunction with AllIndices to force the delete to happen on

/_all/_query

But to get the most consistent experience use the same model type as the generic argument to both calls.

Hope this makes sense.

[PoOoZaQ]

Thanks for the replies!

I've tried what you suggested, first I changed the type of the delete from string to jobject, and afterwards tried adding AllTypes after AllIndices, but still both returned 0 results.

These are my tries:

var response = _client.DeleteByQuery<string>(s => s
    .AllIndices().AllTypes().Query(q => q.Terms("string.ProgramId", ids)));

var response = _client.DeleteByQuery<JObject>(s => s
    .AllIndices().Query(q => q.Terms("string.ProgramId", ids)));

var response = _client.DeleteByQuery<JObject>(s => s
    .AllIndices().AllTypes().Query(q => q.Terms("string.ProgramId", ids)));

[edit]
An important note: my search doesn't return any results with AllIndices / with specific Index I know the results are there, as well. No idea why... When I search without specifying an index I get results.

[Mpdreamz]

Can you paste response.ConnectionStatus.RequestUrl for your search and your deletes?

Also be sure to check that response.IsValid is true for all.

var response = _client.DeleteByQuery<JObject>(s => s
    .Query(q => q.Terms("string.ProgramId", ids)));

Should work and resolve to the same url as your search

I've attached a commit to show what urls they should resolve to:

acfb02b

[PoOoZaQ]

The response.ConnectionStatus.RequestUrl is:

http://localhost:9200/_all/_query

[PoOoZaQ]

Well after a lot of digging and dumb luck ended up noticing that the Delete actually happens! The response returns faulty though, saying it didn't do anything.

Would still appreciate knowing and learning how its possible the delete happens but the response nothing happened.
And also, why does the Search not working with a specific index (or AllIndices())

Thanks

[Mpdreamz]

Hi @PoOoZaQ to my horror i see the response object for DeleteByQuery is blatently wrong, it uses the same response object as Delete() which is simply not true.

The response you'll get from a DeleteByQuery looks like:

{
  "_indices" : {
    "nest_test_data-792_24733294-5f49-4ea4-8627-1905632ad005" : {
      "_shards" : {
        "total" : 2,
        "successful" : 2,
        "failed" : 0
      }
    }
  }
}

I assume you are inspecting the .Found property on the response which is never actually set to true for delete by query requests. You can use IsValid property to validate for now which is what all our integration tests use and why we did not pick up on the wrong mapping for so long.

We will work on mapping _indices to DeleteResponse for now in NEST 1.x and properly split the two responses in NEST 2.0

Sorry this caused you so much pain today.

PS do you mind opening a new ticket for your search question?