ElasticSearch–Should clause

A colleague asked me for help when his ElasticSearch query wasn’t returning the results he expected.

Let me first explain what he tries to achieve using a simplified example:

• In our search index we have products that are linked to a specific category.
• We want to search for specific product values AND only return results for the categories we specified
• In SQL this would translate to a query similar to the following:

‘SELECT * FROM PRODUCTS WHERE (ProductBrand=’Apple’ OR ProductName=’Apple’ ) AND CATEGORY=’Hardware’’ 

Here is the first attempt to write this using NEST:

We noticed however that instead of executing an AND, we got OR behavior instead. We got all products back where category is ‘Hardware’ but not limited to ‘Apple’ products.

Why is this happening?

First important to notice is that the query above does a full text search and is not filter query. Combining multiple clauses with and, or and not logic becomes more subtle. Queries decide not only whether to include a document, but also how relevant that document is.

ElasticSearch interpretes the query above like this:

• The must clause says: Please return me all documents where the Category is ‘Hardware’
• The should clause says: A document is not required to contain ‘Apple’ but if it does, then it should be considered more relevant.

To fix this, you can either switch to a filter instead of a full text search or specify a minimum_should_match level.

From the documentation:

By default, none of the should clauses are required to match, with one exception: if there are no must clauses, then at least one should clause must match.

Just as we can control the precision of the match query, we can control how many should clauses need to match by using the minimum_should_match parameter, either as an absolute number or as a percentage.

Wat je eigenlijk wil is dat minstens 1 should clause een resultaat teruggeeft. Dit kan je doen door de "minimum_should_match" instelling te manipuleren:

Here is the fixed query: