315: Rewrite the indexing benchmarks r=Kerollmops a=irevoire
There was a panic on the benchmark and while I was trying to understand what was happening I decided to rewrite the way the benchmarks were working.
Before we were creating a database with the good setting, and then for each benchmarks we were:
1. Deleting all documents in the database
2. Indexing a batch of documents
Now for each iteration we recreate entirely a new database from scratch.
Since deleting all the documents in a database may not be the same as starting with a fresh new database I prefer this solution.
Co-authored-by: Irevoire <tamo@meilisearch.com>
320: Sort at query time r=Kerollmops a=Kerollmops
Re-introduce the Sort at the query time (https://github.com/meilisearch/milli/issues/305)
Co-authored-by: Clément Renault <renault.cle@gmail.com>
317: Fix the facet string docids filterable deletion bug r=Kerollmops a=Kerollmops
Fixes a bug where the deletion of documents was returning a decoding error. But only when the settings are set with filterable attributes.
This bug was introduced in #254 in which we made the engine faster in returning the facet distribution. We changed the way we were storing the inverted index, we were no more storing only documents ids with the original values but also groups identified with integers, depending on the facet level we were using. This is similar to how facet numbers are already stored.
⚠️ As `@curquiza` already said, we must first revert #309 before merging this!
Related to https://github.com/meilisearch/MeiliSearch/issues/1601.
Co-authored-by: Clément Renault <clement@meilisearch.com>
318: Revert "Sort at query time" r=Kerollmops a=curquiza
Reverts meilisearch/milli#309
We revert this from `main` not because this leads to a bug, but because we don't want to release it now and we have to merge and release an hotfix on `main`.
Cf:
- https://github.com/meilisearch/milli/issues/316
- https://github.com/meilisearch/milli/pull/317
Once the v0.21.0 is released, we should merge again this awesome addition 👌
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
309: Sort at query time r=Kerollmops a=Kerollmops
This PR:
- Makes the `Asc/Desc` criteria work with strings too, it first returns documents ordered by numbers then by strings, and finally the documents that can't be ordered. Note that it is lexicographically ordered and not ordered by character, which means that it doesn't know about wide and short characters i.e. `a`, `丹`, `▲`.
- Changes the syntax for the `Asc/Desc` criterion by now using a colon to separate the name and the order i.e. `title:asc`, `price:desc`.
- Add the `Sort` criterion at the third position in the ranking rules by default.
- Add the `sort_criteria` method to the `Search` builder struct to let the users define the `Asc/Desc` sortable attributes they want to use at query time. Note that we need to check that the fields are registered in the sortable attributes before performing the search.
- Introduce a new `InvalidSortableAttribute` user error that is raised when the sort criteria declared at query time are not part of the sortable attributes.
- `@ManyTheFish` introduced integration tests for the dynamic Sort criterion.
Fixes#305.
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: many <maxime@meilisearch.com>
307: Update version for the next release (v0.10.0) r=Kerollmops a=curquiza
Replaces https://github.com/meilisearch/milli/pull/304
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
302: Update milli to v0.9.0 r=curquiza a=curquiza
Updating the minor and not patch since #300 seems to be breaking: it involves a re-indexation to get the fix, so it involves an additional step from the users, not only downloading the latest version.
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
300: Fix prefix level position docids database r=curquiza a=ManyTheFish
The prefix search was inverted when we generated the DB.
Instead of searching if word had a prefix in prefix fst,
we were searching if the word was a prefix of a prefix contained in the prefix fst.
The indexer, now, iterate over prefix contained in the fst
and search them by prefix in the word-level-position-docids database,
aggregating matches in a sorter.
Fix#299
Co-authored-by: many <maxime@meilisearch.com>
The prefix search was inverted when we generated the DB.
Instead of searching if word had a prefix in prefix fst,
we were searching if the word was a prefix of a prefix contained in the prefix fst.
The indexer, now, iterate over prefix contained in the fst
and search them by prefix in the word-level-position-docids database,
aggregating matches in a sorter.
Fix#299
298: Rename the search benchmarks r=Kerollmops a=irevoire
And fix a bug. As always, I was not closing the env.
Co-authored-by: Tamo <tamo@meilisearch.com>
296: Fix invalid faceted documents ids buffer size r=Kerollmops a=Kerollmops
Fix a bug found by `@irevoire` when benchmarking the search.
Co-authored-by: Kerollmops <clement@meilisearch.com>
