254: Improve the facet string distribution speed r=Kerollmops a=Kerollmops
This pull request creates a data structure similar to the one we use for the faceted numbers, a tetratomic decision tree but this time for the facet strings. This PR also changes the facet distribution behavior by returning one of the original facet values, fixes#260.
This data structure defines bucket-like structures where documents ids are stored under their facet value and helps the search decide if it wants to move to a lower level under a given bucket or not, depending on if the current bucket contains interesting documents or not. The whole format, algorithm, and previous attempts are explained in the [`facet_string.rs` file](ec1cfdd42b/milli/src/search/facet/facet_string.rs).
Note that this data structure **could** be used to sort by string lexicographically, that hypothetically possible. We need more testing, in terms of performance and quality, as we will sort on lowercased versions of the facet values.
- [x] Implement a faster and more precise way to fetch the facet distribution.
- [x] Store and return the original facet string value. We currently return the lowercased version.
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
1521: Sentry was never sending anything r=Kerollmops a=irevoire
@Kerollmops noticed that we had no log of this release in sentry, and it look like I badly tested my code after ignoring the “No space left on device” errors.
Now it should be fixed.
Co-authored-by: Tamo <tamo@meilisearch.com>
290: Add a $HOME to the CI r=Kerollmops a=irevoire
This should fix this issue:
https://github.com/meilisearch/milli/runs/3104228432?check_suite_focus=true
I think a real fix would be to fix the configuration of our github runner but I don't know how to do it.
@curquiza could probably help us on that once she's back from vacation 😄
Co-authored-by: Tamo <tamo@meilisearch.com>
1498: Show the filterable and not the faceted attributes in the settings r=Kerollmops a=Kerollmops
Fixes#1497
Co-authored-by: Clément Renault <clement@meilisearch.com>
1484: Add MeiliSearch version to issue template r=irevoire a=bidoubiwa
It is relevant to know the version of MeiliSearch before any other additional information that might be important to know.
We could also reduce the number of required information asked to the user. I would like to suggest the following:
Instead of the section of `Desktop` and `Smartphone` I would just improve the last section
```
**Additional context**
Additional information that may be relevant to the issue.
[e.g. architecture, device, OS, browser]
```
By applying this, the template final look will be the following:
-----
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**MeiliSearch version:** [e.g. v0.20.0]
**Additional context**
Additional information that may be relevant to the issue.
[e.g. architecture, device, OS, browser]
Co-authored-by: Charlotte Vermandel <charlottevermandel@gmail.com>
287: Add benchmarks for indexing r=Kerollmops a=irevoire
closes#274
I don't really know how much time this will take on our bench machine. I'm afraid the wiki dataset will take a really long time to bench (it takes 1h30 on my computer).
If you are ok with it, I would like to merge this first PR since it introduces a first set of benchmarks and see how much time it takes in reality on our setup.
Co-authored-by: Tamo <tamo@meilisearch.com>
285: Support documents with at most 65536 fields r=Kerollmops a=Kerollmops
Fixes#248.
In this PR I updated the `obkv` crate, it now supports arbitrary key length and therefore I was able to use an `u16` to represent the fields instead of a single byte. It was impressively easy to update the whole codebase 🍡🍔
Co-authored-by: Kerollmops <clement@meilisearch.com>
1481: fix bug in index deletion r=Kerollmops a=MarinPostma
this bug was caused by a heed iterator entry being deleted while still holding a reference to it.
close#1333
Co-authored-by: mpostma <postma.marin@protonmail.com>
283: Use the AlwaysFreePages flag when opening an index r=irevoire a=Kerollmops
We introduced a new flag in our fork of LMDB, this `AlwaysFreePages` flag forces LMDB to always free the single pages it uses before writing to the disk instead of keeping them in a linked list.
Declaring this flag reduces the memory print (leak) we have on memory after indexing a lot of documents.
Fixes#279.
Co-authored-by: Kerollmops <clement@meilisearch.com>
284: [http-ui] Introduce the route `die` r=Kerollmops a=irevoire
This route just `exit` the process. This can come in handy when you run `http-ui` inside of another process (a profiler for example), and you don't want to kill everything
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Irevoire <tamo@meilisearch.com>
