Commit Graph

8139 Commits

Author SHA1 Message Date
Clémentine Urquizar
e9104a0a32
Add --locked in CI tests 2021-06-03 16:23:59 +02:00
Clémentine Urquizar
70229f07c8
Update Cargo.lock 2021-06-03 16:22:43 +02:00
bors[bot]
ee7d291442
Merge #213
213: Fix the benchmarks script and names r=Kerollmops a=Kerollmops

The benchmarks compare script was not using the `--output` flag and was therefore failing the download of the JSON reports. We also modified the criterion benchmarks to use shorter names, it helps in looking at the benchmarks in the terminal.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2021-06-03 14:18:45 +00:00
Kerollmops
29824d05ab
Reduce the length of the benchmarks names 2021-06-03 15:59:43 +02:00
Kerollmops
76a2343639
Fix the compare script of the benchmarks 2021-06-03 15:39:52 +02:00
many
10882bcbce
Introduce integration test on criteria 2021-06-03 14:44:53 +02:00
bors[bot]
a32236c80c
Merge #211
211: Update Cargo.toml for next release v0.3.0 r=Kerollmops a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-03 10:42:52 +00:00
Clémentine Urquizar
3b2b3aeea9
Update Cargo.toml for next release v0.3.0 2021-06-03 12:24:27 +02:00
bors[bot]
39ed133f9f
Merge #193
193: Fix primary key behavior r=Kerollmops a=MarinPostma

this pr:
- Adds early returns on empty document additions, avoiding error messages to be returned when adding no documents and no primary key was set.
- Changes the primary key inference logic to match that of legacy meilisearch.

close #194 

Co-authored-by: Marin Postma <postma.marin@protonmail.com>
Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-03 10:24:21 +00:00
bors[bot]
fd598f060c
Merge #210
210: Check the benchmarks in the CI r=Kerollmops a=Kerollmops

Fixes #209.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2021-06-03 09:16:06 +00:00
Kerollmops
99b45d2aa0
Make sure that all the workspaces crates compile 2021-06-03 10:56:01 +02:00
marin postma
57898d8a90
fix silent deserialize error 2021-06-03 10:42:55 +02:00
Kerollmops
82fb5f0bef
Fix the benchmarks compilation 2021-06-03 10:33:42 +02:00
Kerollmops
6b7841fefc
Make sure that the benchmarks always compile 2021-06-03 10:29:21 +02:00
bors[bot]
9f40896f4a
Merge #175
175: Fix update loop infinite loop r=irevoire a=MarinPostma

fix update loop infinite loop in case of udpate error.

close #169


Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-02 23:02:10 +00:00
bors[bot]
834504aec0
Merge #204
204: Decorrelate Distinct, Asc/Desc, Filterable fields from the faceted fields r=Kerollmops a=Kerollmops

This PR decorrelates the fields that need to be stored in facet databases (big inverted indexes for fast access) from the filterable fields, the previously named faceted fields are now named filterable fields and are the union of the distinct attribute, all the Asc/Desc criteria and, the filterable fields.

I added two tests to make sure that the engine was correctly generating the faceted databases when a distinct attribute or an Asc/Desc criteria were added, and one to make sure that it was impossible to filter on a non-filterable field even if it was a faceted one.

Note that the `AttributesForFacetting` has also been renamed into `FilterableAttributes`. But it will be the Transplant's job to do that on the API, this change is only visible to the milli's library users.

- Related to https://github.com/meilisearch/transplant/issues/187.
- Fixes #161 by returning the documents that don't have the Asc/Desc field at the end of the bucket.
- Fixes #168.
- Fixes #152.

Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: Marin Postma <postma.marin@protonmail.com>
Co-authored-by: many <maxime@meilisearch.com>
2021-06-02 15:43:39 +00:00
many
26a9974667
Make asc/desc criterion return resting documents
Fix #161.2
2021-06-02 17:41:48 +02:00
marin postma
75c0718691
fix update loop infinite loop 2021-06-02 17:29:50 +02:00
bors[bot]
28962bce99
Merge #207
207: Benchmarks r=Kerollmops a=irevoire



Co-authored-by: tamo <tamo@meilisearch.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: Tamo <irevoire@hotmail.fr>
Co-authored-by: Irevoire <tamo@meilisearch.com>
2021-06-02 15:29:09 +00:00
Tamo
6dc08bf45e
remove the nop function 2021-06-02 17:09:21 +02:00
Tamo
087ae64899
add a gitignore to avoid pushing the autogenerated file 2021-06-02 17:03:30 +02:00
Tamo
3db25153e5
fix the faceted_fields one last time 2021-06-02 17:00:58 +02:00
Kerollmops
3c304c89d4
Make sure that we generate the faceted database when required 2021-06-02 16:24:58 +02:00
Kerollmops
b0c0490e85
Make sure that we can add a Asc/Desc field without it being filterable 2021-06-02 16:24:58 +02:00
Kerollmops
3b1cd4c4b4
Rename the FacetCondition into FilterCondition 2021-06-02 16:24:58 +02:00
Kerollmops
c2afdbb1fb
Move and comment some internal facet_condition helper functions 2021-06-02 16:24:58 +02:00
Kerollmops
6476827d3a
Fix the indexer to be sure that distinct and Asc/Desc are also faceted 2021-06-02 16:24:58 +02:00
Kerollmops
c10469ddb6
Patch the http-ui crate to support filterable fields 2021-06-02 16:24:58 +02:00
Marin Postma
1e366dae3e
remove useless lifetime on Distinct Trait 2021-06-02 16:24:58 +02:00
Kerollmops
187c713de5
Remove the MapDistinct struct as now distinct attributes are faceted 2021-06-02 16:24:57 +02:00
Kerollmops
ff440c1d9d
Introduce the faceted fields method to retrieve those that needs faceting 2021-06-02 16:24:57 +02:00
Kerollmops
2a3f9b32ff
Rename the faceted fields into filterable fields 2021-06-02 16:24:57 +02:00
Irevoire
f346805c0c
Update benchmarks/Cargo.toml
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-06-02 15:47:03 +02:00
bors[bot]
509a56a43d
Merge #158
158: Implements the dumps r=irevoire a=irevoire

closes #20

divergence from legacy meilisearch:
- dump v2 added, support loading of pending updates (only works dumps created from v2)
- added time stamps to the dump info
- Dump info are only persisted in an internal data structure, and they are not fetched from fs on demand anymore. This was a potential security flaw. This means that the dump infos are flushed on every restart.

Co-authored-by: tamo <tamo@meilisearch.com>
Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-06-02 12:06:47 +00:00
Clémentine Urquizar
ef1ac8a0cb
Update README 2021-06-02 11:13:22 +02:00
Clémentine Urquizar
edfcdb171c
Update benchmarks/scripts/list.sh
Co-authored-by: Irevoire <tamo@meilisearch.com>
2021-06-02 11:13:22 +02:00
Clémentine Urquizar
3c91a9a551
Update following reviews 2021-06-02 11:13:22 +02:00
Tamo
bc4f4ee829
remove s3cmd as a dependency and provide a script to list all the available benchmarks 2021-06-02 11:13:22 +02:00
Clémentine Urquizar
61fe422a88
Update benchmarks/scripts/compare.sh
Co-authored-by: Irevoire <tamo@meilisearch.com>
2021-06-02 11:13:22 +02:00
Clémentine Urquizar
57ed96622b
Update benchmarks/scripts/compare.sh
Co-authored-by: Irevoire <tamo@meilisearch.com>
2021-06-02 11:13:22 +02:00
Clémentine Urquizar
b3c0d43890
Update benchmarks/scripts/compare.sh
Co-authored-by: Irevoire <tamo@meilisearch.com>
2021-06-02 11:13:22 +02:00
Clémentine Urquizar
0d0e900158
Add CI for benchmarks 2021-06-02 11:13:22 +02:00
tamo
4536dfccd0
add a way to provide primary_key or autogenerate documents ids 2021-06-02 11:13:20 +02:00
tamo
06c414a753
move the benchmarks to another crate so we can download the datasets automatically without adding overhead to the build of milli 2021-06-02 11:11:50 +02:00
tamo
3c84075d2d
uses an env variable to find the datasets 2021-06-02 11:05:07 +02:00
tamo
4969abeaab
update the facets for the benchmarks 2021-06-02 11:05:07 +02:00
tamo
e5dfde88fd
fix the facets conditions 2021-06-02 11:05:07 +02:00
tamo
7c7fba4e57
remove the time limitation to let criterion do what it wants 2021-06-02 11:05:07 +02:00
tamo
5d5d115608
reformat all the files 2021-06-02 11:05:07 +02:00
tamo
7086009f93
improve the base search 2021-06-02 11:05:07 +02:00