Commit Graph

1804 Commits

Author SHA1 Message Date
bors[bot]
d546f6f40e
Merge #563
563: Improve the `estimatedNbHits` when a `distinctAttribute` is specified r=irevoire a=Kerollmops

This PR is related to https://github.com/meilisearch/meilisearch/issues/2532 but it doesn't fix it entirely. It improves it by computing the excluded documents (the ones with an already-seen distinct value) before stopping the loop, I think it was a mistake and should always have been this way.

The reason it doesn't fix the issue is that Meilisearch is lazy, just to be sure not to compute too many things and answer by taking too much time. When we deduplicate the documents by their distinct value we must do it along the water, everytime we see a new document we check that its distinct value of it doesn't collide with an already returned document. 

The reason we can see the correct result when enough documents are fetched is that we were lucky to see all of the different distinct values possible in the dataset and all of the deduplication was done, no document can be returned.

If we wanted to implement that to have a correct `extimatedNbHits` every time we should have done a pass on the whole set of possible distinct values for the distinct attribute and do a big intersection, this could cost a lot of CPU cycles.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-22 12:39:44 +00:00
bors[bot]
38a8d3cae1
Merge #565
565: Bump the milli version to 0.31.0 r=curquiza a=Kerollmops



Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-22 10:09:41 +00:00
Kerollmops
f5c3b951bc
Bump the milli version to 0.31.0 2022-06-22 12:08:16 +02:00
Kerollmops
d2f84a9d9e
Improve the estimatedNbHits when distinct is enabled 2022-06-22 11:39:21 +02:00
bors[bot]
4f547eff02
Merge #560
560: Update version for next release (v0.30.0) r=curquiza a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-06-20 12:37:01 +00:00
bors[bot]
64b833410c
Merge #559
559: Avoid having an ending separator before crop marker r=irevoire a=ManyTheFish

related to https://github.com/meilisearch/meilisearch/issues/2528


Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-06-20 11:06:52 +00:00
Clémentine Urquizar
31f749b5d8
Update version for next release (v0.30.0) 2022-06-20 12:09:57 +02:00
ManyTheFish
a0ab90a4d7 Avoid having an ending separator before crop marker 2022-06-16 18:23:57 +02:00
bors[bot]
a59ae19842
Merge #558
558: Deletion benchmarks r=ManyTheFish a=ManyTheFish

Add benchmarks on the deletion and start rethinking benchmark names.

Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-06-16 09:34:37 +00:00
ManyTheFish
2652310f2a Change delete benchmark names 2022-06-16 10:32:58 +02:00
ManyTheFish
adbb0ff318 Add deletion benchmarks 2022-06-16 10:17:58 +02:00
bors[bot]
0a5d1a445e
Merge #554
554: Enhance tests for soft deletetion r=irevoire a=ManyTheFish

#### tests: (skip in changelog)
- [x] placeholder search shouldn’t return soft deleted
- [x] search shouldn’t return soft deleted
- [x] filtered placeholder search shouldn’t return soft deleted
- [x] geo-filtered placeholder search shouldn’t return soft deleted
- [x] documents list/get shouldn’t return soft deleted
- [x] stats shouldn’t count soft deleted

#### other: (API breaking)
- [x] ensure that Index methods are not bypassed by Meilisearch


Poke `@irevoire,` we may merge this into your branch.

Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-06-14 09:49:37 +00:00
ManyTheFish
447195a27a Replace format by to_string 2022-06-14 10:32:44 +02:00
ManyTheFish
177154828c Extends deletion tests 2022-06-13 17:34:16 +02:00
ManyTheFish
0d1d354052 Ensure that Index methods are not bypassed by Meilisearch 2022-06-13 17:34:11 +02:00
bors[bot]
f1d848bb9a
Merge #552
552: Fix escaped quotes in filter r=Kerollmops a=irevoire

Will fix https://github.com/meilisearch/meilisearch/issues/2380

The issue was that in the evaluation of the filter, I was using the deref implementation instead of calling the `value` method of my token.

To avoid the problem happening again, I removed the deref implementation; now, you need to either call the `lexeme` or the `value` methods but can't rely on a « default » implementation to get a string out of a token.

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-06-09 14:56:44 +00:00
Tamo
676187ba43
bump milli version 2022-06-09 16:53:32 +02:00
Tamo
90afde435b
fix escaped quotes in filter 2022-06-09 16:03:49 +02:00
bors[bot]
19d44142a1
Merge #550
550: Add the two new pagination and faceting settings r=ManyTheFish a=Kerollmops

This PR adds two new settings in the database, those settings are described [in this spec](https://github.com/meilisearch/specifications/pull/157).

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-09 08:16:01 +00:00
Kerollmops
445d5474cc
Add the pagination_limited_to setting to the database 2022-06-08 18:14:27 +02:00
Kerollmops
69931e50d2
Add the max_values_by_facet setting to the database 2022-06-08 17:54:56 +02:00
Kerollmops
52a494bd3b
Add the new pagination.limited_to and faceting.max_values_per_facet settings 2022-06-08 17:15:36 +02:00
bors[bot]
9580b9de79
Merge #549
549: Bump the version to 0.29.2 r=curquiza a=Kerollmops



Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-08 14:29:47 +00:00
bors[bot]
a762d7f462
Merge #548
548: Setup the new limits on the number of facet values to return r=ManyTheFish a=Kerollmops

This PR implements the early draft of the new spec (waiting for it) specifying how the new facet limit feature should work and which limit we apply to the number of facet values to return by facet.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-08 14:00:33 +00:00
Kerollmops
56ee9cc21f
Bump the version to 0.29.2 2022-06-08 16:00:06 +02:00
Kerollmops
2a505503b3
Change the number of facet values returned by default to 100 2022-06-08 15:58:57 +02:00
Kerollmops
bae4007447
Remove the hard limit on the number of facet values returned 2022-06-08 15:58:57 +02:00
bors[bot]
7313d6c533
Merge #547
547: Update version for next release (v0.29.1) r=Kerollmops a=curquiza

A new milli version will be released once this PR is merged https://github.com/meilisearch/milli/pull/543

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-06-08 10:20:24 +00:00
bors[bot]
306d2f37ff
Merge #543
543: Fix wrong internal ids assignments r=irevoire a=irevoire

Fix https://github.com/meilisearch/meilisearch/issues/2470

Co-authored-by: ad hoc <postma.marin@protonmail.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
2022-06-08 09:19:58 +00:00
Clémentine Urquizar
478dbfa45a
Update version for next release (v0.29.1) 2022-06-07 18:59:33 +02:00
Tamo
d0aaa7ff00
Fix wrong internal ids assignments 2022-06-07 15:49:33 +02:00
ad hoc
31776fdc3f
add failing test 2022-06-07 15:49:33 +02:00
bors[bot]
05ae6dbfa4
Merge #541
541: Update version for next release (v0.29.0) r=ManyTheFish a=curquiza

Need to update the version since #540 was merged and breaking

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-06-02 16:53:28 +00:00
bors[bot]
78f76c841d
Merge #542
542: Refactor matching word r=Kerollmops a=ManyTheFish

Simplify MatchingWords API


Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-06-02 16:23:41 +00:00
ManyTheFish
d212dc6b8b Remove useless newline 2022-06-02 18:22:56 +02:00
ManyTheFish
a5c790bf4b Update http-ui 2022-06-02 18:15:36 +02:00
Clémentine Urquizar
6ce1c6487a
Update version for next release (v0.29.0) 2022-06-02 18:07:55 +02:00
ManyTheFish
727d663f28 Update benchmarks 2022-06-02 18:07:10 +02:00
ManyTheFish
7aabe42ae0 Refactor matching words 2022-06-02 17:59:04 +02:00
bors[bot]
dd186533f0
Merge #540
540: Integrate charabia r=Kerollmops a=ManyTheFish

related to https://github.com/meilisearch/meilisearch/issues/2375
related to https://github.com/meilisearch/meilisearch/issues/2144
related to https://github.com/meilisearch/meilisearch/issues/2417

Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-06-02 15:34:33 +00:00
ManyTheFish
4dd7b20c32 Update benchmarks 2022-06-02 17:33:25 +02:00
ManyTheFish
4dd3675d2b Update http-ui 2022-06-02 16:59:11 +02:00
ManyTheFish
86ac8568e6 Use Charabia in milli 2022-06-02 16:59:11 +02:00
ManyTheFish
192e024ada Add Charabia in Cargo.toml 2022-06-02 16:59:07 +02:00
bors[bot]
ac6df0df57
Merge #539
539: Update version to v0.28.1 r=Kerollmops a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-06-01 16:40:12 +00:00
Clémentine Urquizar
c19c17eddb
Update version to v0.28.1 2022-06-01 18:31:02 +02:00
bors[bot]
74d1914a64
Merge #535
535: Reintroduce the max values by facet limit r=ManyTheFish a=Kerollmops

This PR reintroduces the max values by facet limit this is related to https://github.com/meilisearch/meilisearch/issues/2349.

~I would like some help in deciding on whether I keep the default 100 max values in milli and set up the `FacetDistribution` settings in Meilisearch to use 1000 as the new value, I expose the `max_values_by_facet` for this purpose.~

I changed the default value to 1000 and the max to 10000, thank you `@ManyTheFish` for the help!

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-01 14:30:50 +00:00
bors[bot]
582930dbbb
Merge #538
538: speedup exact words r=Kerollmops a=MarinPostma

This PR make `exact_words` return an `Option` instead of an empty set, since set creation is costly, as noticed by `@kerollmops.`

I was not convinces that this was the cause for all of the performance drop we measured, and then realized that methods that initialized it were called recursively which caused initialization times to add up. While the first fix solves the issue when not using exact words, using exact word remained way more expensive that it should be. To address this issue, the exact words are cached into the `Context`, so they are only initialized once.


Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-05-30 08:20:34 +00:00
bors[bot]
9f78e392b1
Merge #536
536: Improves ranking rules error message r=Kerollmops a=matthias-wright

This PR improves the ranking rules error message to properly reflect the case sensitivity.
The issue was highlighted in [meilisearch/issues/2407](https://github.com/meilisearch/meilisearch/issues/2407).
Cheers!

Co-authored-by: Matthias Wright <matthias.s.wright@gmail.com>
2022-05-24 16:43:52 +00:00
ad hoc
25fc576696
review changes 2022-05-24 14:15:33 +02:00