1157 Commits

Author SHA1 Message Date
Irevoire
3b9f1db061
implement the clear of the rtree 2021-09-08 17:51:07 +02:00
Irevoire
d344489c12
implement the deletion of geo points 2021-09-08 17:51:07 +02:00
Irevoire
44d6b6ae9e
Index the geo points 2021-09-08 17:51:07 +02:00
Irevoire
8d9c2c4425
create a new db with getters and setters 2021-09-08 17:51:07 +02:00
bors[bot]
b22aac92ac
Merge #342
342: Let the caller decide what kind of error they want to returns when parsing `AscDesc` r=Kerollmops a=irevoire

This is one possible fix for #339 
We would then need to patch these lines https://github.com/meilisearch/MeiliSearch/blob/main/meilisearch-http/src/index/search.rs#L110-L114 to return the error we want.

Another solution would be to add a parameter to the `from_str` to specify which context we are in.

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-09-08 14:18:57 +00:00
Tamo
932998f5cc
let the caller decide if they want to return an invalidSortName or an
invalidCriterionName error
2021-09-08 16:17:31 +02:00
bors[bot]
86c3b0c8c2
Merge #350
350: Fix mdb val size error r=Kerollmops a=ManyTheFish

Related to [#1677](https://github.com/meilisearch/MeiliSearch/issues/1677)

Co-authored-by: many <maxime@meilisearch.com>
2021-09-08 13:32:15 +00:00
many
e54280fbfc
Skip empty normalized words 2021-09-08 15:25:23 +02:00
many
d18ee58ab9
Check if key are not empty in validator 2021-09-08 15:25:23 +02:00
bors[bot]
63bc231243
Merge #349
349: Enable the grenad tempfile feature back r=ManyTheFish a=Kerollmops

This PR enables the grenad `tempfile` feature back, [when this is feature is disabled the sorter writes the entries in memory](7c082d05bf/src/sorter.rs (L470-L476)) instead of on disk and therefore, consumes more memory. By enabling this feature grenad merges on disk by using the `tempfile` dependency.

This PR also bumps milli to v0.3.1 where `@ManyTheFish` added an assert for when the allocator can't allocate and disable the default snappy compression in the `http-ui` crate.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2021-09-08 13:23:57 +00:00
Kerollmops
68856e5e2f
Disable the default snappy compression for the http-ui crate 2021-09-08 14:17:32 +02:00
Kerollmops
8a088fb99e
Bump grenad to v0.3.1 2021-09-08 14:08:55 +02:00
Kerollmops
20ad43b908
Enable the grenad tempfile feature back 2021-09-08 14:06:28 +02:00
bors[bot]
772e55d174
Merge #347
347: Update version for the next release (v0.13.0) r=curquiza a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-09-08 11:41:15 +00:00
bors[bot]
d160305868
Merge #348
348: Drop sorter before creating a new one r=Kerollmops a=ManyTheFish



Co-authored-by: many <maxime@meilisearch.com>
2021-09-08 11:34:20 +00:00
many
9961b78b06
Drop sorter before creating a new one 2021-09-08 13:30:26 +02:00
Clémentine Urquizar
eb7b9d9dbf
Update version for the next release (v0.13.0) 2021-09-08 10:59:30 +02:00
bors[bot]
f5e418ace7
Merge #345
345: Better dependencies cache for CI r=irevoire a=shekhirin



Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
2021-09-08 08:43:19 +00:00
bors[bot]
48d211b8b0
Merge #344
344: Move the sort ranking rule before the exactness ranking rule r=ManyTheFish a=Kerollmops

This PR moves the sort ranking rule at the 5th position by default, right before the exactness one.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2021-09-07 15:47:15 +00:00
Alexey Shekhirin
dbd91e7151
chore(ci): use smarter dependencies cache 2021-09-07 18:16:33 +03:00
bors[bot]
720becb5e8
Merge #341
341: Throw a query time error when a sort parameter is used but the sort ranking rule is missing r=Kerollmops a=Kerollmops

This PR makes the engine throw an error for when the ranking rules don't contain the `sort` rule, the `sortable_fields` are correctly set but the user tries to use the `sort` query parameter. Doing so will have no effect on the returned documents so we preferred returning an error to help debug this.

That's breaking on the MeiliSearch side as we added a new variant to the `UserError` enum.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2021-09-07 14:45:05 +00:00
Kerollmops
e2cefc9b4f
Move the sort ranking rule before the exactness ranking rule 2021-09-07 16:41:33 +02:00
bors[bot]
a0b3620b05
Merge #346
346: remove unused grenad default features r=Kerollmops a=MarinPostma

Milli is not using any of grenad default features, and it's zstd feature creates conflict with meilisearch. This pr simply remove the unused features.


Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-09-07 14:30:20 +00:00
mpostma
cd043d4461 remove unused grenad default features 2021-09-07 16:21:46 +02:00
Kerollmops
5989528833
Add a test to make sure we throw the right error message 2021-09-07 11:02:00 +02:00
Kerollmops
fd3daa4423
Throw a query time error when a sort param is used but sort ranking rule is missing 2021-09-07 11:02:00 +02:00
Kerollmops
8dca36433c
Introduce the new SortRankingRuleMissing user error variant 2021-09-07 11:01:59 +02:00
bors[bot]
446ed17589
Merge #338
338: Fix string fields sorting r=Kerollmops a=shekhirin

Resolves https://github.com/meilisearch/milli/issues/333

<details>
  <summary>curl checks</summary>
  
 ```console
➜  ~ curl -s 'localhost:7700/indexes/movies/search' -d '{"sort": ["title:asc"], "limit": 30}' | jq -r '.hits | map(.title)[]'
#1 Cheerleader Camp
#Horror
#RealityHigh
#Roxy
#SquadGoals
$5 a Day
$9.99
'71
(2)
(500) Days of Summer
(Girl)Friend
*batteries not included
...And God Created Woman
...And Justice for All
...E fuori nevica!
.45
1
1 Mile To You
1 Night
10
10 Cloverfield Lane
10 giorni senza mamma
10 Items or Less
10 Rillington Place
10 Rules for Sleeping Around
10 Things I Hate About You
10 to Midnight
10 Years
10,000 BC
10,000 Saints

➜  ~ curl -s 'localhost:7700/indexes/movies/search' -d '{"sort": ["title:desc"], "limit": 30}' | jq -r '.hits | map(.title)[]'
크게 될 놈
왓칭
뷰티플 마인드
노무현과 바보들
ハニー
Счастье – это… Часть 2
СОТКА
Смотри мою любовь
Позивний 'Бандерас'
Лошо момиче
Күлүк Хомус
Куда течет море
Каникулы президента
Ακίνητο Ποτάμι
Üç Harfliler: Beddua
È nata una Star?
Æon Flux
Ága
À propos de Nice
À Nos Amours
À l'aventure
¡Three Amigos!
Zulu Dawn
Zulu
Zulu
Zu: Warriors from the Magic Mountain
Zu Warriors
Zorro
Zorba the Greek
Zootopia
```
</details>

Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
2021-09-07 08:28:23 +00:00
Alexey Shekhirin
0be09555f1
test(search): asc/desc criteria for large datasets 2021-09-03 18:00:08 +03:00
Alexey Shekhirin
c2517e7d5f
fix(facet): string fields sorting 2021-09-03 11:58:26 +03:00
bors[bot]
5cbe879325
Merge #308
308: Implement a better parallel indexer r=Kerollmops a=ManyTheFish

Rewrite the indexer:
- enhance memory consumption control
- optimize parallelism using rayon and crossbeam channel
- factorize the different parts and make new DB implementation easier
- optimize and fix prefix databases


Co-authored-by: many <maxime@meilisearch.com>
2021-09-02 15:03:52 +00:00
many
741a4444a9
Remove log in chunk generator 2021-09-02 16:57:46 +02:00
many
7f7fafb857
Make document_chunk_size settable from update builder 2021-09-02 15:25:39 +02:00
many
db0c681bae
Fix Pr comments 2021-09-02 15:17:52 +02:00
bors[bot]
46f7df232a
Merge #337
337: Update version for the next release (v0.12.0) r=Kerollmops a=curquiza

Breaking because of the new indexer that implies DB changes #308 

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-09-02 10:13:31 +00:00
Clémentine Urquizar
285849e3a6
Update version for the next release (v0.12.0) 2021-09-02 10:08:41 +02:00
bors[bot]
a589f6c60b
Merge #335
335: Get sortable_fields from index only if criteria present in query r=Kerollmops a=shekhirin

Seems like we don't need to retrieve `sortable_fields` from the index if there's no any `sort_criteria` in the query.

Small 🤏  optimization opportunity out there.

Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
2021-09-01 16:01:00 +00:00
bors[bot]
3e0a78acf3
Merge #329
329: Run all benchmarks once every friday r=irevoire a=irevoire

All the benchmarks run every Friday on the `main` branch.
To avoid having pending benchmarks everywhere, we execute one benchmark every 8 hours.
Then the results are uploaded as if it was a normal user-run benchmark.

This PR closes #314 and #321 

Co-authored-by: Irevoire <tamo@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
2021-09-01 15:20:49 +00:00
many
4860fd4529
Ignore empty facet values 2021-09-01 16:48:40 +02:00
many
b3a22f31f6
Fix memory consuption in word pair proximity extractor 2021-09-01 16:48:40 +02:00
many
9452fabfb2
Optimize cbo roaring bitmaps merge 2021-09-01 16:48:40 +02:00
many
8f702828ca
Ignore errors comming from crossbeam channel senders 2021-09-01 16:48:40 +02:00
many
e09eec37bc
Handle distance addition with hard separators 2021-09-01 16:48:40 +02:00
many
fc7cc770d4
Add logging timers 2021-09-01 16:48:40 +02:00
many
a2f59a28f7
Remove unwrap sending errors in channel 2021-09-01 16:48:40 +02:00
many
5c962c03dd
Fix and optimize word_prefix_pair_proximity_docids database 2021-09-01 16:48:40 +02:00
many
2d1727697d
Take stop word in account 2021-09-01 16:48:40 +02:00
many
823da19745
Fix test and use progress callback 2021-09-01 16:48:39 +02:00
many
1d314328f0
Plug new indexer 2021-09-01 16:48:36 +02:00
many
3aaf1d62f3
Publish grenad CompressionType type in milli 2021-09-01 16:42:08 +02:00