Commit Graph

9833 Commits

Author SHA1 Message Date
JWSong
6fb36ed30e get rid of the redundant info in document_addition_with_huge_int_primary_key 2024-06-25 23:54:27 +09:00
JWSong
dcdc83946f accept large number as string 2024-06-25 21:41:47 +09:00
meili-bors[bot]
3c4c46377b
Merge #4665
4665: Add missing Korean support r=ManyTheFish a=junhochoi

Some configuration is missing `korean` features and add a test case in `milli/src/search/mod.rs`.

# Pull Request

## Related issue

#3443 #3882 

## What does this PR do?
- Improvement on enabling Korean support

Inspired by the work (#3882) I tried to enable Korean features but have found some missing configurations.
This PR is add those missing configs (mostly Cargo.toml) and added one test case.

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Junho Choi <jh.choi@catenoid.net>
2024-06-25 11:51:21 +00:00
Tamo
7da21bb601 introduce as many custom error message as possible 2024-06-25 12:40:51 +02:00
meili-bors[bot]
13161fd7d0
Merge #4722
4722: Grow by 1TB instead of 1MB r=dureuill a=dureuill

When an index reaches 1TB, increases its size by 1TB rather than 1MB

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-06-25 10:17:58 +00:00
meili-bors[bot]
b81e2951a9
Merge #4723
4723: Fixes for Rust v1.79 r=ManyTheFish a=dureuill

cherry-picked from the `release-v1.9.0` branch

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-06-25 09:21:29 +00:00
Louis Dureuil
d75e0098c7
Fixes for Rust v1.79 2024-06-25 11:16:06 +02:00
Louis Dureuil
27496354e2
Grow by 1TB instead of 1MB 2024-06-25 09:01:11 +02:00
Junho Choi
2e0ff56f3f Add missing Korean support
Some configuration is missing `korean` features and
add a test case in `milli/src/search/mod.rs`.
2024-06-25 12:45:21 +09:00
Tamo
a74fb87d1e start introducing new error messages 2024-06-24 19:00:53 +02:00
Tamo
558b66e535 makes most tests works with variable error messages 2024-06-24 19:00:44 +02:00
Strift
cade18bd47
Update README.md (#4721) 2024-06-24 15:47:10 +02:00
meili-bors[bot]
298c7b0c93
Merge #4715
4715: Build all arroy indexes that need to be built r=dureuill a=irevoire

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4588

## What does this PR do?
- Update arroy
- Ensure we always rebuild the arroy indexes that need to be built


Co-authored-by: Tamo <tamo@meilisearch.com>
2024-06-24 09:32:04 +00:00
Tamo
606e108420 fix all the flaky snapshots 2024-06-24 11:13:45 +02:00
Tamo
7be17b7e4c add the missing snapshots 2024-06-24 10:52:57 +02:00
Tamo
1693332cab Update arroy and always build the tree that need to be built 2024-06-24 10:14:03 +02:00
meili-bors[bot]
ddd564665b
Merge #4713
4713: Speed up facet distribution r=ManyTheFish a=Kerollmops

This PR is akin to #4682, but this time, the same logic is applied to the facets. Bitmaps are not decoded, and we do an intersection on the bytes with the search candidates instead of materializing the RoaringBitmap to destroy it just after the operation.

A prospect raised some slow requests when performing facet searches, and I found out that the disk optimization intersection wasn't performed on the facets.

Co-authored-by: Clément Renault <clement@meilisearch.com>
2024-06-24 05:23:46 +00:00
karribalu
2a38f5c757 Run Rustfmt 2024-06-21 00:14:26 +01:00
karribalu
133d33d72c Merge remote-tracking branch 'origin/main' 2024-06-20 23:55:17 +01:00
karribalu
fb683fe88b Fix bad http status and error message on wrong payload 2024-06-20 23:55:09 +01:00
meili-bors[bot]
4ae11bfd31
Merge #4710
4710: Only spawn thread pool once (v1.9) r=irevoire a=dureuill

# Pull Request

See #4707 

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-06-20 11:45:32 +00:00
Clément Renault
9736e16a88
Make clippy happy 2024-06-20 13:02:44 +02:00
Clément Renault
6fa4da8ae7
Improve facet distribution speed in count mode 2024-06-20 12:58:51 +02:00
Clément Renault
19d7cdc20d
Improve facet distribution speed in lexico mode 2024-06-20 12:57:08 +02:00
meili-bors[bot]
c229200820
Merge #4712
4712: Update mini-dashboard 2.14 r=irevoire a=curquiza

Fixes #4668

Co-authored-by: curquiza <clementine@meilisearch.com>
2024-06-20 08:47:22 +00:00
curquiza
bad28cc9e2 Update mini-dashboard 2.14 2024-06-20 10:01:36 +02:00
Clément Renault
534f696b29
Update the README to link more demos (#4711)
This Pull Request adds two new interesting demos to a brand new list, which replaces the short _Try it_ text just below the Where2Watch showcase image hoping people will notice them.
2024-06-20 09:53:06 +02:00
Louis Dureuil
a04041c8f2
Only spawn the pool once 2024-06-19 16:25:33 +02:00
Clémentine
b347b66619
Revert "Add june 11th webinar banner" (#4705) 2024-06-18 18:45:50 +02:00
meili-bors[bot]
e580d6b98f
Merge #4693
4693: Introduce distinct attributes at search time r=irevoire a=Kerollmops

This PR fixes #4611.

### To Do
- [x] Remove the `distinguishableAttributes` settings (not even a commit about that).
- [x] Use the `filterableAttributes` to be able to use the `distinct` parameter at search.
- [x] Work on the errors and make tests.

Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
2024-06-18 07:45:03 +00:00
Tamo
8ba65e333b add snapshot files 2024-06-17 16:50:26 +02:00
Tamo
43875e6758 fix bug around nested fields 2024-06-17 15:59:30 +02:00
Tamo
d7844a6e45 add a bunch of tests on the errors of the distinct at search time 2024-06-17 15:37:32 +02:00
meili-bors[bot]
e9bf4c43a4
Merge #4649
4649: Don't store the vectors in the documents database r=dureuill a=irevoire

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4607

## What does this PR do?
- Ensure that anything falling under `_vectors` is NOT searchable, filterable or sortable
- [x] per embedder, add a roaring bitmap of documents that provide "userProvided" embeddings
- [x] in the indexing process in extract_vector_points, set the bit corresponding to the document depending on the "userProvided" subfield in the _vectors field.
- [x] in the document DB in typed chunks, when writing the _vectors field, remove all keys corresponding to an embedder

Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-06-17 12:32:03 +00:00
Tamo
a8a0854421
Update meilisearch/src/analytics/segment_analytics.rs 2024-06-17 14:30:50 +02:00
Louis Dureuil
0a8f50695e
Fixes for Rust v1.79 2024-06-13 17:47:44 +02:00
Louis Dureuil
09d9b63e1c
- test case where all vectors were generated
- update tests following changes in behavior from previous commit
2024-06-13 17:16:41 +02:00
Louis Dureuil
b9b938c902
Change retrieveVectors behavior:
- when the feature is disabled, documents are never modified
- when the feature is enabled and `retrieveVectors` is disabled, `_vectors` is removed from documents
- when the feature is enabled and `retrieveVectors` is enabled, vectors from the vectors DB are merged with `_vectors` in documents

Additionally `_vectors` is never displayed when the `displayedAttributes` list does not contain either `*` or `_vectors`

- fixed an issue where `_vectors` was not injected when all vectors in the dataset where always generated
2024-06-13 17:13:36 +02:00
Tamo
6bf07d969e add failing test 2024-06-13 15:49:42 +02:00
Louis Dureuil
e35ef31738
Small changes following review 2024-06-13 14:20:48 +02:00
Louis Dureuil
3f212a8202
Update tests 2024-06-12 18:13:34 +02:00
Louis Dureuil
bc547dad6f
Update dump file 2024-06-12 18:12:56 +02:00
Louis Dureuil
3bc8f81abc
user_provided => regenerate 2024-06-12 18:12:20 +02:00
Louis Dureuil
a89eea233b
Fix vectors injection 2024-06-12 17:10:19 +02:00
Louis Dureuil
34fabed214
Add test for vector writeback 2024-06-12 17:09:34 +02:00
Louis Dureuil
fca9fe39b3
Update test snapshots 2024-06-12 14:50:55 +02:00
Louis Dureuil
f5cf01e7d1
Rework extraction to use EmbedderAction 2024-06-12 14:50:55 +02:00
Louis Dureuil
d1dd7e5d09
In transform for removed embedders, write back their user provided vectors in documents, and clear the writers 2024-06-12 14:50:55 +02:00
Louis Dureuil
d18c1f77d7
Update embedder configs with a finer granularity
- no longer clear vector DB between any two embedder changes
2024-06-12 14:50:55 +02:00
Louis Dureuil
d0b05ae691
Add EmbedderAction to settings 2024-06-12 14:50:54 +02:00