meili-bors[bot]
e9bf4c43a4
Merge #4649
...
4649: Don't store the vectors in the documents database r=dureuill a=irevoire
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4607
## What does this PR do?
- Ensure that anything falling under `_vectors` is NOT searchable, filterable or sortable
- [x] per embedder, add a roaring bitmap of documents that provide "userProvided" embeddings
- [x] in the indexing process in extract_vector_points, set the bit corresponding to the document depending on the "userProvided" subfield in the _vectors field.
- [x] in the document DB in typed chunks, when writing the _vectors field, remove all keys corresponding to an embedder
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-06-17 12:32:03 +00:00
Louis Dureuil
09d9b63e1c
- test case where all vectors were generated
...
- update tests following changes in behavior from previous commit
2024-06-13 17:16:41 +02:00
Louis Dureuil
b9b938c902
Change retrieveVectors behavior:
...
- when the feature is disabled, documents are never modified
- when the feature is enabled and `retrieveVectors` is disabled, `_vectors` is removed from documents
- when the feature is enabled and `retrieveVectors` is enabled, vectors from the vectors DB are merged with `_vectors` in documents
Additionally `_vectors` is never displayed when the `displayedAttributes` list does not contain either `*` or `_vectors`
- fixed an issue where `_vectors` was not injected when all vectors in the dataset where always generated
2024-06-13 17:13:36 +02:00
Louis Dureuil
3f212a8202
Update tests
2024-06-12 18:13:34 +02:00
Louis Dureuil
3bc8f81abc
user_provided => regenerate
2024-06-12 18:12:20 +02:00
Louis Dureuil
fca9fe39b3
Update test snapshots
2024-06-12 14:50:55 +02:00
meili-bors[bot]
e0eff08095
Merge #4685
...
4685: Fix ci tests r=dureuill a=ManyTheFish
# Pull Request
Make the all following CI succeed:
https://github.com/meilisearch/meilisearch/actions/runs/9477183091
## Related issue
Fixes #4629
## What does this PR do?
- Change the test behavior for `swedish-recomposition` feature flag
- Remove the `-v` parameter from grep
Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Many the fish <many@meilisearch.com>
2024-06-12 07:58:33 +00:00
Tamo
3493093c4f
add a batch of tests
2024-06-11 16:03:54 +02:00
Tamo
600e97d9dc
gate the retrieveVectors parameter behind the vectors feature flag
2024-06-10 18:26:12 +02:00
Louis Dureuil
7559dfc814
Merge tag 'v1.8.2' into release-v1.9.0
2024-06-10 15:07:34 +02:00
Louis Dureuil
50f8218a5d
Asynchronously drop permits
2024-06-10 10:19:57 +02:00
ManyTheFish
57d066595b
fix Tests almost all features
2024-06-06 17:24:50 +02:00
Tamo
734d1c53ad
fix a panic in yaup
2024-06-06 16:31:07 +02:00
Tamo
63dded3961
implements the new analytics for the get documents routes
2024-06-06 11:39:29 +02:00
Tamo
2cdcb703d9
fix the deletion of vectors and add a test
2024-06-06 11:39:29 +02:00
Tamo
6607875f49
add the retrieveVectors parameter to the get and fetch documents route
2024-06-06 11:39:29 +02:00
Tamo
31a793d226
fix the regeneration of the embeddings in the search
2024-06-06 11:39:29 +02:00
Tamo
d85ab23b82
rename all occurences of user_defined to user_provided for consistency
2024-06-06 11:39:29 +02:00
Tamo
49fa41ce65
apply first round of review comments
2024-06-06 11:39:29 +02:00
Tamo
400cf3eb92
add api error test on the new retrieveVectors parameter
2024-06-06 11:39:29 +02:00
Tamo
376b3a19a7
makes clippy and fmt happy
2024-06-06 11:39:29 +02:00
Tamo
d92c173fdc
update the new similar tests
2024-06-06 11:39:29 +02:00
Tamo
6b29676e7e
update snapshots
2024-06-06 11:39:29 +02:00
Tamo
caad40964a
implements the analytics
2024-06-06 11:39:29 +02:00
Tamo
cc5dca8321
fix two bug and add a dump test
2024-06-06 11:39:29 +02:00
Tamo
5d50850e12
always push the user defined vectors in arroy
2024-06-06 11:39:29 +02:00
Tamo
04f6523f3c
expose a new parameter to retrieve the embedders at search time
2024-06-06 11:36:11 +02:00
meili-bors[bot]
98e062a714
Merge #4675
...
4675: Update actix-web 4.5.1 -> 4.6.0 r=dureuill a=dureuill
# Pull Request
- actix-web 4.5.1 -> 4.6.0
- actix-http 3.6.0 -> 3.7.0
- actix-web-static-files (commit 2d3b6160) -> 4.0.1
- tracing-actix-web 0.7.9 -> 0.7.10
- brotli 3.4.0 -> 6.0.0
## Related issue
Fixes #4625
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-06-05 07:40:35 +00:00
Louis Dureuil
8412665957
Update actix-web 4.5.1 -> 4.6.0
2024-06-04 09:54:30 +02:00
meili-bors[bot]
fc584f1db3
Merge #4666
...
4666: Add a score threshold search parameter r=ManyTheFish a=dureuill
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4609
## What does this PR do?
- See [usage](https://meilisearch.notion.site/Filter-by-score-usage-224a183ce7b24ca99b6a9a8da755668a?pvs=25#95b76ded400342ba9ab3d67c734836f0 ) and [the known limitation](https://meilisearch.notion.site/Filter-by-score-usage-224a183ce7b24ca99b6a9a8da755668a?pvs=25#e4e32195bf0e4195b5daecdbb7a97a17 )
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-06-03 08:42:44 +00:00
Louis Dureuil
2b6db6541e
Changes after review
2024-06-03 10:30:00 +02:00
meili-bors[bot]
d6bd88ce4f
Merge #4667
...
4667: Frequency matching strategy r=Kerollmops a=ManyTheFish
# Pull Request
## Related issue
Fixes #3773
## What does this PR do?
- add test for matching strategy
- implement frequency matching strategy
See the [PRD for more details](https://www.notion.so/meilisearch/Frequency-Matching-Strategy-0f3ba08833a442a39590a53a1505ab00 ).
[Public API](https://www.notion.so/meilisearch/frequency-matching-strategy-89868fb7fc584026bc56e378eb854a7f ).
Co-authored-by: ManyTheFish <many@meilisearch.com>
2024-05-30 14:53:31 +00:00
Louis Dureuil
c2fb7afe59
fmt
2024-05-30 12:06:46 +02:00
ManyTheFish
3f1a510069
Add tests and fix matching strategy
2024-05-30 12:02:42 +02:00
Louis Dureuil
41976b82b1
Tests for ranking_score_threshold
2024-05-30 11:22:26 +02:00
Louis Dureuil
c36410fcbf
Analytics for ranking score threshold
2024-05-30 11:22:12 +02:00
Louis Dureuil
7ce2691374
Add ranking score threshold to similar API
2024-05-30 11:21:31 +02:00
Louis Dureuil
c26db7878c
Expose rankingScoreThreshold in API
2024-05-30 10:32:35 +02:00
ManyTheFish
1ab88e10b9
Merge branch 'main' into merge-release-v1.8.1-in-main
2024-05-29 16:24:00 +02:00
ManyTheFish
6a4b2516aa
WIP
2024-05-29 16:21:24 +02:00
Louis Dureuil
2cf3e1c80a
Temporarily ignore perform snapshot test under Windows
2024-05-29 12:42:47 +02:00
Many the fish
e1fbfde6c4
Merge branch 'main' into merge-release-v1.8.1-in-main
2024-05-29 11:31:03 +02:00
Louis Dureuil
ca006e38ec
Basic tests
2024-05-28 15:28:19 +02:00
Louis Dureuil
e26bd87780
Error tests for similar routes
2024-05-28 15:28:19 +02:00
Louis Dureuil
c01e498a63
Test server can call similar
2024-05-28 15:28:19 +02:00
Louis Dureuil
ca6cc4654b
Add similar route
2024-05-28 15:28:19 +02:00
Louis Dureuil
3bd9d2478c
Add error codes
2024-05-28 15:27:43 +02:00
Louis Dureuil
54b15059a0
Analytics changes
2024-05-28 15:27:43 +02:00
Louis Dureuil
e172e938e7
add search rules directly takes the filter rather than the searchquery
2024-05-28 15:22:25 +02:00
Louis Dureuil
02b3d82c60
filtered_universe accepts index and txn instead of SearchContext
2024-05-28 15:22:12 +02:00
