meili-bors[bot]
f36f34c2f7
Merge #4717
...
4717: Implement intersection at end on the search pipeline r=Kerollmops a=Kerollmops
This PR is akin to #4713 and #4682 because it uses the new RoaringBitmap method to do the intersections directly on the serialized bytes for the bytes LMDB/heed returns. More work related to this issue can be done, and I listed that in #4780 .
Running the following command shows where we use bitand/intersection operations and where we can potentially apply this optimization.
```sh
rg --type rust --vimgrep '\s&[=\s]' milli/src/search
```
Co-authored-by: Clément Renault <clement@meilisearch.com>
2024-07-10 15:01:33 +00:00
Clément Renault
3bac22fd87
We do not do intersections with the universe when it is related to cache
2024-07-10 16:49:36 +02:00
Clément Renault
ce61cb7fe6
Simplify and speedup an intersection pass
2024-07-10 16:49:36 +02:00
Clément Renault
1693d1a311
Simplify the check to decide to stop a loop
2024-07-10 16:49:36 +02:00
Clément Renault
febea735ca
Remove the unused universe parameter from resolve_negative_phrases
2024-07-10 16:49:36 +02:00
Clément Renault
93ba051094
Remove the invalid get_phrases_docids universe parameter
2024-07-10 16:49:35 +02:00
Clément Renault
cd7a20fa32
Make it work by avoid storing invalid stuff in the cache
2024-07-10 16:49:35 +02:00
Clément Renault
41f51adbec
Do less useless intersections
2024-07-10 16:49:35 +02:00
Clément Renault
0ca1a4e805
Always do the intersections with the universe
2024-07-10 16:49:34 +02:00
Clément Renault
50a7393c55
Modify the compute_query_term_subset_docids function to accept the universe
2024-07-10 16:49:34 +02:00
Clément Renault
837274f853
Restrict even more the Rhai engine
2024-07-10 16:30:18 +02:00
Clément Renault
487997f6ad
Support the new editDocumentsByFunction experimental feature
2024-07-10 16:29:18 +02:00
Clément Renault
94809090a3
Support not specifying a context
2024-07-10 16:29:18 +02:00
Clément Renault
01144b2c74
Make the edit documents by function route experimental
2024-07-10 16:29:18 +02:00
Clément Renault
e97600eead
Improve the analytics for the document edition by function
2024-07-10 16:29:18 +02:00
Clément Renault
767553519d
Create errors for the HTTP route issues
2024-07-10 16:29:18 +02:00
Clément Renault
aace587dd1
Create errors for the internal processing ones
2024-07-10 16:29:18 +02:00
Clément Renault
e706023969
Fix some analytics issues
2024-07-10 16:29:17 +02:00
Clément Renault
bcd0c5f5a4
Support DocumentEdition in dumps
2024-07-10 16:29:17 +02:00
Clément Renault
f35d6710f3
Update rhai to v1.19.0
2024-07-10 16:29:17 +02:00
Clément Renault
b7b8f564c3
delete-me: Simply support generating dump
2024-07-10 16:29:05 +02:00
Clément Renault
862d49e4af
Editing documents requires the documents.all action (add, get, and del)
2024-07-10 16:29:05 +02:00
Clément Renault
81ec0abad1
Use the new rayon-par-bridge library
2024-07-10 16:29:04 +02:00
Clément Renault
b67d385cf0
Parallelize the edition functions
2024-07-10 16:28:54 +02:00
Clément Renault
dfecb25814
Disable the time package
2024-07-10 16:28:37 +02:00
Clément Renault
2eae2015d7
Support aborting documents edition by function
2024-07-10 16:28:15 +02:00
Clément Renault
33fa17bf12
Support deleting documents with functions
2024-07-10 16:28:15 +02:00
Clément Renault
400e6b93ce
Support user-provided context for documents edition
2024-07-10 16:28:15 +02:00
Clément Renault
f32e6c32fc
Rename editionCode to function
2024-07-10 16:28:15 +02:00
Clément Renault
f4add93043
Limit the number of script operations
2024-07-10 16:28:14 +02:00
Clément Renault
f07256971a
Fix tests
2024-07-10 16:28:14 +02:00
Clément Renault
2fae96ac14
Show the actual number of actually edited documents
2024-07-10 16:28:14 +02:00
Clément Renault
246f0e7130
Make the filter field really optional
2024-07-10 16:28:14 +02:00
Clément Renault
45af18ae9c
Check the Rhai syntax before accepting the script
2024-07-10 16:28:13 +02:00
Clément Renault
2d97164d9f
It works perfectly with some Rhai
2024-07-10 16:28:13 +02:00
Clément Renault
efc156a4a4
Executing Lua works correctly
2024-07-10 16:27:36 +02:00
Clément Renault
ba85959642
Support filtering the documents to edit with lua
2024-07-10 16:23:21 +02:00
Clément Renault
1702b5cf44
Prepare for processing documents edition
2024-07-10 16:23:21 +02:00
meili-bors[bot]
2099b4f0dd
Merge #4786
...
4786: Update dependencies r=Kerollmops a=irevoire
# Pull Request
## Related issue
Fixes #4753
## What does this PR do?
- Update all dependencies except rustls
- [x] Release charabia
- [x] Update charabia
- [x] Double check that the docker build works after updating charabia
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
2024-07-10 13:23:54 +00:00
Tamo
0d5bc4578e
Update CONTRIBUTING.md
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2024-07-10 15:21:43 +02:00
Tamo
8f60ad0a23
apply review comments
2024-07-10 14:38:19 +02:00
Tamo
9570139eeb
update contributing.md with the new lindera update
2024-07-10 14:28:43 +02:00
Clément Renault
9d6885793e
Upgrade dependencies
2024-07-10 13:46:24 +02:00
Clément Renault
98cd6a865c
Update dependencies after removing useless ones
2024-07-10 13:37:24 +02:00
Clément Renault
5f4530ce57
Remove more unused dependencies
2024-07-10 13:36:34 +02:00
Tamo
0ecaf861fa
fix ci
2024-07-10 10:06:59 +02:00
Tamo
4d5005b01a
make clippy happy
2024-07-10 10:06:59 +02:00
Tamo
952e742321
update charabia
2024-07-09 23:41:29 +02:00
Tamo
ee9aa63044
update rust version
2024-07-09 23:41:29 +02:00
Tamo
43db4f4242
update fxprof_processed_profile
2024-07-09 23:41:29 +02:00
