Commit Graph

4386 Commits

Author SHA1 Message Date
Loïc Lecrenier
9522b75454
Continue implementation of task deletion
1. Matched tasks are a roaring bitmap
2. Start implementation in meilisearch-http
3. Snapshots use meili-snap
4. Rename to TaskDeletion
2022-10-27 11:34:03 +02:00
Kerollmops
e4d461ecba
Make sure that we do not batch tasks from different indexes 2022-10-27 11:34:03 +02:00
Kerollmops
b029369653
Add a test to check different indexes autobatching 2022-10-27 11:34:03 +02:00
Kerollmops
408d00136c
Extract index creation rights and simplify the autobatcher rules 2022-10-27 11:34:03 +02:00
Kerollmops
2c24c7d403
Fix invalid import of tasks types 2022-10-27 11:34:03 +02:00
Tamo
7034803712
move the API key in meilisearch_types 2022-10-27 11:34:02 +02:00
Tamo
c192146fbe
remove an unused file 2022-10-27 11:34:02 +02:00
Tamo
b6c84e53ba
uncomment a task serialization test 2022-10-27 11:34:02 +02:00
Tamo
2f1eb78b1d
refactor the Task a little bit 2022-10-27 11:34:02 +02:00
Tamo
510ce9fc51
start moving a lot of task types to meilisearch_types 2022-10-27 11:34:01 +02:00
Tamo
fa4c1de019
store md5 instead of the whole snapshots 2022-10-27 11:34:01 +02:00
Loïc Lecrenier
3e4337c91f
Add meili-snap crate to make writing snapshot tests easier 2022-10-27 11:34:01 +02:00
Tamo
0af00f6b32
fix all the import and comment most of the dump v6 2022-10-27 11:34:01 +02:00
Tamo
141a1c9464
push the document_format and settings I forgot in the previous PR 2022-10-27 11:34:00 +02:00
Tamo
667c282e19
get rids of the index crate + the document_types crate 2022-10-27 11:34:00 +02:00
Loïc Lecrenier
9a74ea0943
Fix compiler errors related autobatching option of the index scheduler 2022-10-27 11:34:00 +02:00
Loïc Lecrenier
eabac9676b
Fix typo and remove useless code in tests 2022-10-27 11:34:00 +02:00
Loïc Lecrenier
ab4e649221
Apply suggestions from code review
Co-authored-by: Tamo <tamo@meilisearch.com>
2022-10-27 11:34:00 +02:00
Loïc Lecrenier
568199fc0d
Add more task deletion tests 2022-10-27 11:33:59 +02:00
Loïc Lecrenier
13a72f8757
Use more complete snapshot tests for the index scheduler 2022-10-27 11:33:59 +02:00
Loïc Lecrenier
4c55c30027
Add a DetailsView type and improve index scheduler snapshots
The DetailsView type is necessary because serde incorrectly
deserialises the `Details` type, so the database fails to correctly
decode Tasks
2022-10-27 11:33:59 +02:00
Loïc Lecrenier
dc81992eb2
Implement TaskDeletion in the index scheduler 2022-10-27 11:33:59 +02:00
Kerollmops
fe84f2648b
Allow a user to disable the auto batching system 2022-10-27 11:33:59 +02:00
Kerollmops
e2a766acb5
Add a test to check that it works without autobatching 2022-10-27 11:33:58 +02:00
Kerollmops
db9d1b18ca
Remove the IndexScheduler::notify method 2022-10-27 11:33:58 +02:00
Kerollmops
19c6f8303f
Make sure that the index-scheduler tick loop is rerun after processing 2022-10-27 11:33:58 +02:00
Kerollmops
b311eb3bed
Add a test that verifies that sending multiple tasks works 2022-10-27 11:33:58 +02:00
Tamo
f026ac3115
remove unused files 2022-10-27 11:33:58 +02:00
Tamo
f176382b34
fix the tests 2022-10-27 11:33:57 +02:00
Tamo
4bd9e4d723
write a bunch of tests that goes through the whole compat layers 2022-10-27 11:33:57 +02:00
Tamo
c6f4fb5f7d
remove the warnings 2022-10-27 11:33:57 +02:00
Tamo
2ae0806773
rewrite the update file API 2022-10-27 11:33:57 +02:00
Tamo
7579a363ab
finish the dump reader API, the dump Writer API now needs to be updated 2022-10-27 11:33:57 +02:00
Tamo
0284764b5e
start dumping the update files to a known format 2022-10-27 11:33:56 +02:00
Tamo
9117fde712
fix the compat between v3 and v4 2022-10-27 11:33:56 +02:00
Tamo
f622ef9836
remove the ununsed snapshot files 2022-10-27 11:33:56 +02:00
Tamo
dc0f307d61
remove all warnings 2022-10-27 11:33:56 +02:00
Tamo
06fadb3004
write the compat layer from v2 to v3 2022-10-27 11:33:55 +02:00
Tamo
6107540ad4
remove old compat files 2022-10-27 11:33:55 +02:00
Tamo
7e18f92635
write the dump v2 import 2022-10-27 11:33:55 +02:00
Tamo
43496b97bd
make the open function public 2022-10-27 11:33:55 +02:00
Tamo
6f327a00c7
fix some warnings 2022-10-27 11:33:55 +02:00
Tamo
58ef80a2a7
rebase on main 2022-10-27 11:33:54 +02:00
Tamo
22ffbf3676
write and test the compat layer from v3 to v4 2022-10-27 11:33:54 +02:00
Tamo
089106a970
write and test the dump v3 import 2022-10-27 11:33:54 +02:00
Tamo
026f6fb06a
fix the test once again 2022-10-27 11:33:54 +02:00
Tamo
efe0a5f422
finish the test for the compatibility between v4 and v5 2022-10-27 11:33:53 +02:00
Tamo
47e0288747
rewrite the compat API to something more generic 2022-10-27 11:33:53 +02:00
Tamo
2f47443458
rename a few things for consistency 2022-10-27 11:33:53 +02:00
Tamo
a8128678a4
implement the dump v4 import 2022-10-27 11:33:53 +02:00