Commit Graph

45 Commits

Author SHA1 Message Date
Louis Dureuil 732c52093d
Processing time without autobatching implementation 2023-05-03 17:41:48 +02:00
Louis Dureuil a94e78ffb0
Disable autobatching of additions and deletions 2023-04-12 10:53:00 +02:00
Tamo ea9ac46f28
stop autobatching the deletion without the index creation right with the addition 2023-02-08 21:24:27 +01:00
Tamo 67dda0678f
cleanup the autobatcher a little bit 2023-02-08 18:10:59 +01:00
Tamo 2db6347686
update the autobatcher to batch the addition and deletion together 2023-02-08 18:07:59 +01:00
Tamo c7b2e3be87
apply review comments 2023-01-24 17:54:43 +01:00
Tamo 7d1ebb7295
add test on the autobatcher layer 2023-01-23 20:56:12 +01:00
Tamo 767cb725a5
reimplement the batching of task with or without primary key in the autobatcher 2023-01-23 20:18:22 +01:00
Loïc Lecrenier 1f75caae88
Fix a few index swap bugs.
1. Details of the indexSwap task
2. Query tasks with type=indexUid
3. Synchronous error message for multiple index not found
2022-10-27 11:35:17 +02:00
Kerollmops e0548e42e7
Rename the Snapshot task into SnapshotCreation 2022-10-27 11:35:14 +02:00
Irevoire ecf4e43b3d
rename the dumpExport to dumpCreation 2022-10-27 11:35:10 +02:00
Irevoire e9055f5572
fix clippy 2022-10-27 11:35:08 +02:00
Irevoire 8d1408c65e
fix the import of the dumpv4&v5 when there is no instance-uid + rename the Kind+KindWithContent+Details variant for the DocumentImport and the Setting 2022-10-27 11:35:05 +02:00
Clément Renault 80b2e70ee7
Introduce a rustfmt file 2022-10-27 11:35:05 +02:00
Irevoire 5303bbffab
fix the last rule about merging the allow_index_creation 2022-10-27 11:34:16 +02:00
Irevoire fc944c39a5
simplify the code A LOT and create less false positive 2022-10-27 11:34:16 +02:00
Irevoire a1d4cc673d
add a whole new batch of tests around the index already exists / allow_index_creation 2022-10-27 11:34:16 +02:00
Irevoire 28d9f2c041
fix all the snapshot tests 2022-10-27 11:34:16 +02:00
Irevoire d9218578e3
it probably works but it's also horrendous 2022-10-27 11:34:16 +02:00
Loïc Lecrenier 17cd2a4aa0
Implement POST /indexes-swap 2022-10-27 11:34:15 +02:00
Loïc Lecrenier 169f386418
Add some documentation to the index scheduler 2022-10-27 11:34:15 +02:00
Tamo cff003c928
remove the unused variants from the autobatcher 2022-10-27 11:34:14 +02:00
Kerollmops 725158b454
Introduce the core algorithm of task cancelation 2022-10-27 11:34:11 +02:00
Kerollmops 1ca9a67c49
Introduce the task cancelation task type 2022-10-27 11:34:11 +02:00
Loïc Lecrenier 9522b75454
Continue implementation of task deletion
1. Matched tasks are a roaring bitmap
2. Start implementation in meilisearch-http
3. Snapshots use meili-snap
4. Rename to TaskDeletion
2022-10-27 11:34:03 +02:00
Kerollmops 408d00136c
Extract index creation rights and simplify the autobatcher rules 2022-10-27 11:34:03 +02:00
Tamo 7034803712
move the API key in meilisearch_types 2022-10-27 11:34:02 +02:00
Tamo 2f1eb78b1d
refactor the Task a little bit 2022-10-27 11:34:02 +02:00
Tamo 510ce9fc51
start moving a lot of task types to meilisearch_types 2022-10-27 11:34:01 +02:00
Tamo 667c282e19
get rids of the index crate + the document_types crate 2022-10-27 11:34:00 +02:00
Loïc Lecrenier 568199fc0d
Add more task deletion tests 2022-10-27 11:33:59 +02:00
Loïc Lecrenier dc81992eb2
Implement TaskDeletion in the index scheduler 2022-10-27 11:33:59 +02:00
Clément Renault 068a4b2884
Correctly batch tasks with different index creation rights 2022-10-27 11:33:50 +02:00
Clément Renault 87212cfd20
Use a ControlFlow in the autobatcher function 2022-10-27 11:33:50 +02:00
Kerollmops 9e8242c57d
Remove the IndexRename operation 2022-10-27 11:33:48 +02:00
Kerollmops 5d21c790ef
Make clippy happy 2022-10-27 11:33:47 +02:00
Kerollmops f68906f5dc
Merge both DocumentAddition/Update into one DocumentImport variant 2022-10-27 11:33:46 +02:00
Tamo ae86a8ccd6
slightly refactor the autobatching tests 2022-10-27 11:33:43 +02:00
Tamo 5436b996ab
reduce the size of the snapshots 2022-10-27 11:33:40 +02:00
Tamo b816535e33
greatly reduce the number of warnings 2022-10-27 11:33:40 +02:00
Tamo c97d51a624
add a bunch of tests 2022-10-27 11:33:39 +02:00
Tamo 803f2157af
split the DocumentAdditionOrUpdate in two tasks; DocumentAddition and DocumentUpdate 2022-10-27 11:33:38 +02:00
Tamo b7c5b71a53
starts importing the real tasks 2022-10-27 11:33:38 +02:00
Tamo a0588d6b94
finishes the global skelton of the auto-batcher 2022-10-27 11:33:37 +02:00
Irevoire 448f44f631
move the autobatcher logic to another file 2022-10-27 11:33:36 +02:00