Commit Graph

187 Commits

Author SHA1 Message Date
funilrys
06e7db7a1f
fixup! Extract the dates out of the dumpv4. 2022-11-12 18:28:23 +01:00
funilrys
953b2ec438
fixup! Extract the dates out of the dumpv4. 2022-11-02 17:49:37 +01:00
funilrys
ab3056cc66
Extract the dates out of the dumpv4.
This patch possibly fixes #2987.

This patch introduces a way to fill the IndexMetadata.created_at
and IndexMetadata.updated_at keys from the tasks events.
This is done by reading the creation date of the first event
(created_at) and the creation date of the last event (updated_at).
2022-10-31 18:58:14 +01:00
Irevoire
510afda590
remove unused import 2022-10-30 20:05:20 +01:00
Irevoire
fea9fdcd7e
fix the dump reader process when no instance-uid was specified 2022-10-30 20:00:27 +01:00
Irevoire
225405bb0d
ignore the dump tests 2022-10-27 11:35:22 +02:00
Irevoire
6aa816d96a
use meili-snap in the dump 2022-10-27 11:35:22 +02:00
Irevoire
866a3676eb
reupload the test fix for the dump 2022-10-27 11:35:22 +02:00
Kerollmops
fa84eae0f1
Insta review and fix insta snapshots 2022-10-27 11:35:21 +02:00
Irevoire
3979c9f02b
fix all the dump snasphots 2022-10-27 11:35:20 +02:00
Kerollmops
314b89ca30
Fix insta snapshots 2022-10-27 11:35:20 +02:00
Irevoire
0ba6253eed
fix the sort 2022-10-27 11:35:19 +02:00
Irevoire
e8cd571820
try to convert the OsStr to a rust string to fix the sort 2022-10-27 11:35:19 +02:00
Loïc Lecrenier
e641d08846
Cargo fmt 2022-10-27 11:35:18 +02:00
Loïc Lecrenier
1f75caae88
Fix a few index swap bugs.
1. Details of the indexSwap task
2. Query tasks with type=indexUid
3. Synchronous error message for multiple index not found
2022-10-27 11:35:17 +02:00
Irevoire
a16604af80
fix all the tests 2022-10-27 11:35:17 +02:00
Kerollmops
2de8a0711a
Cargo insta test/review 2022-10-27 11:35:16 +02:00
Kerollmops
7074872a78
cargo insta accept 2022-10-27 11:35:15 +02:00
Kerollmops
e0548e42e7
Rename the Snapshot task into SnapshotCreation 2022-10-27 11:35:14 +02:00
Irevoire
0aca5e84b9
rename received_document_ids to matched_documents in the DocumentDeletion task type (reimplementation of #2826) 2022-10-27 11:35:12 +02:00
Irevoire
7e52f1effb
remove a lot of unecessary clone and ref 2022-10-27 11:35:11 +02:00
Irevoire
ecf4e43b3d
rename the dumpExport to dumpCreation 2022-10-27 11:35:10 +02:00
Irevoire
eb4bdde432
fix clippy 2022-10-27 11:35:09 +02:00
Irevoire
e9055f5572
fix clippy 2022-10-27 11:35:08 +02:00
Irevoire
de7d4200d8
fix the snapshot tests of the dump after renaming a bunch of kinds 2022-10-27 11:35:08 +02:00
Irevoire
8d1408c65e
fix the import of the dumpv4&v5 when there is no instance-uid + rename the Kind+KindWithContent+Details variant for the DocumentImport and the Setting 2022-10-27 11:35:05 +02:00
Clément Renault
80b2e70ee7
Introduce a rustfmt file 2022-10-27 11:35:05 +02:00
Loïc Lecrenier
17cd2a4aa0
Implement POST /indexes-swap 2022-10-27 11:34:15 +02:00
Irevoire
66c3b93ef1
fix all the snapshot tests in the dump 2022-10-27 11:34:15 +02:00
Tamo
6730e190db
fix the dumps tests since we added informations in the DumpTask 2022-10-27 11:34:13 +02:00
Kerollmops
290945e258
Update the canceledBy and finishedAt fields 2022-10-27 11:34:11 +02:00
Kerollmops
725158b454
Introduce the core algorithm of task cancelation 2022-10-27 11:34:11 +02:00
Kerollmops
591527a99d
Prefer using TaskDeletion in the dumps 2022-10-27 11:34:11 +02:00
Kerollmops
1ca9a67c49
Introduce the task cancelation task type 2022-10-27 11:34:11 +02:00
Tamo
6bd6321226
dump the content of the dump tasks instead of recreating at import time with wrong API keys 2022-10-27 11:34:08 +02:00
Tamo
83f3c5ec57
flush the dump-writer only once everything has been inserted 2022-10-27 11:34:08 +02:00
Tamo
78ce29f461
apply most style comments of the review 2022-10-27 11:34:08 +02:00
Tamo
dd70daaae3
Update dump/src/error.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-10-27 11:34:08 +02:00
Tamo
d0e91555d1
rebase on index-scheduler 2022-10-27 11:34:08 +02:00
Tamo
a9eeb070b8
fix all the errors code and settings issues when importing a dump v2 2022-10-27 11:34:07 +02:00
Tamo
3872a1b8d1
fix all the error codes 2022-10-27 11:34:07 +02:00
Tamo
955d3339f0
remove the dbg 2022-10-27 11:34:06 +02:00
Tamo
208c785697
add a bufwriter on the documents 2022-10-27 11:34:06 +02:00
Tamo
d976e680c5
first mostly working version 2022-10-27 11:34:06 +02:00
Tamo
c051166bcc
update the API a little bit 2022-10-27 11:34:05 +02:00
Tamo
72a906ae75
fix the tests 2022-10-27 11:34:05 +02:00
Tamo
b7f9c94f4a
write the dump export 2022-10-27 11:34:05 +02:00
Tamo
7034803712
move the API key in meilisearch_types 2022-10-27 11:34:02 +02:00
Tamo
2f1eb78b1d
refactor the Task a little bit 2022-10-27 11:34:02 +02:00
Tamo
510ce9fc51
start moving a lot of task types to meilisearch_types 2022-10-27 11:34:01 +02:00
Tamo
fa4c1de019
store md5 instead of the whole snapshots 2022-10-27 11:34:01 +02:00
Tamo
0af00f6b32
fix all the import and comment most of the dump v6 2022-10-27 11:34:01 +02:00
Tamo
f026ac3115
remove unused files 2022-10-27 11:33:58 +02:00
Tamo
f176382b34
fix the tests 2022-10-27 11:33:57 +02:00
Tamo
4bd9e4d723
write a bunch of tests that goes through the whole compat layers 2022-10-27 11:33:57 +02:00
Tamo
c6f4fb5f7d
remove the warnings 2022-10-27 11:33:57 +02:00
Tamo
2ae0806773
rewrite the update file API 2022-10-27 11:33:57 +02:00
Tamo
7579a363ab
finish the dump reader API, the dump Writer API now needs to be updated 2022-10-27 11:33:57 +02:00
Tamo
0284764b5e
start dumping the update files to a known format 2022-10-27 11:33:56 +02:00
Tamo
9117fde712
fix the compat between v3 and v4 2022-10-27 11:33:56 +02:00
Tamo
f622ef9836
remove the ununsed snapshot files 2022-10-27 11:33:56 +02:00
Tamo
dc0f307d61
remove all warnings 2022-10-27 11:33:56 +02:00
Tamo
06fadb3004
write the compat layer from v2 to v3 2022-10-27 11:33:55 +02:00
Tamo
6107540ad4
remove old compat files 2022-10-27 11:33:55 +02:00
Tamo
7e18f92635
write the dump v2 import 2022-10-27 11:33:55 +02:00
Tamo
43496b97bd
make the open function public 2022-10-27 11:33:55 +02:00
Tamo
6f327a00c7
fix some warnings 2022-10-27 11:33:55 +02:00
Tamo
58ef80a2a7
rebase on main 2022-10-27 11:33:54 +02:00
Tamo
22ffbf3676
write and test the compat layer from v3 to v4 2022-10-27 11:33:54 +02:00
Tamo
089106a970
write and test the dump v3 import 2022-10-27 11:33:54 +02:00
Tamo
026f6fb06a
fix the test once again 2022-10-27 11:33:54 +02:00
Tamo
efe0a5f422
finish the test for the compatibility between v4 and v5 2022-10-27 11:33:53 +02:00
Tamo
47e0288747
rewrite the compat API to something more generic 2022-10-27 11:33:53 +02:00
Tamo
2f47443458
rename a few things for consistency 2022-10-27 11:33:53 +02:00
Tamo
a8128678a4
implement the dump v4 import 2022-10-27 11:33:53 +02:00
Tamo
c50b44039e
add the compat layer between v5 and v6 2022-10-27 11:33:53 +02:00
Tamo
6dcc5851b5
get rids of the trait in most places 2022-10-27 11:33:52 +02:00
Tamo
0972587cfc
start writting the compat layer between v5 and v6 2022-10-27 11:33:52 +02:00
Tamo
afd5fe0783
test the dump v5 2022-10-27 11:33:52 +02:00
Tamo
1473a71e33
write the v5 dump import 2022-10-27 11:33:52 +02:00
Tamo
101f55ce8b
introduce the index metadata 2022-10-27 11:33:52 +02:00
Tamo
e845cc2b6f
fix the tests 2022-10-27 11:33:51 +02:00
Tamo
7bd6f63001
implement the dump reader v6 2022-10-27 11:33:51 +02:00
Tamo
699ae1b190
start implementing a skeleton of the v1 dump reader 2022-10-27 11:33:51 +02:00
Tamo
f041d474a5
move the DumpWriter and Error to their own module 2022-10-27 11:33:51 +02:00
Tamo
ece6c3f6e7
fix the dump export 2022-10-27 11:33:51 +02:00
Tamo
87a6a337aa
write a dump exporter 2022-10-27 11:33:51 +02:00