189 Commits

Author	SHA1	Message	Date
funilrys	8a14f6f545	Add Task.processed_at.	2022-11-13 10:13:10 +01:00
funilrys	079357ee1f	Fix linting issues.	2022-11-12 20:57:27 +01:00
funilrys	06e7db7a1f	fixup! Extract the dates out of the dumpv4.	2022-11-12 18:28:23 +01:00
funilrys	953b2ec438	fixup! Extract the dates out of the dumpv4.	2022-11-02 17:49:37 +01:00
funilrys	ab3056cc66	Extract the dates out of the dumpv4. ... This patch possibly fixes #2987. This patch introduces a way to fill the IndexMetadata.created_at and IndexMetadata.updated_at keys from the tasks events. This is done by reading the creation date of the first event (created_at) and the creation date of the last event (updated_at).	2022-10-31 18:58:14 +01:00
Irevoire	510afda590	remove unused import	2022-10-30 20:05:20 +01:00
Irevoire	fea9fdcd7e	fix the dump reader process when no instance-uid was specified	2022-10-30 20:00:27 +01:00
Irevoire	225405bb0d	ignore the dump tests	2022-10-27 11:35:22 +02:00
Irevoire	6aa816d96a	use meili-snap in the dump	2022-10-27 11:35:22 +02:00
Irevoire	866a3676eb	reupload the test fix for the dump	2022-10-27 11:35:22 +02:00
Kerollmops	fa84eae0f1	Insta review and fix insta snapshots	2022-10-27 11:35:21 +02:00
Irevoire	3979c9f02b	fix all the dump snasphots	2022-10-27 11:35:20 +02:00
Kerollmops	314b89ca30	Fix insta snapshots	2022-10-27 11:35:20 +02:00
Irevoire	0ba6253eed	fix the sort	2022-10-27 11:35:19 +02:00
Irevoire	e8cd571820	try to convert the OsStr to a rust string to fix the sort	2022-10-27 11:35:19 +02:00
Loïc Lecrenier	e641d08846	Cargo fmt	2022-10-27 11:35:18 +02:00
Loïc Lecrenier	1f75caae88	Fix a few index swap bugs. ... 1. Details of the indexSwap task 2. Query tasks with type=indexUid 3. Synchronous error message for multiple index not found	2022-10-27 11:35:17 +02:00
Irevoire	a16604af80	fix all the tests	2022-10-27 11:35:17 +02:00
Kerollmops	2de8a0711a	Cargo insta test/review	2022-10-27 11:35:16 +02:00
Kerollmops	7074872a78	cargo insta accept	2022-10-27 11:35:15 +02:00
Kerollmops	e0548e42e7	Rename the Snapshot task into SnapshotCreation	2022-10-27 11:35:14 +02:00
Irevoire	0aca5e84b9	rename received_document_ids to matched_documents in the DocumentDeletion task type (reimplementation of #2826)	2022-10-27 11:35:12 +02:00
Irevoire	7e52f1effb	remove a lot of unecessary clone and ref	2022-10-27 11:35:11 +02:00
Irevoire	ecf4e43b3d	rename the dumpExport to dumpCreation	2022-10-27 11:35:10 +02:00
Irevoire	eb4bdde432	fix clippy	2022-10-27 11:35:09 +02:00
Irevoire	e9055f5572	fix clippy	2022-10-27 11:35:08 +02:00
Irevoire	de7d4200d8	fix the snapshot tests of the dump after renaming a bunch of kinds	2022-10-27 11:35:08 +02:00
Irevoire	8d1408c65e	fix the import of the dumpv4&v5 when there is no instance-uid + rename the Kind+KindWithContent+Details variant for the DocumentImport and the Setting	2022-10-27 11:35:05 +02:00
Clément Renault	80b2e70ee7	Introduce a rustfmt file	2022-10-27 11:35:05 +02:00
Loïc Lecrenier	17cd2a4aa0	Implement POST /indexes-swap	2022-10-27 11:34:15 +02:00
Irevoire	66c3b93ef1	fix all the snapshot tests in the dump	2022-10-27 11:34:15 +02:00
Tamo	6730e190db	fix the dumps tests since we added informations in the DumpTask	2022-10-27 11:34:13 +02:00
Kerollmops	290945e258	Update the canceledBy and finishedAt fields	2022-10-27 11:34:11 +02:00
Kerollmops	725158b454	Introduce the core algorithm of task cancelation	2022-10-27 11:34:11 +02:00
Kerollmops	591527a99d	Prefer using TaskDeletion in the dumps	2022-10-27 11:34:11 +02:00
Kerollmops	1ca9a67c49	Introduce the task cancelation task type	2022-10-27 11:34:11 +02:00
Tamo	6bd6321226	dump the content of the dump tasks instead of recreating at import time with wrong API keys	2022-10-27 11:34:08 +02:00
Tamo	83f3c5ec57	flush the dump-writer only once everything has been inserted	2022-10-27 11:34:08 +02:00
Tamo	78ce29f461	apply most style comments of the review	2022-10-27 11:34:08 +02:00
Tamo	dd70daaae3	Update dump/src/error.rs ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2022-10-27 11:34:08 +02:00
Tamo	d0e91555d1	rebase on index-scheduler	2022-10-27 11:34:08 +02:00
Tamo	a9eeb070b8	fix all the errors code and settings issues when importing a dump v2	2022-10-27 11:34:07 +02:00
Tamo	3872a1b8d1	fix all the error codes	2022-10-27 11:34:07 +02:00
Tamo	955d3339f0	remove the dbg	2022-10-27 11:34:06 +02:00
Tamo	208c785697	add a bufwriter on the documents	2022-10-27 11:34:06 +02:00
Tamo	d976e680c5	first mostly working version	2022-10-27 11:34:06 +02:00
Tamo	c051166bcc	update the API a little bit	2022-10-27 11:34:05 +02:00
Tamo	72a906ae75	fix the tests	2022-10-27 11:34:05 +02:00
Tamo	b7f9c94f4a	write the dump export	2022-10-27 11:34:05 +02:00
Tamo	7034803712	move the API key in meilisearch_types	2022-10-27 11:34:02 +02:00
Tamo	2f1eb78b1d	refactor the Task a little bit	2022-10-27 11:34:02 +02:00
Tamo	510ce9fc51	start moving a lot of task types to meilisearch_types	2022-10-27 11:34:01 +02:00
Tamo	fa4c1de019	store md5 instead of the whole snapshots	2022-10-27 11:34:01 +02:00
Tamo	0af00f6b32	fix all the import and comment most of the dump v6	2022-10-27 11:34:01 +02:00
Tamo	f026ac3115	remove unused files	2022-10-27 11:33:58 +02:00
Tamo	f176382b34	fix the tests	2022-10-27 11:33:57 +02:00
Tamo	4bd9e4d723	write a bunch of tests that goes through the whole compat layers	2022-10-27 11:33:57 +02:00
Tamo	c6f4fb5f7d	remove the warnings	2022-10-27 11:33:57 +02:00
Tamo	2ae0806773	rewrite the update file API	2022-10-27 11:33:57 +02:00
Tamo	7579a363ab	finish the dump reader API, the dump Writer API now needs to be updated	2022-10-27 11:33:57 +02:00
Tamo	0284764b5e	start dumping the update files to a known format	2022-10-27 11:33:56 +02:00
Tamo	9117fde712	fix the compat between v3 and v4	2022-10-27 11:33:56 +02:00
Tamo	f622ef9836	remove the ununsed snapshot files	2022-10-27 11:33:56 +02:00
Tamo	dc0f307d61	remove all warnings	2022-10-27 11:33:56 +02:00
Tamo	06fadb3004	write the compat layer from v2 to v3	2022-10-27 11:33:55 +02:00
Tamo	6107540ad4	remove old compat files	2022-10-27 11:33:55 +02:00
Tamo	7e18f92635	write the dump v2 import	2022-10-27 11:33:55 +02:00
Tamo	43496b97bd	make the open function public	2022-10-27 11:33:55 +02:00
Tamo	6f327a00c7	fix some warnings	2022-10-27 11:33:55 +02:00
Tamo	58ef80a2a7	rebase on main	2022-10-27 11:33:54 +02:00
Tamo	22ffbf3676	write and test the compat layer from v3 to v4	2022-10-27 11:33:54 +02:00
Tamo	089106a970	write and test the dump v3 import	2022-10-27 11:33:54 +02:00
Tamo	026f6fb06a	fix the test once again	2022-10-27 11:33:54 +02:00
Tamo	efe0a5f422	finish the test for the compatibility between v4 and v5	2022-10-27 11:33:53 +02:00
Tamo	47e0288747	rewrite the compat API to something more generic	2022-10-27 11:33:53 +02:00
Tamo	2f47443458	rename a few things for consistency	2022-10-27 11:33:53 +02:00
Tamo	a8128678a4	implement the dump v4 import	2022-10-27 11:33:53 +02:00
Tamo	c50b44039e	add the compat layer between v5 and v6	2022-10-27 11:33:53 +02:00
Tamo	6dcc5851b5	get rids of the trait in most places	2022-10-27 11:33:52 +02:00
Tamo	0972587cfc	start writting the compat layer between v5 and v6	2022-10-27 11:33:52 +02:00
Tamo	afd5fe0783	test the dump v5	2022-10-27 11:33:52 +02:00
Tamo	1473a71e33	write the v5 dump import	2022-10-27 11:33:52 +02:00
Tamo	101f55ce8b	introduce the index metadata	2022-10-27 11:33:52 +02:00
Tamo	e845cc2b6f	fix the tests	2022-10-27 11:33:51 +02:00
Tamo	7bd6f63001	implement the dump reader v6	2022-10-27 11:33:51 +02:00
Tamo	699ae1b190	start implementing a skeleton of the v1 dump reader	2022-10-27 11:33:51 +02:00
Tamo	f041d474a5	move the DumpWriter and Error to their own module	2022-10-27 11:33:51 +02:00
Tamo	ece6c3f6e7	fix the dump export	2022-10-27 11:33:51 +02:00
Tamo	87a6a337aa	write a dump exporter	2022-10-27 11:33:51 +02:00