146 Commits

Author	SHA1	Message	Date
Tamo	7b8641a7af	fix the dump tests ... The issue was linked to the fact that the debug implementation of the PhantomData wasn't the same between rust stable and rust nightly. This was causing an issue while snapshsotting the settings and this commit fix it by representing the settings as json which already ignores the PhantomData	2022-11-23 16:59:20 +01:00
funilrys	e81b349658	Fix linting issue.	2022-11-14 18:51:34 +01:00
funilrys	0a102d601c	Update Task.created_at ... Indeed, before this patch we weren't considering the TaskContent::SetingsUpdate while trying to find the creation date.	2022-11-13 10:14:20 +01:00
funilrys	8a14f6f545	Add Task.processed_at.	2022-11-13 10:13:10 +01:00
funilrys	079357ee1f	Fix linting issues.	2022-11-12 20:57:27 +01:00
funilrys	06e7db7a1f	fixup! Extract the dates out of the dumpv4.	2022-11-12 18:28:23 +01:00
funilrys	953b2ec438	fixup! Extract the dates out of the dumpv4.	2022-11-02 17:49:37 +01:00
funilrys	ab3056cc66	Extract the dates out of the dumpv4. ... This patch possibly fixes #2987. This patch introduces a way to fill the IndexMetadata.created_at and IndexMetadata.updated_at keys from the tasks events. This is done by reading the creation date of the first event (created_at) and the creation date of the last event (updated_at).	2022-10-31 18:58:14 +01:00
Irevoire	510afda590	remove unused import	2022-10-30 20:05:20 +01:00
Irevoire	fea9fdcd7e	fix the dump reader process when no instance-uid was specified	2022-10-30 20:00:27 +01:00
curquiza	68f80dbacf	Update version for the next release (v0.30.0) in Cargo.toml files	2022-10-27 11:35:44 +00:00
Irevoire	225405bb0d	ignore the dump tests	2022-10-27 11:35:22 +02:00
Irevoire	6aa816d96a	use meili-snap in the dump	2022-10-27 11:35:22 +02:00
Irevoire	866a3676eb	reupload the test fix for the dump	2022-10-27 11:35:22 +02:00
Kerollmops	fa84eae0f1	Insta review and fix insta snapshots	2022-10-27 11:35:21 +02:00
Irevoire	3979c9f02b	fix all the dump snasphots	2022-10-27 11:35:20 +02:00
Kerollmops	314b89ca30	Fix insta snapshots	2022-10-27 11:35:20 +02:00
Irevoire	0ba6253eed	fix the sort	2022-10-27 11:35:19 +02:00
Irevoire	e8cd571820	try to convert the OsStr to a rust string to fix the sort	2022-10-27 11:35:19 +02:00
Loïc Lecrenier	e641d08846	Cargo fmt	2022-10-27 11:35:18 +02:00
Loïc Lecrenier	1f75caae88	Fix a few index swap bugs. ... 1. Details of the indexSwap task 2. Query tasks with type=indexUid 3. Synchronous error message for multiple index not found	2022-10-27 11:35:17 +02:00
Irevoire	a16604af80	fix all the tests	2022-10-27 11:35:17 +02:00
Kerollmops	2de8a0711a	Cargo insta test/review	2022-10-27 11:35:16 +02:00
Kerollmops	7074872a78	cargo insta accept	2022-10-27 11:35:15 +02:00
Kerollmops	e0548e42e7	Rename the Snapshot task into SnapshotCreation	2022-10-27 11:35:14 +02:00
Irevoire	0aca5e84b9	rename received_document_ids to matched_documents in the DocumentDeletion task type (reimplementation of #2826)	2022-10-27 11:35:12 +02:00
Irevoire	7e52f1effb	remove a lot of unecessary clone and ref	2022-10-27 11:35:11 +02:00
Irevoire	ecf4e43b3d	rename the dumpExport to dumpCreation	2022-10-27 11:35:10 +02:00
Irevoire	eb4bdde432	fix clippy	2022-10-27 11:35:09 +02:00
Irevoire	e9055f5572	fix clippy	2022-10-27 11:35:08 +02:00
Irevoire	de7d4200d8	fix the snapshot tests of the dump after renaming a bunch of kinds	2022-10-27 11:35:08 +02:00
Irevoire	8d1408c65e	fix the import of the dumpv4&v5 when there is no instance-uid + rename the Kind+KindWithContent+Details variant for the DocumentImport and the Setting	2022-10-27 11:35:05 +02:00
Clément Renault	32cfac0cfd	Sort the TOML dependencies	2022-10-27 11:35:05 +02:00
Clément Renault	80b2e70ee7	Introduce a rustfmt file	2022-10-27 11:35:05 +02:00
Loïc Lecrenier	17cd2a4aa0	Implement POST /indexes-swap	2022-10-27 11:34:15 +02:00
Irevoire	66c3b93ef1	fix all the snapshot tests in the dump	2022-10-27 11:34:15 +02:00
Tamo	6730e190db	fix the dumps tests since we added informations in the DumpTask	2022-10-27 11:34:13 +02:00
Kerollmops	290945e258	Update the canceledBy and finishedAt fields	2022-10-27 11:34:11 +02:00
Kerollmops	725158b454	Introduce the core algorithm of task cancelation	2022-10-27 11:34:11 +02:00
Kerollmops	591527a99d	Prefer using TaskDeletion in the dumps	2022-10-27 11:34:11 +02:00
Kerollmops	1ca9a67c49	Introduce the task cancelation task type	2022-10-27 11:34:11 +02:00
Tamo	6bd6321226	dump the content of the dump tasks instead of recreating at import time with wrong API keys	2022-10-27 11:34:08 +02:00
Tamo	83f3c5ec57	flush the dump-writer only once everything has been inserted	2022-10-27 11:34:08 +02:00
Tamo	78ce29f461	apply most style comments of the review	2022-10-27 11:34:08 +02:00
Tamo	dd70daaae3	Update dump/src/error.rs ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2022-10-27 11:34:08 +02:00
Tamo	d0e91555d1	rebase on index-scheduler	2022-10-27 11:34:08 +02:00
Tamo	a9eeb070b8	fix all the errors code and settings issues when importing a dump v2	2022-10-27 11:34:07 +02:00
Tamo	3872a1b8d1	fix all the error codes	2022-10-27 11:34:07 +02:00
Tamo	955d3339f0	remove the dbg	2022-10-27 11:34:06 +02:00
Tamo	208c785697	add a bufwriter on the documents	2022-10-27 11:34:06 +02:00
Tamo	d976e680c5	first mostly working version	2022-10-27 11:34:06 +02:00
Tamo	c051166bcc	update the API a little bit	2022-10-27 11:34:05 +02:00
Tamo	72a906ae75	fix the tests	2022-10-27 11:34:05 +02:00
Tamo	b7f9c94f4a	write the dump export	2022-10-27 11:34:05 +02:00
Tamo	7034803712	move the API key in meilisearch_types	2022-10-27 11:34:02 +02:00
Tamo	c192146fbe	remove an unused file	2022-10-27 11:34:02 +02:00
Tamo	2f1eb78b1d	refactor the Task a little bit	2022-10-27 11:34:02 +02:00
Tamo	510ce9fc51	start moving a lot of task types to meilisearch_types	2022-10-27 11:34:01 +02:00
Tamo	fa4c1de019	store md5 instead of the whole snapshots	2022-10-27 11:34:01 +02:00
Tamo	0af00f6b32	fix all the import and comment most of the dump v6	2022-10-27 11:34:01 +02:00
Tamo	667c282e19	get rids of the index crate + the document_types crate	2022-10-27 11:34:00 +02:00
Tamo	f026ac3115	remove unused files	2022-10-27 11:33:58 +02:00
Tamo	f176382b34	fix the tests	2022-10-27 11:33:57 +02:00
Tamo	4bd9e4d723	write a bunch of tests that goes through the whole compat layers	2022-10-27 11:33:57 +02:00
Tamo	c6f4fb5f7d	remove the warnings	2022-10-27 11:33:57 +02:00
Tamo	2ae0806773	rewrite the update file API	2022-10-27 11:33:57 +02:00
Tamo	7579a363ab	finish the dump reader API, the dump Writer API now needs to be updated	2022-10-27 11:33:57 +02:00
Tamo	0284764b5e	start dumping the update files to a known format	2022-10-27 11:33:56 +02:00
Tamo	9117fde712	fix the compat between v3 and v4	2022-10-27 11:33:56 +02:00
Tamo	f622ef9836	remove the ununsed snapshot files	2022-10-27 11:33:56 +02:00
Tamo	dc0f307d61	remove all warnings	2022-10-27 11:33:56 +02:00
Tamo	06fadb3004	write the compat layer from v2 to v3	2022-10-27 11:33:55 +02:00
Tamo	6107540ad4	remove old compat files	2022-10-27 11:33:55 +02:00
Tamo	7e18f92635	write the dump v2 import	2022-10-27 11:33:55 +02:00
Tamo	43496b97bd	make the open function public	2022-10-27 11:33:55 +02:00
Tamo	6f327a00c7	fix some warnings	2022-10-27 11:33:55 +02:00
Tamo	58ef80a2a7	rebase on main	2022-10-27 11:33:54 +02:00
Tamo	22ffbf3676	write and test the compat layer from v3 to v4	2022-10-27 11:33:54 +02:00
Tamo	089106a970	write and test the dump v3 import	2022-10-27 11:33:54 +02:00
Tamo	026f6fb06a	fix the test once again	2022-10-27 11:33:54 +02:00
Tamo	efe0a5f422	finish the test for the compatibility between v4 and v5	2022-10-27 11:33:53 +02:00
Tamo	47e0288747	rewrite the compat API to something more generic	2022-10-27 11:33:53 +02:00
Tamo	2f47443458	rename a few things for consistency	2022-10-27 11:33:53 +02:00
Tamo	a8128678a4	implement the dump v4 import	2022-10-27 11:33:53 +02:00
Tamo	c50b44039e	add the compat layer between v5 and v6	2022-10-27 11:33:53 +02:00
Tamo	6dcc5851b5	get rids of the trait in most places	2022-10-27 11:33:52 +02:00
Tamo	0972587cfc	start writting the compat layer between v5 and v6	2022-10-27 11:33:52 +02:00
Tamo	afd5fe0783	test the dump v5	2022-10-27 11:33:52 +02:00
Tamo	1473a71e33	write the v5 dump import	2022-10-27 11:33:52 +02:00
Tamo	101f55ce8b	introduce the index metadata	2022-10-27 11:33:52 +02:00
Tamo	e845cc2b6f	fix the tests	2022-10-27 11:33:51 +02:00
Tamo	7bd6f63001	implement the dump reader v6	2022-10-27 11:33:51 +02:00
Tamo	699ae1b190	start implementing a skeleton of the v1 dump reader	2022-10-27 11:33:51 +02:00
Tamo	f041d474a5	move the DumpWriter and Error to their own module	2022-10-27 11:33:51 +02:00
Tamo	ece6c3f6e7	fix the dump export	2022-10-27 11:33:51 +02:00
Tamo	87a6a337aa	write a dump exporter	2022-10-27 11:33:51 +02:00