196 Commits

Author	SHA1	Message	Date
marin postma	112cd1787c	change error message for uuid resolver	2021-06-21 10:59:40 +02:00
marin postma	58f9974be4	remove anyhow refs & implement missing errors	2021-06-21 10:59:38 +02:00
bors[bot]	c1b6f0e833	Merge #183 ... 183: Add cropping and update `_formatted` behavior r=curquiza a=MarinPostma TODO: - [x] Solves #5 - [x] Solves #203 - [x] integrate the new milli highlight (according to the query words) Co-authored-by: Marin Postma <postma.marin@protonmail.com> Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>	2021-06-18 11:18:37 +00:00
Clémentine Urquizar	e4b3d35ed8	Fix clippy errors	2021-06-17 17:03:43 +02:00
marin postma	5a47cef9a8	bump milli to 0.4.0	2021-06-16 17:15:56 +02:00
Tamo	f068d7f978	makes clippy happy	2021-06-14 14:47:53 +02:00
Tamo	18d4d6097a	implements the synonyms in transplant	2021-06-14 14:47:51 +02:00
bors[bot]	b119bb4ab0	Merge #197 ... 197: Update milli (v0.3.1) with filterable attributes r=MarinPostma a=curquiza Fixes #187 and #70 Also fixes #195 Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>	2021-06-14 12:19:42 +00:00
bors[bot]	d65b5db97f	Merge #144 #173 ... 144: Concurrent update run loop (refactor) r=MarinPostma a=MarinPostma This PR allows multiple request to the update store to be performed concurently (i.e, one can list updates while an updates in being written to the update store). 173: Convert UpdateStatus to legacy meilisearch format r=MarinPostma a=MarinPostma Returns the update statuses with the same format as legacy meilisearch. The number of documents in a document addition/deletion is not known before processing, so it is only returned when the update is `processed`. close #78 associated milli PR: https://github.com/meilisearch/milli/pull/178 Co-authored-by: marin postma <postma.marin@protonmail.com> Co-authored-by: Marin Postma <postma.marin@protonmail.com>	2021-06-14 11:30:44 +00:00
Clémentine Urquizar	88bf867a3e	Rename attributes for faceting into filterable attributes	2021-06-14 13:20:43 +02:00
Clémentine Urquizar	aa04124bfc	Add changes according to milli update	2021-06-14 13:20:37 +02:00
marin postma	0f767e3743	conccurrent update run loop	2021-06-14 10:57:14 +02:00
Marin Postma	e8bd5ea4e0	convert UpdateStatus to legacy meilisearch format	2021-06-14 10:21:57 +02:00
mpostma	3ef0830c5d	review changes	2021-06-10 16:11:52 +02:00
mpostma	eb7616ca0f	remove dbg	2021-06-10 16:03:48 +02:00
Irevoire	20e1caef47	makes clippy happy	2021-06-10 16:03:48 +02:00
Irevoire	99551fc21b	fix encoding bug	2021-06-10 16:03:48 +02:00
mpostma	2716c1aebb	fix update store lock	2021-06-09 16:19:45 +02:00
mpostma	1a65eed724	fix index creation bug	2021-06-09 11:52:36 +02:00
marin postma	75c0718691	fix update loop infinite loop	2021-06-02 17:29:50 +02:00
Tamo	d0552e765e	forbid deserialization of Setting<Checked>	2021-06-01 20:41:45 +02:00
marin	df6ba0e824	Apply suggestions from code review ... Co-authored-by: Irevoire <tamo@meilisearch.com>	2021-06-01 11:18:37 +02:00
Marin Postma	6609f9e3be	review edits	2021-05-31 18:41:37 +02:00
Marin Postma	1c4f0b2ccf	clippy, fmt & tests	2021-05-31 16:03:39 +02:00
Marin Postma	10fc870684	improve dump info reports	2021-05-31 15:49:04 +02:00
Marin Postma	b3c8f0e1f6	fix empty index error	2021-05-31 10:58:51 +02:00
Marin Postma	bc5a5e37ea	fix dump v1	2021-05-31 10:42:31 +02:00
Marin Postma	33c6c4f0ee	add timestamos to dump info	2021-05-30 15:55:17 +02:00
Marin Postma	39c16c0fe4	fix dump import	2021-05-30 12:35:17 +02:00
Marin Postma	1cb64caae4	dump content is now only uuid	2021-05-29 00:08:17 +02:00
Marin Postma	b258f4f394	fix dump import	2021-05-27 14:30:20 +02:00
Marin Postma	c47369839b	dump meta	2021-05-27 10:51:19 +02:00
Marin Postma	b924e897f1	load index dump	2021-05-27 10:27:47 +02:00
Marin Postma	e818c33fec	implement load uuid_resolver	2021-05-26 20:42:09 +02:00
Marin Postma	9278a6fe59	integrate in dump actor	2021-05-25 18:14:11 +02:00
Marin Postma	3593ebb8aa	dump updates	2021-05-25 16:44:58 +02:00
Marin Postma	464639aa0f	udpate actor error improvements	2021-05-25 16:44:58 +02:00
Marin Postma	4acbe8e473	implement index dump	2021-05-25 16:44:58 +02:00
Marin Postma	7ad553670f	index error handling	2021-05-25 16:44:58 +02:00
Marin Postma	2185fb8367	dump uuid resolver	2021-05-25 16:44:54 +02:00
tamo	89846d1656	improve panic message	2021-05-25 15:47:57 +02:00
tamo	1a6dcec83a	crash when the actor have no inbox	2021-05-25 15:23:13 +02:00
tamo	991d8e1ec6	fix the error printing	2021-05-25 10:48:57 +02:00
tamo	49a0e8aa19	use a RwLock instead of a Mutex	2021-05-24 18:19:34 +02:00
tamo	912f0286b3	remove the dump_inner trickery	2021-05-24 18:06:20 +02:00
tamo	dcf29e1081	fix the error handling in case there is a panic while creating a dump	2021-05-24 17:33:42 +02:00
tamo	529f7962f4	handle parallel requests for the dump actor	2021-05-24 15:42:12 +02:00
tamo	8a11c6c429	Implements the legacy behaviour of the dump ... When asked if a dump exists we check if it's the current dump, and if it's not then we check on the filesystem for any file matching our `uid.dump`	2021-05-24 12:35:46 +02:00
tamo	4cbf866821	merge with main	2021-05-12 18:12:37 +02:00
Marin Postma	295f496e8a	atomic index dump load	2021-05-12 16:21:37 +02:00
Marin Postma	1b671d4302	fix-snapshot	2021-05-11 13:57:18 +02:00
tamo	c30b32e173	add the criterion attribute when importing dumps from the v1	2021-05-11 13:21:36 +02:00
tamo	9e798fea75	fix the import of dump without unprocessing updates	2021-05-11 13:03:47 +02:00
tamo	92a7c8cd17	make clippy happy	2021-05-11 00:27:22 +02:00
tamo	8b7735c20a	move the import of the updates in the v2 and ignore the v1 for now	2021-05-11 00:20:55 +02:00
tamo	7d748fa384	integrate the new Settings in the dumps	2021-05-10 20:48:06 +02:00
tamo	d767990424	fix the import of the updates in the dump	2021-05-10 20:25:12 +02:00
tamo	ef438852cd	fix the v1	2021-05-10 20:25:12 +02:00
tamo	40ced3ff8d	first working version	2021-05-10 20:25:12 +02:00
tamo	5f5402a3ab	provide a way to access the internal content path of all processing State	2021-05-10 20:25:12 +02:00
tamo	26dcb9e66d	bump milli version and fix a performance issue for large dumps	2021-05-10 20:25:12 +02:00
Marin Postma	956012da95	fix dump lock	2021-05-10 20:25:12 +02:00
tamo	24192fc550	fix tests	2021-05-10 20:25:12 +02:00
tamo	efca63f9ce	[WIP] rebase on main	2021-05-10 20:25:09 +02:00
tamo	c3552cecdf	WIP rebase on main	2021-05-10 20:24:18 +02:00
tamo	0f94ef8abc	WIP: dump	2021-05-10 20:24:18 +02:00
tamo	0275b36fb0	[WIP] rebase on main	2021-05-10 20:24:14 +02:00
tamo	1b5fc61eb6	[WIP] rebase on main	2021-05-10 20:23:12 +02:00
tamo	0fee81678e	[WIP] rebase on main	2021-05-10 20:22:18 +02:00
tamo	c4d898a265	split the dumps between v1 and v2	2021-05-10 20:20:57 +02:00
tamo	e389c088eb	WIP: rebasing on master	2021-05-10 20:20:57 +02:00
Marin Postma	706643dfed	type setting struct	2021-05-10 17:30:09 +02:00
Marin Postma	ec7eb7798f	remove facet setting	2021-05-04 22:36:31 +02:00
Marin Postma	a961f0ce75	fix clippy warnings	2021-04-27 18:28:46 +02:00
Marin Postma	4fe2a13c71	rewrite update store	2021-04-27 15:20:52 +02:00
Marin Postma	51829ad85e	review fixes	2021-04-27 15:10:57 +02:00
Marin Postma	c78f351300	fix tests	2021-04-27 15:10:57 +02:00
Marin Postma	ee675eadf1	fix stats	2021-04-27 15:10:55 +02:00
Marin Postma	33830d5ecf	fix snapshots	2021-04-27 15:09:55 +02:00
Marin Postma	2b154524bb	fix filtered out pending update	2021-04-27 15:09:23 +02:00
Marin Postma	b626d02ffe	simplify index actor run loop	2021-04-27 15:09:22 +02:00
Marin Postma	9ce68d11a7	single update store instance	2021-04-27 15:09:21 +02:00
Marin Postma	5a38f13cae	multi_index udpate store	2021-04-27 15:07:13 +02:00
Clémentine Urquizar	1ba46f8f77	Disable clippy rule	2021-04-27 12:43:00 +02:00
Clémentine Urquizar	f80ea24d2b	Add tests on every platform and fix clippy errors	2021-04-27 12:42:59 +02:00
Alexey Shekhirin	adfdb99abc	feat(http): calculate updates' and uuids' dbs size	2021-04-09 15:59:12 +03:00
Alexey Shekhirin	ae1655586c	fixes after review	2021-04-09 14:40:48 +03:00
Alexey Shekhirin	698a1ea582	feat(http): store processing as RwLock<Option<Uuid>> in index_actor	2021-04-09 14:34:43 +03:00
Alexey Shekhirin	87412f63ef	feat(http): implement is_indexing for stats	2021-04-09 14:34:42 +03:00
Alexey Shekhirin	dd9eae8c26	feat(http): stats route	2021-04-09 14:34:42 +03:00
Clémentine Urquizar	cb23775d18	Rename pending into enqueued	2021-04-07 19:46:36 +02:00
tamo	0f2143e7fd	remove the now useless dead_code flags	2021-03-26 14:15:12 +01:00
mpostma	d7c077cffb	atomic snapshot import	2021-03-25 14:48:51 +01:00
mpostma	7d6ec7f3d3	resolve merge	2021-03-25 14:21:05 +01:00
mpostma	f3dc853be3	Merge remote-tracking branch 'origin/main' into snapshots	2021-03-25 13:45:07 +01:00
mpostma	48507460b2	add snapshot tests	2021-03-25 12:02:10 +01:00
mpostma	d029464de8	fix snapshot path	2021-03-25 10:23:31 +01:00
mpostma	79d09705d8	perform snapshot on startup	2021-03-25 09:35:15 +01:00
bors[bot]	868658f3d8	Merge #109 ... 109: Make updates atomic r=curquiza a=MarinPostma Until now, the index_uid->uuid mapping was done before the update was written to disk in the case of automatic index creation. This was an issue when the update failed, and the index would still exists in the uuid resolver. This is fixed by this pr, by first creating the update with an uuid if the index does not exist, and then register this uuid to the uuid resolver. This is preliminary work to the implementation of snapshots (#19). This pr also changes the `resolve` method on the `UuidResolver` to `get` to make it clearer. The `create_uuid` method may be bound to disappear when the index name resolution is handled by a remote machine. Co-authored-by: mpostma <postma.marin@protonmail.com>	2021-03-24 12:24:32 +00:00
mpostma	d892a2643e	fix clippy	2021-03-24 12:38:59 +01:00