509 Commits

Author	SHA1	Message	Date
Tamo	0ba1c46e19	fix a deadlock	2022-10-27 11:33:44 +02:00
Tamo	22bfb5a7a0	remove Clone from the IndexScheduler	2022-10-27 11:33:44 +02:00
Tamo	d8d3499aec	remove a bunch of comments	2022-10-27 11:33:44 +02:00
Tamo	64e132ce53	move as many fields as possible out of the IndexScheduler	2022-10-27 11:33:44 +02:00
Tamo	9e1f38ec7c	move the test function in the test module	2022-10-27 11:33:44 +02:00
Tamo	6f4dcc0c38	start implementing some logic to test the internal states of the scheduler	2022-10-27 11:33:43 +02:00
Tamo	84cd5cef0b	fix the tests	2022-10-27 11:33:43 +02:00
Tamo	ae86a8ccd6	slightly refactor the autobatching tests	2022-10-27 11:33:43 +02:00
Tamo	ce2dfecc03	connect the new scheduler to meilisearch-http officially. ... I can index documents and do search	2022-10-27 11:33:43 +02:00
Tamo	cb4feabca2	implements the get_tasks	2022-10-27 11:33:43 +02:00
Tamo	19154e48fe	fix all compilation errors	2022-10-27 11:33:42 +02:00
Irevoire	8d51c1f389	wip integrating the scheduler in meilisearch-http	2022-10-27 11:33:42 +02:00
Irevoire	250410495c	start integrating the index-scheduler in meilisearch-lib	2022-10-27 11:33:42 +02:00
Irevoire	8f0fd35358	add insta::json for later	2022-10-27 11:33:42 +02:00
Irevoire	8770e07397	I can index documents without meilisearch	2022-10-27 11:33:42 +02:00
Tamo	edd8344dc9	wip	2022-10-27 11:33:42 +02:00
Tamo	e547552702	create the end Batch type for all Index* operations	2022-10-27 11:33:41 +02:00
Tamo	925971809a	create the end Batch type for all Document* operation	2022-10-27 11:33:41 +02:00
Tamo	1ea9c0b4c0	write most of the run loop	2022-10-27 11:33:41 +02:00
Tamo	9ff0fe952e	split the run function in two	2022-10-27 11:33:41 +02:00
Tamo	a8b18b2c96	fix the register test	2022-10-27 11:33:40 +02:00
Tamo	5436b996ab	reduce the size of the snapshots	2022-10-27 11:33:40 +02:00
Tamo	7d0c8a3379	test the register tasks	2022-10-27 11:33:40 +02:00
Tamo	fc098022c7	start integrating the index-scheduler in the meilisearch codebase	2022-10-27 11:33:40 +02:00
Tamo	b816535e33	greatly reduce the number of warnings	2022-10-27 11:33:40 +02:00
Tamo	38e4ffe73c	fix smol typo	2022-10-27 11:33:40 +02:00
Tamo	366a344474	get rids of the horrendous spinlock in favor of synchronoise	2022-10-27 11:33:39 +02:00
Tamo	7b6673dc1d	implement the index swap in the index mapper	2022-10-27 11:33:39 +02:00
Tamo	03aca2e452	move the index mapping logic in another structure	2022-10-27 11:33:39 +02:00
Tamo	4129783019	migrate the index handling code in a different file + implements the create index	2022-10-27 11:33:39 +02:00
Tamo	1804416afa	reintroduce the uuid mapping for the indexes	2022-10-27 11:33:39 +02:00
Tamo	c97d51a624	add a bunch of tests	2022-10-27 11:33:39 +02:00
Tamo	803f2157af	split the DocumentAdditionOrUpdate in two tasks; DocumentAddition and DocumentUpdate	2022-10-27 11:33:38 +02:00
Tamo	b7c5b71a53	starts importing the real tasks	2022-10-27 11:33:38 +02:00
Tamo	5cc8f96237	get rids of the auto-generated mains	2022-10-27 11:33:38 +02:00
Tamo	94e29a9f5f	extract the index abstraction out of the index-scheduler in its own module	2022-10-27 11:33:38 +02:00
Tamo	48138c21a9	rename the update-file-store to file-store since it can store any kind of file	2022-10-27 11:33:38 +02:00
Tamo	76597fc382	import the update_file_store in the index-scheduler	2022-10-27 11:33:37 +02:00
Tamo	a0588d6b94	finishes the global skelton of the auto-batcher	2022-10-27 11:33:37 +02:00
Tamo	b3c9b128d9	polish the global structure of the batch creation	2022-10-27 11:33:37 +02:00
Irevoire	448f44f631	move the autobatcher logic to another file	2022-10-27 11:33:36 +02:00
Tamo	f638774764	add the document format file	2022-10-27 11:33:36 +02:00
Tamo	516860f342	fix the create_new_batch method	2022-10-27 11:33:36 +02:00
Tamo	6b9689a1c0	fix the whole batchKind thingy	2022-10-27 11:33:36 +02:00
Tamo	af0f5d6c0c	implements most operations	2022-10-27 11:33:36 +02:00
Tamo	5a7fcf2688	fix a few typos	2022-10-27 11:33:35 +02:00
Tamo	30d2b24689	implements the index deletion, creation and swap	2022-10-27 11:33:35 +02:00
Tamo	72b2e68de4	makes the updates getters smoother to uses	2022-10-27 11:33:35 +02:00
Tamo	7879189c6b	make the project compile again	2022-10-27 11:33:35 +02:00
Tamo	46b8ebcab4	fix the file store	2022-10-27 11:33:35 +02:00
Tamo	fa742f60e8	make the file store entirely synchronous, including the file deletion	2022-10-27 11:33:35 +02:00
Tamo	a7aa92df5f	fix most of the index module	2022-10-27 11:33:34 +02:00
Irevoire	d8b8e04ad1	wip porting the index back in the scheduler	2022-10-27 11:33:34 +02:00
Irevoire	fe330e1be9	add a little bit of documentation	2022-10-27 11:33:34 +02:00
Tamo	2c4e5ce8be	implements the filter query	2022-10-27 11:33:34 +02:00
Tamo	705af94fd7	add the task to the index db in the register task	2022-10-27 11:33:34 +02:00
Tamo	ed745591e1	split the scheduler into multiples files	2022-10-27 11:33:34 +02:00
Tamo	22d24dba56	implement the get_batch method	2022-10-27 11:33:33 +02:00
Tamo	1a47949063	START THE REWRITE OF THE INDEX SCHEDULER: index & register has been implemented	2022-10-27 11:33:33 +02:00