67 Commits

Author	SHA1	Message	Date
Louis Dureuil	0647f75e6b	Add borrow_mut_or_yield extension method	2024-10-16 17:36:41 +02:00
Louis Dureuil	c75de1f391	Remove TODO	2024-10-16 11:18:59 +02:00
Louis Dureuil	f9a6c624a7	Put primary key, and use provided key in operation	2024-10-16 09:27:00 +02:00
Louis Dureuil	152683083b	Change document operation to use method in primary key	2024-10-15 14:08:37 +02:00
Louis Dureuil	c283c95f6a	Support nested primary keys	2024-10-15 14:08:37 +02:00
Clément Renault	9a0e1dc375	Fix the prefix deletion	2024-10-15 11:20:09 +02:00
Clément Renault	1e81d72b5f	Use the fixed version of the Rhai crate	2024-10-14 18:18:59 +02:00
Clément Renault	52b95c4e59	Make sure we edit the task statuses	2024-10-14 16:48:15 +02:00
Clément Renault	7e1dc8439b	Introduce the new update by function	2024-10-14 16:32:50 +02:00
Louis Dureuil	96658ec775	Make de public	2024-10-14 15:41:58 +02:00
Louis Dureuil	c01ee7b732	external changes	2024-10-14 15:41:58 +02:00
Louis Dureuil	6ad3f57bc1	Changes to de	2024-10-14 15:41:58 +02:00
ManyTheFish	d675e73af1	Finish prefix databases	2024-10-14 11:12:10 +02:00
ManyTheFish	a2fbf2ea21	set updated at at the end of the indexing	2024-10-14 11:05:25 +02:00
ManyTheFish	132916f62c	Only run word pair proximity docids extraction if proximity_precision enables it	2024-10-14 11:05:25 +02:00
Louis Dureuil	8371819114	Some clippy related fixes	2024-10-14 10:58:37 +02:00
Louis Dureuil	6028d6ba43	Remove somme warnings	2024-10-10 22:42:37 +02:00
Louis Dureuil	68a2502388	Introduce indexer level bumpalo	2024-10-10 22:23:05 +02:00
Clément Renault	39b27e42be	Plug the deletion pipeline	2024-10-08 16:04:19 +02:00
ManyTheFish	35f78b5423	TO REMOVE: usefull debug prints	2024-10-03 11:13:01 +02:00
ManyTheFish	14261f8f04	Integrate facet level bulk update ... Only the facet bulk update has been added so far, the incremental must be completely rewritten Factorize facet merging Fix facet level extraction	2024-10-03 11:13:00 +02:00
ManyTheFish	d79f75f630	Compute and Write external-documents-ids database	2024-10-03 11:11:56 +02:00
Clément Renault	b7a5ba100e	Move the ParallelIteratorExt into the parallel_iterator_ext module	2024-10-01 11:11:52 +02:00
Clément Renault	00e045b249	Rename and use the try_arc_for_each_try_init method	2024-10-01 11:11:25 +02:00
Clément Renault	d83c9a4074	Introduce the try_for_each_try_init method to be used with Arced Errors	2024-10-01 11:11:25 +02:00
Clément Renault	f3356ddaa4	Fix the errors when using the try_map_try_init method	2024-10-01 11:11:10 +02:00
Clément Renault	31de5c747e	WIP using try_map_try_init	2024-10-01 11:10:53 +02:00
Louis Dureuil	8cb5e7437d	try using try_map_try_init	2024-10-01 11:10:53 +02:00
ManyTheFish	bb7a503e5d	Compute prefix databases ... We are now computing the prefix FST and a prefix delta in the Merger thread, after all the databases are written, the main thread will recompute the prefix databases based on the prefix delta without needing any grenad temporary file anymore	2024-10-01 09:57:06 +02:00
Louis Dureuil	64589278ac	Appease *some* of clippy warnings	2024-09-30 16:08:29 +02:00
ManyTheFish	2b51a63418	Remove dead code	2024-09-30 11:42:36 +02:00
Louis Dureuil	3d8024fb2b	write the weighted fields ids map	2024-09-30 11:35:03 +02:00
Clément Renault	e0c7067355	Expose an IndexedParallelIterator to the index function	2024-09-24 17:24:59 +02:00
Clément Renault	013acb3d93	Measure merger writer channel contention	2024-09-23 11:07:59 +02:00
ManyTheFish	7ba49b849e	Extract and write facet databases	2024-09-16 09:35:16 +02:00
Clément Renault	f7652186e1	WIP geo fields	2024-09-12 18:01:02 +02:00
Clément Renault	b2f4e67c9a	Do not store useless updates	2024-09-12 15:38:31 +02:00
Clément Renault	ff5d3b59f5	Move the document id extraction to the primary key code	2024-09-12 12:01:42 +02:00
Clément Renault	3e9198ebaa	Support guessing primary key again	2024-09-11 17:25:40 +02:00
Clément Renault	2a0ad0982f	Fix the document counter	2024-09-11 15:59:36 +02:00
ManyTheFish	39b5990f64	Mutualize tokenization	2024-09-11 10:22:38 +02:00
Clément Renault	8287c2644f	Support CSV again	2024-09-10 21:10:28 +01:00
Clément Renault	c1c44a0b81	Impl serialize on TopLevelMap	2024-09-10 19:32:03 +01:00
Clément Renault	04596f3616	Move the TopLevelMap into a dedicated module	2024-09-10 18:01:17 +01:00
Clément Renault	24cb5839ad	Move the document changes sorting logic to a new trait	2024-09-10 17:37:52 +01:00
Clément Renault	8fd0afaaaa	Make sure we iterate over the payload documents in order	2024-09-06 08:09:08 +02:00
Clément Renault	72c6a21a30	Use raw JSON to read the payloads	2024-09-05 20:08:23 +02:00
Clément Renault	8412be4a7d	Cleanup CowStr and TopLevelMap struct	2024-09-05 18:32:55 +02:00
Louis Dureuil	10f09c531f	add some commented code to read from json with raw values	2024-09-05 18:22:16 +02:00
ManyTheFish	8fd99b111b	Add tracing timers logs	2024-09-05 18:00:22 +02:00