10114 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Clément Renault	a7a01646cf	Remove the useless Manually drop	2024-10-03 15:57:31 +02:00
Clément Renault	0409a26cd8	Replace the concurrent vec by a linked list	2024-10-03 15:15:29 +02:00
ManyTheFish	35f78b5423	TO REMOVE: usefull debug prints	2024-10-03 11:13:01 +02:00
ManyTheFish	14261f8f04	Integrate facet level bulk update ... Only the facet bulk update has been added so far, the incremental must be completely rewritten Factorize facet merging Fix facet level extraction	2024-10-03 11:13:00 +02:00
ManyTheFish	774ed28539	Fix Prefix FST when a document is modified	2024-10-03 11:12:26 +02:00
ManyTheFish	d79f75f630	Compute and Write external-documents-ids database	2024-10-03 11:11:56 +02:00
Clément Renault	ccf01c2471	Merge pull request #4969 from meilisearch/indexer-edition-2024-try-map ... Indexer edition 2024 try map	2024-10-02 11:25:05 +02:00
Clément Renault	b7a5ba100e	Move the ParallelIteratorExt into the parallel_iterator_ext module	2024-10-01 11:11:52 +02:00
Clément Renault	dead7a56a3	Keep the caches in the AppendOnlyVec	2024-10-01 11:11:39 +02:00
Clément Renault	0a8cb471df	Introduce the AppendOnlyVec struct for the parallel computing	2024-10-01 11:11:25 +02:00
Clément Renault	00e045b249	Rename and use the try_arc_for_each_try_init method	2024-10-01 11:11:25 +02:00
Clément Renault	d83c9a4074	Introduce the try_for_each_try_init method to be used with Arced Errors	2024-10-01 11:11:25 +02:00
Clément Renault	f3356ddaa4	Fix the errors when using the try_map_try_init method	2024-10-01 11:11:10 +02:00
Clément Renault	31de5c747e	WIP using try_map_try_init	2024-10-01 11:10:53 +02:00
Clément Renault	3843240940	Prefer using Ars instead of Options	2024-10-01 11:10:53 +02:00
Louis Dureuil	8cb5e7437d	try using try_map_try_init	2024-10-01 11:10:53 +02:00
Louis Dureuil	5b776556fe	Add ParallelIteratorExt	2024-10-01 11:10:53 +02:00
ManyTheFish	bb7a503e5d	Compute prefix databases ... We are now computing the prefix FST and a prefix delta in the Merger thread, after all the databases are written, the main thread will recompute the prefix databases based on the prefix delta without needing any grenad temporary file anymore	2024-10-01 09:57:06 +02:00
Louis Dureuil	64589278ac	Appease *some* of clippy warnings	2024-09-30 16:08:29 +02:00
ManyTheFish	8df6daf308	Remove fid_wordcount_docids.rs	2024-09-30 11:52:31 +02:00
ManyTheFish	5b552caf42	Fix position in insertions	2024-09-30 11:46:32 +02:00
ManyTheFish	2b51a63418	Remove dead code	2024-09-30 11:42:36 +02:00
Louis Dureuil	3d8024fb2b	write the weighted fields ids map	2024-09-30 11:35:03 +02:00
Louis Dureuil	4b0da0ff24	Fix inversion of field_id and position	2024-09-30 11:34:50 +02:00
Louis Dureuil	079f2b5de0	Format error messages consistently	2024-09-30 11:34:31 +02:00
ManyTheFish	960060ebdf	Fix fst builder when their is no previous FST	2024-09-25 16:53:00 +02:00
Clément Renault	3d244451df	Reduce the lru key size from 8 to 12 bytes	2024-09-25 16:14:13 +02:00
Clément Renault	5f53935c8a	Fix a bug in the Lru	2024-09-25 16:09:34 +02:00
Clément Renault	29a7623c3f	Fxi some logs	2024-09-25 15:57:50 +02:00
Clément Renault	e97041f7d0	Replace the Lru free list by a simple increment	2024-09-25 15:55:52 +02:00
Clément Renault	52d7f3ed1c	Reduce the lru key size from 20 to 8 bytes	2024-09-25 15:37:13 +02:00
Clément Renault	86d5e6d9ff	Use the new Lru	2024-09-25 14:54:56 +02:00
Clément Renault	759b9b1546	Introduce a new custom Lru	2024-09-25 14:49:12 +02:00
ManyTheFish	3f7a500f3b	Build prefix fst	2024-09-25 14:36:06 +02:00
ManyTheFish	974272f2e9	Merge branch 'main' into indexer-edition-2024	2024-09-25 07:41:16 +02:00
Clément Renault	7ad037841f	Move the tracing info to eprintln	2024-09-24 18:21:58 +02:00
Clément Renault	e0c7067355	Expose an IndexedParallelIterator to the index function	2024-09-24 17:24:59 +02:00
ManyTheFish	6e87332410	Change the way the FST is built	2024-09-24 16:28:31 +02:00
Clément Renault	2d1caf27df	Use eprintln to log	2024-09-24 15:59:50 +02:00
Clément Renault	92678383d6	Update charabia	2024-09-24 15:37:56 +02:00
Clément Renault	7f148c127c	Measure the SmallVec efficacity	2024-09-24 15:32:15 +02:00
Clément Renault	4ce5d3d66d	Do not check before pushing in bitmaps	2024-09-24 09:43:16 +02:00
Clément Renault	ff931edb55	Update roaring to inline max calls	2024-09-23 16:53:42 +02:00
Clément Renault	42b093687d	Introduce the new PushOptimizedBitmap	2024-09-23 16:38:21 +02:00
Clément Renault	835c5f98f9	Remove the debug symbols	2024-09-23 15:49:24 +02:00
Clément Renault	f00664247d	Add more stats about the channel message sent	2024-09-23 15:13:52 +02:00
Clément Renault	3c63d4a1e5	Fix charabia Zho	2024-09-23 14:50:17 +02:00
Clément Renault	4551abf6d4	Update roaring to the latest version	2024-09-23 14:35:33 +02:00
Clément Renault	193d7f5d34	Add the mutualized charabia normalization	2024-09-23 14:24:25 +02:00
Clément Renault	013acb3d93	Measure merger writer channel contention	2024-09-23 11:07:59 +02:00