23 Commits

Author	SHA1	Message	Date
ManyTheFish	4e97e38177	Serialize docids bitmap one time	2024-10-14 11:12:10 +02:00
ManyTheFish	14261f8f04	Integrate facet level bulk update ... Only the facet bulk update has been added so far, the incremental must be completely rewritten Factorize facet merging Fix facet level extraction	2024-10-03 11:13:00 +02:00
ManyTheFish	d79f75f630	Compute and Write external-documents-ids database	2024-10-03 11:11:56 +02:00
ManyTheFish	bb7a503e5d	Compute prefix databases ... We are now computing the prefix FST and a prefix delta in the Merger thread, after all the databases are written, the main thread will recompute the prefix databases based on the prefix delta without needing any grenad temporary file anymore	2024-10-01 09:57:06 +02:00
Louis Dureuil	64589278ac	Appease *some* of clippy warnings	2024-09-30 16:08:29 +02:00
Clément Renault	7ad037841f	Move the tracing info to eprintln	2024-09-24 18:21:58 +02:00
Clément Renault	f00664247d	Add more stats about the channel message sent	2024-09-23 15:13:52 +02:00
Clément Renault	013acb3d93	Measure merger writer channel contention	2024-09-23 11:07:59 +02:00
ManyTheFish	7ba49b849e	Extract and write facet databases	2024-09-16 09:35:16 +02:00
Clément Renault	f7652186e1	WIP geo fields	2024-09-12 18:01:02 +02:00
Clément Renault	73ce67862d	Use the word pair proximity and fid word count docids extractors ... Co-authored-by: ManyTheFish <many@meilisearch.com>	2024-09-05 10:56:22 +02:00
Clément Renault	1d59c19cd2	Send the WordsFst by using an Mmap	2024-09-04 14:30:09 +02:00
Clément Renault	98e48371c3	Factorize some stuff	2024-09-04 12:17:13 +02:00
Clément Renault	6d74fb0229	Introduce the WordFidWordDocids database	2024-09-04 11:40:55 +02:00
Clément Renault	27b4cab857	Extract and write the documents and words fst in the database	2024-09-04 09:59:19 +02:00
ManyTheFish	c50d3edc4a	Integrate first searchable exctrator	2024-09-03 11:02:39 +02:00
Clément Renault	521775f788	I push for Many	2024-09-02 15:10:21 +02:00
Clément Renault	e639ec79d1	Move the indexers into their own modules	2024-09-02 10:42:19 +02:00
Clément Renault	794ebcd582	Replace grenad with the new grenad various-improvement branch	2024-08-30 11:53:59 +02:00
Clément Renault	0c57cf7565	Replace obkv with the temporary new version of it	2024-08-30 11:53:58 +02:00
Clément Renault	27df9e6c73	Introduce the indexer::index function that runs the indexation	2024-08-30 11:53:58 +02:00
Clément Renault	45c060831e	Introduce typed channels and the merger loop	2024-08-30 11:53:58 +02:00
Clément Renault	874c1ac538	First channels types	2024-08-30 11:53:58 +02:00