2663 Commits

Author	SHA1	Message	Date
Louis Dureuil	10f49f0d75	Post processing of the merge	2024-11-06 17:50:12 +01:00
Louis Dureuil	ee03743355	Merge branch 'indexer-edition-2024' into indexer-edition-2024-doc-chunks	2024-11-06 15:50:53 +01:00
Louis Dureuil	3658f57f93	Add progress	2024-11-04 15:10:40 +01:00
Louis Dureuil	a77d5ea8c1	Pass embedders to documents	2024-10-30 14:03:29 +01:00
Louis Dureuil	c9082130c8	support vectors or array of vectors	2024-10-30 13:50:51 +01:00
Louis Dureuil	df5bc3c9fd	Reintroduce vector errors	2024-10-30 10:55:57 +01:00
Louis Dureuil	0f6a1dbce7	habemus field distribution	2024-10-30 10:06:46 +01:00
Louis Dureuil	4ebedf4dc8	clippy fixes	2024-10-30 10:06:38 +01:00
Louis Dureuil	1075dd34bb	Vectors	2024-10-29 17:43:36 +01:00
Louis Dureuil	7058959a46	Write into documents	2024-10-28 16:18:48 +01:00
Louis Dureuil	9cbb2b066a	WIP vector extraction	2024-10-28 14:23:54 +01:00
Louis Dureuil	5efd70c251	Allow random access to fields in documents	2024-10-28 14:23:38 +01:00
Louis Dureuil	65470e26e0	Document trait changes	2024-10-28 14:23:20 +01:00
Louis Dureuil	bbb67ae0a8	todo channel	2024-10-28 14:23:02 +01:00
Louis Dureuil	af9f96e2af	Update older embedding	2024-10-28 14:22:45 +01:00
Louis Dureuil	1960003805	Remove some warnings	2024-10-28 14:22:19 +01:00
Louis Dureuil	2a91849660	Remove primary key from top id map	2024-10-28 14:21:50 +01:00
Louis Dureuil	c8189e975c	Add rendering based on document trait	2024-10-28 14:10:55 +01:00
Louis Dureuil	9e7c455a01	GlobalFieldIdMap manages metadata	2024-10-28 14:09:48 +01:00
Louis Dureuil	c22dc55694	Add embed_chunks_ref	2024-10-28 14:08:54 +01:00
Louis Dureuil	89243f7df0	WIP vector extraction	2024-10-21 10:39:40 +02:00
Louis Dureuil	9fe5122176	Fixup imports	2024-10-21 10:39:31 +02:00
Louis Dureuil	aff8ca4397	Add raw versions of parsed vectors	2024-10-21 10:39:05 +02:00
Louis Dureuil	1a3f4e719d	Vector document trait	2024-10-21 10:38:21 +02:00
Louis Dureuil	c278024709	Add vectors field and geo field to document trait	2024-10-21 10:37:40 +02:00
Louis Dureuil	73e29ee155	EmbeddingSender stub	2024-10-21 10:35:56 +02:00
Clément Renault	9c1e54a2c8	Move crates under a sub folder to clean up the code	2024-10-21 08:18:43 +02:00
Louis Dureuil	cd378e5bd2	Add chunking	2024-10-17 10:18:00 +02:00
Louis Dureuil	c1fcb2ebc6	add some warning	2024-10-17 09:43:11 +02:00
Louis Dureuil	0749633618	Don't sort in parallel in sorters of the new indexer	2024-10-17 09:30:18 +02:00
Louis Dureuil	0647f75e6b	Add borrow_mut_or_yield extension method	2024-10-16 17:36:41 +02:00
Louis Dureuil	c75de1f391	Remove TODO	2024-10-16 11:18:59 +02:00
Louis Dureuil	f9a6c624a7	Put primary key, and use provided key in operation	2024-10-16 09:27:00 +02:00
Louis Dureuil	017757004e	Add PrimaryKey::new_or_insert	2024-10-16 09:26:18 +02:00
Louis Dureuil	152683083b	Change document operation to use method in primary key	2024-10-15 14:08:37 +02:00
Louis Dureuil	c283c95f6a	Support nested primary keys	2024-10-15 14:08:37 +02:00
Clément Renault	9a0e1dc375	Fix the prefix deletion	2024-10-15 11:20:09 +02:00
Clément Renault	1e81d72b5f	Use the fixed version of the Rhai crate	2024-10-14 18:18:59 +02:00
Clément Renault	52b95c4e59	Make sure we edit the task statuses	2024-10-14 16:48:15 +02:00
Clément Renault	7e1dc8439b	Introduce the new update by function	2024-10-14 16:32:50 +02:00
Louis Dureuil	96658ec775	Make de public	2024-10-14 15:41:58 +02:00
Louis Dureuil	c01ee7b732	external changes	2024-10-14 15:41:58 +02:00
Louis Dureuil	6ad3f57bc1	Changes to de	2024-10-14 15:41:58 +02:00
Louis Dureuil	28d92c521a	External docids to &'bump str	2024-10-14 15:41:58 +02:00
Louis Dureuil	7df20d8282	Changes to primary key	2024-10-14 15:41:57 +02:00
ManyTheFish	b4102741e6	Fix duplicated fields when a document is modified	2024-10-14 14:59:40 +02:00
ManyTheFish	a525598ad6	Fix facet string indexing	2024-10-14 11:12:10 +02:00
ManyTheFish	4e97e38177	Serialize docids bitmap one time	2024-10-14 11:12:10 +02:00
ManyTheFish	d675e73af1	Finish prefix databases	2024-10-14 11:12:10 +02:00
ManyTheFish	a2fbf2ea21	set updated at at the end of the indexing	2024-10-14 11:05:25 +02:00