206 Commits

Author	SHA1	Message	Date
Louis Dureuil	f87747f4d3	Remove unwraps	2024-03-25 11:23:04 +01:00
Louis Dureuil	ac52c857e8	Update ollama and openai impls to use the rest embedder internally	2024-03-25 11:23:03 +01:00
Louis Dureuil	b11df7ec34	Meilisearch: fix some wrong spans	2024-03-05 10:11:43 +01:00
ManyTheFish	3beda8833d	Fix and add logs	2024-02-14 11:46:30 +01:00
Many the fish	e5e811e2c9	Update milli/src/update/index_documents/extract/mod.rs ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2024-02-13 14:22:21 +01:00
ManyTheFish	be1b054b05	Compute chunk size based on the input data size ant the number of indexing threads	2024-02-08 17:28:37 +01:00
Tamo	bf43a3f60a	fix typo	2024-02-08 15:04:06 +01:00
Tamo	1502382316	use debug instead of debug_span	2024-02-08 15:04:06 +01:00
Tamo	08af0e690c	Structures a bunch of logs	2024-02-08 15:04:06 +01:00
Tamo	e773dfa9ba	get rids of log in milli and add logs for the bucket sort	2024-02-08 15:04:05 +01:00
Louis Dureuil	5d7061682e	Add tracing to milli	2024-02-08 15:03:31 +01:00
Louis Dureuil	fbf5f2a392	Don't use a runtime in extract_embedder, use it only for OpenAI	2024-02-01 10:33:27 +01:00
Tamo	9f8f3105d5	make clippy happy	2024-02-01 10:33:27 +01:00
Tamo	318843aacd	add a bunch of tests and fix the error message when adding the geosearch as filterable/sortable while there is malformed documents in the DB	2024-02-01 10:33:27 +01:00
Tamo	c1bf33a112	Revert "Remove panic on the geosearch"	2024-01-25 18:51:19 +01:00
Tamo	0887186ecf	make clippy happy	2024-01-17 16:07:10 +01:00
Tamo	7d190d8078	add a bunch of tests and fix the error message when adding the geosearch as filterable/sortable while there is malformed documents in the DB	2024-01-17 15:51:52 +01:00
Clément Renault	9f9ad4cc05	Fix Clippy warnings	2024-01-16 15:27:24 +01:00
Clément Renault	3ee7682fa7	Fix some integer comparisons	2024-01-16 15:22:23 +01:00
Many the fish	9e1b458010	Merge branch 'main' into change-proximity-precision-settings	2023-12-18 09:08:47 +01:00
ManyTheFish	6425996e36	Change the naming of attributeScale and wordScale into byAttribute and byWord	2023-12-14 16:31:00 +01:00
Louis Dureuil	e0cc775dc4	Various changes ... - DistributionShift in Search object (to be set from model in embed?) - Fix issue where embedder index wasn't computed at search time - Accept as default embedder either the "default" one, or the only embedder when there is only one	2023-12-14 16:08:41 +01:00
Louis Dureuil	12940d79a9	WIP ... - manual embedder - multi embedders OK - clippy + tests OK	2023-12-14 16:08:41 +01:00
Louis Dureuil	922a640188	WIP multi embedders ... fixed template bugs	2023-12-14 16:08:41 +01:00
Louis Dureuil	687d92f217	prompt bifluor+	2023-12-14 16:07:49 +01:00
Louis Dureuil	fb539f61fe	WIP	2023-12-14 16:07:49 +01:00
Louis Dureuil	dde3a04679	WIP arroy integration	2023-12-14 16:07:49 +01:00
Louis Dureuil	13c2c6c16b	Small commit to add hybrid search and autoembedding	2023-12-14 16:07:48 +01:00
ManyTheFish	467b49153d	Implement proximityPrecision setting on milli side	2023-12-06 15:49:02 +01:00
ManyTheFish	bddc168d83	List TODOs	2023-12-06 14:59:23 +01:00
Clément Renault	0dbf1a16ff	Make clippy happy	2023-11-23 14:11:38 +01:00
Clément Renault	0d4482625a	Make the changes to use heed v0.20-alpha.6	2023-11-23 11:43:58 +01:00
ManyTheFish	39cbb499c2	Small fixes	2023-11-20 10:20:39 +01:00
ManyTheFish	263e825619	Fix typos in comments	2023-11-20 10:06:29 +01:00
Louis Dureuil	cbaa54cafd	Fix clippy issues	2023-11-06 11:19:31 +01:00
Louis Dureuil	1ad1fcc8c8	Remove all warnings	2023-11-06 10:31:14 +01:00
Clément Renault	ff522c919d	Fix the vector extractions for the diff indexing	2023-11-02 15:58:08 +01:00
ManyTheFish	1b4ff991c0	update typed chunks	2023-11-02 15:26:20 +01:00
ManyTheFish	4b64c33aa2	update vector extractor	2023-11-02 15:26:20 +01:00
Louis Dureuil	f19332466e	Extract field value as values instead of Option<Value>	2023-10-31 16:36:27 +01:00
Louis Dureuil	be395c7944	Change order of arguments to tokenizer_builder	2023-10-30 16:26:29 +01:00
Louis Dureuil	abf424ebfc	Remove unused FromIterator	2023-10-30 11:41:56 +01:00
ManyTheFish	762b0b47e6	Use deladd merging function in chunks mergers	2023-10-30 11:40:20 +01:00
Louis Dureuil	cda6ca1ee6	Remove TypedChunk::NewDocumentIds	2023-10-30 11:40:18 +01:00
ManyTheFish	476e4d3dbe	Use value buffer instead of the initial value when writting the final result in the sorter	2023-10-30 11:39:31 +01:00
Clément Renault	576fa9c6da	Remove useless comment	2023-10-30 11:39:31 +01:00
Kerollmops	77dcbff6b2	Remove and Insert the DelAdd geo points	2023-10-30 11:39:31 +01:00
Kerollmops	544440c363	Ignore geo fields when the Del and Add content is the same	2023-10-30 11:39:31 +01:00
Clément Renault	a3dae4db9b	Extract the geo fields DelAdd and generate a new DelAdd obkv with it	2023-10-30 11:39:31 +01:00
ManyTheFish	ba90a5ec0e	update extract fid word count docids	2023-10-30 11:39:31 +01:00