14 Commits

Author	SHA1	Message	Date
Louis Dureuil	303e601b87	HuggingFace: Clearer error message when a model is not supported	2024-07-23 15:13:22 +02:00
meili-bors[bot]	339a5e3431	Merge #4549 ... 4549: Hugging Face embedder improvements r=dureuill a=dureuill Architectural changes/Internal improvements ### 1. Prefer safetensors weights over pytorch weights when available safetensors weights are memory mapped, which reduces memory usage of supported models. ### 2. Update candle Updates candle to `0.4.1`, now targeting crates.io and the tokenizers to `v0.15.2` (still on github). This might fix https://github.com/meilisearch/meilisearch/issues/4399 thanks to the now included https://github.com/huggingface/candle/issues/1454 Co-authored-by: Louis Dureuil <louis@meilisearch.com>	2024-04-04 13:47:18 +00:00
Louis Dureuil	a1eccc762a	Prefer safetensors to pytorch when both are available	2024-04-03 11:05:59 +02:00
Louis Dureuil	afd1da5642	Add distribution to all embedders	2024-03-27 11:50:22 +01:00
Louis Dureuil	a1db342f01	Expose REST embedder to the API	2024-03-25 11:23:15 +01:00
Louis Dureuil	c3d02f092d	OpenAI sync	2024-03-25 11:23:03 +01:00
Tamo	e773dfa9ba	get rids of log in milli and add logs for the bucket sort	2024-02-08 15:04:05 +01:00
Louis Dureuil	1555870088	Truncate HuggingFace vectors that are too long	2024-02-01 10:33:27 +01:00
Louis Dureuil	f692021bfc	Implement PR comments	2024-01-22 10:25:56 +01:00
Louis Dureuil	84f49d76cd	Add cuda feature	2024-01-22 10:25:16 +01:00
Louis Dureuil	87bba98bd8	Various changes ... - fixed seed for arroy - check vector dimensions as soon as it is provided to search - don't embed whitespace	2023-12-14 16:08:42 +01:00
Louis Dureuil	5b51cb04af	Remove some settings	2023-12-14 16:08:42 +01:00
Louis Dureuil	922a640188	WIP multi embedders ... fixed template bugs	2023-12-14 16:08:41 +01:00
Louis Dureuil	13c2c6c16b	Small commit to add hybrid search and autoembedding	2023-12-14 16:07:48 +01:00