59 Commits

Author	SHA1	Message	Date
Louis Dureuil	7a9290aaae	Use new incremental facet indexing and enable sanity checks in debug	2025-01-06 15:08:48 +01:00
Tamo	fa885e75b4	rename the send_progress in progress	2024-12-11 18:13:12 +01:00
Tamo	867e6a8f1d	rename the send_progress field to progress since it s not sending anything	2024-12-11 16:25:01 +01:00
Tamo	df9b68f8ed	inital implementation of the progress	2024-12-11 16:25:01 +01:00
Kerollmops	a751972c57	Prefer using a stable than a random hash builder	2024-12-10 14:25:53 +01:00
Kerollmops	89637bcaaf	Use bumparaw-collections in Meilisearch/milli	2024-12-10 11:52:20 +01:00
Kerollmops	f5dd8dfc3e	Rollback max memory usage changes	2024-12-09 10:26:30 +01:00
Louis Dureuil	3a11e39c01	Force max_memory to a min of 100MiB	2024-12-04 17:53:30 +01:00
Kerollmops	8ecb726683	Fix the minimun BBQueue channel threshold	2024-12-03 15:49:11 +01:00
Clément Renault	d5c07ef7b3	Manage key length conversion error correctly	2024-12-02 11:03:00 +01:00
Clément Renault	30eb0e5b5b	Rename recv and read methods to recv_action and recv_frame	2024-12-02 10:08:01 +01:00
Clément Renault	14ee7aa84c	Make sure the BBQueue is at least 50 MiB	2024-11-28 18:02:48 +01:00
Clément Renault	8a35cd1743	Adjust the BBQueue buffers to use 2% instead of 10%	2024-11-28 16:00:15 +01:00
Clément Renault	3c7ac093d3	Take the BBQueue capacity into account in the max memory	2024-11-28 15:43:14 +01:00
Clément Renault	b57dd5c58e	Remove the Vector variant and use the Vectors	2024-11-28 15:20:43 +01:00
Clément Renault	096a28656e	Fix a bug around deleting all the vectors of a doc	2024-11-28 15:15:06 +01:00
Clément Renault	cc4bd54669	Correctly construct the Embeddings struct	2024-11-28 13:53:25 +01:00
Clément Renault	58eab9a018	Send large payload through crossbeam	2024-11-28 12:01:06 +01:00
Clément Renault	e83534a430	Fix the indexer::index to correctly use the rayon::ThreadPool	2024-11-27 16:27:43 +01:00
Clément Renault	98d4a2909e	Fix the way we spawn the rayon threadpool	2024-11-27 16:05:44 +01:00
Clément Renault	cc63802115	Modify and return the IndexEmbeddings to write them later	2024-11-27 14:58:03 +01:00
Clément Renault	acec45ad7c	Send a WakeUp when writing data in the BBQueue buffers	2024-11-27 14:33:23 +01:00
Clément Renault	70802eb7c7	Fix most issues with the lifetimes	2024-11-27 14:32:42 +01:00
Clément Renault	6ac5b3b136	Finish most of the channels types	2024-11-27 14:32:26 +01:00
Clément Renault	2094ce8a9a	Move the arroy building after the writing loop	2024-11-27 14:30:33 +01:00
Clément Renault	8442db8101	Implement mostly all senders	2024-11-27 14:16:35 +01:00
meili-bors[bot]	a2f64f6552	Merge #5095 ... 5095: Span to measure the part of db writes that is after the merge/extraction r=curquiza a=dureuill Co-authored-by: Louis Dureuil <louis@meilisearch.com>	2024-11-27 11:10:00 +00:00
meili-bors[bot]	d0b2c0a523	Merge #5091 ... 5091: Settings opt out r=Kerollmops a=ManyTheFish # Pull Request Related PRD: https://www.notion.so/meilisearch/API-usage-Settings-to-opt-out-indexing-features-fff4b06b651f8108ade3f858aeb16b14?pvs=4 ## Related issue Fixes #4979 - [x] Add setting opt-out - [x] Add analytics - [x] Add tests Co-authored-by: ManyTheFish <many@meilisearch.com> Co-authored-by: Many the fish <many@meilisearch.com>	2024-11-26 15:50:28 +00:00
Louis Dureuil	8f57b4fdf4	Span to measure the part of db writes that is after the merge/extraction	2024-11-26 14:46:36 +01:00
Louis Dureuil	aa460819a7	Add more precise spans	2024-11-26 09:45:36 +01:00
ManyTheFish	d66dc363ed	Test and implement settings opt-out	2024-11-25 18:23:22 +01:00
Clément Renault	25aac45fc7	Expose better error messages	2024-11-25 15:54:43 +01:00
Clément Renault	5727e00374	Remove useless geo skipped	2024-11-21 16:47:08 +01:00
Clément Renault	9b60843831	Remove commented lines	2024-11-21 16:47:07 +01:00
ManyTheFish	94b260fd25	Remove orphan span	2024-11-21 12:12:07 +01:00
Clément Renault	ab2c83f868	Use the disk less when computing prefixes	2024-11-21 10:45:37 +01:00
Louis Dureuil	e0864f1b21	Separate side effect and debug asserts	2024-11-20 16:25:17 +01:00
Clément Renault	a38344acb3	Replace eprintlns by tracing	2024-11-20 15:29:51 +01:00
Louis Dureuil	651c30899e	Allow fetching embedders from inside tests	2024-11-20 13:16:36 +01:00
Louis Dureuil	c782c09208	Move step to a dedicated mod and replace it with an enum	2024-11-18 18:22:13 +01:00
Louis Dureuil	75943a5a9b	Add TODO to remember replacing steps with an enum	2024-11-18 17:40:51 +01:00
Louis Dureuil	04c38220ca	Move MostlySend, ThreadLocal, FullySend to their own commit	2024-11-18 16:43:05 +01:00
Clément Renault	5b4c06c24c	Plug the grenad max memory parameter	2024-11-18 11:28:04 +01:00
Louis Dureuil	c202f3dbe2	fix tests and revert change in behavior when primary_key_from_op != primary_key_from_db && index.is_empty()	2024-11-18 10:59:05 +01:00
Clément Renault	83865d2ebd	Expose intermediate errors when processing batches	2024-11-18 10:59:05 +01:00
Clément Renault	9e8367f1e6	Move the rayon thread pool outside the extract method	2024-11-14 10:40:32 +01:00
ManyTheFish	51b6293738	Add linear facet databases	2024-11-13 17:43:02 +01:00
Clément Renault	b17896d899	Finialize the GeoExtractor	2024-11-13 17:43:02 +01:00
Louis Dureuil	6094bb299a	Fix user_provided vectors	2024-11-12 10:15:55 +01:00
Louis Dureuil	8a314ab81d	Fix primary key fid order	2024-11-08 00:05:12 +01:00