2284 Commits

Author	SHA1	Message	Date
ManyTheFish	d29d4f88da	Skip iterating over documents when the faceted field list doesn't change	2024-06-04 15:31:24 +02:00
ManyTheFish	17c5ceeb9d	iterate over the faceted fields instead of over the whole document	2024-06-04 14:04:20 +02:00
Clément Renault	b9a0ff0dd6	Cache a lot of operations to know if a field must be indexed	2024-05-30 16:18:23 +02:00
Clément Renault	75496af985	Add a span for the prepare_for_documents_reindexing	2024-05-30 12:14:22 +02:00
Clément Renault	0e9eb9eedb	Add a span for the settings diff creation	2024-05-30 12:08:27 +02:00
Clément Renault	3a78e988da	Reduce the number of complex calls to settings diff functions	2024-05-30 11:23:07 +02:00
Clément Renault	d9e5074189	Introduce a new way to determine the operations to perform on the fields	2024-05-30 11:23:07 +02:00
Clément Renault	bc210bdc00	Introduce a dedicated function to write proximity entries in database	2024-05-30 11:23:06 +02:00
Clément Renault	4bf83f701c	Give the settings diff to the write_typed_chunk_into_index function	2024-05-30 11:23:06 +02:00
Clément Renault	db3887929f	Fix an issue with settings diff and * in the searchable attributes	2024-05-30 11:22:50 +02:00
Clément Renault	9af103a88e	Introducing a new into_del_add_obkv_conditional_operation function	2024-05-30 11:22:49 +02:00
Clément Renault	99211eb375	Introduce the SettingDiff only_additional_fields method	2024-05-30 11:22:49 +02:00
ManyTheFish	1ab88e10b9	Merge branch 'main' into merge-release-v1.8.1-in-main	2024-05-29 16:24:00 +02:00
Many the fish	e1fbfde6c4	Merge branch 'main' into merge-release-v1.8.1-in-main	2024-05-29 11:31:03 +02:00
ManyTheFish	27b75ec648	merge main into v1.8.1	2024-05-29 11:26:07 +02:00
Louis Dureuil	ca6cc4654b	Add similar route	2024-05-28 15:28:19 +02:00
Louis Dureuil	d35278320e	Add support functions for accessing arroy writers and readers	2024-05-28 15:27:43 +02:00
Louis Dureuil	02b3d82c60	filtered_universe accepts index and txn instead of SearchContext	2024-05-28 15:22:12 +02:00
Louis Dureuil	fd2c95999d	Change validate_document_id to public and remove extra layer of result	2024-05-28 15:21:19 +02:00
Clément Renault	dc949ab46a	Remove puffin usage	2024-05-27 15:59:14 +02:00
Clément Renault	7f3e51349e	Remove puffin for the dependencies	2024-05-27 15:53:06 +02:00
meili-bors[bot]	19acc65ad2	Merge #4646 ... 4646: Reduce `Transform`'s disk usage r=Kerollmops a=Kerollmops This PR implements what is described in #4485. It reduces the number of disk writes and disk usage. Co-authored-by: Clément Renault <clement@meilisearch.com>	2024-05-23 16:06:50 +00:00
Clément Renault	fe17c0f52e	Construct the minimal OBKVs according to the settings diff	2024-05-23 11:23:57 +02:00
Clément Renault	bc5663e673	FieldIdsMap no longer useful thanks to #4631	2024-05-22 16:06:15 +02:00
Louis Dureuil	8a941c0241	Smaller review changes	2024-05-22 14:44:42 +02:00
Louis Dureuil	3412e7fbcf	"[]" is deserialized as 0 embedding rather than 1 embedding of dim 0	2024-05-22 12:25:21 +02:00
Louis Dureuil	16037e2169	Don't remove embedders that are not in the config from the document DB	2024-05-22 12:24:51 +02:00
Louis Dureuil	8f7c8ca7f0	Remove now unused error variant	2024-05-22 12:23:43 +02:00
Clément Renault	500ddc76b5	Make the flattened sorter optional	2024-05-21 16:16:36 +02:00
Clément Renault	943f8dba0c	Make clippy happy	2024-05-21 14:58:41 +02:00
Clément Renault	1aa8ed9ef7	Make the original sorter optional	2024-05-21 14:53:26 +02:00
ManyTheFish	f762307838	Fix clippy	2024-05-21 13:44:20 +02:00
ManyTheFish	3e94a90722	Fixes	2024-05-21 13:39:46 +02:00
Louis Dureuil	b17cb56dee	Test array of vectors	2024-05-20 14:44:10 +02:00
ManyTheFish	fc7e817221	Index geo points based on the settings differences	2024-05-20 12:27:26 +02:00
Louis Dureuil	d05d49ffd8	Fix tests	2024-05-20 10:36:18 +02:00
Louis Dureuil	0462ebbe58	Don't write an empty _vectors field	2024-05-20 10:36:18 +02:00
Louis Dureuil	2f7a8a4efb	Don't write vectors that weren't autogenerated in document DB	2024-05-20 10:36:18 +02:00
Louis Dureuil	52d9cb6e5a	Refactor vector indexing ... - use the parsed_vectors module - only parse `_vectors` once per document, instead of once per embedder per document	2024-05-20 10:36:17 +02:00
Louis Dureuil	261de888b7	Add function to get the embeddings of a document in an index	2024-05-20 10:36:17 +02:00
Louis Dureuil	98c811247e	Add parsed vectors module	2024-05-20 10:25:59 +02:00
Tamo	273c6e8c5c	uses the latest version of heed to get rid of unsafe code	2024-05-16 18:31:32 +02:00
Tamo	897d25780e	update milli to latest version	2024-05-16 18:31:32 +02:00
Tamo	f2d0a59f1d	when no searchable attributes are defined, makes all the weight equals to zero	2024-05-16 01:06:33 +02:00
Tamo	c78a2fa4f5	rename method and variable around the attributes to search on feature	2024-05-15 18:04:42 +02:00
Tamo	5542f1d9f1	get back to what we were doingb efore in the DB cache and with the restricted field id	2024-05-15 18:00:39 +02:00
Tamo	ad4d8502b3	stops storing the whole fieldids weights map when no searchable are defined	2024-05-15 17:16:10 +02:00
Tamo	7ec4e2a3fb	apply all style review comments	2024-05-15 15:02:26 +02:00
Tamo	9fffb8e83d	make clippy happy	2024-05-14 17:36:32 +02:00
Tamo	caa6a7149a	make the attribute ranking rule use the weights and fix the tests	2024-05-14 17:36:32 +02:00