MeiliSearch/crates/milli/src
meili-bors[bot] 796acd1aee
Merge #5288
5288: Improve AI logging r=dureuill a=Kerollmops

This PR fixes #5285 and brings the changes from #5233 to simplify debugging indexation and search performance issues related to AI. The following texts can be found in the logs to debug and understand performance issues:

 - `embed_one: search` represents the time we spent waiting for the embedding generation, i.e., OpenAI, local HuggingFace, Ollama.
 - `filtered_universe: search::universe` the time spent filtering the documents.
 - ~`next_bucket: search::vector_sort` is the time spent finding the nearest neighbors (ANNs) in the vector store (arroy), locally~ was being triggered too many times.
 - `indexing::vectors` is the time arroy spends indexing the new vectors for a batch.
 - `documents::extract vectors` and `documents::merge vectors` to see the time spent generating and writing the embeddings.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2025-02-04 10:20:45 +00:00
..
documents User failure for documents with docid of ==512 bytes 2024-12-10 11:06:24 +01:00
facet Move crates under a sub folder to clean up the code 2024-10-21 08:18:43 +02:00
fields_ids_map First batch of PR comment 2024-11-21 16:38:11 +01:00
heed_codec fmt 2025-01-23 16:51:20 +01:00
prompt Return docid in case of errors while rendering the document template 2024-12-11 15:26:18 +01:00
search Move back the search-side logging to tracing 2025-02-04 11:16:17 +01:00
snapshots/index.rs Move crates under a sub folder to clean up the code 2024-10-21 08:18:43 +02:00
update Merge #5288 2025-02-04 10:20:45 +00:00
vector Merge #5288 2025-02-04 10:20:45 +00:00
asc_desc.rs Fix the format and linter error 2024-12-28 20:35:55 +08:00
constants.rs Add the new tasks with most of the job done 2025-01-23 16:48:32 +01:00
criterion.rs Replace hardcoded string with constants 2024-12-28 20:35:55 +08:00
error.rs move the version check to the task queue 2025-01-23 16:48:32 +01:00
external_documents_ids.rs Move crates under a sub folder to clean up the code 2024-10-21 08:18:43 +02:00
fieldids_weights_map.rs Fix the format and linter error 2024-12-28 20:35:55 +08:00
fields_ids_map.rs Merge branch 'indexer-edition-2024' into indexer-edition-2024-doc-chunks 2024-11-06 15:50:53 +01:00
index.rs fix the bad index version on opening 2025-01-23 16:51:24 +01:00
lib.rs Remove fuzzing feature 2025-01-08 15:59:03 +01:00
localized_attributes_rules.rs Implements the get and delete tasks route 2025-01-07 16:26:04 +01:00
order_by_map.rs Move crates under a sub folder to clean up the code 2024-10-21 08:18:43 +02:00
progress.rs fix the field distribution when upgrading from the v1_12 2025-01-23 16:51:19 +01:00
proximity.rs Move crates under a sub folder to clean up the code 2024-10-21 08:18:43 +02:00
score_details.rs Move crates under a sub folder to clean up the code 2024-10-21 08:18:43 +02:00
snapshot_tests.rs Move crates under a sub folder to clean up the code 2024-10-21 08:18:43 +02:00
thread_pool_no_abort.rs Do not create too many rayon tasks 2025-01-30 16:36:12 +01:00