mirror of
https://github.com/meilisearch/MeiliSearch
synced 2024-12-01 17:15:46 +01:00
19e6f675b3
4900: Indexer edition 2024 r=Kerollmops a=dureuill This PR is implementing the indexer edition 2024, largely inspired by [the ideas from this blog post](https://blog.kerollmops.com/meilisearch-is-too-slow). Fixes https://github.com/meilisearch/meilisearch/issues/4985 ## Features - Stream-first approach to reading documents. - Minimum disk write operations. - RAM usage-first approach to avoid modifying common bitmaps on disk but in memory. - Reduced LMDB fragmentation by writing entries only once... - ...computing the final version of the entries in parallel... - ...and storing them in write-optimized data structures before sending them to the BTree (LMDB). - Indexing in multiple transactions to improve large dataset support (dumps). Co-authored-by: ManyTheFish <many@meilisearch.com> Co-authored-by: Clément Renault <clement@meilisearch.com> Co-authored-by: Louis Dureuil <louis@meilisearch.com> |
||
|---|---|---|
| .. | ||
| assets | ||
| auth | ||
| batches | ||
| common | ||
| dashboard | ||
| documents | ||
| dumps | ||
| features | ||
| index | ||
| logs | ||
| search | ||
| settings | ||
| similar | ||
| snapshot | ||
| stats | ||
| swap_indexes | ||
| tasks | ||
| vector | ||
| content_type.rs | ||
| integration.rs | ||