mirror of
https://github.com/meilisearch/MeiliSearch
synced 2024-12-12 22:36:29 +01:00
9a9383643f
5125: Change the default max memory usage to 5% of the total memory r=ManyTheFish a=Kerollmops After thorough testing, we found that giving 5% of the total available memory to allocate resident memory (caches and channels) is the best approach. The main reason is that the new indexer is highly memory-map oriented, with LMDB, and reads the database while performing the indexation. So, by allowing the maximum amount of memory available to LMDB and the OS, it will perform the key-value store reads and all other indexation operations faster by keeping more pages hot in the cache. In #5124, we also sorted the entries to merge to improve the read speed of LMDB. This is common in database management systems: Reading stuff on the disk is much faster when done in lexicographic order (the default sorted order of key values). The entries have a great chance of already being in the OS memory cache, as they were loaded in a previous read, and reading stuff on the disk is very slow compared to reading memory. Co-authored-by: Kerollmops <clement@meilisearch.com> |
||
|---|---|---|
| .. | ||
| benchmarks | ||
| build-info | ||
| dump | ||
| file-store | ||
| filter-parser | ||
| flatten-serde-json | ||
| fuzzers | ||
| index-scheduler | ||
| json-depth-checker | ||
| meili-snap | ||
| meilisearch | ||
| meilisearch-auth | ||
| meilisearch-types | ||
| meilitool | ||
| milli | ||
| permissive-json-pointer | ||
| tracing-trace | ||
| xtask | ||