mirror of
https://github.com/meilisearch/MeiliSearch
synced 2024-11-26 23:04:26 +01:00
5e754b3ee0
708: Reduce memory usage of the MatchingWords structure r=ManyTheFish a=loiclec # Pull Request ## Related issue Fixes (partially) https://github.com/meilisearch/meilisearch/issues/3115 ## What does this PR do? 1. Reduces the memory usage caused by the creation of a 10-word query tree by 20x. This is done by deduplicating the `MatchingWord` values, which are heavy because of their inner DFA. The deduplication works by wrapping each `MatchingWord` in a reference-counted box and using a hash map to determine whether a `MatchingWord` DFA already exists for a certain signature, or whether a new one needs to be built. 2. Avoid the worst-case scenario of creating a `MatchingWord` for extremely long words that cannot be indexed by milli. Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com> |
||
|---|---|---|
| .. | ||
| documents | ||
| facet | ||
| heed_codec | ||
| search | ||
| snapshots/index.rs/initial_field_distribution | ||
| update | ||
| asc_desc.rs | ||
| criterion.rs | ||
| error.rs | ||
| external_documents_ids.rs | ||
| fields_ids_map.rs | ||
| index.rs | ||
| lib.rs | ||
| proximity.rs | ||
| snapshot_tests.rs | ||