mirror of
https://github.com/meilisearch/MeiliSearch
synced 2024-11-22 21:04:27 +01:00
39869be23b
590: Optimise facets indexing r=Kerollmops a=loiclec # Pull Request ## What does this PR do? Fixes #589 ## Notes I added documentation for the whole module which attempts to explain the shape of the databases and their purpose. However, I realise there is already some documentation about this, so I am not sure if we want to keep it. ## Benchmarks We get a ~1.15x speed up on the geo_point benchmark. ``` group indexing_main_57042355 indexing_optimise-facets-indexation_5728619a ----- ---------------------- -------------------------------------------- indexing/-geo-delete-facetedNumber-facetedGeo-searchable- 1.00 1862.7±294.45µs ? ?/sec 1.58 2.9±1.32ms ? ?/sec indexing/-movies-delete-facetedString-facetedNumber-searchable- 1.11 8.9±2.44ms ? ?/sec 1.00 8.0±1.42ms ? ?/sec indexing/-movies-delete-facetedString-facetedNumber-searchable-nested- 1.00 12.8±3.32ms ? ?/sec 1.32 16.9±6.98ms ? ?/sec indexing/-songs-delete-facetedString-facetedNumber-searchable- 1.09 43.8±4.78ms ? ?/sec 1.00 40.3±3.79ms ? ?/sec indexing/-wiki-delete-searchable- 1.08 287.4±28.72ms ? ?/sec 1.00 264.9±9.46ms ? ?/sec indexing/Indexing geo_point 1.14 61.2±0.39s ? ?/sec 1.00 53.8±0.57s ? ?/sec indexing/Indexing movies in three batches 1.00 16.6±0.12s ? ?/sec 1.00 16.5±0.10s ? ?/sec indexing/Indexing movies with default settings 1.00 14.1±0.30s ? ?/sec 1.00 14.0±0.28s ? ?/sec indexing/Indexing nested movies with default settings 1.10 10.9±0.50s ? ?/sec 1.00 10.0±0.10s ? ?/sec indexing/Indexing nested movies without any facets 1.01 9.6±0.23s ? ?/sec 1.00 9.5±0.06s ? ?/sec indexing/Indexing songs in three batches with default settings 1.07 66.3±0.55s ? ?/sec 1.00 61.8±0.63s ? ?/sec indexing/Indexing songs with default settings 1.03 58.8±0.82s ? ?/sec 1.00 57.1±1.22s ? ?/sec indexing/Indexing songs without any facets 1.00 53.6±1.09s ? ?/sec 1.01 54.0±0.58s ? ?/sec indexing/Indexing songs without faceted numbers 1.02 58.0±1.29s ? ?/sec 1.00 57.1±1.43s ? ?/sec indexing/Indexing wiki 1.00 1064.1±21.20s ? ?/sec 1.00 1068.0±20.49s ? ?/sec indexing/Indexing wiki in three batches 1.00 1182.5±9.62s ? ?/sec 1.01 1191.2±10.96s ? ?/sec indexing/Reindexing geo_point 1.12 68.0±0.21s ? ?/sec 1.00 60.5±0.82s ? ?/sec indexing/Reindexing movies with default settings 1.01 14.1±0.21s ? ?/sec 1.00 14.0±0.26s ? ?/sec indexing/Reindexing songs with default settings 1.04 61.6±0.57s ? ?/sec 1.00 59.2±0.87s ? ?/sec indexing/Reindexing wiki 1.00 1734.0±11.38s ? ?/sec 1.01 1746.6±22.48s ? ?/sec ``` Co-authored-by: Loïc Lecrenier <loic@meilisearch.com> |
||
|---|---|---|
| .. | ||
| fuzz | ||
| src | ||
| tests | ||
| Cargo.toml | ||
| README.md | ||
Milli
Fuzzing milli
Currently you can only fuzz the indexation. To execute the fuzzer run:
cargo +nightly fuzz run indexing
To execute the fuzzer on multiple thread you can also run:
cargo +nightly fuzz run -j4 indexing
Since the fuzzer is going to create a lot of temporary file to let milli index its documents I would also recommand to execute it on a ramdisk. Here is how to setup a ramdisk on linux:
sudo mount -t tmpfs none path/to/your/ramdisk
And then set the TMPDIR environment variable to make the fuzzer create its file in it:
export TMPDIR=path/to/your/ramdisk