mirror of
https://github.com/meilisearch/MeiliSearch
synced 2024-11-05 12:38:55 +01:00
Merge #359
359: Improve the benchmark comparison script r=irevoire a=irevoire This modification allow us to compare more than 2 benchmarks or to only print the results of one benchmark Co-authored-by: Irevoire <tamo@meilisearch.com> Co-authored-by: Tamo <tamo@meilisearch.com>
This commit is contained in:
commit
203aa727a7
@ -54,7 +54,7 @@ MILLI_BENCH_DATASETS_PATH=~/datasets cargo bench --bench songs # the code is com
|
||||
|
||||
## Comparison between benchmarks
|
||||
|
||||
The benchmark reports we push are generated with `critcmp`. Thus, we use `critcmp` to generate comparison results between 2 benchmarks.
|
||||
The benchmark reports we push are generated with `critcmp`. Thus, we use `critcmp` to show the result of a benchmark, or compare results between multiple benchmarks.
|
||||
|
||||
We provide a script to download and display the comparison report.
|
||||
|
||||
@ -71,12 +71,18 @@ List the available file in the DO Space:
|
||||
```bash
|
||||
songs_main_09a4321.json
|
||||
songs_geosearch_24ec456.json
|
||||
search_songs_main_cb45a10b.json
|
||||
```
|
||||
|
||||
Run the comparison script:
|
||||
|
||||
```bash
|
||||
# we get the result of ONE benchmark, this give you an idea of how much time an operation took
|
||||
./benchmarks/scripts/compare.sh son songs_geosearch_24ec456.json
|
||||
# we compare two benchmarks
|
||||
./benchmarks/scripts/compare.sh songs_main_09a4321.json songs_geosearch_24ec456.json
|
||||
# we compare three benchmarks
|
||||
./benchmarks/scripts/compare.sh songs_main_09a4321.json songs_geosearch_24ec456.json search_songs_main_cb45a10b.json
|
||||
```
|
||||
|
||||
## Datasets
|
||||
|
@ -17,21 +17,9 @@ if [[ "$?" -ne 0 ]]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [[ $# -ne 2 ]]
|
||||
then
|
||||
echo 'Need 2 arguments.'
|
||||
echo 'Usage: '
|
||||
echo ' $ ./compare.sh old new'
|
||||
echo 'Ex:'
|
||||
echo ' $ ./compare.sh songs_main_09a4321.json songs_geosearch_24ec456.json'
|
||||
exit 1
|
||||
fi
|
||||
|
||||
old_file="$1"
|
||||
new_file="$2"
|
||||
s3_url='https://milli-benchmarks.fra1.digitaloceanspaces.com/critcmp_results'
|
||||
|
||||
for file in $old_file $new_file
|
||||
for file in $@
|
||||
do
|
||||
file_s3_url="$s3_url/$file"
|
||||
file_local_path="/tmp/$file"
|
||||
@ -45,6 +33,12 @@ do
|
||||
fi
|
||||
done
|
||||
|
||||
# Print the diff changes between the old and new benchmarks
|
||||
# by only displaying the lines that have a diff of more than 5%.
|
||||
critcmp --threshold 5 "/tmp/$old_file" "/tmp/$new_file"
|
||||
path_list=$(echo " $@" | sed 's/ / \/tmp\//g')
|
||||
|
||||
if [[ ${#@} -gt 1 ]]; then
|
||||
# Print the diff changes between the old and new benchmarks
|
||||
# by only displaying the lines that have a diff of more than 5%.
|
||||
critcmp --threshold 5 $path_list
|
||||
else
|
||||
critcmp $path_list
|
||||
fi
|
||||
|
Loading…
Reference in New Issue
Block a user