2851: Upgrade clap to 4.0 r=loiclec a=choznerol
# Pull Request
## Related issue
Fixes#2846
This PR is draft based on #2847 to avoid conflict. I will rebase and mark as 'Ready for review' after #2847 is merged.
## What does this PR do?
1. Upgrade clap to the latest version or 4.0 (4.0.9 as of today) by following the [migrating instruction](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#migrating) from [4.0 changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#migrating)
2. Fix an `ArgGroup` typo that can only be caught after upgrading to 4.0 in 20a715e29ed17c5a76229c98fb31504ada873597
## Notable changes
### The `--help` message
The format, ordering and indentation of `--help` message was changed in 4.0. I recorded the output of `cargo run -- --help` before and after upgrade to 4.0 for reference.
<details>
<summary>diff</summary>
Output of `diff --ignore-all-space --text --unified --new-file help-message-before.txt help-message-after.txt`:
```diff
--- help-message-before.txt 2022-10-14 16:45:36.000000000 +0800
+++ help-message-after.txt 2022-10-14 16:36:53.000000000 +0800
`@@` -1,12 +1,8 `@@`
-meilisearch-http 0.29.1
+Usage: meilisearch [OPTIONS]
-USAGE:
- meilisearch [OPTIONS]
-
-OPTIONS:
+Options:
--config-file-path <CONFIG_FILE_PATH>
- Set the path to a configuration file that should be used to setup the engine. Format
- must be TOML
+ Set the path to a configuration file that should be used to setup the engine. Format must be TOML
--db-path <DB_PATH>
Designates the location where database files will be created and retrieved
`@@` -26,15 +22,14 `@@`
[default: dumps/]
--env <ENV>
- Configures the instance's environment. Value must be either `production` or
- `development`
+ Configures the instance's environment. Value must be either `production` or `development`
[env: MEILI_ENV=]
[default: development]
[possible values: development, production]
-h, --help
- Print help information
+ Print help information (use `-h` for a summary)
--http-addr <HTTP_ADDR>
Sets the HTTP address and port Meilisearch will use
`@@` -43,63 +38,53 `@@`
[default: 127.0.0.1:7700]
--http-payload-size-limit <HTTP_PAYLOAD_SIZE_LIMIT>
- Sets the maximum size of accepted payloads. Value must be given in bytes or explicitly
- stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
+ Sets the maximum size of accepted payloads. Value must be given in bytes or explicitly stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
[env: MEILI_HTTP_PAYLOAD_SIZE_LIMIT=]
[default: 100000000]
--ignore-dump-if-db-exists
- Prevents a Meilisearch instance with an existing database from throwing an error when
- using `--import-dump`. Instead, the dump will be ignored and Meilisearch will launch
- using the existing database.
+ Prevents a Meilisearch instance with an existing database from throwing an error when using `--import-dump`. Instead, the dump will be ignored and Meilisearch will launch using the existing database.
This option will trigger an error if `--import-dump` is not defined.
[env: MEILI_IGNORE_DUMP_IF_DB_EXISTS=]
--ignore-missing-dump
- Prevents Meilisearch from throwing an error when `--import-dump` does not point to a
- valid dump file. Instead, Meilisearch will start normally without importing any dump.
+ Prevents Meilisearch from throwing an error when `--import-dump` does not point to a valid dump file. Instead, Meilisearch will start normally without importing any dump.
This option will trigger an error if `--import-dump` is not defined.
[env: MEILI_IGNORE_MISSING_DUMP=]
--ignore-missing-snapshot
- Prevents a Meilisearch instance from throwing an error when `--import-snapshot` does not
- point to a valid snapshot file.
+ Prevents a Meilisearch instance from throwing an error when `--import-snapshot` does not point to a valid snapshot file.
This command will throw an error if `--import-snapshot` is not defined.
[env: MEILI_IGNORE_MISSING_SNAPSHOT=]
--ignore-snapshot-if-db-exists
- Prevents a Meilisearch instance with an existing database from throwing an error when
- using `--import-snapshot`. Instead, the snapshot will be ignored and Meilisearch will
- launch using the existing database.
+ Prevents a Meilisearch instance with an existing database from throwing an error when using `--import-snapshot`. Instead, the snapshot will be ignored and Meilisearch will launch using the existing database.
This command will throw an error if `--import-snapshot` is not defined.
[env: MEILI_IGNORE_SNAPSHOT_IF_DB_EXISTS=]
--import-dump <IMPORT_DUMP>
- Imports the dump file located at the specified path. Path must point to a `.dump` file.
- If a database already exists, Meilisearch will throw an error and abort launch
+ Imports the dump file located at the specified path. Path must point to a `.dump` file. If a database already exists, Meilisearch will throw an error and abort launch
[env: MEILI_IMPORT_DUMP=]
--import-snapshot <IMPORT_SNAPSHOT>
- Launches Meilisearch after importing a previously-generated snapshot at the given
- filepath
+ Launches Meilisearch after importing a previously-generated snapshot at the given filepath
[env: MEILI_IMPORT_SNAPSHOT=]
--log-level <LOG_LEVEL>
Defines how much detail should be present in Meilisearch's logs.
- Meilisearch currently supports five log levels, listed in order of increasing verbosity:
- ERROR, WARN, INFO, DEBUG, TRACE.
+ Meilisearch currently supports five log levels, listed in order of increasing verbosity: ERROR, WARN, INFO, DEBUG, TRACE.
[env: MEILI_LOG_LEVEL=]
[default: INFO]
`@@` -110,31 +95,25 `@@`
[env: MEILI_MASTER_KEY=]
--max-index-size <MAX_INDEX_SIZE>
- Sets the maximum size of the index. Value must be given in bytes or explicitly stating a
- base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
+ Sets the maximum size of the index. Value must be given in bytes or explicitly stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
[env: MEILI_MAX_INDEX_SIZE=]
[default: 107374182400]
--max-indexing-memory <MAX_INDEXING_MEMORY>
- Sets the maximum amount of RAM Meilisearch can use when indexing. By default,
- Meilisearch uses no more than two thirds of available memory
+ Sets the maximum amount of RAM Meilisearch can use when indexing. By default, Meilisearch uses no more than two thirds of available memory
[env: MEILI_MAX_INDEXING_MEMORY=]
[default: "21.33 TiB"]
--max-indexing-threads <MAX_INDEXING_THREADS>
- Sets the maximum number of threads Meilisearch can use during indexation. By default,
- the indexer avoids using more than half of a machine's total processing units. This
- ensures Meilisearch is always ready to perform searches, even while you are updating an
- index
+ Sets the maximum number of threads Meilisearch can use during indexation. By default, the indexer avoids using more than half of a machine's total processing units. This ensures Meilisearch is always ready to perform searches, even while you are updating an index
[env: MEILI_MAX_INDEXING_THREADS=]
[default: 5]
--max-task-db-size <MAX_TASK_DB_SIZE>
- Sets the maximum size of the task database. Value must be given in bytes or explicitly
- stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
+ Sets the maximum size of the task database. Value must be given in bytes or explicitly stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
[env: MEILI_MAX_TASK_DB_SIZE=]
[default: 107374182400]
```
- ~[help-message-before.txt](https://github.com/meilisearch/meilisearch/files/9715683/help-message-before.txt)~ [help-message-before.txt](https://github.com/meilisearch/meilisearch/files/9784156/help-message-before-2.txt)
- ~[help-message-after.txt](https://github.com/meilisearch/meilisearch/files/9715682/help-message-after.txt)~ [help-message-after.txt](https://github.com/meilisearch/meilisearch/files/9784091/help-message-after.txt)
</details>
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Lawrence Chou <choznerol@protonmail.com>
Refactored tests code to allow to specify compression (content-encoding) algorithm.
Added tests to verify what actix actually handle different content encodings properly.
2745: Config file support r=curquiza a=mlemesle
# Pull Request
## What does this PR do?
Fixes#2558
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
2789: Fix typos r=Kerollmops a=kianmeng
# Pull Request
## What does this PR do?
Found via `codespell -L crate,nam,hart`.
## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
2814: Skip dashboard test if mini-dashboard feature is disabled r=Kerollmops a=jirutka
Fixes#2813
Fixes the following error:
cargo test --no-default-features
...
error: couldn't read target/debug/build/meilisearch-http-ec029d8c902cf2cb/out/generated.rs: No such file or directory (os error 2)
--> meilisearch-http/tests/dashboard/mod.rs:8:9
|
8 | include!(concat!(env!("OUT_DIR"), "/generated.rs"));
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: this error originates in the macro `include` (in Nightly builds, run with -Z macro-backtrace for more info)
error: could not compile `meilisearch-http` due to previous error
2826: Rename receivedDocumentIds into matchedDocuments r=Kerollmops a=Ugzuzg
# Pull Request
## What does this PR do?
Fixes#2799
Changes DocumentDeletion task details response.
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Tested with curl:
```
curl \
-X POST 'http://localhost:7700/indexes/movies/documents/delete-batch' \
-H 'Content-Type: application/json' \
--data-binary '[
23488,
153738,
437035,
363869
]'
{"taskUid":1,"indexUid":"movies","status":"enqueued","type":"documentDeletion","enqueuedAt":"2022-10-01T20:06:37.105416054Z"}%
curl \
-X GET 'http://localhost:7700/tasks/1'
{"uid":1,"indexUid":"movies","status":"succeeded","type":"documentDeletion","details":{"matchedDocuments":4,"deletedDocuments":2},"duration":"PT0.005708322S","enqueuedAt":"2022-10-01T20:06:37.105416054Z","startedAt":"2022-10-01T20:06:37.115562733Z","finishedAt":"2022-10-01T20:06:37.121271055Z"}
```
Co-authored-by: mlemesle <lemesle.martin@hotmail.fr>
Co-authored-by: Kian-Meng Ang <kianmeng@cpan.org>
Co-authored-by: Jakub Jirutka <jakub@jirutka.cz>
Co-authored-by: Jarasłaŭ Viktorčyk <ugzuzg@gmail.com>
(Some of) these specialized tokenizations include huge dictionaries
that currently account for 90% (!) of the meilisearch binary size.
This commit adds chinese, hebrew, japanese, and thai feature flags
that are propagated via milli down to the charabia crate. To keep it
backward compatible, they are enabled by default.
Related to meilisearch/milli#632
2689: Use mimalloc as the global allocator r=Kerollmops a=loiclec
milli has switched its global allocator to mimalloc already, and we have seen some performance gains as a result. Furthermore, we can use mimalloc as the global allocator on all platforms whereas jemalloc was only activated on Linux.
This PR brings mimalloc to Meilisearch as well.
2690: Add LTO and codegen-units=1 to release compile options r=Kerollmops a=loiclec
This PR brings Meilisearch's release compile options in line with milli (see https://github.com/meilisearch/milli/pull/606 ).
Adding LTO and codegen=units=1 will make compile times longer, but they also speed up the final binary significantly.
Co-authored-by: Loïc Lecrenier <loic@meilisearch.com>
2584: Format API keys in hexa instead of base64 r=curquiza a=ManyTheFish
This PR:
- Changes API key generation and formatting to ease the generation of the key made by our users
- updates the `uuid` crate version
The API key can now be generated in bash as below:
```sh
echo -n $HYPHENATED_UUID | openssl dgst -sha256 -hmac $MASTER_KEY
```
fixes the issue raised in [product/discussion#421](https://github.com/meilisearch/product/discussions/421#discussioncomment-3079410), this should not impact anything in documentation nor integration but ease the key generation on the user sides.
poke `@gmourier`
Co-authored-by: ManyTheFish <many@meilisearch.com>
Move `meilisearch_error` to `meilisearch_types::error`
Move `meilisearch_lib::index_resolver::IndexUid` to `meilisearch_types::index_uid`
Add a new `InvalidIndexUid` error in `meilisearch_types::index_uid`
2173: chore(all): replace chrono with time r=irevoire a=irevoire
Chrono has been unmaintained for a few month now and there is a CVE on it.
Also I updated all the error messages related to the API key as you can see here: https://github.com/meilisearch/specifications/pull/114fix#2172
Co-authored-by: Irevoire <tamo@meilisearch.com>
