2876: Full support for compressed (Gzip, Brotli, Zlib) API requests r=Kerollmops a=mou
# Pull Request
## Related issue
Fixes#2802
## What does this PR do?
- Adds missed content-encoding support for streamed requests (documents)
- Adds additional tests to validate content-encoding support is in place
- Adds new tests to validate content-encoding support for previously existing code (built-in actix functionality for unmarshaling JSON payloads)
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Andrey "MOU" Larionov <anlarionov@gmail.com>
2896: Fix CI to send signal to Cloud team r=Kerollmops a=curquiza
Hello `@eskombro` 👋
I realized the CI we recently created together was not good when we release the official Meilisearch version. Indeed, in this case `steps.meta.outputs.tags` contains several tags, see: https://github.com/meilisearch/meilisearch/actions/runs/3197492898/jobs/5220776456
You might want to ask: why the CI, including the Cloud team signal, was available when releasing v0.29.1 and not v0.29.0? Good question, thanks for asking it Sam! It was a mistake on our side, it should not have been available for v0.29.1, and this is how I found out v0.29.1 was broken and contained commits it should not have. So I deleted everything and started the release process again for v0.29.1.
Anyway, since I had the chance to see the bug in this release mess, I want to take the opportunity to fix it. Now, we will only send the real tag.
Here you have more documentation about what `github.ref_name` is: https://docs.github.com/en/actions/learn-github-actions/contexts
Bisous bisous!
Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
662: Enhance word splitting strategy r=ManyTheFish a=akki1306
# Pull Request
## Related issue
Fixes#648
## What does this PR do?
- [split_best_frequency](55d889522b/milli/src/search/query_tree.rs (L282-L301)) to use frequency of word pairs near together with proximity value of 1 instead of considering the frequency of individual words. Word pairs having max frequency are considered.
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Akshay Kulkarni <akshayk.gj@gmail.com>
Alongside request encoding (compression) support, it is helpful to verify that the server respect `Accept-Encoding` headers and apply the corresponding compression to responses.
604: Speed up debug builds r=Kerollmops a=loiclec
Note: this draft PR is based on https://github.com/meilisearch/milli/pull/601 , for no particular reason.
## What does this PR do?
Make a series of changes with the goal of speeding up debug builds:
1. Add an `all_languages` feature which compiles charabia with its `default` features activated.
The `all_languages` feature is activated by default. But running:
```
cargo build --no-default-features
```
on `milli` is now much faster.
2. Reduce the debug optimisation level from 3 to 0, except for a few critical dependencies.
3. Compile the build dependencies quicker as well. Previously, all build dependencies were compiled with `opt-level = 3`. Now, only the critical build dependencies are compiled with optimisations.
4. Reduce the amount of code generated by the `documents!` macro
5. Make the "progress update" closure provided to indexing functions a trait object instead of a generic parameter. This avoids monomorphising the indexing code multiple times needlessly.
## Results
Initial build times on my computer before and after these changes:
| | cargo check | cargo check --no-default-features | cargo test | cargo test --lib | cargo test --no-default-features | cargo test --lib --no-default-features |
|--------|-------------|-----------------------------------|------------|------------------|----------------------------------|----------------------------------------|
| before | 1m05s | 1m05s | 2m06s | 1m47s | 2m06 | 1m47s |
| after | 28.9s | 13.1s | 40s | 38s | 23s | 21s |
Co-authored-by: Loïc Lecrenier <loic@meilisearch.com>
2890: fix: add handle dumpCreation query on tasks request r=Kerollmops a=washbin
# Pull Request
## Related issue
Fixes#2874
## What does this PR do?
- add missing `DumpCreation` type in tasks route
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Co-authored-by: washbin <76929116+washbin@users.noreply.github.com>
658: Add proximity calculation for the same word r=ManyTheFish a=msvaljek
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/milli/issues/647
## What does this PR do?
- During [the increase of the current word position](d94339a858/milli/src/update/index_documents/extract/extract_word_pair_proximity_docids.rs (L129-L135)) we extract the proximity between the current position and the next one.
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: msvaljek <marko.svaljek@commercetools.com>
2862: Use Ubuntu 18.04 for all CI tasks that previously used Ubuntu 20.04 r=curquiza a=loiclec
This is to prevent linking with a version of glibc that is too recent.
With meilisearch v0.29.0 we inadvertently bumped the minimum supported glibc version to 2.29, which means it couldn't be run from Debian 10 (for example) anymore. By using Ubuntu 18.04, which uses glibc 2.27, we restore support for older Linux distros.
Fixes#2850
Co-authored-by: Loïc Lecrenier <loic@meilisearch.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
This is to prevent linking with a version of glibc that is too recent.
With meilisearch v0.29.0 we inadvertently bumped the minimum supported
glibc version to 2.29, which means it couldn't be run from Debian 10
(for example) anymore. By using Ubuntu 18.04, which uses glibc 2.27, we
restore support for older Linux distros.
2868: Uncomment cache steps in Github CI r=curquiza a=AM1TB
# Pull Request
Uncomment cache steps as they were previously affected by an issue with Github actions: https://www.githubstatus.com/incidents/gq1x0j8bv67v
## Related issue
Fixes#2864
## What does this PR do?
- Reintroduce the rust-cache steps within Github CI.
## PR checklist
Please check if your PR fulfills the following requirements:
- [X] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [X] Have you read the contributing guidelines?
- [X] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Amit Banerjee <amit.banerjee.jsr@gmail.com>
Actix provides different content encodings out of the box, but only if we use built-in content wrappers and containers. This patch wraps its own Payload implementation with an actix decoder, which enables request compression support.
Refactored tests code to allow to specify compression (content-encoding) algorithm.
Added tests to verify what actix actually handle different content encodings properly.
2861: Change default bind address to localhost r=Kerollmops a=Fall1ngStar
# Pull Request
## Related issue
Fixes#2782
## What does this PR do?
- Change the default bind address to `localhost` so that it can be accessed with IPv6
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Fall1ngStar <fall1ngstar.public@gmail.com>
Close#2800
This is an alternative to the `--config-file-path` option. If both `--config-file-path` and `MEILI_CONFIG_FILE_PATH` are present, `--config-file-path` takes precedence according to the "Priority order" section of #2558.
Not sure why but the compiler didn't catch this until clap is upgraded to v4.
Follwoing are the error from 'cargo test':
running 2 tests
test routes::indexes::search::test::test_fix_sort_query_parameters ... ok
test option::test::test_valid_opt ... FAILED
failures:
---- option::test::test_valid_opt stdout ----
thread 'option::test::test_valid_opt' panicked at 'Command meilisearch-http: Argument or group 'import-snapshot' specified in 'requires*' for 'ignore_missing_snapshot' does not exist', /Users/ychou/.cargo/registry/src/github.com-1ecc6299db9ec823/clap-4.0.9/src/builder/debug_asserts.rs:152:13
note: run with environment variable to display a backtrace
failures:
option::test::test_valid_opt
test result: FAILED. 1 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
2839: Update internal CLI documentation r=curquiza a=jeertmans
# Pull Request
## Related issue
Fixes#2810
## What does this PR do?
- Make internal CLI documentation match the online one
## Remarks
- Scope is limited to `meilisearch/meilisearch-http/src/option.rs`, not the flattened structs: should I also take care of them?
- Could not find online docs for `enable_metrics_route`
- Max column width wrapping was done by hand, so may not be perfect
- I removed the links from the internal doc: should I put them back?
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Co-authored-by: Jérome Eertmans <jeertmans@icloud.com>
