2178: Refacto docker r=irevoire a=irevoire
closes#2166 and #2085
-----------
I noticed many people had issues with the default configuration of our Dockerfile.
Some examples:
- #2166: If you use ubuntu and mount your `data.ms` in a volume (as shown in the [doc](https://docs.meilisearch.com/learn/getting_started/installation.html#download-and-launch)), you can't run meilisearch
- #2085: Here, meilisearch was not able to erase the `data.ms` when loading a dump because it's the mount point
Currently, we don't show how to use the snapshot and dumps with docker in the documentation. And it's quite hard to do:
- You either send a big command to meilisearch to change the dump-path, snapshot-path and db-path a single directory and then mount that one
- Or you mount three volumes
- And there were other issues on the slack community
I think this PR solve the problem.
Now the image contains the `meilisearch` binary in the `/bin` directory, so it's easy to find and always in the `PATH`.
It creates a `data` directory and moves the working-dir in it.
So now you can find the `dumps`, `snapshots` and `data.ms` directory in `/data`.
Here is the new command to run meilisearch with a volume:
```
docker run -it --rm -v $PWD/meili_data:/data -p 7700:7700 getmeili/meilisearch:latest
```
And if you need to import a dump or a snapshot, you don't need to restart your container and mount another volume. You can directly hit the `POST /dumps` route and then run:
```
docker run -it --rm -v $PWD/meili_data:/data -p 7700:7700 getmeili/meilisearch:latest meilisearch --import-dump dumps/20220217-152115159.dump
```
-------
You can already try this PR with the following docker image:
```
getmeili/meilisearch:test-docker-v0.26.0
```
If you want to use the v0.25.2 I created another image;
```
getmeili/meilisearch:test-docker-v0.25.2
```
------
If you're using helm I created a branch [here](https://github.com/meilisearch/meilisearch-kubernetes/tree/test-docker-v0.26.0) that use the v0.26.0 image with the good volume 👍
If you use this conf with the v0.25.2, it should also work.
Co-authored-by: Tamo <tamo@meilisearch.com>
2298: Nested fields r=irevoire a=irevoire
There are a few things that I want to fix _AFTER_ merging this PR.
For the following RCs.
## Stop the useless conversion
In the `search.rs` I convert a `Document` to a `Value`, and then the `Value` to a `Document` and then back to a `Value` etc. I should stop doing all these conversion and stick to one format.
Probably by merging my `permissive-json-pointer` crate into meilisearch.
That would also give me the opportunity to work directly with obkvs and stops deserializing fields I don't need.
## Add more test specific to the nested
Everything seems to works but I should write tests to double check that the nested works well with the `formatted` field.
## See how I could stop iterating on hashmap and instead fill them correctly
This is related to milli. I really often needs to iterate over hashmap to see if a field is a subset of another field. I could probably generate a structure containing all the possible key values.
ie. the user say `doggo` is an attribute to retrieve. Instead of iterating on all the attributes to retrieve to check if `doggo.name` is a subset of `doggo`. I should insert `doggo.name` in the attributes to retrieve map.
Co-authored-by: Tamo <tamo@meilisearch.com>
2304: chore(bors): comments clippy out r=curquiza a=irevoire
There is currently an issue with clippy that stops us from merging PRs.
https://github.com/rust-lang/rust-clippy/issues/8662#issuecomment-1093899755
We can't use clippy in the CI while that's not merged
Co-authored-by: Tamo <tamo@meilisearch.com>
2297: Feat(Search): Enhance formating search results r=ManyTheFish a=ManyTheFish
Add new settings and change crop_len behavior to count words instead of characters.
- [x] `highlightPreTag`
- [x] `highlightPostTag`
- [x] `cropMarker`
- [x] `cropLength` count word instead of chars
- [x] `cropLength` 0 is now considered as no `cropLength`
- [ ] ~smart crop finding the best matches interval~ (postponed)
Partially fixes #2214. (no smart crop)
Co-authored-by: ManyTheFish <many@meilisearch.com>
2271: Simplify Dockerfile r=ManyTheFish a=Thearas
# Pull Request
## What does this PR do?
1. Fixes#2234
2. Replace `$TARGETPLATFORM` with `apk --print-arch` to make Dockerfile available for `docker build` as well, not just `docker buildx` (inspired by [rust-lang/docker-rust](https://github.com/rust-lang/docker-rust/blob/master/1.59.0/alpine3.14/Dockerfile#L13))
PTAL `@curquiza`
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Thearas <thearas850@gmail.com>
2296: disable typo for attributes r=curquiza a=MarinPostma
Introduce the disable typos on attribute feature as per https://github.com/meilisearch/specifications/pull/117.
Co-authored-by: ad hoc <postma.marin@protonmail.com>
2249: feat(all): introduce disable typos r=irevoire a=MarinPostma
Introduce the disable typo setting, that allows disabling typos for an index.
waiting on https://github.com/meilisearch/milli/pull/469
Co-authored-by: ad hoc <postma.marin@protonmail.com>
2277: fix(http): fix panic when sending document update without content type header r=MarinPostma a=MarinPostma
I found a panic when pushing documents without a content-type. This fixes is by returning unknown instead of crashing.
Co-authored-by: ad hoc <postma.marin@protonmail.com>
2207: Fix: avoid embedding the user input into the error response. r=Kerollmops a=CNLHC
# Pull Request
## What does this PR do?
Fix#2107.
The problem is meilisearch embeds the user input to the error message.
The reason for this problem is `milli` throws a `serde_json: Error` whose `Display` implementation will do this embedding.
I tried to solve this problem in this PR by manually implementing the `Display` trait for `DocumentFormatError` instead of deriving automatically.
<!-- Please link the issue you're trying to fix with this PR, if none then please create an issue first. -->
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Liu Hancheng <cn_lhc@qq.com>
Co-authored-by: LiuHanCheng <2463765697@qq.com>
2281: Hard limit the number of results returned by a search r=Kerollmops a=Kerollmops
This PR fixes#2133 by hard-limiting the number of results that a search request can return at any time. I would like the guidance of `@MarinPostma` to test that, should I use a mocking test here? Or should I do anything else?
I talked about touching the _nb_hits_ value with `@qdequele` and we concluded that it was not correct to do so.
Could you please confirm that it is the right place to change that?
Co-authored-by: Kerollmops <clement@meilisearch.com>
2267: Add instance options for RAM and CPU usage r=Kerollmops a=2shiori17
# Pull Request
## What does this PR do?
Fixes#2212
<!-- Please link the issue you're trying to fix with this PR, if none then please create an issue first. -->
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: 2shiori17 <98276492+2shiori17@users.noreply.github.com>
Co-authored-by: shiori <98276492+2shiori17@users.noreply.github.com>
2254: Test with default CLI opts r=Kerollmops a=Kerollmops
Fixes#2252.
This PR makes sure that we test the HTTP engine with the default CLI parameters and removes some useless internal CLI options.
Co-authored-by: Kerollmops <clement@meilisearch.com>
2280: Bump the milli dependency to 0.24.1 r=curquiza a=Kerollmops
We had issues with lindera recently, it was unable to download the official dictionaries from Google Drive and this was causing issues with our CIs (and other users' CIs too). The maintainer changed the source to download the dictionaries to get it from Sourceforge and it is much better and stable now.
This PR bumps the milli dependency to the latest version which includes the latest version of the tokenizer which, itself, includes the latest version of lindera, I advise that we rebase the currently opened pull requests to include this PR when it is merged on main.
Co-authored-by: Kerollmops <clement@meilisearch.com>
2263: Upgrade the config of the issue management r=curquiza a=curquiza
To reduce the feature feedback in the meilisearch repo
Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
2253: refactor authentication key extraction r=ManyTheFish a=MarinPostma
I am concerned that the part of the code that performs the key prefix extraction from the jwt token migh be misused in the future. Since this is a critical part of the code, I moved it into it's own function. Since we deserialized the payload twice anyway, I reordered the verifications, and we now use the data from the validated token.
Co-authored-by: ad hoc <postma.marin@protonmail.com>
Tamo