Commit Graph

265 Commits

Author SHA1 Message Date
ad hoc
46cdc17701
make scheduler accept multiple batch handlers 2022-05-25 11:13:34 +02:00
ad hoc
6a0231cb28
perform dump method 2022-05-25 11:13:33 +02:00
ad hoc
7fa3eb1003
register dump tasks 2022-05-25 11:13:33 +02:00
ad hoc
2f0625a984
register and insert dump task in scheduler 2022-05-25 11:13:33 +02:00
ad hoc
737b891a41
introduce Dump TaskListIdentifier variant 2022-05-25 11:13:33 +02:00
ad hoc
5a5066023b
introduce TaskListIdentifier 2022-05-25 11:13:33 +02:00
ad hoc
aa50acb031
make Task index_uid an option
Not all task relate to an index. Tasks that don't have an index_uid set
to None
2022-05-25 11:13:32 +02:00
bors[bot]
341756a0eb
Merge #2357
2357: chore(dump): add dump tests r=Kerollmops a=irevoire

Add tests on the import of dump v1, v2, v3 and v4.

Since the dumps are slow to decompress, I made the `flate2` crate always compile in optimized.
And since they're also slow to index, I also made the `milli` crate always compile in optimized. What do you think of this `@MarinPostma?`
Should we keep milli unoptimized in case it could help us debug some things? 👀 

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-05-24 12:24:29 +00:00
Tamo
5f0e9b63d2
chore(dump): add tests 2022-05-24 14:21:56 +02:00
Irevoire
4e9accdeb7
chore(search): rename in the search endpoint
Fix ##2376
2022-05-19 16:31:37 +02:00
ManyTheFish
50763aac82 Fix clippy 2022-05-19 11:23:22 +02:00
ManyTheFish
0250ea9157 Intergrate smart crop in Meilisearch 2022-05-18 18:35:51 +02:00
bors[bot]
6d221058f1
Merge #2404
2404: Bring `release-v0.27.1` into `main` r=curquiza a=curquiza

Following the v0.27.1 hotfixes

Co-authored-by: ad hoc <postma.marin@protonmail.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-05-17 16:11:17 +00:00
Clémentine Urquizar
49e857776c
Update version for next release (v0.27.1) 2022-05-17 11:59:35 +02:00
Tamo
85d19bfb3e
chore: bump milli 2022-05-16 18:43:35 +02:00
ad hoc
5670b4d012
fix dump import error 2022-05-16 14:33:33 +02:00
bors[bot]
b9b9cba154
Merge #2383
2383: v0.27.0: bring `stable` into `main` r=Kerollmops a=curquiza

Bring `stable` into `main`

Co-authored-by: ad hoc <postma.marin@protonmail.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Paul Sanders <psanders1@gmail.com>
Co-authored-by: Irevoire <tamo@meilisearch.com>
Co-authored-by: Morgane Dubus <30866152+mdubus@users.noreply.github.com>
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
2022-05-16 08:35:25 +00:00
Clémentine Urquizar
c46f3587de
Bump milli to v0.26.4 2022-05-04 11:25:36 +02:00
ad hoc
6025372565
fix(lib): Check db presence after dumps 2022-04-27 10:41:09 +02:00
Clémentine Urquizar
38d681c230
Change Nelson path 2022-04-21 18:42:34 +02:00
Clémentine Urquizar - curqui
6ff8bf823d
Revert "[TEST PURPOSE] Bump meilisearch to version 9000.0.0" 2022-04-21 16:36:56 +02:00
releasemops
f1cd6b6ee8 bump meilisearch to v9000.0.0 2022-04-21 14:26:40 +00:00
Clémentine Urquizar - curqui
acf3357cf3
Revert "[TEST PURPOSE] Bump meilisearch to version 8000.1.0" 2022-04-21 16:24:27 +02:00
releasemops
0714551101 bump meilisearch to v8000.1.0 2022-04-21 13:23:46 +00:00
Tamo
1ef87cc6d0
chore: move permissive-json-pointer in the meilisearch repository
Update permissive-json-pointer/src/lib.rs

Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-04-20 19:24:41 +02:00
bors[bot]
4a9000bb96
Merge #2332
2332: fix(search): formatted field r=curquiza a=irevoire

fix #2318

Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-04-20 14:59:41 +00:00
bors[bot]
754c49f991
Merge #2326
2326: rename min word lenght for typo r=irevoire a=MarinPostma

rename `minWordLengthForTypo` to `minWordSizeForTypos` as specified.

discussed here: https://github.com/meilisearch/specifications/pull/117#discussion_r850795714

Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-04-20 11:54:10 +00:00
Clémentine Urquizar
a7fd199ded
Fix typo reseting by upgrading milli to v0.26.2 2022-04-20 12:24:46 +02:00
Irevoire
58a1124e9a
fix(search): formatted field 2022-04-20 11:30:01 +02:00
ad hoc
9b064e53e7
fix(http, lib): rename_min_word_length_for_typo into rename_min_word_size_for_typo 2022-04-17 10:02:56 +02:00
bors[bot]
289bfd46ff
Merge #2321
2321: Bump milli r=curquiza a=irevoire



Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-04-14 11:51:15 +00:00
Irevoire
64b0a50a58
chore: bump milli 2022-04-14 12:12:54 +02:00
bors[bot]
b1333ab5b0
Merge #2320
2320: chore(http, lib): rename typo to typo_tolerance r=irevoire a=MarinPostma

fix #2319


Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-04-14 09:50:39 +00:00
ad hoc
276dc6043a
chore(http, lib): rename typo to typo_tolerance 2022-04-14 10:42:06 +02:00
bors[bot]
6c06fb226d
Merge #2307
2307: Feat(Analytics): Add analytics for search format options r=irevoire a=ManyTheFish

Specification: [#120](https://github.com/meilisearch/specifications/pull/120) ([f5c6a8e](f5c6a8e183))

fix #2308

Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-04-13 12:01:52 +00:00
bors[bot]
049cf0fcee
Merge #2313
2313: fix(search): remove the back and forth between the IndexMap and the serde_json::Map r=irevoire a=irevoire

This is ok because we're using the preserve_order feature in serde_json which is already internally using an IndexMap.

See https://github.com/meilisearch/meilisearch/pull/2298#discussion_r845228412_


Co-authored-by: Tamo <tamo@meilisearch.com>
2022-04-12 14:17:26 +00:00
Tamo
2ee210483f
fix(search): remove the back and forth between the IndexMap and the serde_json::Map
This is ok because we're using the preserve_order feature in serde_json which is already internally using an IndexMap.
2022-04-12 16:12:52 +02:00
Clémentine Urquizar
b3661bf8ec
Change version for the next release (v0.27.0) 2022-04-11 16:25:15 +02:00
ManyTheFish
0990e95830 Feat(Analytics): Add analytics for search format options 2022-04-11 14:53:15 +02:00
Tamo
69d312209e
feat(search): Implements the nested fields
See https://github.com/meilisearch/specifications/pull/121
2022-04-07 19:47:20 +02:00
bors[bot]
013fe4cbc9
Merge #2297
2297: Feat(Search): Enhance formating search results r=ManyTheFish a=ManyTheFish

Add new settings and change crop_len behavior to count words instead of characters.

- [x] `highlightPreTag`
- [x] `highlightPostTag`
- [x] `cropMarker`
- [x] `cropLength` count word instead of chars
- [x] `cropLength` 0 is now considered as no `cropLength`
- [ ] ~smart crop finding the best matches interval~ (postponed)

Partially fixes  #2214. (no smart crop)


Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-04-07 13:29:56 +00:00
ManyTheFish
dc2cc1ee89 Feat(Search): Enhance formating search results 2022-04-07 15:04:08 +02:00
ad hoc
67dea08a0a
feat(http, lib): enable disable typos on attributes 2022-04-06 19:25:12 +02:00
ad hoc
e9f66b8766
feat(all): introduce disable typo on words 2022-04-06 19:16:36 +02:00
ad hoc
dd43ba6234
feat(all): introduce disable typos 2022-04-06 19:10:12 +02:00
ad hoc
27a88bcd47
feat(all): introduce minWordLengthForTypo
fix typo in settting

skip serializing not set typo settings
2022-04-06 19:03:24 +02:00
ad hoc
981fba5b44
feat(all): introduce disable typos 2022-04-06 15:47:48 +02:00
ad hoc
a523828f61
chore(lib): bump milli to 0.25.0 2022-04-06 15:03:10 +02:00
bors[bot]
9e344f6576
Merge #2207
2207: Fix: avoid embedding the user input into the error response. r=Kerollmops a=CNLHC

# Pull Request

## What does this PR do?
Fix #2107. 

The problem is meilisearch embeds the user input to the error message. 

The reason for this problem is `milli` throws a `serde_json: Error` whose `Display` implementation will do this embedding.  

I tried to solve this problem in this PR by manually implementing the `Display` trait for `DocumentFormatError` instead of deriving automatically.

<!-- Please link the issue you're trying to fix with this PR, if none then please create an issue first. -->

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Liu Hancheng <cn_lhc@qq.com>
Co-authored-by: LiuHanCheng <2463765697@qq.com>
2022-04-04 17:35:17 +00:00
bors[bot]
09a72cee03
Merge #2281
2281: Hard limit the number of results returned by a search r=Kerollmops a=Kerollmops

This PR fixes #2133 by hard-limiting the number of results that a search request can return at any time. I would like the guidance of `@MarinPostma` to test that, should I use a mocking test here? Or should I do anything else?

I talked about touching the _nb_hits_ value with `@qdequele` and we concluded that it was not correct to do so.

Could you please confirm that it is the right place to change that?

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-04-04 17:19:05 +00:00
Liu Hancheng
7ece7a9d9e change truncate strategy and coresponding test 2022-03-31 10:39:21 +08:00
LiuHanCheng
b28aa8e666
Update meilisearch-lib/src/document_formats.rs
Co-authored-by: Clément Renault <renault.cle@gmail.com>
2022-03-31 10:14:13 +08:00
2shiori17
98107565c0 Add more detailed comments for max_indexing_threads 2022-03-31 09:32:45 +09:00
2shiori17
a2d7c16f91 Remove indexing_jobs option 2022-03-31 09:27:29 +09:00
shiori
9edd407a88
Merge branch 'main' into add-instance-options 2022-03-31 02:38:07 +09:00
Kerollmops
8bc6e8dcf9
Make sure that offsets are clamped too 2022-03-30 10:06:15 -07:00
Kerollmops
b3a11e04af
Implement Default on IndexerOpts again 2022-03-29 11:37:08 -07:00
Kerollmops
acdb10a307
Remove some useless indexer options 2022-03-29 11:37:08 -07:00
Kerollmops
8fecc6238d
Make the test use the default CLI options 2022-03-29 11:37:08 -07:00
Kerollmops
405af09fc8
Hard limit the number of results returned by a search 2022-03-29 11:27:53 -07:00
Kerollmops
94f04e79eb
Bump the milli dependency to 0.24.1 2022-03-29 09:17:25 -07:00
LiuHanCheng
13a0e78d3f
Update meilisearch-lib/src/document_formats.rs
Co-authored-by: Clément Renault <renault.cle@gmail.com>
2022-03-28 14:58:00 +08:00
LiuHanCheng
80d8ac40af
Update meilisearch-lib/src/document_formats.rs
Co-authored-by: Clément Renault <renault.cle@gmail.com>
2022-03-28 14:57:51 +08:00
Liu Hancheng
c7b489f8cb tidy 2022-03-25 21:36:11 +08:00
Liu Hancheng
3c72f4dc51 fix test and add truncate test. 2022-03-25 21:31:23 +08:00
Liu Hancheng
ce85981a4e add truncate logic 2022-03-25 20:53:28 +08:00
Liu Hancheng
193c666bf9 Merge branch 'main' of github.com:meilisearch/meilisearch into CNLHC/change_json_error_message 2022-03-25 19:53:13 +08:00
2shiori17
705d10a96d Add instance options for RAM and CPU usage 2022-03-24 18:52:36 +00:00
Kerollmops
3df542f072
Export milli's heed from meilisearch-lib 2022-03-24 15:30:10 +01:00
Kerollmops
ee6be4f6b9
Import milli from meilisearch-lib in meilisearch-http 2022-03-24 14:45:37 +01:00
Kerollmops
ac48860bbb
Upgrade the workspace dependencies 2022-03-17 11:03:31 +01:00
Kerollmops
86c1e83ea1
Remove three unused dependencies 2022-03-17 11:00:24 +01:00
bors[bot]
bb9372114c
Merge #2244
2244: chore(all): bump milli r=curquiza a=MarinPostma

continues the work initiated by `@psvnlsaikumar` in #2228

Co-authored-by: Sai Kumar <psvnlsaikumar@gmail.com>
2022-03-16 17:15:10 +00:00
Sai Kumar
e271395971
chore(all): bump milli
* updates to Use the milli's heed dependency #2210

* Update index.rs

* Update store.rs

* Update mod.rs

* cargo fmt
2022-03-16 16:34:44 +01:00
ad hoc
32843f30d9
bug(http): fix panic on startup 2022-03-16 13:33:37 +01:00
LiuHanCheng
c8895cab77
Update meilisearch-lib/src/document_formats.rs
Co-authored-by: Clément Renault <renault.cle@gmail.com>
2022-03-08 12:03:59 +08:00
ad hoc
62ce8e0bda
chore(http): rename auto batching cli option 2022-03-07 15:19:19 +01:00
Liu Hancheng
b138b92d39 show detailed error message 2022-03-04 15:31:11 +08:00
Liu Hancheng
58e2903177 first try 2022-03-04 10:46:59 +08:00
ManyTheFish
c2b58720d1 Fix(dumps): Explicitly define serde for time 2022-03-02 11:37:48 +01:00
Rob Ede
15150db957
clippy 2022-02-28 19:03:38 +01:00
Rob Ede
3b2e467ca6
update actix-web dependency to 4.0 2022-02-28 19:03:37 +01:00
Rob Ede
8d624b3800
clippy 2022-02-28 13:43:22 +00:00
ad hoc
4fbb83a34d
bug(snapshot): Correctly open environments in snapshots 2022-02-28 12:37:30 +01:00
Rob Ede
961e22493c
update actix-web dependency to 4.0 2022-02-25 23:28:55 +00:00
ad hoc
7e832105d7
bug(snapshot): Correctly open environments in snapshots 2022-02-23 17:12:08 +01:00
Tamo
21d277a0ef
fix(all): fix two dates that were wrongly formatted 2022-02-22 11:29:11 +01:00
bors[bot]
c3e3c900f2
Merge #2173
2173: chore(all): replace chrono with time r=irevoire a=irevoire

Chrono has been unmaintained for a few month now and there is a CVE on it.

Also I updated all the error messages related to the API key as you can see here: https://github.com/meilisearch/specifications/pull/114

fix #2172

Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-02-17 14:12:23 +00:00
Irevoire
05c8d81e65
chore: get rid of chrono in favor of time
Chrono has been unmaintened for a few month now and there is a CVE on it.

make clippy happy

bump milli
2022-02-16 18:14:29 +01:00
Clémentine Urquizar
e2a9414c7a
Update version (v0.26.0) 2022-02-14 16:11:07 +01:00
Clémentine Urquizar
1a87b2f37d
Bump milli to v0.22.1 2022-02-08 11:21:44 +01:00
mpostma
c9a236b0af
feat(lib): auto-batching 2022-02-01 18:06:20 +01:00
bors[bot]
622c15e825
Merge #2096
2096: feat(auth): Tenant token r=Kerollmops a=ManyTheFish

Make meilisearch support JWT authentication signed with meilisearch API keys
using HS256, HS384 or HS512 algorithms.

Related spec: [specifications#89](https://github.com/meilisearch/specifications/pull/89) [rendered](https://github.com/meilisearch/specifications/blob/scoped-api-keys/text/0089-tenant-tokens.md)
Fix #1991 


Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-01-27 10:38:41 +00:00
ManyTheFish
7ca647f0d0 feat(auth): Implement Tenant token
Make meilisearch support JWT authentication signed with meilisearch API keys
using HS256, HS384 or HS512 algorithms.

Related spec: https://github.com/meilisearch/specifications/pull/89
Fix #1991
2022-01-27 08:25:39 +01:00
Clémentine Urquizar - curqui
aa50fcb1f0
Merge branch 'main' into stable 2022-01-26 20:17:41 +01:00
Clémentine Urquizar
8a959da120
Update MeiliSearch into Meilisearch everywhere 2022-01-26 17:43:16 +01:00
bors[bot]
c8bb54cd94
Merge #2098
2098: feat(dump): Provide the same cli options as the snapshots r=MarinPostma a=irevoire

Add two cli options for the dump:
- `--ignore-missing-dump`
- `--ignore-dump-if-db-exists`

Fix #2087

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-01-26 14:32:23 +00:00
Tamo
bad4bed439
feat(dump): Provide the same cli options as the snapshots
Add two cli options for the dump:
- `--ignore-missing-dump`
- `--ignore-dump-if-db-exists`

Fix #2087
2022-01-26 14:34:06 +01:00
bors[bot]
8ae504bfb0
Merge #2101
2101: chore(all): update actix-web dependency to 4.0.0-beta.21 r=MarinPostma a=robjtede

# Pull Request

## What does this PR do?
I don't expect any more breaking changes to Actix Web that will affect Meilisearch so bump to latest beta.

Fixes #N/A?
<!-- Please link the issue you're trying to fix with this PR, if none then please create an issue first. -->

## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to MeiliSearch!


Co-authored-by: Rob Ede <robjtede@icloud.com>
2022-01-24 14:33:46 +00:00
bors[bot]
5981e6c57c
Merge #2095
2095: feat(error): Update the error message when you have no version file r=MarinPostma a=irevoire

Following this [issue](https://github.com/meilisearch/meilisearch-kubernetes/issues/95) we decided to change the error message from:
```
Version file is missing or the previous MeiliSearch engine version was below 0.24.0. Use a dump to update MeiliSearch.
```
to
```
Version file is missing or the previous MeiliSearch engine version was below 0.25.0. Use a dump to update MeiliSearch.
```

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-01-24 14:09:13 +00:00