Commit Graph

526 Commits

Author SHA1 Message Date
Tamo
2f1eb78b1d
refactor the Task a little bit 2022-10-27 11:34:02 +02:00
Tamo
510ce9fc51
start moving a lot of task types to meilisearch_types 2022-10-27 11:34:01 +02:00
Tamo
fa4c1de019
store md5 instead of the whole snapshots 2022-10-27 11:34:01 +02:00
Loïc Lecrenier
3e4337c91f
Add meili-snap crate to make writing snapshot tests easier 2022-10-27 11:34:01 +02:00
Tamo
667c282e19
get rids of the index crate + the document_types crate 2022-10-27 11:34:00 +02:00
Loïc Lecrenier
4c55c30027
Add a DetailsView type and improve index scheduler snapshots
The DetailsView type is necessary because serde incorrectly
deserialises the `Details` type, so the database fails to correctly
decode Tasks
2022-10-27 11:33:59 +02:00
Tamo
47e0288747
rewrite the compat API to something more generic 2022-10-27 11:33:53 +02:00
Tamo
e845cc2b6f
fix the tests 2022-10-27 11:33:51 +02:00
Tamo
7bd6f63001
implement the dump reader v6 2022-10-27 11:33:51 +02:00
Tamo
87a6a337aa
write a dump exporter 2022-10-27 11:33:51 +02:00
Kerollmops
91e13c2824
Implement ErrorCode on the milli::Error type 2022-10-27 11:33:48 +02:00
Tamo
2c8f1a43e9
get rids of meilisearch-lib 2022-10-27 11:33:44 +02:00
Tamo
6f4dcc0c38
start implementing some logic to test the internal states of the scheduler 2022-10-27 11:33:43 +02:00
Tamo
84cd5cef0b
fix the tests 2022-10-27 11:33:43 +02:00
Irevoire
8d51c1f389
wip integrating the scheduler in meilisearch-http 2022-10-27 11:33:42 +02:00
Irevoire
250410495c
start integrating the index-scheduler in meilisearch-lib 2022-10-27 11:33:42 +02:00
Irevoire
8f0fd35358
add insta::json for later 2022-10-27 11:33:42 +02:00
Irevoire
8770e07397
I can index documents without meilisearch 2022-10-27 11:33:42 +02:00
Tamo
7d0c8a3379
test the register tasks 2022-10-27 11:33:40 +02:00
Tamo
fc098022c7
start integrating the index-scheduler in the meilisearch codebase 2022-10-27 11:33:40 +02:00
Tamo
366a344474
get rids of the horrendous spinlock in favor of synchronoise 2022-10-27 11:33:39 +02:00
Tamo
c97d51a624
add a bunch of tests 2022-10-27 11:33:39 +02:00
Tamo
94e29a9f5f
extract the index abstraction out of the index-scheduler in its own module 2022-10-27 11:33:38 +02:00
Tamo
76597fc382
import the update_file_store in the index-scheduler 2022-10-27 11:33:37 +02:00
Tamo
2afb381f95
get rids of nelson 2022-10-27 11:33:37 +02:00
Tamo
a9844bd4f6
move the update file store to another crate with as little dependencies as possible 2022-10-27 11:33:37 +02:00
Tamo
46b8ebcab4
fix the file store 2022-10-27 11:33:35 +02:00
Tamo
a7aa92df5f
fix most of the index module 2022-10-27 11:33:34 +02:00
Irevoire
d8b8e04ad1
wip porting the index back in the scheduler 2022-10-27 11:33:34 +02:00
Tamo
1a47949063
START THE REWRITE OF THE INDEX SCHEDULER: index & register has been implemented 2022-10-27 11:33:33 +02:00
ManyTheFish
4afed4de4f stabilize milli 2022-10-24 14:16:41 +02:00
ManyTheFish
c02ae4dfc0 Update roaring 2022-10-19 14:25:43 +02:00
ManyTheFish
062d17fbc0 Use a milli version that compute exhaustivelly the number of hits 2022-10-19 14:05:42 +02:00
bors[bot]
b0749407f3
Merge #2804
2804: Add environement variable `MEILI_CONFIG_FILE_PATH` to define the config file path r=Kerollmops a=choznerol

# Pull Request

## What does this PR do?
Fixes #2800

~This is a draft PR base on the code in #2745. I will `rebase` and mark it ready for review only after #2745 merge.~ Done rebase


## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?


## Demo

With `config.toml`, `config_copy1.toml` and `config_copy2.toml` present:
> <img width="692" alt="image" src="https://user-images.githubusercontent.com/12410942/192566891-6b3c9d26-736f-4e23-a09b-687fca1cb50d.png">

`MEILI_CONFIG_FILE_PATH` works:
> <img width="773" alt="image" src="https://user-images.githubusercontent.com/12410942/192567023-f751536e-992a-4e90-a176-cb19122248be.png">

`--config-file-path` still works:
> <img width="768" alt="image" src="https://user-images.githubusercontent.com/12410942/192567318-88c80b24-7873-4cec-8d08-16fe4d228055.png">

When both present, `--config-file-path` taks precedence:
> <img width="1214" alt="image" src="https://user-images.githubusercontent.com/12410942/192567477-8a7cffe1-96f0-42a9-a348-6dbec20dc1e7.png">



Co-authored-by: Lawrence Chou <choznerol@protonmail.com>
2022-10-17 10:40:05 +00:00
bors[bot]
5d895dd7da
Merge #2851
2851: Upgrade clap to 4.0 r=loiclec a=choznerol

# Pull Request

## Related issue
Fixes #2846 

This PR is draft based on #2847 to avoid conflict. I will rebase and mark as 'Ready for review' after #2847 is merged.

## What does this PR do?
1. Upgrade clap to the latest version or 4.0 (4.0.9 as of today) by following the [migrating instruction](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#migrating) from [4.0 changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#migrating)
2. Fix an `ArgGroup` typo that can only be caught after upgrading to 4.0 in 20a715e29ed17c5a76229c98fb31504ada873597

## Notable changes

### The `--help` message

The format, ordering and indentation of `--help` message was changed in 4.0. I recorded the output of `cargo run -- --help` before and after upgrade to 4.0 for reference.

<details>
<summary>diff</summary>

Output of `diff --ignore-all-space --text --unified --new-file help-message-before.txt help-message-after.txt`:

```diff
--- help-message-before.txt	2022-10-14 16:45:36.000000000 +0800
+++ help-message-after.txt	2022-10-14 16:36:53.000000000 +0800
`@@` -1,12 +1,8 `@@`
-meilisearch-http 0.29.1
+Usage: meilisearch [OPTIONS]
 
-USAGE:
-    meilisearch [OPTIONS]
-
-OPTIONS:
+Options:
         --config-file-path <CONFIG_FILE_PATH>
-            Set the path to a configuration file that should be used to setup the engine. Format
-            must be TOML
+          Set the path to a configuration file that should be used to setup the engine. Format must be TOML
 
         --db-path <DB_PATH>
             Designates the location where database files will be created and retrieved
`@@` -26,15 +22,14 `@@`
             [default: dumps/]
 
         --env <ENV>
-            Configures the instance's environment. Value must be either `production` or
-            `development`
+          Configures the instance's environment. Value must be either `production` or `development`
             
             [env: MEILI_ENV=]
             [default: development]
             [possible values: development, production]
 
     -h, --help
-            Print help information
+          Print help information (use `-h` for a summary)
 
         --http-addr <HTTP_ADDR>
             Sets the HTTP address and port Meilisearch will use
`@@` -43,63 +38,53 `@@`
             [default: 127.0.0.1:7700]
 
         --http-payload-size-limit <HTTP_PAYLOAD_SIZE_LIMIT>
-            Sets the maximum size of accepted payloads. Value must be given in bytes or explicitly
-            stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
+          Sets the maximum size of accepted payloads. Value must be given in bytes or explicitly stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
             
             [env: MEILI_HTTP_PAYLOAD_SIZE_LIMIT=]
             [default: 100000000]
 
         --ignore-dump-if-db-exists
-            Prevents a Meilisearch instance with an existing database from throwing an error when
-            using `--import-dump`. Instead, the dump will be ignored and Meilisearch will launch
-            using the existing database.
+          Prevents a Meilisearch instance with an existing database from throwing an error when using `--import-dump`. Instead, the dump will be ignored and Meilisearch will launch using the existing database.
             
             This option will trigger an error if `--import-dump` is not defined.
             
             [env: MEILI_IGNORE_DUMP_IF_DB_EXISTS=]
 
         --ignore-missing-dump
-            Prevents Meilisearch from throwing an error when `--import-dump` does not point to a
-            valid dump file. Instead, Meilisearch will start normally without importing any dump.
+          Prevents Meilisearch from throwing an error when `--import-dump` does not point to a valid dump file. Instead, Meilisearch will start normally without importing any dump.
             
             This option will trigger an error if `--import-dump` is not defined.
             
             [env: MEILI_IGNORE_MISSING_DUMP=]
 
         --ignore-missing-snapshot
-            Prevents a Meilisearch instance from throwing an error when `--import-snapshot` does not
-            point to a valid snapshot file.
+          Prevents a Meilisearch instance from throwing an error when `--import-snapshot` does not point to a valid snapshot file.
             
             This command will throw an error if `--import-snapshot` is not defined.
             
             [env: MEILI_IGNORE_MISSING_SNAPSHOT=]
 
         --ignore-snapshot-if-db-exists
-            Prevents a Meilisearch instance with an existing database from throwing an error when
-            using `--import-snapshot`. Instead, the snapshot will be ignored and Meilisearch will
-            launch using the existing database.
+          Prevents a Meilisearch instance with an existing database from throwing an error when using `--import-snapshot`. Instead, the snapshot will be ignored and Meilisearch will launch using the existing database.
             
             This command will throw an error if `--import-snapshot` is not defined.
             
             [env: MEILI_IGNORE_SNAPSHOT_IF_DB_EXISTS=]
 
         --import-dump <IMPORT_DUMP>
-            Imports the dump file located at the specified path. Path must point to a `.dump` file.
-            If a database already exists, Meilisearch will throw an error and abort launch
+          Imports the dump file located at the specified path. Path must point to a `.dump` file. If a database already exists, Meilisearch will throw an error and abort launch
             
             [env: MEILI_IMPORT_DUMP=]
 
         --import-snapshot <IMPORT_SNAPSHOT>
-            Launches Meilisearch after importing a previously-generated snapshot at the given
-            filepath
+          Launches Meilisearch after importing a previously-generated snapshot at the given filepath
             
             [env: MEILI_IMPORT_SNAPSHOT=]
 
         --log-level <LOG_LEVEL>
             Defines how much detail should be present in Meilisearch's logs.
             
-            Meilisearch currently supports five log levels, listed in order of increasing verbosity:
-            ERROR, WARN, INFO, DEBUG, TRACE.
+          Meilisearch currently supports five log levels, listed in order of increasing verbosity: ERROR, WARN, INFO, DEBUG, TRACE.
             
             [env: MEILI_LOG_LEVEL=]
             [default: INFO]
`@@` -110,31 +95,25 `@@`
             [env: MEILI_MASTER_KEY=]
 
         --max-index-size <MAX_INDEX_SIZE>
-            Sets the maximum size of the index. Value must be given in bytes or explicitly stating a
-            base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
+          Sets the maximum size of the index. Value must be given in bytes or explicitly stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
             
             [env: MEILI_MAX_INDEX_SIZE=]
             [default: 107374182400]
 
         --max-indexing-memory <MAX_INDEXING_MEMORY>
-            Sets the maximum amount of RAM Meilisearch can use when indexing. By default,
-            Meilisearch uses no more than two thirds of available memory
+          Sets the maximum amount of RAM Meilisearch can use when indexing. By default, Meilisearch uses no more than two thirds of available memory
             
             [env: MEILI_MAX_INDEXING_MEMORY=]
             [default: "21.33 TiB"]
 
         --max-indexing-threads <MAX_INDEXING_THREADS>
-            Sets the maximum number of threads Meilisearch can use during indexation. By default,
-            the indexer avoids using more than half of a machine's total processing units. This
-            ensures Meilisearch is always ready to perform searches, even while you are updating an
-            index
+          Sets the maximum number of threads Meilisearch can use during indexation. By default, the indexer avoids using more than half of a machine's total processing units. This ensures Meilisearch is always ready to perform searches, even while you are updating an index
             
             [env: MEILI_MAX_INDEXING_THREADS=]
             [default: 5]
 
         --max-task-db-size <MAX_TASK_DB_SIZE>
-            Sets the maximum size of the task database. Value must be given in bytes or explicitly
-            stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
+          Sets the maximum size of the task database. Value must be given in bytes or explicitly stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
             
             [env: MEILI_MAX_TASK_DB_SIZE=]
             [default: 107374182400]
```

- ~[help-message-before.txt](https://github.com/meilisearch/meilisearch/files/9715683/help-message-before.txt)~ [help-message-before.txt](https://github.com/meilisearch/meilisearch/files/9784156/help-message-before-2.txt)
- ~[help-message-after.txt](https://github.com/meilisearch/meilisearch/files/9715682/help-message-after.txt)~ [help-message-after.txt](https://github.com/meilisearch/meilisearch/files/9784091/help-message-after.txt)

</details>


## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Lawrence Chou <choznerol@protonmail.com>
2022-10-17 10:08:27 +00:00
Lawrence Chou
3c3ae3ff98
Impeove invalid config_file_path handling
1. Besides opt.config_file_path, also consider MEILI_CONFIG_FILE_PATH in the Err path because they are both user input.
2. Print out the incorrect file path in error message.
3. Add tests
https://github.com/meilisearch/meilisearch/pull/2804#discussion_r991999888
2022-10-12 12:04:48 +08:00
Andrey "MOU" Larionov
11b986a81d
Added support for specifying compression in tests
Refactored tests code to allow to specify compression (content-encoding) algorithm.

Added tests to verify what actix actually handle different content encodings properly.
2022-10-09 22:09:29 +02:00
Lawrence Chou
9e5ef8eb69
Upgrade clap to v4
Close #2846

4.0.0 changelog: 'https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#400---2022-09-28'

I followed the [Migrating steps](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#migrating) and the only issue I encountered are:
1. The typo problem in previous commit "Fix clap ArgGroup typo"
2. I can't say I am 100% sure every [Subtle changes](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#breaking-changes) is fine for our use case, but at least after a quick read I didn't notice anything actionable.
2022-10-07 00:32:25 +08:00
bors[bot]
1b72eba1f3
Merge #2867
2867: Bring back `stable` into `main` r=Kerollmops a=curquiza

Following hotfix for v0.29.1

Co-authored-by: Loïc Lecrenier <loic@meilisearch.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
2022-10-06 14:14:23 +00:00
Loïc Lecrenier
05af8f0e46 Update version for next release (v0.29.1) 2022-10-06 10:27:11 +02:00
Loïc Lecrenier
a2c91a87fe Upgrade dependencies
Except:
- clap stays on 3.0 because it is more complicated to upgrade
- enum_iterator goes up to 1.1.2 instead of 1.2 because of the vergen
dependency
2022-10-05 15:53:02 +02:00
Clémentine Urquizar - curqui
a7d2c9572e
Merge branch 'main' into stable 2022-10-04 14:20:10 +02:00
bors[bot]
fcedce8578
Merge #2745 #2789 #2814 #2826
2745: Config file support r=curquiza a=mlemesle

# Pull Request

## What does this PR do?
Fixes #2558

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


2789: Fix typos r=Kerollmops a=kianmeng

# Pull Request

## What does this PR do?

Found via `codespell -L crate,nam,hart`.

## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


2814: Skip dashboard test if mini-dashboard feature is disabled r=Kerollmops a=jirutka

Fixes #2813

Fixes the following error:

    cargo test --no-default-features
    ...
    error: couldn't read target/debug/build/meilisearch-http-ec029d8c902cf2cb/out/generated.rs: No such file or directory (os error 2)
     --> meilisearch-http/tests/dashboard/mod.rs:8:9
      |
    8 |         include!(concat!(env!("OUT_DIR"), "/generated.rs"));
      |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |
      = note: this error originates in the macro `include` (in Nightly builds, run with -Z macro-backtrace for more info)

    error: could not compile `meilisearch-http` due to previous error

2826: Rename receivedDocumentIds into matchedDocuments r=Kerollmops a=Ugzuzg

# Pull Request

## What does this PR do?

Fixes #2799 
Changes DocumentDeletion task details response.

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Tested with curl:
```
curl \
  -X POST 'http://localhost:7700/indexes/movies/documents/delete-batch' \
  -H 'Content-Type: application/json' \
  --data-binary '[
    23488,
    153738,
    437035,
    363869
  ]'

{"taskUid":1,"indexUid":"movies","status":"enqueued","type":"documentDeletion","enqueuedAt":"2022-10-01T20:06:37.105416054Z"}%

curl \
  -X GET 'http://localhost:7700/tasks/1'

{"uid":1,"indexUid":"movies","status":"succeeded","type":"documentDeletion","details":{"matchedDocuments":4,"deletedDocuments":2},"duration":"PT0.005708322S","enqueuedAt":"2022-10-01T20:06:37.105416054Z","startedAt":"2022-10-01T20:06:37.115562733Z","finishedAt":"2022-10-01T20:06:37.121271055Z"}
```

Co-authored-by: mlemesle <lemesle.martin@hotmail.fr>
Co-authored-by: Kian-Meng Ang <kianmeng@cpan.org>
Co-authored-by: Jakub Jirutka <jakub@jirutka.cz>
Co-authored-by: Jarasłaŭ Viktorčyk <ugzuzg@gmail.com>
2022-10-03 09:48:34 +00:00
Jakub Jirutka
5b57114771 Bump milli from 0.33.0 to 0.33.4 2022-09-14 20:52:11 +02:00
Clémentine Urquizar
268d59ccb1
Update milli version to v0.33.4 2022-09-13 18:01:09 +02:00
Clémentine Urquizar
aabd67a9fa
Update patch version to remove CVE 2022-09-12 14:36:45 +02:00
Kerollmops
441492f1c8
Bump milli to v0.33.3 2022-09-07 18:23:49 +02:00
mlemesle
403226a029 Add support for config file 2022-09-07 16:09:00 +02:00
ManyTheFish
50434d35d0 Update milli v0.33.2 2022-09-01 13:15:05 +02:00
evpeople
833ade80a6 cargo update 2022-08-31 13:58:53 +08:00
ManyTheFish
aff4b64265 Update dependencies 2022-08-23 16:39:39 +02:00
bors[bot]
0a2ef0037f
Merge #2689 #2690
2689: Use mimalloc as the global allocator r=Kerollmops a=loiclec

milli has switched its global allocator to mimalloc already, and we have seen some performance gains as a result. Furthermore, we can use mimalloc as the global allocator on all platforms whereas jemalloc was only activated on Linux. 

This PR brings mimalloc to Meilisearch as well. 

2690: Add LTO and codegen-units=1 to release compile options r=Kerollmops a=loiclec

This PR brings Meilisearch's release compile options in line with milli (see https://github.com/meilisearch/milli/pull/606 ). 

Adding LTO and codegen=units=1 will make compile times longer, but they also speed up the final binary significantly.

Co-authored-by: Loïc Lecrenier <loic@meilisearch.com>
2022-08-23 12:05:02 +00:00
Clémentine Urquizar
6fe3f285ce
Update version for next release (v0.29.0) 2022-08-23 13:39:56 +02:00
Loïc Lecrenier
e659c08ac4 Use mimalloc as the global allocator 2022-08-23 12:58:10 +02:00
mohandasspat
4bee0565e8 prometheus and grafana dashboards implemented 2022-08-22 13:21:22 +05:30
Clément Renault
9b2036ac05
Accept either an array of documents or a single document 2022-08-18 11:55:14 +02:00
bors[bot]
b5f91b91c3
Merge #2523
2523: Improve the tasks error reporting when processed in batches r=irevoire a=Kerollmops

This fixes #2478 by changing the behavior of the task handler when there is an error in a batch of document addition or update.

What changes is that when there is a user error in a task in a batch we now report this task as failed with the right error message but we continue to process the other tasks. A user error can be when a geo field is invalid, a document id is invalid, or missing.

fixes #2582, #2478

Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-08-16 14:15:30 +00:00
ManyTheFish
cf955a77db Fix(cli): Clamp databases max size to a multiple of system page size
fix #2659
2022-08-11 10:44:47 +02:00
Kerollmops
fe32097964 Update milli v0.32 2022-07-28 14:45:10 +02:00
Clémentine Urquizar
9cf6acb671
Fix highlight issue by updating milli to v0.31.2 2022-07-21 14:11:24 +04:00
Clémentine Urquizar
d01a3ab889
Update version for next release (v0.28.1) 2022-07-20 15:46:53 +04:00
ManyTheFish
a146fd45b9 Format API keys in hexa instead of base64 2022-07-05 16:14:18 +02:00
Irevoire
05ee2eff01
add more tests on the formatted route 2022-06-28 13:17:55 +02:00
bors[bot]
9e261b996f
Merge #2543
2543: fix all the array on the search get route and improve the tests r=curquiza a=irevoire

fix #2527

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-06-23 14:51:36 +00:00
Kerollmops
7feb15df28
Bump milli to 0.31.1 2022-06-23 10:47:48 +02:00
Tamo
c47369b502
fix all the array on the search get route and improve the tests 2022-06-22 14:33:44 +02:00
Clémentine Urquizar
32c8846514
Rollback 0.0.0 versionning 2022-06-22 12:20:12 +02:00
Clémentine Urquizar
7490383d4f
Update the not-released version in Cargo.toml files 2022-06-21 19:17:33 +02:00
Clémentine Urquizar
c6ed756dbc
Update script after review 2022-06-21 10:46:32 +02:00
Tamo
2063fbd985
chore: bump milli 2022-06-09 18:34:03 +02:00
pierre-l
36cb09eb25 Add a new meilisearch_types crate
Move `meilisearch_error` to `meilisearch_types::error`
Move `meilisearch_lib::index_resolver::IndexUid` to `meilisearch_types::index_uid`
Add a new `InvalidIndexUid` error in `meilisearch_types::index_uid`
2022-06-09 16:14:13 +02:00
bors[bot]
b9b32d65a8
Merge #2494
2494: Introduce the new faceting and pagination settings r=ManyTheFish a=Kerollmops

This PR introduces two new settings following the newly created spec https://github.com/meilisearch/specifications/pull/157:
 - The `faceting.max_values_per_facet` one describes the maximum number of values (each with a count) associated with a value in a facet distribution query.
 - The `pagination.limited_to` one describes the maximum number of documents that a search query can ever return.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-09 12:09:21 +00:00
Kerollmops
c924614527
Bump milli to 0.29.2 2022-06-09 10:54:28 +02:00
Clémentine Urquizar
fc4990b968
Update version for next release (v0.28.0) 2022-06-08 17:59:18 +02:00
bors[bot]
6171f17f1d
Merge #2468
2468: Update milli 0.29 r=Kerollmops a=ManyTheFish

- [x] Update milli to 0.29
- [x] Integrate charabia
- [x] Set disabled_words to default when Index::exact_words returns None
- [x] Fix ranking rules integration test

fixes #2375
fixes #2144
fixes #2417
fixes #2407

Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-06-08 14:29:20 +00:00
ManyTheFish
8d09772334 Update milli 2022-06-08 14:38:05 +02:00
ManyTheFish
987a7f8926 Wrap sha256 in HMAC instead of directly use sha256 2022-06-08 14:25:12 +02:00
Kerollmops
277a0a7967
Bump serde-cs to simplify our usage of the star_or function 2022-06-06 10:17:33 +02:00
bors[bot]
cf2d8de48a
Merge #2452
2452: Change http verbs r=ManyTheFish a=Kerollmops

This PR fixes #2419 by updating the HTTP verbs used to update the settings and every single setting parameter.

- [x] `PATCH /indexes/{indexUid}` instead of `PUT`
- [x] `PATCH /indexes/{indexUid}/settings`  instead of `POST`
- [x] `PATCH /indexes/{indexUid}/settings/typo-tolerance`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/displayed-attributes`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/distinct-attribute`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/filterable-attributes`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/ranking-rules`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/searchable-attributes`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/sortable-attributes`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/stop-words`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/synonyms`  instead of `POST`


Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-02 11:46:17 +00:00
bors[bot]
c9cd1738a5
Merge #2445
2445: Seek-based tasks list r=Kerollmops a=Kerollmops

This PR implements the seek-based pagination for the tasks list following [the spec](https://github.com/meilisearch/specifications/pull/115).

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-02 10:25:54 +00:00
Kerollmops
bcb51905d7
Fix the authorization tests 2022-06-02 12:16:46 +02:00
bors[bot]
8b8c3e32f0
Merge #2450
2450: Bump the dependencies r=ManyTheFish a=Kerollmops

In order to use [the latest version of grenad](https://docs.rs/grenad) I bump the dependencies here. We also use the latest versions of all our other dependencies now.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-02 08:53:12 +00:00
Kerollmops
e769043576
Bump the dependencies 2022-06-01 17:59:05 +02:00
ManyTheFish
7652295d2c Encode key in base64 instead of hexa 2022-06-01 16:17:47 +02:00
ManyTheFish
96a5791e39 Add uid and name fields in keys 2022-06-01 16:07:44 +02:00
Kerollmops
c11d21879a
Introduce tasks limit and after to the tasks route 2022-06-01 13:26:36 +02:00
Kerollmops
3684c822f1
Add indexUid filtering on the /tasks route 2022-05-31 11:33:20 +02:00
Tamo
5f0e9b63d2
chore(dump): add tests 2022-05-24 14:21:56 +02:00
ManyTheFish
0250ea9157 Intergrate smart crop in Meilisearch 2022-05-18 18:35:51 +02:00
bors[bot]
6d221058f1
Merge #2404
2404: Bring `release-v0.27.1` into `main` r=curquiza a=curquiza

Following the v0.27.1 hotfixes

Co-authored-by: ad hoc <postma.marin@protonmail.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-05-17 16:11:17 +00:00
Clémentine Urquizar
49e857776c
Update version for next release (v0.27.1) 2022-05-17 11:59:35 +02:00
Tamo
85d19bfb3e
chore: bump milli 2022-05-16 18:43:35 +02:00
bors[bot]
b9b9cba154
Merge #2383
2383: v0.27.0: bring `stable` into `main` r=Kerollmops a=curquiza

Bring `stable` into `main`

Co-authored-by: ad hoc <postma.marin@protonmail.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Paul Sanders <psanders1@gmail.com>
Co-authored-by: Irevoire <tamo@meilisearch.com>
Co-authored-by: Morgane Dubus <30866152+mdubus@users.noreply.github.com>
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
2022-05-16 08:35:25 +00:00
Clémentine Urquizar
c46f3587de
Bump milli to v0.26.4 2022-05-04 11:25:36 +02:00
Clémentine Urquizar
38d681c230
Change Nelson path 2022-04-21 18:42:34 +02:00
Clémentine Urquizar - curqui
6ff8bf823d
Revert "[TEST PURPOSE] Bump meilisearch to version 9000.0.0" 2022-04-21 16:36:56 +02:00
releasemops
f1cd6b6ee8 bump meilisearch to v9000.0.0 2022-04-21 14:26:40 +00:00
Clémentine Urquizar - curqui
acf3357cf3
Revert "[TEST PURPOSE] Bump meilisearch to version 8000.1.0" 2022-04-21 16:24:27 +02:00
releasemops
0714551101 bump meilisearch to v8000.1.0 2022-04-21 13:23:46 +00:00
Tamo
1ef87cc6d0
chore: move permissive-json-pointer in the meilisearch repository
Update permissive-json-pointer/src/lib.rs

Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-04-20 19:24:41 +02:00
Clémentine Urquizar
a7fd199ded
Fix typo reseting by upgrading milli to v0.26.2 2022-04-20 12:24:46 +02:00
Irevoire
64b0a50a58
chore: bump milli 2022-04-14 12:12:54 +02:00
Clémentine Urquizar
b3661bf8ec
Change version for the next release (v0.27.0) 2022-04-11 16:25:15 +02:00
Tamo
69d312209e
feat(search): Implements the nested fields
See https://github.com/meilisearch/specifications/pull/121
2022-04-07 19:47:20 +02:00
ad hoc
981fba5b44
feat(all): introduce disable typos 2022-04-06 15:47:48 +02:00
ad hoc
a523828f61
chore(lib): bump milli to 0.25.0 2022-04-06 15:03:10 +02:00
Kerollmops
94f04e79eb
Bump the milli dependency to 0.24.1 2022-03-29 09:17:25 -07:00
Kerollmops
09212abdf7
Update the cargo lock 2022-03-24 14:53:08 +01:00
Kerollmops
ee6be4f6b9
Import milli from meilisearch-lib in meilisearch-http 2022-03-24 14:45:37 +01:00
Kerollmops
ac48860bbb
Upgrade the workspace dependencies 2022-03-17 11:03:31 +01:00
Kerollmops
46e6d23dd2
Remove the zstd dependency comming from actix-web/http 2022-03-17 11:00:25 +01:00
Kerollmops
86c1e83ea1
Remove three unused dependencies 2022-03-17 11:00:24 +01:00
Sai Kumar
e271395971
chore(all): bump milli
* updates to Use the milli's heed dependency #2210

* Update index.rs

* Update store.rs

* Update mod.rs

* cargo fmt
2022-03-16 16:34:44 +01:00
ad hoc
b57c59baa4
sequential extractor 2022-03-04 20:43:12 +01:00
Rob Ede
15150db957
clippy 2022-02-28 19:03:38 +01:00
Rob Ede
3b2e467ca6
update actix-web dependency to 4.0 2022-02-28 19:03:37 +01:00
bors[bot]
c3e3c900f2
Merge #2173
2173: chore(all): replace chrono with time r=irevoire a=irevoire

Chrono has been unmaintained for a few month now and there is a CVE on it.

Also I updated all the error messages related to the API key as you can see here: https://github.com/meilisearch/specifications/pull/114

fix #2172

Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-02-17 14:12:23 +00:00
Irevoire
05c8d81e65
chore: get rid of chrono in favor of time
Chrono has been unmaintened for a few month now and there is a CVE on it.

make clippy happy

bump milli
2022-02-16 18:14:29 +01:00
Clémentine Urquizar
e2a9414c7a
Update version (v0.26.0) 2022-02-14 16:11:07 +01:00
Clémentine Urquizar
1a87b2f37d
Bump milli to v0.22.1 2022-02-08 11:21:44 +01:00
mpostma
c9a236b0af
feat(lib): auto-batching 2022-02-01 18:06:20 +01:00
bors[bot]
622c15e825
Merge #2096
2096: feat(auth): Tenant token r=Kerollmops a=ManyTheFish

Make meilisearch support JWT authentication signed with meilisearch API keys
using HS256, HS384 or HS512 algorithms.

Related spec: [specifications#89](https://github.com/meilisearch/specifications/pull/89) [rendered](https://github.com/meilisearch/specifications/blob/scoped-api-keys/text/0089-tenant-tokens.md)
Fix #1991 


Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-01-27 10:38:41 +00:00
ManyTheFish
7ca647f0d0 feat(auth): Implement Tenant token
Make meilisearch support JWT authentication signed with meilisearch API keys
using HS256, HS384 or HS512 algorithms.

Related spec: https://github.com/meilisearch/specifications/pull/89
Fix #1991
2022-01-27 08:25:39 +01:00
Clémentine Urquizar - curqui
aa50fcb1f0
Merge branch 'main' into stable 2022-01-26 20:17:41 +01:00
Tamo
7828da15c3
feat(analytics): send the whole set of cli options instead of only the snapshot 2022-01-26 13:52:41 +01:00
Rob Ede
9f5fee404b
chore(all): update actix-web dependency to 4.0.0-beta.21 2022-01-21 20:44:17 +00:00
Tamo
436f61a7f4
chore: bump meilisearch 2022-01-18 12:27:15 +01:00
Tamo
3fab5869fa
chore: bump milli 2022-01-18 11:50:17 +01:00
mpostma
f6d53e03f1 chore(http): migrate from structopt to clap3 2022-01-12 14:07:19 +01:00
many
ee7970f603 feat(auth): Extend API keys
- Add API keys in snapshots
- Add API keys in dumps
- Rename action indexes.add to indexes.create
- fix QA #1979

fix #1979
fix #1995
fix #2001
fix #2003
related to #1890
2021-12-14 17:33:39 +01:00
Clémentine Urquizar
ae73386723
Update version for the next release (v0.25.0) 2021-12-07 14:00:43 +01:00
many
ae2b0e7aa7
Use milli reexported tokenizer instead of importing meilisearch-tokenizer dependency 2021-12-06 17:18:28 +01:00
many
ffefd0caf2
feat(auth): API keys
implements:
https://github.com/meilisearch/specifications/blob/develop/text/0085-api-keys.md

- Add tests on API keys management route (meilisearch-http/tests/auth/api_keys.rs)
- Add tests checking authorizations on each meilisearch routes (meilisearch-http/tests/auth/authorization.rs)
- Implement API keys management routes (meilisearch-http/src/routes/api_key.rs)
- Create module to manage API keys and authorizations (meilisearch-auth)
- Reimplement GuardedData to extend authorizations (meilisearch-http/src/extractors/authentication/mod.rs)
- Change X-MEILI-API-KEY by Authorization Bearer (meilisearch-http/src/extractors/authentication/mod.rs)
- Change meilisearch routes to fit to the new authorization feature (meilisearch-http/src/routes/)

- close #1867
2021-12-06 09:52:41 +01:00
Marin Postma
a30e02c18c feat(all): Task store
implements:
https://github.com/meilisearch/specifications/blob/develop/text/0060-refashion-updates-apis.md

linked PR:

- #1889
- #1891
- #1892
- #1902
- #1906
- #1911
- #1914
- #1915
- #1916
- #1918
- #1924
- #1925
- #1926
- #1930
- #1936
- #1937
- #1942
- #1944
- #1945
- #1946
- #1947
- #1950
- #1951
- #1957
- #1959
- #1960
- #1961
- #1962
- #1964

- https://github.com/meilisearch/milli/pull/414
- https://github.com/meilisearch/milli/pull/409
- https://github.com/meilisearch/milli/pull/406
- https://github.com/meilisearch/milli/pull/418

- close #1687
- close #1786
- close #1940
- close #1948
- close #1949
- close #1932
- close #1956
2021-12-02 20:14:42 +01:00
many
cc6306c0e1
Update milli version 2021-11-04 14:57:45 +01:00
many
b664a46e91
Update milli version 2021-11-03 16:11:20 +01:00
Tamo
c4737749ab
bump segment to be able to display a user 2021-10-29 17:25:53 +02:00
Tamo
de35a9a605
use an official release of segment 2021-10-29 17:25:53 +02:00
Tamo
9be90011c6
save the user-id in the config dir of the OS 2021-10-29 17:25:51 +02:00
Tamo
664d09e86a
makes the analytics works with the option and the feature 2021-10-29 17:25:47 +02:00
Tamo
e226b1a87f
rewrite the main analytics module and the information sent in the tick 2021-10-29 17:25:42 +02:00
bors[bot]
cf67964133
Merge #1848
1848: Error format and Definition r=MarinPostma a=ManyTheFish



Co-authored-by: many <maxime@meilisearch.com>
2021-10-28 14:15:35 +00:00
many
3a29cbf0ae
Use milli v0.20.0 2021-10-28 15:59:06 +02:00
many
cbaca2b579
Fix PR comments 2021-10-28 15:42:42 +02:00
Clémentine Urquizar
a76d9b15c9
Update version for the next release (v0.24.0) 2021-10-28 12:24:49 +02:00
many
ff0908d3fa
Ignore errors tests that show unrelated bugs 2021-10-28 11:41:59 +02:00
many
7464720426
Fix some errors 2021-10-28 10:47:59 +02:00
marin postma
4ac005b094
optimize document transform
fix error types

bump milli
2021-10-26 13:51:15 +02:00
Clémentine Urquizar
0f342ac46e
Update MeiliSearch version 2021-10-12 16:43:31 +02:00
Clémentine Urquizar
29ac324e90
Update milli version to v0.17.3 2021-10-12 16:12:16 +02:00