Commit Graph

711 Commits

Author SHA1 Message Date
Irevoire
250410495c
start integrating the index-scheduler in meilisearch-lib 2022-10-27 11:33:42 +02:00
Irevoire
8f0fd35358
add insta::json for later 2022-10-27 11:33:42 +02:00
Irevoire
8770e07397
I can index documents without meilisearch 2022-10-27 11:33:42 +02:00
Tamo
7d0c8a3379
test the register tasks 2022-10-27 11:33:40 +02:00
Tamo
fc098022c7
start integrating the index-scheduler in the meilisearch codebase 2022-10-27 11:33:40 +02:00
Tamo
366a344474
get rids of the horrendous spinlock in favor of synchronoise 2022-10-27 11:33:39 +02:00
Tamo
c97d51a624
add a bunch of tests 2022-10-27 11:33:39 +02:00
Tamo
94e29a9f5f
extract the index abstraction out of the index-scheduler in its own module 2022-10-27 11:33:38 +02:00
Tamo
76597fc382
import the update_file_store in the index-scheduler 2022-10-27 11:33:37 +02:00
Tamo
2afb381f95
get rids of nelson 2022-10-27 11:33:37 +02:00
Tamo
a9844bd4f6
move the update file store to another crate with as little dependencies as possible 2022-10-27 11:33:37 +02:00
Tamo
46b8ebcab4
fix the file store 2022-10-27 11:33:35 +02:00
Tamo
a7aa92df5f
fix most of the index module 2022-10-27 11:33:34 +02:00
Irevoire
d8b8e04ad1
wip porting the index back in the scheduler 2022-10-27 11:33:34 +02:00
Tamo
1a47949063
START THE REWRITE OF THE INDEX SCHEDULER: index & register has been implemented 2022-10-27 11:33:33 +02:00
ManyTheFish
4afed4de4f stabilize milli 2022-10-24 14:16:41 +02:00
ManyTheFish
c02ae4dfc0 Update roaring 2022-10-19 14:25:43 +02:00
ManyTheFish
062d17fbc0 Use a milli version that compute exhaustivelly the number of hits 2022-10-19 14:05:42 +02:00
bors[bot]
b0749407f3
Merge #2804
2804: Add environement variable `MEILI_CONFIG_FILE_PATH` to define the config file path r=Kerollmops a=choznerol

# Pull Request

## What does this PR do?
Fixes #2800

~This is a draft PR base on the code in #2745. I will `rebase` and mark it ready for review only after #2745 merge.~ Done rebase


## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?


## Demo

With `config.toml`, `config_copy1.toml` and `config_copy2.toml` present:
> <img width="692" alt="image" src="https://user-images.githubusercontent.com/12410942/192566891-6b3c9d26-736f-4e23-a09b-687fca1cb50d.png">

`MEILI_CONFIG_FILE_PATH` works:
> <img width="773" alt="image" src="https://user-images.githubusercontent.com/12410942/192567023-f751536e-992a-4e90-a176-cb19122248be.png">

`--config-file-path` still works:
> <img width="768" alt="image" src="https://user-images.githubusercontent.com/12410942/192567318-88c80b24-7873-4cec-8d08-16fe4d228055.png">

When both present, `--config-file-path` taks precedence:
> <img width="1214" alt="image" src="https://user-images.githubusercontent.com/12410942/192567477-8a7cffe1-96f0-42a9-a348-6dbec20dc1e7.png">



Co-authored-by: Lawrence Chou <choznerol@protonmail.com>
2022-10-17 10:40:05 +00:00
bors[bot]
5d895dd7da
Merge #2851
2851: Upgrade clap to 4.0 r=loiclec a=choznerol

# Pull Request

## Related issue
Fixes #2846 

This PR is draft based on #2847 to avoid conflict. I will rebase and mark as 'Ready for review' after #2847 is merged.

## What does this PR do?
1. Upgrade clap to the latest version or 4.0 (4.0.9 as of today) by following the [migrating instruction](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#migrating) from [4.0 changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#migrating)
2. Fix an `ArgGroup` typo that can only be caught after upgrading to 4.0 in 20a715e29ed17c5a76229c98fb31504ada873597

## Notable changes

### The `--help` message

The format, ordering and indentation of `--help` message was changed in 4.0. I recorded the output of `cargo run -- --help` before and after upgrade to 4.0 for reference.

<details>
<summary>diff</summary>

Output of `diff --ignore-all-space --text --unified --new-file help-message-before.txt help-message-after.txt`:

```diff
--- help-message-before.txt	2022-10-14 16:45:36.000000000 +0800
+++ help-message-after.txt	2022-10-14 16:36:53.000000000 +0800
`@@` -1,12 +1,8 `@@`
-meilisearch-http 0.29.1
+Usage: meilisearch [OPTIONS]
 
-USAGE:
-    meilisearch [OPTIONS]
-
-OPTIONS:
+Options:
         --config-file-path <CONFIG_FILE_PATH>
-            Set the path to a configuration file that should be used to setup the engine. Format
-            must be TOML
+          Set the path to a configuration file that should be used to setup the engine. Format must be TOML
 
         --db-path <DB_PATH>
             Designates the location where database files will be created and retrieved
`@@` -26,15 +22,14 `@@`
             [default: dumps/]
 
         --env <ENV>
-            Configures the instance's environment. Value must be either `production` or
-            `development`
+          Configures the instance's environment. Value must be either `production` or `development`
             
             [env: MEILI_ENV=]
             [default: development]
             [possible values: development, production]
 
     -h, --help
-            Print help information
+          Print help information (use `-h` for a summary)
 
         --http-addr <HTTP_ADDR>
             Sets the HTTP address and port Meilisearch will use
`@@` -43,63 +38,53 `@@`
             [default: 127.0.0.1:7700]
 
         --http-payload-size-limit <HTTP_PAYLOAD_SIZE_LIMIT>
-            Sets the maximum size of accepted payloads. Value must be given in bytes or explicitly
-            stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
+          Sets the maximum size of accepted payloads. Value must be given in bytes or explicitly stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
             
             [env: MEILI_HTTP_PAYLOAD_SIZE_LIMIT=]
             [default: 100000000]
 
         --ignore-dump-if-db-exists
-            Prevents a Meilisearch instance with an existing database from throwing an error when
-            using `--import-dump`. Instead, the dump will be ignored and Meilisearch will launch
-            using the existing database.
+          Prevents a Meilisearch instance with an existing database from throwing an error when using `--import-dump`. Instead, the dump will be ignored and Meilisearch will launch using the existing database.
             
             This option will trigger an error if `--import-dump` is not defined.
             
             [env: MEILI_IGNORE_DUMP_IF_DB_EXISTS=]
 
         --ignore-missing-dump
-            Prevents Meilisearch from throwing an error when `--import-dump` does not point to a
-            valid dump file. Instead, Meilisearch will start normally without importing any dump.
+          Prevents Meilisearch from throwing an error when `--import-dump` does not point to a valid dump file. Instead, Meilisearch will start normally without importing any dump.
             
             This option will trigger an error if `--import-dump` is not defined.
             
             [env: MEILI_IGNORE_MISSING_DUMP=]
 
         --ignore-missing-snapshot
-            Prevents a Meilisearch instance from throwing an error when `--import-snapshot` does not
-            point to a valid snapshot file.
+          Prevents a Meilisearch instance from throwing an error when `--import-snapshot` does not point to a valid snapshot file.
             
             This command will throw an error if `--import-snapshot` is not defined.
             
             [env: MEILI_IGNORE_MISSING_SNAPSHOT=]
 
         --ignore-snapshot-if-db-exists
-            Prevents a Meilisearch instance with an existing database from throwing an error when
-            using `--import-snapshot`. Instead, the snapshot will be ignored and Meilisearch will
-            launch using the existing database.
+          Prevents a Meilisearch instance with an existing database from throwing an error when using `--import-snapshot`. Instead, the snapshot will be ignored and Meilisearch will launch using the existing database.
             
             This command will throw an error if `--import-snapshot` is not defined.
             
             [env: MEILI_IGNORE_SNAPSHOT_IF_DB_EXISTS=]
 
         --import-dump <IMPORT_DUMP>
-            Imports the dump file located at the specified path. Path must point to a `.dump` file.
-            If a database already exists, Meilisearch will throw an error and abort launch
+          Imports the dump file located at the specified path. Path must point to a `.dump` file. If a database already exists, Meilisearch will throw an error and abort launch
             
             [env: MEILI_IMPORT_DUMP=]
 
         --import-snapshot <IMPORT_SNAPSHOT>
-            Launches Meilisearch after importing a previously-generated snapshot at the given
-            filepath
+          Launches Meilisearch after importing a previously-generated snapshot at the given filepath
             
             [env: MEILI_IMPORT_SNAPSHOT=]
 
         --log-level <LOG_LEVEL>
             Defines how much detail should be present in Meilisearch's logs.
             
-            Meilisearch currently supports five log levels, listed in order of increasing verbosity:
-            ERROR, WARN, INFO, DEBUG, TRACE.
+          Meilisearch currently supports five log levels, listed in order of increasing verbosity: ERROR, WARN, INFO, DEBUG, TRACE.
             
             [env: MEILI_LOG_LEVEL=]
             [default: INFO]
`@@` -110,31 +95,25 `@@`
             [env: MEILI_MASTER_KEY=]
 
         --max-index-size <MAX_INDEX_SIZE>
-            Sets the maximum size of the index. Value must be given in bytes or explicitly stating a
-            base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
+          Sets the maximum size of the index. Value must be given in bytes or explicitly stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
             
             [env: MEILI_MAX_INDEX_SIZE=]
             [default: 107374182400]
 
         --max-indexing-memory <MAX_INDEXING_MEMORY>
-            Sets the maximum amount of RAM Meilisearch can use when indexing. By default,
-            Meilisearch uses no more than two thirds of available memory
+          Sets the maximum amount of RAM Meilisearch can use when indexing. By default, Meilisearch uses no more than two thirds of available memory
             
             [env: MEILI_MAX_INDEXING_MEMORY=]
             [default: "21.33 TiB"]
 
         --max-indexing-threads <MAX_INDEXING_THREADS>
-            Sets the maximum number of threads Meilisearch can use during indexation. By default,
-            the indexer avoids using more than half of a machine's total processing units. This
-            ensures Meilisearch is always ready to perform searches, even while you are updating an
-            index
+          Sets the maximum number of threads Meilisearch can use during indexation. By default, the indexer avoids using more than half of a machine's total processing units. This ensures Meilisearch is always ready to perform searches, even while you are updating an index
             
             [env: MEILI_MAX_INDEXING_THREADS=]
             [default: 5]
 
         --max-task-db-size <MAX_TASK_DB_SIZE>
-            Sets the maximum size of the task database. Value must be given in bytes or explicitly
-            stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
+          Sets the maximum size of the task database. Value must be given in bytes or explicitly stating a base unit (for instance: 107374182400, '107.7Gb', or '107374 Mb')
             
             [env: MEILI_MAX_TASK_DB_SIZE=]
             [default: 107374182400]
```

- ~[help-message-before.txt](https://github.com/meilisearch/meilisearch/files/9715683/help-message-before.txt)~ [help-message-before.txt](https://github.com/meilisearch/meilisearch/files/9784156/help-message-before-2.txt)
- ~[help-message-after.txt](https://github.com/meilisearch/meilisearch/files/9715682/help-message-after.txt)~ [help-message-after.txt](https://github.com/meilisearch/meilisearch/files/9784091/help-message-after.txt)

</details>


## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Lawrence Chou <choznerol@protonmail.com>
2022-10-17 10:08:27 +00:00
Lawrence Chou
3c3ae3ff98
Impeove invalid config_file_path handling
1. Besides opt.config_file_path, also consider MEILI_CONFIG_FILE_PATH in the Err path because they are both user input.
2. Print out the incorrect file path in error message.
3. Add tests
https://github.com/meilisearch/meilisearch/pull/2804#discussion_r991999888
2022-10-12 12:04:48 +08:00
Andrey "MOU" Larionov
11b986a81d
Added support for specifying compression in tests
Refactored tests code to allow to specify compression (content-encoding) algorithm.

Added tests to verify what actix actually handle different content encodings properly.
2022-10-09 22:09:29 +02:00
Lawrence Chou
9e5ef8eb69
Upgrade clap to v4
Close #2846

4.0.0 changelog: 'https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#400---2022-09-28'

I followed the [Migrating steps](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#migrating) and the only issue I encountered are:
1. The typo problem in previous commit "Fix clap ArgGroup typo"
2. I can't say I am 100% sure every [Subtle changes](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md#breaking-changes) is fine for our use case, but at least after a quick read I didn't notice anything actionable.
2022-10-07 00:32:25 +08:00
bors[bot]
1b72eba1f3
Merge #2867
2867: Bring back `stable` into `main` r=Kerollmops a=curquiza

Following hotfix for v0.29.1

Co-authored-by: Loïc Lecrenier <loic@meilisearch.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
2022-10-06 14:14:23 +00:00
Loïc Lecrenier
05af8f0e46 Update version for next release (v0.29.1) 2022-10-06 10:27:11 +02:00
Loïc Lecrenier
a2c91a87fe Upgrade dependencies
Except:
- clap stays on 3.0 because it is more complicated to upgrade
- enum_iterator goes up to 1.1.2 instead of 1.2 because of the vergen
dependency
2022-10-05 15:53:02 +02:00
Clémentine Urquizar - curqui
a7d2c9572e
Merge branch 'main' into stable 2022-10-04 14:20:10 +02:00
bors[bot]
fcedce8578
Merge #2745 #2789 #2814 #2826
2745: Config file support r=curquiza a=mlemesle

# Pull Request

## What does this PR do?
Fixes #2558

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


2789: Fix typos r=Kerollmops a=kianmeng

# Pull Request

## What does this PR do?

Found via `codespell -L crate,nam,hart`.

## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


2814: Skip dashboard test if mini-dashboard feature is disabled r=Kerollmops a=jirutka

Fixes #2813

Fixes the following error:

    cargo test --no-default-features
    ...
    error: couldn't read target/debug/build/meilisearch-http-ec029d8c902cf2cb/out/generated.rs: No such file or directory (os error 2)
     --> meilisearch-http/tests/dashboard/mod.rs:8:9
      |
    8 |         include!(concat!(env!("OUT_DIR"), "/generated.rs"));
      |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |
      = note: this error originates in the macro `include` (in Nightly builds, run with -Z macro-backtrace for more info)

    error: could not compile `meilisearch-http` due to previous error

2826: Rename receivedDocumentIds into matchedDocuments r=Kerollmops a=Ugzuzg

# Pull Request

## What does this PR do?

Fixes #2799 
Changes DocumentDeletion task details response.

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Tested with curl:
```
curl \
  -X POST 'http://localhost:7700/indexes/movies/documents/delete-batch' \
  -H 'Content-Type: application/json' \
  --data-binary '[
    23488,
    153738,
    437035,
    363869
  ]'

{"taskUid":1,"indexUid":"movies","status":"enqueued","type":"documentDeletion","enqueuedAt":"2022-10-01T20:06:37.105416054Z"}%

curl \
  -X GET 'http://localhost:7700/tasks/1'

{"uid":1,"indexUid":"movies","status":"succeeded","type":"documentDeletion","details":{"matchedDocuments":4,"deletedDocuments":2},"duration":"PT0.005708322S","enqueuedAt":"2022-10-01T20:06:37.105416054Z","startedAt":"2022-10-01T20:06:37.115562733Z","finishedAt":"2022-10-01T20:06:37.121271055Z"}
```

Co-authored-by: mlemesle <lemesle.martin@hotmail.fr>
Co-authored-by: Kian-Meng Ang <kianmeng@cpan.org>
Co-authored-by: Jakub Jirutka <jakub@jirutka.cz>
Co-authored-by: Jarasłaŭ Viktorčyk <ugzuzg@gmail.com>
2022-10-03 09:48:34 +00:00
Jakub Jirutka
5b57114771 Bump milli from 0.33.0 to 0.33.4 2022-09-14 20:52:11 +02:00
Clémentine Urquizar
268d59ccb1
Update milli version to v0.33.4 2022-09-13 18:01:09 +02:00
Clémentine Urquizar
aabd67a9fa
Update patch version to remove CVE 2022-09-12 14:36:45 +02:00
Kerollmops
441492f1c8
Bump milli to v0.33.3 2022-09-07 18:23:49 +02:00
mlemesle
403226a029 Add support for config file 2022-09-07 16:09:00 +02:00
ManyTheFish
50434d35d0 Update milli v0.33.2 2022-09-01 13:15:05 +02:00
evpeople
833ade80a6 cargo update 2022-08-31 13:58:53 +08:00
ManyTheFish
aff4b64265 Update dependencies 2022-08-23 16:39:39 +02:00
bors[bot]
0a2ef0037f
Merge #2689 #2690
2689: Use mimalloc as the global allocator r=Kerollmops a=loiclec

milli has switched its global allocator to mimalloc already, and we have seen some performance gains as a result. Furthermore, we can use mimalloc as the global allocator on all platforms whereas jemalloc was only activated on Linux. 

This PR brings mimalloc to Meilisearch as well. 

2690: Add LTO and codegen-units=1 to release compile options r=Kerollmops a=loiclec

This PR brings Meilisearch's release compile options in line with milli (see https://github.com/meilisearch/milli/pull/606 ). 

Adding LTO and codegen=units=1 will make compile times longer, but they also speed up the final binary significantly.

Co-authored-by: Loïc Lecrenier <loic@meilisearch.com>
2022-08-23 12:05:02 +00:00
Clémentine Urquizar
6fe3f285ce
Update version for next release (v0.29.0) 2022-08-23 13:39:56 +02:00
Loïc Lecrenier
e659c08ac4 Use mimalloc as the global allocator 2022-08-23 12:58:10 +02:00
mohandasspat
4bee0565e8 prometheus and grafana dashboards implemented 2022-08-22 13:21:22 +05:30
Clément Renault
9b2036ac05
Accept either an array of documents or a single document 2022-08-18 11:55:14 +02:00
bors[bot]
b5f91b91c3
Merge #2523
2523: Improve the tasks error reporting when processed in batches r=irevoire a=Kerollmops

This fixes #2478 by changing the behavior of the task handler when there is an error in a batch of document addition or update.

What changes is that when there is a user error in a task in a batch we now report this task as failed with the right error message but we continue to process the other tasks. A user error can be when a geo field is invalid, a document id is invalid, or missing.

fixes #2582, #2478

Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-08-16 14:15:30 +00:00
ManyTheFish
cf955a77db Fix(cli): Clamp databases max size to a multiple of system page size
fix #2659
2022-08-11 10:44:47 +02:00
Kerollmops
fe32097964 Update milli v0.32 2022-07-28 14:45:10 +02:00
Clémentine Urquizar
9cf6acb671
Fix highlight issue by updating milli to v0.31.2 2022-07-21 14:11:24 +04:00
Clémentine Urquizar
d01a3ab889
Update version for next release (v0.28.1) 2022-07-20 15:46:53 +04:00
ManyTheFish
a146fd45b9 Format API keys in hexa instead of base64 2022-07-05 16:14:18 +02:00
Irevoire
05ee2eff01
add more tests on the formatted route 2022-06-28 13:17:55 +02:00
bors[bot]
9e261b996f
Merge #2543
2543: fix all the array on the search get route and improve the tests r=curquiza a=irevoire

fix #2527

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-06-23 14:51:36 +00:00
Kerollmops
7feb15df28
Bump milli to 0.31.1 2022-06-23 10:47:48 +02:00
Tamo
c47369b502
fix all the array on the search get route and improve the tests 2022-06-22 14:33:44 +02:00
Clémentine Urquizar
32c8846514
Rollback 0.0.0 versionning 2022-06-22 12:20:12 +02:00
Clémentine Urquizar
7490383d4f
Update the not-released version in Cargo.toml files 2022-06-21 19:17:33 +02:00
Clémentine Urquizar
c6ed756dbc
Update script after review 2022-06-21 10:46:32 +02:00
Tamo
2063fbd985
chore: bump milli 2022-06-09 18:34:03 +02:00
pierre-l
36cb09eb25 Add a new meilisearch_types crate
Move `meilisearch_error` to `meilisearch_types::error`
Move `meilisearch_lib::index_resolver::IndexUid` to `meilisearch_types::index_uid`
Add a new `InvalidIndexUid` error in `meilisearch_types::index_uid`
2022-06-09 16:14:13 +02:00
bors[bot]
b9b32d65a8
Merge #2494
2494: Introduce the new faceting and pagination settings r=ManyTheFish a=Kerollmops

This PR introduces two new settings following the newly created spec https://github.com/meilisearch/specifications/pull/157:
 - The `faceting.max_values_per_facet` one describes the maximum number of values (each with a count) associated with a value in a facet distribution query.
 - The `pagination.limited_to` one describes the maximum number of documents that a search query can ever return.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-09 12:09:21 +00:00
Kerollmops
c924614527
Bump milli to 0.29.2 2022-06-09 10:54:28 +02:00
Clémentine Urquizar
fc4990b968
Update version for next release (v0.28.0) 2022-06-08 17:59:18 +02:00
bors[bot]
6171f17f1d
Merge #2468
2468: Update milli 0.29 r=Kerollmops a=ManyTheFish

- [x] Update milli to 0.29
- [x] Integrate charabia
- [x] Set disabled_words to default when Index::exact_words returns None
- [x] Fix ranking rules integration test

fixes #2375
fixes #2144
fixes #2417
fixes #2407

Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-06-08 14:29:20 +00:00
ManyTheFish
8d09772334 Update milli 2022-06-08 14:38:05 +02:00
ManyTheFish
987a7f8926 Wrap sha256 in HMAC instead of directly use sha256 2022-06-08 14:25:12 +02:00
Kerollmops
277a0a7967
Bump serde-cs to simplify our usage of the star_or function 2022-06-06 10:17:33 +02:00
bors[bot]
cf2d8de48a
Merge #2452
2452: Change http verbs r=ManyTheFish a=Kerollmops

This PR fixes #2419 by updating the HTTP verbs used to update the settings and every single setting parameter.

- [x] `PATCH /indexes/{indexUid}` instead of `PUT`
- [x] `PATCH /indexes/{indexUid}/settings`  instead of `POST`
- [x] `PATCH /indexes/{indexUid}/settings/typo-tolerance`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/displayed-attributes`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/distinct-attribute`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/filterable-attributes`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/ranking-rules`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/searchable-attributes`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/sortable-attributes`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/stop-words`  instead of `POST`
- [x] `PUT /indexes/{indexUid}/settings/synonyms`  instead of `POST`


Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-02 11:46:17 +00:00
bors[bot]
c9cd1738a5
Merge #2445
2445: Seek-based tasks list r=Kerollmops a=Kerollmops

This PR implements the seek-based pagination for the tasks list following [the spec](https://github.com/meilisearch/specifications/pull/115).

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-02 10:25:54 +00:00
Kerollmops
bcb51905d7
Fix the authorization tests 2022-06-02 12:16:46 +02:00
bors[bot]
8b8c3e32f0
Merge #2450
2450: Bump the dependencies r=ManyTheFish a=Kerollmops

In order to use [the latest version of grenad](https://docs.rs/grenad) I bump the dependencies here. We also use the latest versions of all our other dependencies now.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-02 08:53:12 +00:00
Kerollmops
e769043576
Bump the dependencies 2022-06-01 17:59:05 +02:00
ManyTheFish
7652295d2c Encode key in base64 instead of hexa 2022-06-01 16:17:47 +02:00
ManyTheFish
96a5791e39 Add uid and name fields in keys 2022-06-01 16:07:44 +02:00
Kerollmops
c11d21879a
Introduce tasks limit and after to the tasks route 2022-06-01 13:26:36 +02:00
Kerollmops
3684c822f1
Add indexUid filtering on the /tasks route 2022-05-31 11:33:20 +02:00
Tamo
5f0e9b63d2
chore(dump): add tests 2022-05-24 14:21:56 +02:00
ManyTheFish
0250ea9157 Intergrate smart crop in Meilisearch 2022-05-18 18:35:51 +02:00
bors[bot]
6d221058f1
Merge #2404
2404: Bring `release-v0.27.1` into `main` r=curquiza a=curquiza

Following the v0.27.1 hotfixes

Co-authored-by: ad hoc <postma.marin@protonmail.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-05-17 16:11:17 +00:00
Clémentine Urquizar
49e857776c
Update version for next release (v0.27.1) 2022-05-17 11:59:35 +02:00
Tamo
85d19bfb3e
chore: bump milli 2022-05-16 18:43:35 +02:00
bors[bot]
b9b9cba154
Merge #2383
2383: v0.27.0: bring `stable` into `main` r=Kerollmops a=curquiza

Bring `stable` into `main`

Co-authored-by: ad hoc <postma.marin@protonmail.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Paul Sanders <psanders1@gmail.com>
Co-authored-by: Irevoire <tamo@meilisearch.com>
Co-authored-by: Morgane Dubus <30866152+mdubus@users.noreply.github.com>
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
2022-05-16 08:35:25 +00:00
Clémentine Urquizar
c46f3587de
Bump milli to v0.26.4 2022-05-04 11:25:36 +02:00
Clémentine Urquizar
38d681c230
Change Nelson path 2022-04-21 18:42:34 +02:00
Clémentine Urquizar - curqui
6ff8bf823d
Revert "[TEST PURPOSE] Bump meilisearch to version 9000.0.0" 2022-04-21 16:36:56 +02:00
releasemops
f1cd6b6ee8 bump meilisearch to v9000.0.0 2022-04-21 14:26:40 +00:00
Clémentine Urquizar - curqui
acf3357cf3
Revert "[TEST PURPOSE] Bump meilisearch to version 8000.1.0" 2022-04-21 16:24:27 +02:00
releasemops
0714551101 bump meilisearch to v8000.1.0 2022-04-21 13:23:46 +00:00
Tamo
1ef87cc6d0
chore: move permissive-json-pointer in the meilisearch repository
Update permissive-json-pointer/src/lib.rs

Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-04-20 19:24:41 +02:00
Clémentine Urquizar
a7fd199ded
Fix typo reseting by upgrading milli to v0.26.2 2022-04-20 12:24:46 +02:00
Irevoire
64b0a50a58
chore: bump milli 2022-04-14 12:12:54 +02:00
Clémentine Urquizar
b3661bf8ec
Change version for the next release (v0.27.0) 2022-04-11 16:25:15 +02:00
Tamo
69d312209e
feat(search): Implements the nested fields
See https://github.com/meilisearch/specifications/pull/121
2022-04-07 19:47:20 +02:00
ad hoc
981fba5b44
feat(all): introduce disable typos 2022-04-06 15:47:48 +02:00
ad hoc
a523828f61
chore(lib): bump milli to 0.25.0 2022-04-06 15:03:10 +02:00
Kerollmops
94f04e79eb
Bump the milli dependency to 0.24.1 2022-03-29 09:17:25 -07:00
Kerollmops
09212abdf7
Update the cargo lock 2022-03-24 14:53:08 +01:00
Kerollmops
ee6be4f6b9
Import milli from meilisearch-lib in meilisearch-http 2022-03-24 14:45:37 +01:00
Kerollmops
ac48860bbb
Upgrade the workspace dependencies 2022-03-17 11:03:31 +01:00
Kerollmops
46e6d23dd2
Remove the zstd dependency comming from actix-web/http 2022-03-17 11:00:25 +01:00
Kerollmops
86c1e83ea1
Remove three unused dependencies 2022-03-17 11:00:24 +01:00
Sai Kumar
e271395971
chore(all): bump milli
* updates to Use the milli's heed dependency #2210

* Update index.rs

* Update store.rs

* Update mod.rs

* cargo fmt
2022-03-16 16:34:44 +01:00
ad hoc
b57c59baa4
sequential extractor 2022-03-04 20:43:12 +01:00
Rob Ede
15150db957
clippy 2022-02-28 19:03:38 +01:00
Rob Ede
3b2e467ca6
update actix-web dependency to 4.0 2022-02-28 19:03:37 +01:00
bors[bot]
c3e3c900f2
Merge #2173
2173: chore(all): replace chrono with time r=irevoire a=irevoire

Chrono has been unmaintained for a few month now and there is a CVE on it.

Also I updated all the error messages related to the API key as you can see here: https://github.com/meilisearch/specifications/pull/114

fix #2172

Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-02-17 14:12:23 +00:00
Irevoire
05c8d81e65
chore: get rid of chrono in favor of time
Chrono has been unmaintened for a few month now and there is a CVE on it.

make clippy happy

bump milli
2022-02-16 18:14:29 +01:00
Clémentine Urquizar
e2a9414c7a
Update version (v0.26.0) 2022-02-14 16:11:07 +01:00
Clémentine Urquizar
1a87b2f37d
Bump milli to v0.22.1 2022-02-08 11:21:44 +01:00
mpostma
c9a236b0af
feat(lib): auto-batching 2022-02-01 18:06:20 +01:00
bors[bot]
622c15e825
Merge #2096
2096: feat(auth): Tenant token r=Kerollmops a=ManyTheFish

Make meilisearch support JWT authentication signed with meilisearch API keys
using HS256, HS384 or HS512 algorithms.

Related spec: [specifications#89](https://github.com/meilisearch/specifications/pull/89) [rendered](https://github.com/meilisearch/specifications/blob/scoped-api-keys/text/0089-tenant-tokens.md)
Fix #1991 


Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-01-27 10:38:41 +00:00
ManyTheFish
7ca647f0d0 feat(auth): Implement Tenant token
Make meilisearch support JWT authentication signed with meilisearch API keys
using HS256, HS384 or HS512 algorithms.

Related spec: https://github.com/meilisearch/specifications/pull/89
Fix #1991
2022-01-27 08:25:39 +01:00
Clémentine Urquizar - curqui
aa50fcb1f0
Merge branch 'main' into stable 2022-01-26 20:17:41 +01:00
Tamo
7828da15c3
feat(analytics): send the whole set of cli options instead of only the snapshot 2022-01-26 13:52:41 +01:00
Rob Ede
9f5fee404b
chore(all): update actix-web dependency to 4.0.0-beta.21 2022-01-21 20:44:17 +00:00
Tamo
436f61a7f4
chore: bump meilisearch 2022-01-18 12:27:15 +01:00
Tamo
3fab5869fa
chore: bump milli 2022-01-18 11:50:17 +01:00
mpostma
f6d53e03f1 chore(http): migrate from structopt to clap3 2022-01-12 14:07:19 +01:00
many
ee7970f603 feat(auth): Extend API keys
- Add API keys in snapshots
- Add API keys in dumps
- Rename action indexes.add to indexes.create
- fix QA #1979

fix #1979
fix #1995
fix #2001
fix #2003
related to #1890
2021-12-14 17:33:39 +01:00
Clémentine Urquizar
ae73386723
Update version for the next release (v0.25.0) 2021-12-07 14:00:43 +01:00
many
ae2b0e7aa7
Use milli reexported tokenizer instead of importing meilisearch-tokenizer dependency 2021-12-06 17:18:28 +01:00
many
ffefd0caf2
feat(auth): API keys
implements:
https://github.com/meilisearch/specifications/blob/develop/text/0085-api-keys.md

- Add tests on API keys management route (meilisearch-http/tests/auth/api_keys.rs)
- Add tests checking authorizations on each meilisearch routes (meilisearch-http/tests/auth/authorization.rs)
- Implement API keys management routes (meilisearch-http/src/routes/api_key.rs)
- Create module to manage API keys and authorizations (meilisearch-auth)
- Reimplement GuardedData to extend authorizations (meilisearch-http/src/extractors/authentication/mod.rs)
- Change X-MEILI-API-KEY by Authorization Bearer (meilisearch-http/src/extractors/authentication/mod.rs)
- Change meilisearch routes to fit to the new authorization feature (meilisearch-http/src/routes/)

- close #1867
2021-12-06 09:52:41 +01:00
Marin Postma
a30e02c18c feat(all): Task store
implements:
https://github.com/meilisearch/specifications/blob/develop/text/0060-refashion-updates-apis.md

linked PR:

- #1889
- #1891
- #1892
- #1902
- #1906
- #1911
- #1914
- #1915
- #1916
- #1918
- #1924
- #1925
- #1926
- #1930
- #1936
- #1937
- #1942
- #1944
- #1945
- #1946
- #1947
- #1950
- #1951
- #1957
- #1959
- #1960
- #1961
- #1962
- #1964

- https://github.com/meilisearch/milli/pull/414
- https://github.com/meilisearch/milli/pull/409
- https://github.com/meilisearch/milli/pull/406
- https://github.com/meilisearch/milli/pull/418

- close #1687
- close #1786
- close #1940
- close #1948
- close #1949
- close #1932
- close #1956
2021-12-02 20:14:42 +01:00
many
cc6306c0e1
Update milli version 2021-11-04 14:57:45 +01:00
many
b664a46e91
Update milli version 2021-11-03 16:11:20 +01:00
Tamo
c4737749ab
bump segment to be able to display a user 2021-10-29 17:25:53 +02:00
Tamo
de35a9a605
use an official release of segment 2021-10-29 17:25:53 +02:00
Tamo
9be90011c6
save the user-id in the config dir of the OS 2021-10-29 17:25:51 +02:00
Tamo
664d09e86a
makes the analytics works with the option and the feature 2021-10-29 17:25:47 +02:00
Tamo
e226b1a87f
rewrite the main analytics module and the information sent in the tick 2021-10-29 17:25:42 +02:00
bors[bot]
cf67964133
Merge #1848
1848: Error format and Definition r=MarinPostma a=ManyTheFish



Co-authored-by: many <maxime@meilisearch.com>
2021-10-28 14:15:35 +00:00
many
3a29cbf0ae
Use milli v0.20.0 2021-10-28 15:59:06 +02:00
many
cbaca2b579
Fix PR comments 2021-10-28 15:42:42 +02:00
Clémentine Urquizar
a76d9b15c9
Update version for the next release (v0.24.0) 2021-10-28 12:24:49 +02:00
many
ff0908d3fa
Ignore errors tests that show unrelated bugs 2021-10-28 11:41:59 +02:00
many
7464720426
Fix some errors 2021-10-28 10:47:59 +02:00
marin postma
4ac005b094
optimize document transform
fix error types

bump milli
2021-10-26 13:51:15 +02:00
Clémentine Urquizar
0f342ac46e
Update MeiliSearch version 2021-10-12 16:43:31 +02:00
Clémentine Urquizar
29ac324e90
Update milli version to v0.17.3 2021-10-12 16:12:16 +02:00
Clémentine Urquizar
3edbc74430
Merge branch 'main' into stable 2021-10-11 18:30:10 +02:00
Clémentine Urquizar
60473637fe
Update milli version 2021-10-11 16:21:19 +02:00
Tom Parker-Shemilt
6c46fbbc57 Remove memmap dependency 2021-10-10 22:33:40 +01:00
bors[bot]
ddbcf449da
Merge #1763
1763: Index tests r=MarinPostma a=MarinPostma

This pr aims to test more thorougly the usage on index in the meilisearch database, by writing unit tests.

work included:
- [x] Create index mock and stub methods
- [x] Test snapshot creation
- [x] Test Dumps
- [x] Test search

Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-10-06 14:39:53 +00:00
mpostma
85ae34cf9f test snapshots 2021-10-06 14:10:23 +02:00
Tamo
fca686e7f8
bump meilisearch 2021-10-04 13:52:37 +02:00
Clémentine Urquizar
b17dae9ac0
Update version for the next release (v0.23.0) 2021-09-29 18:40:35 +02:00
bors[bot]
5fad37aebd
Merge #1711
1711: MeiliSearch refactor introducing OBKV format r=MarinPostma a=MarinPostma

This PR refactor some multiple components of meilisearch, and introduce the obkv document format to meilisearch

- [x] Split meilisearch-http and meilisearch-lib
- [x] Replace `IndexActor` and `UuidResolver` with `IndexResolver`
- [x] Remove mentions to Actor
- [x] Remove Actor traits to simplify code
- [x] Integrate obkv document format
- [x] Remove `Data`
- [x] Restore all route
- [x] Replace `Box<dyn error>` with `anyhow::Error`
- [x] Introduce update file store
- [x] Update file store error handling
- [x] Fix dumps
- [x] Fix snapshots
- [x] Fix tests
- [x] Update module documentation
- [x] add csv suppport (feat `@ManyTheFish` #1729 )
- [x] add jsonl support
- [x] integrate geosearch (feat `@irevoire` #1725) 

partially implements #1691 and #1690. The error handling is very basic now, I will finish it in the next pr.

Some unit tests have been disabled, I will re-enable them ASAP, but they need a bit more work.

close #1531 


P.S: sorry for this monstrous PR :'(

Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: many <maxime@meilisearch.com>
2021-09-29 14:38:55 +00:00
mpostma
311933614e bump milli to v0.17.0 2021-09-29 15:44:54 +02:00
mpostma
1f537e1b60 jsonl support 2021-09-29 11:28:02 +02:00
mpostma
6e8a3fe8de move csv parsing to document_formats 2021-09-28 22:58:48 +02:00
many
2a14948123 Use an existing revision of milli 2021-09-28 22:30:34 +02:00
many
d30830a55c Add csv deserializer for documents 2021-09-28 22:28:13 +02:00
mpostma
5fa9bc67d7 remove unused dependencies 2021-09-28 22:16:18 +02:00
mpostma
3503fbf7fe re-export milli from meilisearch_lib 2021-09-28 22:08:03 +02:00
mpostma
692c676625 fix tests 2021-09-28 18:57:36 +02:00
Tamo
654f49ccec
[WIP] put milli on branch main 2021-09-28 14:50:26 +02:00
Tamo
c1376a9f2a
add the geosearch to Meilisearch 2021-09-28 14:50:26 +02:00
mpostma
5353be74c3 refactor index actor 2021-09-22 15:07:04 +02:00
Tamo
2e99fa8251
remove the cargo.lock again 2021-09-22 11:30:33 +02:00
mpostma
aa6c5df0bc Implement documents format
document reader transform

remove update format

support document sequences

fix document transform

clean transform

improve error handling

add documents! macro

fix transform bug

fix tests

remove csv dependency

Add comments on the transform process

replace search cli

fmt

review edits

fix http ui

fix clippy warnings

Revert "fix clippy warnings"

This reverts commit a1ce3cd96e603633dbf43e9e0b12b2453c9c5620.

fix review comments

remove smallvec in transform loop

review edits
2021-09-21 16:58:33 +02:00
mpostma
60518449fc split meilisearch-http and meilisearch-lib 2021-09-21 13:23:22 +02:00
mpostma
e14640e530 refactor meilisearch 2021-09-20 14:54:20 +02:00
happysalada
770b6d25ae deps: unify pest dependency 2021-09-15 12:15:44 +09:00
Felix Yan
a57943b77e
Use tikv-jemallocator instead of jemallocator
`jemallocator` has been abandoned for nearly two years, and `rustc`
itself moved to use `tikv-jemallocator` instead:
3965773ae7

Let's switch to a better maintained version.
2021-09-14 18:30:24 +03:00
mpostma
0f7625e29a bump dependencies 2021-09-13 15:17:08 +02:00
Clémentine Urquizar
6d2f7af642
Bump milli: fix some crashes 2021-09-13 12:14:54 +02:00
Tamo
49c918defa
bump milli 2021-09-08 17:41:47 +02:00
Clémentine Urquizar
33514b28be
Merge pull request #1588 from meilisearch/test-new-indexer
Integrate the new indexer
2021-09-06 10:21:42 +02:00
Clément Renault
7e80337e5b
Bump milli to v0.12.0 2021-09-02 18:19:12 +02:00
many
8d4723d91b
Update lock file 2021-09-02 18:19:12 +02:00
Clémentine Urquizar
63e67f72e3
Update tokenizer and new milli version 2021-09-02 18:19:00 +02:00
Kerollmops
0cd66c3a89
Bump the milli version 2021-09-02 18:19:00 +02:00
Clément Renault
b092a624ed
Introduce the MaxMemory struct that defaults to 2/3 of the available memory 2021-09-02 18:18:59 +02:00
Clémentine Urquizar
24e84d7ca1
Test new indexer 2021-09-02 18:11:20 +02:00
Clémentine Urquizar
90116155b4
Update version for the next release (v0.22.0) 2021-09-01 12:33:30 +02:00
bors[bot]
19c5c74291
Merge #1652 #1654 #1657
1652: Remove dependabot r=MarinPostma a=curquiza

Fixes #1649 

Dependabot for vulnerability and security updates is still activated.

1654: Add Script for Windows r=MarinPostma a=singh08prashant

fixes #1570 

changes:

1. added script for detecting windows os running git bash
2. appended `.exe` to `$release_file` for windows as listed [here](https://github.com/meilisearch/MeiliSearch/releases/)
3. removed global `$BINARY_NAME='meilisearch'` as windows require `.exe` file

1657: Bring vergen hotfix from `stable` to `main` r=MarinPostma a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: singh08prashant <singh08prashant@gmail.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
2021-08-31 14:31:42 +00:00
Kerollmops
b8c954eb3f
Bump the MeiliSearch version to v0.21.1 2021-08-30 17:41:25 +02:00
Kerollmops
1782753387
Bump vergen and remove unused build feature 2021-08-30 15:03:45 +02:00
Kerollmops
51387b2c80
Introduce the new invalid sortable error codes 2021-08-25 17:29:30 +02:00
Irevoire
672fcee8aa
remove sentry 2021-08-24 12:38:31 +02:00
Clémentine Urquizar
149f46c184
Fix pacic when decoding 2021-08-23 12:37:51 +02:00
Clémentine Urquizar
38fc876704
Update tokenizer and new milli version with new tags 2021-08-18 14:55:10 +02:00
Clémentine Urquizar
39d5a99095
Update milli and tokenizer version 2021-08-18 12:09:34 +02:00
bors[bot]
2beb306834
Merge #1577
1577: Update milli dependency: fix facet values bugs r=Kerollmops a=curquiza

Fixes #1576 

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-08-16 16:13:42 +00:00
Clémentine Urquizar
f3e595e2f0
Update milli dependency 2021-08-16 13:36:42 +02:00
Clémentine Urquizar
7531280764
Update tokenizer version to v0.2.4 2021-08-12 13:55:47 +02:00
bors[bot]
7e3b2ddff2
Merge #1554
1554: Fix dump v1 (attributesForFaceting, and criteria) r=curquiza a=MarinPostma

close #1553


Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-08-05 19:45:52 +00:00
mpostma
8f05d8d546 fix clippy warnings 2021-08-05 16:00:47 +02:00
Clémentine Urquizar
31fe263356
Update milli to v0.9.0 2021-08-05 13:08:27 +02:00
bors[bot]
9810f6b695
Merge #1540
1540: Update milli to version 0.8.1 r=curquiza a=curquiza

Integrates this fix into MeiliSearch https://github.com/meilisearch/milli/pull/296

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-07-29 17:15:52 +00:00
mpostma
53b4b2fcbc Use serdeval for validating json format. 2021-07-29 18:02:54 +02:00
Clémentine Urquizar
9a8629a6a9
Update milli 2021-07-29 17:45:31 +02:00
Clémentine Urquizar
e5dabf265a
Update milli version to v0.8.0 2021-07-28 10:52:47 +02:00
Tamo
0038b3848a
add a simple github cache 2021-07-26 15:31:26 +02:00
Tamo
f79b8287f5
update vergen 2021-07-26 15:25:30 +02:00
Kerollmops
0353fbb5df
Bump the tokenizer version to v0.2.4 2021-07-22 17:14:45 +02:00
Kerollmops
838ed1cd32
Use an u16 field id instead of one byte 2021-07-06 11:58:03 +02:00
Kerollmops
91c5d0c042
Use the AlwaysFreePages flag when opening an index 2021-07-05 16:36:13 +02:00
Tamo
9949a2a930
bump milli to 0.7.2 2021-07-05 12:19:27 +02:00
Kerollmops
a6b4069172
Bump to v0.7.2 2021-07-05 10:54:53 +02:00
Tamo
fc09d77e89
fix the benchmarks dependcies 2021-07-01 11:38:30 +02:00
Clémentine Urquizar
3c149d8a43
Update tokenizer version to v0.2.3 2021-06-30 18:41:35 +02:00
Clémentine Urquizar
4f0d3b065f
Update milli 2021-06-30 18:39:06 +02:00
Clémentine Urquizar
01de7f9e36
Update version 2021-06-30 15:59:59 +02:00
Clémentine Urquizar
b489515f4d
Update milli version to v0.7.1 2021-06-30 13:52:46 +02:00
Tamo
ec809ca487
use rustls instead of openssl and remove all default-features of reqwest 2021-06-29 13:07:40 +02:00
Clémentine Urquizar
c09e610bb5
Update heed and milli 2021-06-29 10:25:47 +02:00
Clément Renault
80c6aaf1fd
Bump milli to 0.7.0 2021-06-28 18:31:56 +02:00
Clément Renault
bdc5599b73
Bump heed to use the git repo with v0.12.0 2021-06-28 18:26:20 +02:00
Tamo
a59f437ee3
use only half of the computer threads for the indexation by default 2021-06-28 14:35:50 +02:00
marin postma
74bb748a4e
bump milli to 0.6.0 2021-06-23 18:40:19 +02:00
Kerollmops
98285b4b18
Bump milli to 0.6.0 2021-06-23 17:30:26 +02:00
Clémentine Urquizar
9885fb4159
Update version for the next release (v0.5.1) 2021-06-23 14:05:20 +02:00
Clémentine Urquizar
0de696feaf
Update version for alpha 6 2021-06-22 18:40:51 +02:00
Clémentine Urquizar
e2844f3a92
Update tokenizer version to v0.2.3 2021-06-22 15:57:47 +02:00
Clémentine Urquizar
2e3d85c31a
Update milli version to v0.5.0 2021-06-22 15:57:46 +02:00
Tamo
77eb37934f
add jemalloc to http-ui and the benchmarks 2021-06-22 14:17:56 +02:00
Clémentine Urquizar
320670f8fe
Update version for the next release (v0.5.0) 2021-06-21 15:59:17 +02:00
marin postma
0bfdf9a785
bump milli 2021-06-21 12:11:09 +02:00
marin postma
abdf642d68
integrate milli errors 2021-06-21 12:11:08 +02:00
Clémentine Urquizar
35fcc351a0
Update version for the next release (v0.4.2) 2021-06-20 17:37:24 +02:00
Kerollmops
ccd6f13793
Update version to the next release (0.4.1) 2021-06-17 15:01:20 +02:00
bors[bot]
2f2484e186
Merge #212
212: bump milli to 0.4.0 r=MarinPostma a=MarinPostma



Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-16 15:42:34 +00:00
bors[bot]
2062b10b79
Merge #209
209: Integrate amplitude r=MarinPostma a=irevoire

And merge the sentry and amplitude usage under one “Enable analytics” flag

closes #180


Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Irevoire <tamo@meilisearch.com>
2021-06-16 15:25:31 +00:00
marin postma
5a47cef9a8
bump milli to 0.4.0 2021-06-16 17:15:56 +02:00
Clémentine Urquizar
f5ff3e8e19
Update version for the next release (v0.4.0) 2021-06-16 14:01:05 +02:00
Tamo
def1596eaf
Integrate amplitude
And merge the sentry and amplitude usage under one “Enable analytics”
flag
2021-06-15 15:36:30 +02:00
Clémentine Urquizar
fe5a494035
Update alpha for the next release 2021-06-14 17:55:04 +02:00
Kerollmops
312c2d1d8e
Use the Error enum everywhere in the project 2021-06-14 16:58:38 +02:00
bors[bot]
b119bb4ab0
Merge #197
197: Update milli (v0.3.1) with filterable attributes r=MarinPostma a=curquiza

Fixes #187 and #70
Also fixes #195 

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-14 12:19:42 +00:00
Clémentine Urquizar
9996c59183
Update with milli 0.3.1 2021-06-14 13:20:43 +02:00
Clémentine Urquizar
aa04124bfc
Add changes according to milli update 2021-06-14 13:20:37 +02:00
marin postma
0f767e3743
conccurrent update run loop 2021-06-14 10:57:14 +02:00
bors[bot]
1e659bb17b
Merge #194
194: Bump sentry version r=MarinPostma a=irevoire

closes #102 

Co-authored-by: tamo <tamo@meilisearch.com>
2021-06-14 08:34:04 +00:00
Clémentine Urquizar
7d5395c12b
Update Tokenizer version to v0.2.3 2021-06-10 17:00:04 +02:00
Clémentine Urquizar
dc64e139b9
Update version for the next release (v0.3.1) 2021-06-09 14:39:21 +02:00
Kerollmops
103dddba2f
Move the UpdateStore into the http-ui crate 2021-06-08 17:59:51 +02:00
Kerollmops
29824d05ab
Reduce the length of the benchmarks names 2021-06-03 15:59:43 +02:00
tamo
4536dfccd0
add a way to provide primary_key or autogenerate documents ids 2021-06-02 11:13:20 +02:00
tamo
dffbaca63b
bump sentry version 2021-05-31 13:59:31 +02:00
tamo
26dcb9e66d
bump milli version and fix a performance issue for large dumps 2021-05-10 20:25:12 +02:00
Clémentine Urquizar
1e11578ef0
Update version for the next release (v0.2.1) 2021-05-05 14:57:34 +02:00
Clémentine Urquizar
eb03a3ccb1
Upgrade Milli and Tokenizer 2021-05-04 17:56:19 +02:00
Marin Postma
928fb34eff
bump milli and fix tests 2021-05-04 15:10:22 +02:00
Clémentine Urquizar
a8680887d8
Upgrade Milli version (v0.2.0) 2021-05-03 14:50:47 +02:00
Clémentine Urquizar
34e02aba42
Upgrade Tokenizer version (v0.2.2) 2021-05-03 10:55:55 +02:00
Marin Postma
78217bcf18
Fix cors authentication issue 2021-04-29 16:28:12 +02:00
Marin Postma
a961f0ce75
fix clippy warnings 2021-04-27 18:28:46 +02:00
Marin Postma
4fe2a13c71
rewrite update store 2021-04-27 15:20:52 +02:00
Marin Postma
5a38f13cae
multi_index udpate store 2021-04-27 15:07:13 +02:00
many
0d7d3ce802
Update roaring package 2021-04-27 14:39:53 +02:00
many
71740805a7
Fix forgotten typo tests 2021-04-27 14:39:53 +02:00
Kerollmops
0f4c0beffd
Introduce the Attribute criterion 2021-04-27 14:25:34 +02:00
Clémentine Urquizar
f3b6bf55a6
Update version for the next release (alpha4) 2021-04-26 19:05:16 +02:00
Marin Postma
e4bd1bc5ce
update actix-web-static-file rev 2021-04-22 11:42:41 +02:00
Marin Postma
6bcf20c70e
serve static site 2021-04-22 10:26:54 +02:00
Marin Postma
bb79695e44
load mini-dashboard assets 2021-04-22 10:26:54 +02:00
Marin Postma
ea5517bc8c
add mini-dashboard feature 2021-04-22 10:26:54 +02:00
Marin Postma
c6bb36efa5
implement _formated 2021-04-20 21:23:28 +02:00
Clémentine Urquizar
b8e535579f
Update version for the next release (alpha3) 2021-04-20 16:11:07 +02:00
Marin Postma
f8c338e3a7
add test for dedicated distinct route 2021-04-20 15:49:17 +02:00
mpostma
ec230c2835
enable distinct 2021-04-20 11:29:06 +02:00
Marin Postma
45665245dc
bump actix 2021-04-20 11:07:23 +02:00
Kerollmops
c9b2d3ae1a
Warn instead of returning an error when a conversion fails 2021-04-20 10:23:31 +02:00
Kerollmops
51767725b2
Simplify integer and float functions trait bounds 2021-04-20 10:23:31 +02:00
Kerollmops
efbfa81fa7
Merge the Float and Integer enum variant into the Number one 2021-04-20 10:23:30 +02:00
Clémentine Urquizar
2c380731b9
Update milli version to v0.1.1 2021-04-19 16:03:39 +02:00
Clémentine Urquizar
127d3d028e
Update version for the next release (v0.1.1) 2021-04-19 14:48:13 +02:00
Clémentine Urquizar
b0717b75d9
Update tokenizer to v0.2.1 2021-04-14 19:06:18 +02:00
Clémentine Urquizar
2c5c79d68e
Update Tokenizer version to v0.2.1 2021-04-14 18:54:04 +02:00
Alexey Shekhirin
3af8fa194c
test(http): combine settings assert_(ser|de)_tokens into 1 test 2021-04-10 12:13:59 +03:00
Alexey Shekhirin
ae1655586c
fixes after review 2021-04-09 14:40:48 +03:00
Alexey Shekhirin
dd9eae8c26
feat(http): stats route 2021-04-09 14:34:42 +03:00
Alexey Shekhirin
51ba1bd7d3
fix(http, index): init analyzer with optional stop words
Next release

update tokenizer
2021-04-08 17:16:13 +03:00
Alexey Shekhirin
84c1dda39d
test(http): setting enum serialize/deserialize 2021-04-08 17:03:40 +03:00
Alexey Shekhirin
dc636d190d
refactor(http, update): introduce setting enum 2021-04-08 17:03:40 +03:00
tamo
40ef9a3c6a
push a first implementation of the stop_words 2021-04-06 16:29:04 +02:00
Clémentine Urquizar
4ee6ce7871
Next release 2021-04-01 17:16:16 +02:00
tamo
62a8f1d707
bump the version of the tokenizer 2021-04-01 13:49:22 +02:00
tamo
73dcdb27f6
select a specific release of the tokenizer instead of using the latests git commit 2021-03-25 15:00:18 +01:00
mpostma
46293546f3
add tests and mocks 2021-03-24 12:38:59 +01:00
mpostma
8b99860e85
use write sender for updates 2021-03-18 08:32:05 +01:00
bors[bot]
ca3b343b1f
Merge #96
96: Check json payload on document addition r=curquiza a=MarinPostma

Check if the json payload in updates is valid. It uses a json validator to avoid allocation, and only serializes the json in case of error, to return a pretty message.

Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-03-16 17:20:44 +00:00
mpostma
204c743bcc
add json payload check on document addition 2021-03-16 14:28:13 +01:00
mpostma
6a742ee62c
restore version route 2021-03-15 19:11:27 +01:00
mpostma
c29b86849b
use actix cors git dependency 2021-03-15 17:40:20 +01:00
mpostma
f727dcc8c6
update milli 2021-03-15 14:26:59 +01:00
mpostma
80d0f9c49d
methods to update index time metadata 2021-03-15 14:05:47 +01:00
mpostma
adc71a70ce
fix displayed attributes in document retrieval 2021-03-15 10:17:41 +01:00
mpostma
3f68460d6c
fix update dedup 2021-03-11 20:58:51 +01:00
mpostma
79a4bc8129
use meta from milli 2021-03-11 19:40:18 +01:00
mpostma
a56e8c1a0c
fix tests 2021-03-10 14:47:04 +01:00
mpostma
5ecf514d28
restructure project 2021-03-10 13:46:49 +01:00
mpostma
562da9dd3f
fix test compilation 2021-03-10 11:56:51 +01:00
Clément Renault
b18ec00a7a
Add a logging_timer macro to te criterion next methods 2021-03-08 16:12:06 +01:00
Kerollmops
636a9df177
Temporarily fix the tinytemplate doc hidden issue 2021-03-08 15:57:45 +01:00
mpostma
7d28f8cff0
implement get single udpate 2021-03-06 10:51:52 +01:00
mpostma
f090f42e7a
multi index store
create two channels for Index handler, one for writes and one for reads,
so write are processed one at a time, while reads are processed in
parallel.
2021-03-04 19:18:01 +01:00
Kerollmops
07784c8990
Tune the words prefixes threshold to compute for 1/1000 instead 2021-03-03 15:51:28 +01:00
Kerollmops
79a143b32f
Introduce the query tree data structure 2021-03-03 13:40:18 +01:00
mpostma
62532b8f79
WIP concurent index store 2021-03-02 14:05:03 +01:00
Clément Renault
9423310816
Introduce an helpers crate that export the database to stdout 2021-03-01 19:55:04 +01:00
mpostma
61ce749122
update tokio and disable all routes 2021-02-26 09:10:04 +01:00
Kerollmops
519b1cb5c9
Update dependencies 2021-02-21 10:26:04 +01:00
mpostma
91d6e90d5d
enable faceted searches 2021-02-16 19:20:39 +01:00
Clément Renault
fecf3d6fc1
Move the command lines helpers into different crates 2021-02-14 18:55:15 +01:00
Clément Renault
d8f3421608
Update the dependencies and remove the unused ones 2021-02-14 18:32:46 +01:00
Clément Renault
e8639517da
Change the project to become a workspace with milli as a default-member 2021-02-12 16:15:09 +01:00
mpostma
f8f02af23e
incorporate review changes 2021-02-04 13:21:15 +01:00
mpostma
9af0a08122
post review fixes 2021-02-02 17:34:06 +01:00
mpostma
17c463ca61
remove unused deps 2021-02-01 13:32:21 +01:00
mpostma
e9c95f6623
remove useless files 2021-01-28 19:43:54 +01:00
mpostma
6c63ee6798
implement list all indexes 2021-01-28 18:32:24 +01:00
mpostma
74410d8c6b
architecture rework 2021-01-28 14:12:34 +01:00
Clément Renault
433ac8c38a
Remove the ordered-float serde feature 2021-01-27 14:11:10 +01:00
Kerollmops
61dbcfa44a
Bump the roaring to 0.6.4 2021-01-26 14:38:43 +01:00
Clément Renault
51a37de885
Introduce the FacetValue enum type 2021-01-26 14:09:09 +01:00
mpostma
87a56d2bc9
Fix settings bug
replace ids with str in settings

This allows for better maintainability of the settings code, since
updating the searchable attributes is now straightforward.

criterion use string

fix reindexing fieldid remaping

add tests for primary_key compute

fix tests

fix http-ui

fixup! add tests for primary_key compute

code improvements settings

update deps

fixup! code improvements settings

fixup! refactor settings updates and fix bug

fixup! Fix settings bug

fixup! Fix settings bug

fixup! Fix settings bug

Update src/update/index_documents/transform.rs

Co-authored-by: Clément Renault <clement@meilisearch.com>

fixup! Fix settings bug
2021-01-26 13:53:08 +01:00
mpostma
6a3f625e11
WIP: refactor IndexController
change the architecture of the index controller to allow it to own an
index store.
2021-01-16 15:09:48 +01:00
mpostma
686f987180
fix compile errors 2021-01-14 11:27:07 +01:00
mpostma
d22fab5bae
implement open index 2021-01-13 18:20:14 +01:00
mpostma
ddd7789713
WIP: IndexController 2021-01-13 17:50:36 +01:00
mpostma
4f7f7538f7
highlight with new tokenizer 2021-01-11 21:59:37 +01:00
mpostma
1ae761311e
integrate with meilisearch tokenizer 2021-01-07 16:14:27 +01:00
mpostma
b4d447b5cb temp 2021-01-01 16:59:49 +01:00
mpostma
d1e9ded76f
setting builder takes ownership 2020-12-31 00:50:30 +01:00
mpostma
d9dc2036a7 support error & return document count on addition 2020-12-30 18:44:33 +01:00
mpostma
54861335a0 retrieve update status 2020-12-30 18:16:07 +01:00
mpostma
0cd9e62fc6 search first iteration 2020-12-24 12:58:34 +01:00
mpostma
1a38bfd31f data add documents 2020-12-23 13:52:28 +01:00
mpostma
55e1552957 update queue refactor, first iteration 2020-12-22 17:13:50 +01:00
mpostma
7c9eaaeadb clean code, and fix errors 2020-12-22 14:02:41 +01:00
Kerollmops
77e951e933
Use the byte-unit crate to ease library usage 2020-12-20 12:00:37 +01:00
Clément Renault
e7f2ab9138
Bump grenad to fix an indexing bug 2020-12-05 16:39:15 +01:00
Clément Renault
0959e1501f
Introduce the FacetRevRange Iterator struct 2020-12-04 12:02:23 +01:00
Clément Renault
61b383f422
Introduce the criteria update setting 2020-12-04 12:02:22 +01:00
Clément Renault
a0adfb5e8e
Introduce a real pest parser and support every facet filter conditions 2020-11-23 16:43:55 +01:00
Clément Renault
07a0c82790
Bump heed to 0.10.4 to use be able to lazily decode roaring bitmaps 2020-11-23 16:43:53 +01:00
Clément Renault
38c76754ef
Make the facet level search system generic on f64 and i64 2020-11-23 16:43:52 +01:00
Clément Renault
b255be93fa
Bump heed to 0.10.3 2020-11-23 16:43:49 +01:00
Clément Renault
a18d9a1f87
Parse and store the faceted fields 2020-11-13 16:13:51 +01:00
Clément Renault
640c7d748a
Modify the highlight function to support any JSON type 2020-11-05 13:59:32 +01:00
Clément Renault
0408c9d66a
Move the http server into its own sub-module 2020-11-05 11:16:39 +01:00
Clément Renault
4fded5bd0e
Bump heed to be able to reference a RoTxn from multiple threads 2020-11-02 12:49:23 +01:00
Clément Renault
f0d028d3a4
Update the Transform struct to support JSON updates 2020-10-31 20:52:49 +01:00
Clément Renault
9d47ee52b4
Generate a uuid v4 based document id when missing 2020-10-31 15:11:06 +01:00
Clément Renault
085d3b9d94
Update heed to 0.10.0 2020-10-30 11:42:00 +01:00
Clément Renault
b5d52b6b45
Prefer using a smallstr instead of a real String to reduce allocations 2020-10-29 14:32:32 +01:00
Clément Renault
98fc24cbdf
Bump heed to fix a prefix iter bug 2020-10-28 10:55:21 +01:00
Clément Renault
b44b04d25b
Serialize the CSV record values as JSON strings 2020-10-24 14:43:46 +02:00
Clément Renault
802e925fd7
Switch to a JSON protocol for the front page 2020-10-21 18:26:29 +02:00
Clément Renault
2210818114
Introduce the obkv heed codec 2020-10-21 15:51:48 +02:00
Clément Renault
f948a03be2
Optimise the merge functions to avoid allocations 2020-10-20 16:40:50 +02:00
Clément Renault
cde8478388
Replace the panic in the merge function by actual errors 2020-10-20 16:19:07 +02:00