Commit Graph

190 Commits

Author SHA1 Message Date
Tamo
8fb7b1d10f
bump deserr 2023-02-14 20:04:30 +01:00
James
62358bd31c Fix metrics feature
As reported the metrics feature was broken by still using and old reference to `meilisearch_auth::actions`. This commit switches to the new location, `meilisearch_types::keys::actions`.

Resolves: #3469
See also: #2763
2023-02-14 17:29:38 +00:00
Clément Renault
47748395dc
Update an authentication comment
Co-authored-by: Many the fish <many@meilisearch.com>
2023-02-13 17:20:08 +01:00
Clément Renault
4570d5bf3a
Merge remote-tracking branch 'origin/main' into temp-wildcard 2023-02-09 13:14:05 +01:00
Kerollmops
a015e232ab
Merge remote-tracking branch 'origin/release-v1.0.0' into bring-v1-changes 2023-02-06 16:41:10 +01:00
Kerollmops
a36b1dbd70
Fix the tasks with the new patterns 2023-02-01 18:21:45 +01:00
Kerollmops
d563ed8a39
Making it work with index uid patterns 2023-02-01 17:51:30 +01:00
Louis Dureuil
924d5d4c11
clippy: remove needless lifetimes 2023-01-31 10:40:48 +01:00
Louis Dureuil
3296cf7ae6
clippy: remove needless lifetimes 2023-01-31 09:32:40 +01:00
Kerollmops
184b8afd9e
Make it work in the CreateApiKey struct 2023-01-25 15:01:50 +01:00
Tamo
a858531574 apply review comments 2023-01-25 14:51:36 +01:00
Tamo
04c4487660 udpate the analytics with the new stats method 2023-01-25 11:25:04 +01:00
Tamo
c92948b143 Compute the size of the auth-controller, index-scheduler and all update files in the global stats 2023-01-25 11:25:02 +01:00
bors[bot]
217504fff3
Merge #3406
3406: Master Key: Implements errors and warnings from the specification r=irevoire a=dureuill

<sub>Now in technicolor</sub>

# Pull Request

## What does this PR do?
- Uses `atty` and `termcolor` as dependency
- Use these dependencies to print colored background for warning messages
- Update messages to match https://github.com/meilisearch/specifications/pull/209

## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [ ] Have you read the contributing guidelines?
- [ ] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2023-01-23 16:39:18 +00:00
Louis Dureuil
a9b3f91467
Add missing space
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
2023-01-23 10:33:30 +01:00
Louis Dureuil
0de9a3ffe7
Implements errors and warnings from the specification
Now in technicolor
2023-01-19 18:04:45 +01:00
bors[bot]
8fb685f5aa
Merge #3401
3401: improve the error messages for the immutable fields r=dureuill a=irevoire

Fix https://github.com/meilisearch/meilisearch/issues/3400

Co-authored-by: Tamo <tamo@meilisearch.com>
2023-01-19 15:52:50 +00:00
Tamo
e3742a38d4
improve the error messages for the immutable fields 2023-01-19 16:49:44 +01:00
Louis Dureuil
72e2b220ed
Fix tests 2023-01-19 15:48:20 +01:00
bors[bot]
6f7e0c431a
Merge #3341
3341: add functionnal + error tests on the swap_indexes route and fix a confusing error message r=loiclec a=irevoire

Fix https://github.com/meilisearch/meilisearch/issues/3340
Fix part of https://github.com/meilisearch/meilisearch/issues/3325
Fix https://github.com/meilisearch/meilisearch/issues/3381

Test both the functionality and the error codes

Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2023-01-18 16:32:22 +00:00
Tamo
00f6af6475
fix a wrong error message 2023-01-18 17:26:48 +01:00
Tamo
a4476c20f8
fix a wrong error code and add tests on the document resource 2023-01-18 15:28:02 +01:00
Loïc Lecrenier
e225608337 Use invalid_index_uid error code in more places 2023-01-17 15:28:06 +01:00
Loïc Lecrenier
56e79fa850 Update task snapshot test and clean up details 2023-01-17 13:19:04 +01:00
Loïc Lecrenier
b781f9a0f9 cargo fmt 2023-01-17 11:07:07 +01:00
Loïc Lecrenier
9194508a0f Refactor query parameter deserialisation logic 2023-01-17 11:07:07 +01:00
Loïc Lecrenier
766dd830ae Update deserr to latest version + add new error codes for missing fields
- missing_api_key_indexes
- missing_api_key_actions
- missing_api_key_expires_at

- missing_swap_indexes_indexes
2023-01-17 09:43:07 +01:00
Loïc Lecrenier
436ae4e466 Improve error messages generated by deserr
Split Json and Query Parameter error types
2023-01-17 09:43:07 +01:00
Kerollmops
cde62fcb5b
Merge remote-tracking branch 'origin/release-v1.0.0' into import-milli 2023-01-16 17:35:18 +01:00
Kerollmops
4fb47492e5
Make clippy happy 2023-01-16 16:35:58 +01:00
bors[bot]
a5c4fbbcea
Merge #3334
3334: Add specific error codes `immutable_...` r=irevoire a=loiclec

Add the following error codes:

When an immutable field of API key is sent to the `PATCH /keys` route: 
- `ImmutableApiKeyUid` 
- `ImmutableApiKeyKey`
- `ImmutableApiKeyActions`
- `ImmutableApiKeyIndexes`
- `ImmutableApiKeyExpiresAt`
- `ImmutableApiKeyCreatedAt`
- `ImmutableApiKeyUpdatedAt`

When an immutable field of Index is sent to the `PATCH /indexes/{uid}` route:
- `ImmutableIndexUid`
- `ImmutableIndexCreatedAt`
- `ImmutableIndexUpdatedAt`

Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
2023-01-12 15:31:38 +00:00
Tamo
21b8cd53b7
reformat 2023-01-12 16:20:24 +01:00
Loïc Lecrenier
7f80b116bc
Add specific immutable_field error codes 2023-01-12 16:20:14 +01:00
Tamo
a524dfb713
fix the analytics 2023-01-12 14:49:50 +01:00
Tamo
705fcaa3b8
reformat the imports 2023-01-12 14:09:15 +01:00
Loïc Lecrenier
a09b6a341d Move tasks route to deserr 2023-01-12 13:57:29 +01:00
Tamo
387874ea26
test the error codes on the task routes 2023-01-12 13:46:19 +01:00
Tamo
6d658f4c52
fix a wrong error code + update some error messages 2023-01-11 19:14:11 +01:00
Tamo
a68ac3a1dc
reformat the headers 2023-01-11 19:08:39 +01:00
Loïc Lecrenier
b0b7ad7caf
Apply review suggestions 2023-01-11 19:08:39 +01:00
Loïc Lecrenier
1fc11264e8
Refactor deserr integration 2023-01-11 19:08:39 +01:00
Tamo
d0a85057a3 fix the bad filter test 2023-01-11 11:37:12 +01:00
Tamo
ff843881c5
remove the documentation of the query parameter extractor module 2023-01-09 15:14:48 +01:00
Loïc Lecrenier
ae08fba76e Remove forgotten comment 2023-01-09 13:45:03 +01:00
Loïc Lecrenier
af6d4b3031 Remove unused deserr extractor 2023-01-09 13:43:16 +01:00
Tamo
b03ee54fe0
makes clippy turbo-happy 2023-01-09 13:04:31 +01:00
Loïc Lecrenier
9ab791bedc Update error codes on the api key routes 2023-01-09 12:30:25 +01:00
Loïc Lecrenier
96105a5e8d Update error codes on the documents/ routes 2023-01-09 12:30:25 +01:00
Tamo
3c630891bb
fix the error code for the swap index 2023-01-05 21:25:20 +01:00
Tamo
50ce0409bc
Integrate deserr on the most important routes 2023-01-05 20:48:29 +01:00
bors[bot]
839b05c43d
Merge #3305
3305: Remove hidden but usable CLI arguments r=Kerollmops a=Kerollmops

`@curquiza` found out that we were exposing some internal CLI arguments: `nb-max-chunks` and `log-every-n`. In this PR I removed those two, the only two ones that I found. Those options shouldn't be accessible as non-documented in the documentation or the `--help` message.

Fixes https://github.com/meilisearch/meilisearch/issues/3307

Co-authored-by: Clément Renault <clement@meilisearch.com>
2023-01-05 17:11:58 +00:00
Clément Renault
aa4b813237
Derive Default on IndexerOpts 2023-01-05 16:00:45 +01:00
Louis Dureuil
eb08a0fb0b
Remove --generate-master-key option 2023-01-05 14:55:24 +01:00
Clément Renault
cda529c07b
Remove hidden but usable CLI arguments 2023-01-05 14:25:41 +01:00
bors[bot]
32f7cfa5cb
Merge #3295
3295: Adjust Master Key-related messages r=dureuill a=dureuill

# Pull Request

## Related issue
Follow up for #3272 

## What does this PR do?
- Consistently capitalize "master key" (instead of "Master Key" sometimes) (see https://github.com/meilisearch/specifications/pull/209#discussion_r1060081094)
- Clarify that the counted unit for master key length is bytes, not characters (see https://github.com/meilisearch/documentation/issues/2069#issuecomment-1368873167)

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2023-01-05 08:43:23 +00:00
bors[bot]
502d9e4b24
Merge #3278
3278: Remove `--max-index-size` and `--max-task-db-size` flags r=Kerollmops a=dureuill

# Pull Request

## Related issue
Fixes #3231 

## What does this PR do?
- Remove `--max-index-size` and `--max-task-db-size` flags from the CLI, config file and environment variable
- Set the size of all indexes to **500GiB** and the size of the task DB to **10GiB**.  Reviewers might want to review these values carefully.

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2023-01-04 16:44:27 +00:00
Louis Dureuil
a85ff1f690
Fix documentation
Co-authored-by: Clément Renault <clement@meilisearch.com>
2023-01-04 17:20:03 +01:00
Louis Dureuil
233372abea
Remove --max-index-size and --max-task-db-size 2023-01-04 17:20:01 +01:00
bors[bot]
13d4ae264a
Merge #3269
3269: Simplify primary key inference r=dureuill a=dureuill

# Pull Request

## Related issue
Related to https://github.com/meilisearch/meilisearch/issues/3233

## What does this PR do?
- Integrates https://github.com/meilisearch/milli/pull/752 in meilisearch
- Remove `Serialize` and `Deserialize` from `error::Code` as it is unused.
- No longer filter on `milli` logs when `--log-level` is "info".
  - `milli` only has the newly-added inference log at the `info` level (from greping `info` in the codebase)
  - the default value for `--log-level` is "INFO" and not "info" since `v0.30` so the filter is not active by default.
- updates milli to v0.38.0

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2023-01-04 16:14:36 +00:00
Louis Dureuil
5cb566b165
No longer filter out milli logs when --log-level is "info" 2023-01-04 14:24:20 +01:00
Louis Dureuil
c4aa5cc7d0
Merge --schedule-snapshot and --snapshot-interval-sec options 2023-01-04 14:13:54 +01:00
Loïc Lecrenier
d082ded7ad Explicitly restrict log level options to those that are documented
Fixes https://github.com/meilisearch/meilisearch/issues/3292
2023-01-04 09:40:24 +01:00
bors[bot]
947f08793a
Merge #3296
3296: Remove `--disable-auto-batching` CLI option r=gmourier a=loiclec

Fixes #3294 

The `index-scheduler` code is not modified, only the CLI options have changed.

Co-authored-by: Loïc Lecrenier <loic.lecrenier@me.com>
2023-01-03 16:57:14 +00:00
Louis Dureuil
17dac72464
Characters -> bytes 2023-01-03 15:31:02 +01:00
Loïc Lecrenier
b821c72459 Remove --disable-auto-batching CLI option 2023-01-03 15:01:04 +01:00
Louis Dureuil
7b2575c646
Master Key -> master key 2023-01-03 14:45:23 +01:00
bors[bot]
6425e06cf2
Merge #3274
3274: Reject master keys that are less than 16 bytes and add `--generate-master-key` CLI option r=irevoire a=dureuill

# Pull Request

## Related issue
Fix #3272 
Fix #3287

## What does this PR do?

### User standpoint

---

- Adds a `--generate-master-key` CLI flag to generate a fresh Master Key and exit.

<img width="1351" alt="Capture d’écran 2022-12-22 à 14 18 58" src="https://user-images.githubusercontent.com/41078892/209142778-eab52eeb-eaa8-409b-897a-c0d5728c8aaa.png">

---

(relevant fragment of the `--help` message)

<img width="1351" alt="Capture d’écran 2022-12-22 à 14 19 40" src="https://user-images.githubusercontent.com/41078892/209142891-ebfa2ed6-f231-4f76-a3ae-b7542c7aef04.png">

---

- When `meilisearch` is started in the `development` environment and no Master Key has been provided, then the binary prints a warning before starting.

<img width="1351" alt="Capture d’écran 2022-12-22 à 14 14 49" src="https://user-images.githubusercontent.com/41078892/209142158-54eba3b7-bf71-4f3f-8840-0600b13a1a9f.png">

---

- When `meilisearch` is started in the `development` environment and the provided Master Key is shorter than 16 bytes, then the binary prints a warning before starting.

<img width="1351" alt="Capture d’écran 2022-12-22 à 14 15 58" src="https://user-images.githubusercontent.com/41078892/209142295-0209fe47-c03b-424f-a73f-cee9b633137a.png">

---

- When `meilisearch` is started in the `production` environment, and no Master Key is provided, the error message is altered to generate a fresh Master Key.

<img width="1351" alt="Capture d’écran 2022-12-22 à 17 29 02" src="https://user-images.githubusercontent.com/41078892/209180540-0def5798-15db-47f0-a6ec-8cfa081dea77.png">


---

- When `meilisearch` is started in the `production` environment, and the provided Master Key is shorter than 16 bytes, then the binary exits with an error.

<img width="1351" alt="Capture d’écran 2022-12-22 à 17 28 47" src="https://user-images.githubusercontent.com/41078892/209180567-fa54fe33-fbc4-4b9f-b281-7dfb7b33af85.png">


---

This implements the solution B described here: https://github.com/meilisearch/product/discussions/538#discussioncomment-4391346 

### Implementation standpoint

- Add a new `meilisearch-auth::generate_master_key` function that uses a Cryptographic Random Number Generator (CRNG) to fill a vector of 32 bytes before encoding these bytes as base64

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
2023-01-02 16:00:40 +00:00
Tamo
1692f58b83
slightly update the message associated with the cli parameter + accept an env variable 2023-01-02 16:49:35 +01:00
Tamo
9ba4d0f921
update the error messages according to the spec 2023-01-02 16:43:23 +01:00
Louis Dureuil
3cba476a9f
Add --generate-master-key CLI option 2022-12-26 10:36:45 +01:00
Louis Dureuil
57e851d8a9
Check for key length 2022-12-26 10:36:45 +01:00
amab8901
9a39c4e40d Get date from IndexMetaData 2022-12-22 11:46:17 +01:00
amab8901
df176aaf01 Insert dump_reader.date() into create_raw_index(_) argument 2022-12-21 15:16:31 +01:00
amab8901
0893b175dc Merge branch 'main' into 2983-forward-date-to-milli 2022-12-21 14:31:19 +01:00
amab8901
d5978d11e1 Refactor 2022-12-21 14:28:00 +01:00
Louis Dureuil
869d331680
Clippy fixes after updating Rust to v1.66 2022-12-19 14:17:12 +01:00
Louis Dureuil
ce84a59873
Re-apply some changes from #3132 2022-12-14 20:02:39 +01:00
Tamo
d66bb3a53f rename the two new functions 2022-12-14 17:27:43 +01:00
Tamo
6c0b8edab5 Fix typos
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2022-12-14 17:27:37 +01:00
Tamo
fbbc6eaeca Fix the import of dumps and snapshot.
Some flags were badly applied + the database wrongly deleted when they shouldn't
2022-12-14 17:27:28 +01:00
Kerollmops
7b2f2a4f9c
Do only one convertion to u64 2022-12-13 15:31:55 +01:00
Kerollmops
5d5615ef45
Rename the ReceivePayload error variant 2022-12-13 15:07:35 +01:00
jiangbo212
23c1b223b3 Merge branch 'fix-3037' of github.com:jiangbo212/meilisearch into fix-3037 2022-12-13 10:41:50 +08:00
jiangbo212
87ae0032bf review change 2022-12-13 10:41:43 +08:00
jiangbo212
7c24fea9f2 Merge branch 'main' into fix-3037 2022-12-13 05:16:03 +08:00
jiangbo212
fa46dfb7bb fmt fix 2022-12-12 22:02:56 +08:00
jiangbo212
169682d3ec Merge branch 'main' into fix-3037-new 2022-12-12 21:36:10 +08:00
Tamo
6f1c30b247 Fix the instance-uid in the data.ms
We were writing the instance-uid as bytes instead of string in the data.ms and thus we were unable to parse it later.
Also it was less practical for our user to retrieve it and send it to us.
2022-12-08 18:16:43 +01:00
Colby Allen
2262766494 chore: run fmt nightly on project 2022-12-08 08:31:15 -07:00
Colby Allen
ad2b1467da Renames meilisearch-http to meilisearch 2022-12-08 08:22:53 -07:00