Commit Graph

3318 Commits

Author SHA1 Message Date
bors[bot]
712abf4c5f
Merge #246
246: Stop logging the no space left on device error r=curquiza a=irevoire

closes #208
@qdequele what do you think of that?
Are there any other errors we need to ignore?

As you can see in the code, once we are in `Sentry` the error has already been converted to a `String` so the only thing we can do to see if we need to send the error or not is to match the `String` against our error message. 
If we have a lot of other logs we want to ignore I would suggest prefixing all the logs with something like:
```
User error: No space left on device
```
So in Sentry, we could just check if the log start by `User error:` and ignore all these errors at once

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-29 08:20:49 +00:00
bors[bot]
261df4b386
Merge #252
252: Fix docker run r=curquiza a=curquiza

Not the most beautiful fix since I cannot update alpine to version 3.14 without being flooded with errors.

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-28 15:47:24 +00:00
bors[bot]
b0f399a51d
Merge #249
249: Use half of the computer threads for the indexing process by default r=Kerollmops a=irevoire

closes #241 
By default, we use only half of the CPU threads when indexing documents; this allows the user to use the search while indexing. Also, the machine will not appear unresponsive when indexing a large batch of documents.

On the special case where a user only has one core, we use it entirely 😄 

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-28 15:25:11 +00:00
Clémentine Urquizar
348d112388
Fix docker run 2021-06-28 16:55:29 +02:00
bors[bot]
5c35a5d9fc
Merge #250
250: Update mini-dashboard to v.0.1.3 r=curquiza a=mdubus

Should fix #245 

Co-authored-by: Morgane Dubus <30866152+mdubus@users.noreply.github.com>
2021-06-28 13:42:34 +00:00
Morgane Dubus
a26bb50d62
Update mini-dashboard to v.0.1.3 2021-06-28 15:13:52 +02:00
Tamo
a59f437ee3
use only half of the computer threads for the indexation by default 2021-06-28 14:35:50 +02:00
Tamo
d74c698adc
stop logging the no space left on device error 2021-06-28 13:59:48 +02:00
bors[bot]
8d8fe8fd29
Merge #248
248: Unused borrow that must be used r=curquiza a=irevoire

I noticed #228 introduced a warning while compiling

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-28 11:53:22 +00:00
Tamo
c1c50f6714
unused borrow that must be used 2021-06-28 13:35:25 +02:00
bors[bot]
d7ca68d8e9
Merge #228
228: Authentication rework r=curquiza a=MarinPostma

In an attempt to fix #201, I ended up rewriting completely the authentication system we use. This is because actix doesn't allow to wrap a single route into a middleware, so we initially put each route into it's own service to use the authentication middleware. Routes are now grouped in resources, fixing #201.

As for the authentication, I decided to take a very different approach, and ditch middleware altogether. Instead, I decided to use actix's [extractor](https://actix.rs/docs/extractors/). `Data` is now wrapped in a `GuardedData<P: Policy, T>` (where `T` is `Data`) in each route. The `Policy` trait, thanks to the `authenticate` method tell if a request is authorized to access the resources in the route. Concretely, before the server starts, it is configured with a `AuthConfig` instance that can either be `AuthConfig::NoAuth` when no auth is required at runtime, or `AuthConfig::Auth(Policies)`, where `Policies` maps the `Policy` type to it singleton instance.

In the current implementation, and this to match the legacy meilisearch behaviour, each policy implementation contains a `HashSet` of token (`Vec<u8>` for now), that represents the user it can authenticate. When starting the program, each key (identified as a user) is given a set of `Policy`, representing its roles. The later is facilitated by the `create_users` macro, like so:

```rust
create_users!(
    policies,
    master_key.as_bytes() => { Admin, Private, Public },
    private_key.as_bytes() => { Private, Public },
    public_key.as_bytes() => { Public }
);
```

This is some groundwork for later development on a full fledged authentication system for meilisearch.


fix #201

Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-28 08:38:59 +00:00
marin postma
01b09c065b
change route to service<resource> 2021-06-24 19:02:28 +02:00
bors[bot]
08104fd49c
Merge #242
242: Fix docker build r=MarinPostma a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-24 15:30:27 +00:00
marin postma
3b601f615a
declare new authentication related errors 2021-06-24 16:53:20 +02:00
Clémentine Urquizar
b1f7fe24f6
Fix docker build 2021-06-24 16:45:51 +02:00
marin postma
fbd58f2eec
clippy 2021-06-24 16:36:22 +02:00
marin postma
79fc3bb84e
fmt 2021-06-24 16:36:22 +02:00
marin postma
8e4928c7ea
fix tests 2021-06-24 16:36:22 +02:00
marin postma
d078cbf39b
remove authentication middleware 2021-06-24 16:36:21 +02:00
marin postma
561596d8bc
update stats routes 2021-06-24 16:36:18 +02:00
marin postma
549b489c8a
update settings routes 2021-06-24 16:35:48 +02:00
marin postma
1e9f374ff8
update running route 2021-06-24 16:35:12 +02:00
marin postma
817fcfdd88
update keys route 2021-06-24 16:35:12 +02:00
marin postma
fab50256bc
update index routes 2021-06-24 16:35:04 +02:00
marin postma
b044608b25
update health route 2021-06-24 16:32:45 +02:00
marin postma
ce4fb8ce20
update dump route 2021-06-24 16:32:43 +02:00
marin postma
adf91d286b
update documents and search routes 2021-06-24 16:32:15 +02:00
marin postma
0c1c7a3dd9
implement authentication policies 2021-06-24 16:31:30 +02:00
marin postma
5b71751391
policies macros 2021-06-24 16:31:30 +02:00
marin postma
12f6709e1c
move authencation to extractor mod 2021-06-24 16:31:28 +02:00
marin postma
5229f1e220
experimental auth extractor 2021-06-24 16:30:15 +02:00
bors[bot]
b6ca7929eb
Merge #240
240: Rework error messages r=irevoire a=MarinPostma

Simplify the error messages, and make them more compliant with legacy Meilisearch.

Basically, stop composing the messages, and simply forward the message of inner errors.


Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-24 11:36:11 +00:00
bors[bot]
43204ca67b
Merge #230
230: Logs r=MarinPostma a=irevoire

closes #193 

Since we can't really print the body of requests in actix-web, I logged the parameters of every request and what we were returning to the client.

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-24 09:23:24 +00:00
Tamo
ad8d9a97d6
debug the body of every http request 2021-06-24 11:22:11 +02:00
Tamo
36f32f58d4
add the log_level variable to the cli and reduce the log level of milli and grenad 2021-06-24 11:20:52 +02:00
Tamo
b4fd4212ad
reduce the log level of some info! 2021-06-24 11:20:52 +02:00
marin postma
a1d34faaad
decompose error messages 2021-06-24 10:57:28 +02:00
bors[bot]
a2368db154
Merge #239
239: Bump milli to 0.6.0 r=MarinPostma a=MarinPostma

fix #231


Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-24 08:08:41 +00:00
bors[bot]
381e07b7b6
Merge #1415
1415: Fix README.md typos r=curquiza a=dichotommy

Just fixing some typos and such.
Kanji -> Hanzi
Kanji refers only to the Japanese versions of Chinese characters, and since we don't have a Japanese tokenization pipeline I think it could be misunderstood.

Co-authored-by: Tommy <68053732+dichotommy@users.noreply.github.com>
2021-06-24 07:46:28 +00:00
marin postma
74bb748a4e
bump milli to 0.6.0 2021-06-23 18:40:19 +02:00
Tommy
09113fc73c
Update README.md
Just fixing some typos and such.
Kanji refers only to Japanese versions of the Chinese characters, and since we don't have a Japanese tokenization pipeline I think it could be misleading.
2021-06-23 18:30:48 +02:00
bors[bot]
8638c9ab77
Merge #232
232: Fix payload size limit r=MarinPostma a=MarinPostma

Fix #223

This was due to the fact that Payload ignores the limit payload size limit. I fixed it by implementing my own `Payload` extractor that checks that the size of the payload is not too large.

I also refactored the `create_app` a bit.

Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-23 16:06:08 +00:00
bors[bot]
b676b10cfe
Merge #238
238: Fix settings subroutes get r=MarinPostma a=MarinPostma

Fix #225 

Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-23 15:45:50 +00:00
marin postma
f68c257452
move flush in write_to_file function 2021-06-23 16:49:25 +02:00
marin postma
880fc069bd
remove dbg 2021-06-23 16:49:25 +02:00
marin postma
a838238a63
move payload to own module 2021-06-23 16:49:25 +02:00
marin postma
834995b130
clippy + fmt 2021-06-23 16:49:23 +02:00
marin postma
b000ae7614
remove file if write to update file fails 2021-06-23 16:48:33 +02:00
marin postma
f62779671b
change error message for payload size limit 2021-06-23 16:48:33 +02:00
marin postma
4b292c6e9b
add payload limit to app config 2021-06-23 16:48:33 +02:00