Commit Graph

2832 Commits

Author SHA1 Message Date
Clémentine Urquizar
73db1b3822
Merge remote-tracking branch 'transplant/main' 2021-06-30 15:30:08 +02:00
Clémentine Urquizar
abca68bf24
Remove legacy source code 2021-06-30 15:20:17 +02:00
bors[bot]
eeca841a21
Merge #259
259: Run rustfmt one the whole project and add it to the CI r=curquiza a=irevoire

Since there is currently no other PR modifying the code, I think it's a good time to reformat everything and add rustfmt to the ci.

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-30 11:55:30 +00:00
Tamo
3a9b86ad55
add rustfmt to bors 2021-06-30 10:49:10 +02:00
bors[bot]
f1cc141f6c
Merge #258
258: Use rustls instead of openssl r=curquiza a=irevoire

I also removed all the `default-features` of reqwest since we are only using the JSON one.
Fix #255

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-29 14:42:25 +00:00
Tamo
3011209e28
bump alpine version 2021-06-29 16:36:41 +02:00
Tamo
29bf6a8d42
run rustfmt one the whole project and add it to the CI 2021-06-29 15:25:18 +02:00
Tamo
c282466750
remove the libressl dependency from our docker file 2021-06-29 15:22:11 +02:00
bors[bot]
de9ea94f57
Merge #257
257: Accept no content-type as json r=curquiza a=irevoire

closes #253 

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-29 12:54:33 +00:00
Tamo
fe7640555d
fix the content-type 2021-06-29 13:16:56 +02:00
Tamo
ec809ca487
use rustls instead of openssl and remove all default-features of reqwest 2021-06-29 13:07:40 +02:00
Tamo
1dc99ea451
accept no content-type as json 2021-06-29 11:59:25 +02:00
bors[bot]
f12ace3fbf
Merge #256
256: Update heed and milli r=irevoire a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-29 08:49:22 +00:00
Clémentine Urquizar
c09e610bb5
Update heed and milli 2021-06-29 10:25:47 +02:00
bors[bot]
712abf4c5f
Merge #246
246: Stop logging the no space left on device error r=curquiza a=irevoire

closes #208
@qdequele what do you think of that?
Are there any other errors we need to ignore?

As you can see in the code, once we are in `Sentry` the error has already been converted to a `String` so the only thing we can do to see if we need to send the error or not is to match the `String` against our error message. 
If we have a lot of other logs we want to ignore I would suggest prefixing all the logs with something like:
```
User error: No space left on device
```
So in Sentry, we could just check if the log start by `User error:` and ignore all these errors at once

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-29 08:20:49 +00:00
bors[bot]
261df4b386
Merge #252
252: Fix docker run r=curquiza a=curquiza

Not the most beautiful fix since I cannot update alpine to version 3.14 without being flooded with errors.

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-28 15:47:24 +00:00
bors[bot]
b0f399a51d
Merge #249
249: Use half of the computer threads for the indexing process by default r=Kerollmops a=irevoire

closes #241 
By default, we use only half of the CPU threads when indexing documents; this allows the user to use the search while indexing. Also, the machine will not appear unresponsive when indexing a large batch of documents.

On the special case where a user only has one core, we use it entirely 😄 

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-28 15:25:11 +00:00
Clémentine Urquizar
348d112388
Fix docker run 2021-06-28 16:55:29 +02:00
bors[bot]
5c35a5d9fc
Merge #250
250: Update mini-dashboard to v.0.1.3 r=curquiza a=mdubus

Should fix #245 

Co-authored-by: Morgane Dubus <30866152+mdubus@users.noreply.github.com>
2021-06-28 13:42:34 +00:00
Morgane Dubus
a26bb50d62
Update mini-dashboard to v.0.1.3 2021-06-28 15:13:52 +02:00
Tamo
a59f437ee3
use only half of the computer threads for the indexation by default 2021-06-28 14:35:50 +02:00
Tamo
d74c698adc
stop logging the no space left on device error 2021-06-28 13:59:48 +02:00
bors[bot]
8d8fe8fd29
Merge #248
248: Unused borrow that must be used r=curquiza a=irevoire

I noticed #228 introduced a warning while compiling

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-28 11:53:22 +00:00
Tamo
c1c50f6714
unused borrow that must be used 2021-06-28 13:35:25 +02:00
bors[bot]
d7ca68d8e9
Merge #228
228: Authentication rework r=curquiza a=MarinPostma

In an attempt to fix #201, I ended up rewriting completely the authentication system we use. This is because actix doesn't allow to wrap a single route into a middleware, so we initially put each route into it's own service to use the authentication middleware. Routes are now grouped in resources, fixing #201.

As for the authentication, I decided to take a very different approach, and ditch middleware altogether. Instead, I decided to use actix's [extractor](https://actix.rs/docs/extractors/). `Data` is now wrapped in a `GuardedData<P: Policy, T>` (where `T` is `Data`) in each route. The `Policy` trait, thanks to the `authenticate` method tell if a request is authorized to access the resources in the route. Concretely, before the server starts, it is configured with a `AuthConfig` instance that can either be `AuthConfig::NoAuth` when no auth is required at runtime, or `AuthConfig::Auth(Policies)`, where `Policies` maps the `Policy` type to it singleton instance.

In the current implementation, and this to match the legacy meilisearch behaviour, each policy implementation contains a `HashSet` of token (`Vec<u8>` for now), that represents the user it can authenticate. When starting the program, each key (identified as a user) is given a set of `Policy`, representing its roles. The later is facilitated by the `create_users` macro, like so:

```rust
create_users!(
    policies,
    master_key.as_bytes() => { Admin, Private, Public },
    private_key.as_bytes() => { Private, Public },
    public_key.as_bytes() => { Public }
);
```

This is some groundwork for later development on a full fledged authentication system for meilisearch.


fix #201

Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-28 08:38:59 +00:00
marin postma
01b09c065b
change route to service<resource> 2021-06-24 19:02:28 +02:00
bors[bot]
08104fd49c
Merge #242
242: Fix docker build r=MarinPostma a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-24 15:30:27 +00:00
marin postma
3b601f615a
declare new authentication related errors 2021-06-24 16:53:20 +02:00
Clémentine Urquizar
b1f7fe24f6
Fix docker build 2021-06-24 16:45:51 +02:00
marin postma
fbd58f2eec
clippy 2021-06-24 16:36:22 +02:00
marin postma
79fc3bb84e
fmt 2021-06-24 16:36:22 +02:00
marin postma
8e4928c7ea
fix tests 2021-06-24 16:36:22 +02:00
marin postma
d078cbf39b
remove authentication middleware 2021-06-24 16:36:21 +02:00
marin postma
561596d8bc
update stats routes 2021-06-24 16:36:18 +02:00
marin postma
549b489c8a
update settings routes 2021-06-24 16:35:48 +02:00
marin postma
1e9f374ff8
update running route 2021-06-24 16:35:12 +02:00
marin postma
817fcfdd88
update keys route 2021-06-24 16:35:12 +02:00
marin postma
fab50256bc
update index routes 2021-06-24 16:35:04 +02:00
marin postma
b044608b25
update health route 2021-06-24 16:32:45 +02:00
marin postma
ce4fb8ce20
update dump route 2021-06-24 16:32:43 +02:00
marin postma
adf91d286b
update documents and search routes 2021-06-24 16:32:15 +02:00
marin postma
0c1c7a3dd9
implement authentication policies 2021-06-24 16:31:30 +02:00
marin postma
5b71751391
policies macros 2021-06-24 16:31:30 +02:00
marin postma
12f6709e1c
move authencation to extractor mod 2021-06-24 16:31:28 +02:00
marin postma
5229f1e220
experimental auth extractor 2021-06-24 16:30:15 +02:00
bors[bot]
b6ca7929eb
Merge #240
240: Rework error messages r=irevoire a=MarinPostma

Simplify the error messages, and make them more compliant with legacy Meilisearch.

Basically, stop composing the messages, and simply forward the message of inner errors.


Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-24 11:36:11 +00:00
bors[bot]
43204ca67b
Merge #230
230: Logs r=MarinPostma a=irevoire

closes #193 

Since we can't really print the body of requests in actix-web, I logged the parameters of every request and what we were returning to the client.

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-24 09:23:24 +00:00
Tamo
ad8d9a97d6
debug the body of every http request 2021-06-24 11:22:11 +02:00
Tamo
36f32f58d4
add the log_level variable to the cli and reduce the log level of milli and grenad 2021-06-24 11:20:52 +02:00
Tamo
b4fd4212ad
reduce the log level of some info! 2021-06-24 11:20:52 +02:00