Commit Graph

3772 Commits

Author SHA1 Message Date
Kerollmops
8509243e68
Implement the status and type filtering on the tasks route 2022-05-31 11:33:20 +02:00
Kerollmops
3684c822f1
Add indexUid filtering on the /tasks route 2022-05-31 11:33:20 +02:00
Kerollmops
80f7d87356
Remove the /indexes/:indexUid/tasks/... routes 2022-05-31 11:33:20 +02:00
Kerollmops
d2f457a076
Rename the uid to taskUid in asynchronous response 2022-05-31 11:33:20 +02:00
Kerollmops
e5ef5a6f9c
Remove an unused updates.rs file 2022-05-31 11:33:19 +02:00
bors[bot]
5450fecaef
Merge #2444
2444: add boilerplate for dump v5 r=MarinPostma a=MarinPostma

add the boilerplate files for dump v5


Co-authored-by: ad hoc <postma.marin@protonmail.com>
Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-05-31 08:56:52 +00:00
ManyTheFish
deba0cc096 Make v4::load_dump copy each part a the dump 2022-05-31 10:24:44 +02:00
ad hoc
26e7bdf702
add boilerplate for dump v5 2022-05-30 17:25:29 +02:00
bors[bot]
3441cc6c36
Merge #2410
2410: Make dump a task r=Kerollmops a=MarinPostma

This PR transforms the dump task into a proper task.
The `GET /dumps/:dump_uid` is removed.


Some changes were made to make this work, and a bit a refactoring was necessary.
- The `dump_actor` module has been renamed do `dumps` and moved to the root
- There isn't a `DumpActor` anymore, and the dump process is handled by the `DumpHandler`.
- The `TaskPerformer` is renamed to `BatchHandler`
- The `BatchHandler` trait no longer has a `perform_job` method, but instead has a `accept` method returning whether a handler can proccess a batch
- The scheduler now accept a list of `BatchHandler`, and iterates trhough them until it finds one to accept the current batch.
- `Job` doesn't exist anymore, and everything in now inside of the `BatchContent` enum.
- The `Vec<TaskId>` from `Batch` is replaced with a `BatchContent` enum which hints at the content.
- The Scheduler is slightly modified to accept batch, and prioritize them before regular tasks.
- The `TaskList` are not identified by a `String` representing the index uid anymore, but by a `TaskListIdentifier` which also works for dumps which are not targeting any specific indexes.
- The `GET /dump/:dump_id` no longer exists
- `DumpActorError` is renamed to `DumpError`


close #2410 

Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-05-30 14:09:43 +00:00
bors[bot]
c7711c7816
Merge #2429
2429: Send the analytics to `telemetry.meilisearch.com` instead of segment r=MarinPostma a=irevoire

Fix #2425

Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-05-30 13:18:55 +00:00
Irevoire
d47b997120
chore(analytics): update the url used to send our analytics 2022-05-30 15:13:10 +02:00
ad hoc
1e310ecc7d
fix typo in docstring
Co-authored-by: Tamo <tamo@meilisearch.com>
2022-05-30 14:34:49 +02:00
ad hoc
4cb2c6ef1e
use map_or instead of map + unwrap_or 2022-05-30 12:30:15 +02:00
ad hoc
a9ef399a6b
processing::Nothing return BatchContent::Empty instead of panic 2022-05-26 12:04:27 +02:00
ad hoc
5a2972fc19
use TaskEvent method instead of variants in BatchHandler impl 2022-05-26 11:51:58 +02:00
0x0x1
ba51ca83ec
Update docker volume path
Makes docker volume same as Dockerfile
2022-05-26 10:29:27 +08:00
ad hoc
1647ca3c1f
fix clipy warnings 2022-05-25 15:07:52 +02:00
ad hoc
74a1f88d88
add test for dump processing order 2022-05-25 14:57:36 +02:00
ad hoc
f58507379a
fix dump priority in scheduler 2022-05-25 14:50:14 +02:00
ad hoc
6b2016b350
remove typo in BatchContent variant 2022-05-25 14:39:07 +02:00
ad hoc
3015265bde
remove useless dump errors 2022-05-25 14:37:10 +02:00
ad hoc
49d8fadb52
test dump handler 2022-05-25 14:32:12 +02:00
ad hoc
127171c812
impl Default on Processing 2022-05-25 14:10:39 +02:00
bors[bot]
67b6f4340a
Merge #2422
2422: Update url of movies.json r=curquiza a=0x0x1

URL `https://bit.ly/2PAcw9l` is a notion site.

Co-authored-by: 0x0x1 <101086451+0x0x1@users.noreply.github.com>
2022-05-25 11:21:56 +00:00
ad hoc
986a99296d
remove useless dump test 2022-05-25 11:25:11 +02:00
ad hoc
92d86ce6aa
add tests to IndexResolver BatchHandler 2022-05-25 11:13:36 +02:00
ad hoc
3c85b29865
add doc to BatchHandler 2022-05-25 11:13:35 +02:00
ad hoc
8349f38197
remove unused file 2022-05-25 11:13:35 +02:00
ad hoc
64654ef7c3
rename batch_handler to handler 2022-05-25 11:13:35 +02:00
ad hoc
0f9c134114
fix tests 2022-05-25 11:13:35 +02:00
ad hoc
7b47e4e87a
snapshot batch handler 2022-05-25 11:13:35 +02:00
ad hoc
8743d73973
move DumpHandler to own module 2022-05-25 11:13:35 +02:00
ad hoc
f0aceb4fba
remove unused files 2022-05-25 11:13:35 +02:00
ad hoc
61035a3ea4
create dump v5 2022-05-25 11:13:34 +02:00
ad hoc
4778884105
remove dump status route 2022-05-25 11:13:34 +02:00
ad hoc
57fde30b91
handle dump 2022-05-25 11:13:34 +02:00
ad hoc
56eb2907c9
dump indexes 2022-05-25 11:13:34 +02:00
ad hoc
414d0907ce
register dump handler 2022-05-25 11:13:34 +02:00
ad hoc
60a8249de6
add dump batch handler 2022-05-25 11:13:34 +02:00
ad hoc
46cdc17701
make scheduler accept multiple batch handlers 2022-05-25 11:13:34 +02:00
ad hoc
6a0231cb28
perform dump method 2022-05-25 11:13:33 +02:00
ad hoc
7fa3eb1003
register dump tasks 2022-05-25 11:13:33 +02:00
ad hoc
2f0625a984
register and insert dump task in scheduler 2022-05-25 11:13:33 +02:00
ad hoc
737b891a41
introduce Dump TaskListIdentifier variant 2022-05-25 11:13:33 +02:00
ad hoc
5a5066023b
introduce TaskListIdentifier 2022-05-25 11:13:33 +02:00
ad hoc
aa50acb031
make Task index_uid an option
Not all task relate to an index. Tasks that don't have an index_uid set
to None
2022-05-25 11:13:32 +02:00
bors[bot]
9935db86c7
Merge #2424
2424: Uncomment clippy from the ci check r=curquiza a=irevoire

The issue has been fixed in the latest release of rust. See https://github.com/rust-lang/rust-clippy/issues/8662
Fix #2305


Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-05-24 17:50:55 +00:00
Irevoire
f65116b208
chore(ci): uncomment clippy from the ci check
The issue has been fixed in the latest release of rust. See https://github.com/rust-lang/rust-clippy/issues/8662
Fix #2305
2022-05-24 15:03:11 +02:00
bors[bot]
341756a0eb
Merge #2357
2357: chore(dump): add dump tests r=Kerollmops a=irevoire

Add tests on the import of dump v1, v2, v3 and v4.

Since the dumps are slow to decompress, I made the `flate2` crate always compile in optimized.
And since they're also slow to index, I also made the `milli` crate always compile in optimized. What do you think of this `@MarinPostma?`
Should we keep milli unoptimized in case it could help us debug some things? 👀 

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-05-24 12:24:29 +00:00
Tamo
5f0e9b63d2
chore(dump): add tests 2022-05-24 14:21:56 +02:00