Commit Graph

7550 Commits

Author SHA1 Message Date
Kerollmops
c11d21879a
Introduce tasks limit and after to the tasks route 2022-06-01 13:26:36 +02:00
bors[bot]
d6dd234914
Merge #2434
2434: Update docker volume path r=curquiza a=0x0x1

Currently, the args `$(pwd)/data.ms:/data.ms` cannot share data from the container, makes docker volume same as the [Dockerfile](67b6f4340a/Dockerfile (L40)) to fixing it.



Co-authored-by: 0x0x1 <101086451+0x0x1@users.noreply.github.com>
Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
2022-06-01 10:31:27 +00:00
Clémentine Urquizar - curqui
4970525541
Update README.md
Co-authored-by: Tamo <irevoire@protonmail.ch>
2022-06-01 12:29:36 +02:00
Kerollmops
461b91fd13
Introduce the fetch_unfinished_tasks function to fetch tasks 2022-06-01 12:09:52 +02:00
Kerollmops
004c8b6be3
Add the new limit and after fields in the dump tests 2022-06-01 12:09:52 +02:00
Kerollmops
9d5cc88cd5
Implement the seek-based tasks list pagination 2022-06-01 12:09:52 +02:00
bors[bot]
d22f07f5b2
Merge #2448
2448: docs(security): Fix `Supported` r=MarinPostma a=ryanrussell

Signed-off-by: Ryan Russell <git@ryanrussell.org>

# Pull Request

## What does this PR do?
- Fix typo in security `Suported` -> `Supported`

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?


Co-authored-by: Ryan Russell <git@ryanrussell.org>
2022-05-31 19:59:37 +00:00
bors[bot]
e81c7aa2e6
Merge #2423
2423: Paginate the index resource r=MarinPostma a=irevoire

Fix #2373


Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-05-31 19:25:25 +00:00
Ryan Russell
39db6ea42b
docs(security): Fix Supported
Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-05-31 14:21:34 -05:00
bors[bot]
47007fa71b
Merge #2446
2446: rename Succeded to Succeeded r=irevoire a=MarinPostma

this pr renames `TaskEvent::Succeded` to `TaskEvent::Succeeded` and apply the migration to the dumps


Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-05-31 18:27:02 +00:00
Irevoire
627f13df85
feat(http): paginate the index resource
Fix #2373
2022-05-31 18:11:45 +02:00
bors[bot]
97c14f6fcc
Merge #2427
2427: Update the documents resource r=MarinPostma a=irevoire

- Return Documents API resources on `/documents` in an array in the results field.
- Add limit, offset, and total in the response body.
- Rename `attributesToRetrieve` into `fields` (only for the `/documents` endpoints, not for the `/search` ones).
- The `displayedAttributes` settings do not impact anymore the displayed fields returned in the `/documents` endpoints. These settings only impact the `/search` endpoint.
- make the `/document/:uid` route accept the `fields` query parameter

Fix #2372


Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-05-31 15:33:42 +00:00
ad hoc
446f1f31e0
rename Succeded to Succeeded 2022-05-31 17:22:37 +02:00
Irevoire
ddad6cc069
feat(http): update the documents resource
- Return Documents API resources on `/documents` in an array in the the results field.
- Add limit, offset and total in the response body.
- Rename `attributesToRetrieve` into `fields` (only for the `/documents` endpoints, not for the `/search` ones).
- The `displayedAttributes` settings does not impact anymore the displayed fields returned in the `/documents` endpoints. These settings only impacts the `/search` endpoint.

Fix #2372
2022-05-31 16:40:40 +02:00
bors[bot]
ab39df9693
Merge #2399
2399: Update the tasks endpoints r=MarinPostma a=Kerollmops

This PR wraps all the changes related to the `tasks` endpoints, it is related to https://github.com/meilisearch/meilisearch/issues/2377 but doesn't close it. I will create a new PR to work on [the seek-based pagination](https://github.com/meilisearch/specifications/pull/115).

I wanted to do something cool with Github: being able to merge multiple PR in this one, to help review changes one by one, unfortunately, Github doesn't allow creating empty PRs. I also struggled with git itself when it comes to merging things in the right order, so I decided that I would add all of the changes in this single PR. I will list the changes and references to the specs here.

 - [x] Tasks statuses and types must be case insensitive
 - [x] Tasks statuses, types and indexUid must accept the `*` selector
 - [ ] Rename the `TaskDetails` struct fields

## Changes

- [ ] Add seek-based pagination following [the spec](https://github.com/meilisearch/specifications/pull/115) 
- [x] Add filtering on the `/tasks` endpoint following [this spec](https://github.com/meilisearch/specifications/pull/116)
  - [x] Add filtering capabilities on `type`, `status` and `indexUid` for `GET` `task` lists endpoints.
  - [x] It is possible to specify several values for a filter using the `,` character. e.g. `?status=enqueued,processing`
  - [x] Between two different filters, an AND operation is applied. e.g. `?status=enqueued&type=indexCreation` is equivalent to `status=enqueued AND type = indexCreation`
- [x] Remove `GET /indexes/:indexUid/tasks`. It can be replaced by `GET /tasks?indexUid=:indexUid`
- [x] Remove `GET /indexes/:indexUid/tasks/:taskUid`.
- [x] Rename `uid` to `taskUid` in the `202 - Accepted` task response return by every asynchronous tasks (ex: index creation, document addition...)
- [x] Rename some task properties
  - [x] `documentPartial`-> `documentAdditionOrUpdate`
  - [x] `documentAddition`-> `documentAdditionOrUpdate`
  - [x] `clearAll` -> `documentDeletion` 

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-05-31 09:40:40 +00:00
Kerollmops
1465b5e0ff
Refactorize the tasks filters by moving the match inside 2022-05-31 11:33:21 +02:00
Kerollmops
8800b348f0
Implement the StarOr on all the tasks filters 2022-05-31 11:33:21 +02:00
Kerollmops
082d6b89ff
Make the StarOrIndexUid Generic and call it StarOr 2022-05-31 11:33:21 +02:00
Kerollmops
b82c86c8f5
Allow users to filter indexUid with a * 2022-05-31 11:33:20 +02:00
Kerollmops
36d94257d8
Make clippy happy 2022-05-31 11:33:20 +02:00
Kerollmops
3f80468f18
Rename the Tasks Types 2022-05-31 11:33:20 +02:00
Kerollmops
8509243e68
Implement the status and type filtering on the tasks route 2022-05-31 11:33:20 +02:00
Kerollmops
3684c822f1
Add indexUid filtering on the /tasks route 2022-05-31 11:33:20 +02:00
Kerollmops
80f7d87356
Remove the /indexes/:indexUid/tasks/... routes 2022-05-31 11:33:20 +02:00
Kerollmops
d2f457a076
Rename the uid to taskUid in asynchronous response 2022-05-31 11:33:20 +02:00
Kerollmops
e5ef5a6f9c
Remove an unused updates.rs file 2022-05-31 11:33:19 +02:00
bors[bot]
5450fecaef
Merge #2444
2444: add boilerplate for dump v5 r=MarinPostma a=MarinPostma

add the boilerplate files for dump v5


Co-authored-by: ad hoc <postma.marin@protonmail.com>
Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-05-31 08:56:52 +00:00
ManyTheFish
deba0cc096 Make v4::load_dump copy each part a the dump 2022-05-31 10:24:44 +02:00
ad hoc
26e7bdf702
add boilerplate for dump v5 2022-05-30 17:25:29 +02:00
bors[bot]
3441cc6c36
Merge #2410
2410: Make dump a task r=Kerollmops a=MarinPostma

This PR transforms the dump task into a proper task.
The `GET /dumps/:dump_uid` is removed.


Some changes were made to make this work, and a bit a refactoring was necessary.
- The `dump_actor` module has been renamed do `dumps` and moved to the root
- There isn't a `DumpActor` anymore, and the dump process is handled by the `DumpHandler`.
- The `TaskPerformer` is renamed to `BatchHandler`
- The `BatchHandler` trait no longer has a `perform_job` method, but instead has a `accept` method returning whether a handler can proccess a batch
- The scheduler now accept a list of `BatchHandler`, and iterates trhough them until it finds one to accept the current batch.
- `Job` doesn't exist anymore, and everything in now inside of the `BatchContent` enum.
- The `Vec<TaskId>` from `Batch` is replaced with a `BatchContent` enum which hints at the content.
- The Scheduler is slightly modified to accept batch, and prioritize them before regular tasks.
- The `TaskList` are not identified by a `String` representing the index uid anymore, but by a `TaskListIdentifier` which also works for dumps which are not targeting any specific indexes.
- The `GET /dump/:dump_id` no longer exists
- `DumpActorError` is renamed to `DumpError`


close #2410 

Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-05-30 14:09:43 +00:00
bors[bot]
c7711c7816
Merge #2429
2429: Send the analytics to `telemetry.meilisearch.com` instead of segment r=MarinPostma a=irevoire

Fix #2425

Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-05-30 13:18:55 +00:00
Irevoire
d47b997120
chore(analytics): update the url used to send our analytics 2022-05-30 15:13:10 +02:00
ad hoc
1e310ecc7d
fix typo in docstring
Co-authored-by: Tamo <tamo@meilisearch.com>
2022-05-30 14:34:49 +02:00
ad hoc
4cb2c6ef1e
use map_or instead of map + unwrap_or 2022-05-30 12:30:15 +02:00
bors[bot]
582930dbbb
Merge #538
538: speedup exact words r=Kerollmops a=MarinPostma

This PR make `exact_words` return an `Option` instead of an empty set, since set creation is costly, as noticed by `@kerollmops.`

I was not convinces that this was the cause for all of the performance drop we measured, and then realized that methods that initialized it were called recursively which caused initialization times to add up. While the first fix solves the issue when not using exact words, using exact word remained way more expensive that it should be. To address this issue, the exact words are cached into the `Context`, so they are only initialized once.


Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-05-30 08:20:34 +00:00
ad hoc
a9ef399a6b
processing::Nothing return BatchContent::Empty instead of panic 2022-05-26 12:04:27 +02:00
ad hoc
5a2972fc19
use TaskEvent method instead of variants in BatchHandler impl 2022-05-26 11:51:58 +02:00
0x0x1
ba51ca83ec
Update docker volume path
Makes docker volume same as Dockerfile
2022-05-26 10:29:27 +08:00
ad hoc
1647ca3c1f
fix clipy warnings 2022-05-25 15:07:52 +02:00
ad hoc
74a1f88d88
add test for dump processing order 2022-05-25 14:57:36 +02:00
ad hoc
f58507379a
fix dump priority in scheduler 2022-05-25 14:50:14 +02:00
ad hoc
6b2016b350
remove typo in BatchContent variant 2022-05-25 14:39:07 +02:00
ad hoc
3015265bde
remove useless dump errors 2022-05-25 14:37:10 +02:00
ad hoc
49d8fadb52
test dump handler 2022-05-25 14:32:12 +02:00
ad hoc
127171c812
impl Default on Processing 2022-05-25 14:10:39 +02:00
bors[bot]
67b6f4340a
Merge #2422
2422: Update url of movies.json r=curquiza a=0x0x1

URL `https://bit.ly/2PAcw9l` is a notion site.

Co-authored-by: 0x0x1 <101086451+0x0x1@users.noreply.github.com>
2022-05-25 11:21:56 +00:00
ad hoc
986a99296d
remove useless dump test 2022-05-25 11:25:11 +02:00
ad hoc
92d86ce6aa
add tests to IndexResolver BatchHandler 2022-05-25 11:13:36 +02:00
ad hoc
3c85b29865
add doc to BatchHandler 2022-05-25 11:13:35 +02:00
ad hoc
8349f38197
remove unused file 2022-05-25 11:13:35 +02:00