Commit Graph

234 Commits

Author SHA1 Message Date
marin postma
12f6709e1c
move authencation to extractor mod 2021-06-24 16:31:28 +02:00
marin postma
5229f1e220
experimental auth extractor 2021-06-24 16:30:15 +02:00
Tamo
ad8d9a97d6
debug the body of every http request 2021-06-24 11:22:11 +02:00
bors[bot]
8638c9ab77
Merge #232
232: Fix payload size limit r=MarinPostma a=MarinPostma

Fix #223

This was due to the fact that Payload ignores the limit payload size limit. I fixed it by implementing my own `Payload` extractor that checks that the size of the payload is not too large.

I also refactored the `create_app` a bit.

Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-23 16:06:08 +00:00
marin postma
a838238a63
move payload to own module 2021-06-23 16:49:25 +02:00
marin postma
834995b130
clippy + fmt 2021-06-23 16:49:23 +02:00
marin postma
1c13100948
implement custom payload 2021-06-23 16:48:31 +02:00
marin postma
71226feb74
refactor create_app macro 2021-06-23 16:47:15 +02:00
marin postma
322d6b8cfe
fix serialization bug in settings 2021-06-23 15:25:56 +02:00
Clémentine Urquizar
6d24a4744f
Roll back facetsDistribution 2021-06-23 10:04:01 +02:00
marin postma
3456a78552
refactor formatter
share the analyzer instance between the formatter and the
compute_matches function
2021-06-22 18:28:20 +02:00
marin postma
97ef4a6c22
implement matches 2021-06-22 18:12:52 +02:00
marin postma
9cc31c2258
fix get search crop len 2021-06-22 16:01:40 +02:00
marin postma
1e4592dd7e
enable errors in updates 2021-06-21 18:42:47 +02:00
marin postma
763ee521be
fix rebase errors 2021-06-21 12:11:09 +02:00
marin postma
02277ec2cf
reintroduce anyhow 2021-06-21 12:11:06 +02:00
marin postma
439db1aae0
enable response error for search routes 2021-06-21 11:00:14 +02:00
marin postma
8afbb9c462
enable response error for documents routes 2021-06-21 10:59:41 +02:00
marin postma
5c52a1393f
enable response error for settings routes 2021-06-21 10:59:41 +02:00
marin postma
d1550670a8
enable response error for index routes 2021-06-21 10:59:40 +02:00
marin postma
58f9974be4
remove anyhow refs & implement missing errors 2021-06-21 10:59:38 +02:00
Clémentine Urquizar
623b71e81e
Fix clippy errors 2021-06-17 18:02:25 +02:00
Clémentine Urquizar
dc5a3d4a62
Use BTreeSet instead of HashSet 2021-06-16 16:20:10 +02:00
Clémentine Urquizar
65130d9ee7
Change crop_length type from Option(usize) to usize 2021-06-15 17:40:44 +02:00
Clémentine Urquizar
0da8fa115e
Add custom croplength for attributes to crop 2021-06-15 17:40:43 +02:00
marin postma
a780cff8fd
fix clippy warning 2021-06-14 14:53:47 +02:00
Tamo
18d4d6097a
implements the synonyms in transplant 2021-06-14 14:47:51 +02:00
bors[bot]
b119bb4ab0
Merge #197
197: Update milli (v0.3.1) with filterable attributes r=MarinPostma a=curquiza

Fixes #187 and #70
Also fixes #195 

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-14 12:19:42 +00:00
bors[bot]
d65b5db97f
Merge #144 #173
144: Concurrent update run loop (refactor) r=MarinPostma a=MarinPostma

This PR allows multiple request to the update store to be performed concurently (i.e, one can list updates while an updates in being written to the update store).


173: Convert UpdateStatus to legacy meilisearch format r=MarinPostma a=MarinPostma

Returns the update statuses with the same format as legacy meilisearch.

The number of documents in a document addition/deletion is not known before processing, so it is only returned when the update is `processed`.

close #78 

associated milli PR: https://github.com/meilisearch/milli/pull/178


Co-authored-by: marin postma <postma.marin@protonmail.com>
Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-06-14 11:30:44 +00:00
Clémentine Urquizar
88bf867a3e
Rename attributes for faceting into filterable attributes 2021-06-14 13:20:43 +02:00
Clémentine Urquizar
aa04124bfc
Add changes according to milli update 2021-06-14 13:20:37 +02:00
Marin Postma
11c81ab4cb
fix tests 2021-06-14 11:17:49 +02:00
Marin Postma
e8bd5ea4e0
convert UpdateStatus to legacy meilisearch format 2021-06-14 10:21:57 +02:00
bors[bot]
d765397c82
Merge #179
179: Enable filter paramater during search r=MarinPostma a=MarinPostma

This pr makes the necessary changes to transplant in accordance with the specification on filters.

More precisely, it:
- Removes the `filters` parameter
- Renames `facetFilters` to `filter`
- Allows either a string or an array to be passed to the filter param.

It doesn't allow the mixed syntax, that needs to be handled by milli.

close #81
close #140


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-06-14 08:11:30 +00:00
Marin Postma
1c4f0b2ccf
clippy, fmt & tests 2021-05-31 16:03:39 +02:00
tamo
6d837e3e07
the route to create a dump must return a 202 2021-05-11 17:34:34 +02:00
tamo
efca63f9ce
[WIP] rebase on main 2021-05-10 20:25:09 +02:00
tamo
c3552cecdf
WIP rebase on main 2021-05-10 20:24:18 +02:00
tamo
0275b36fb0
[WIP] rebase on main 2021-05-10 20:24:14 +02:00
tamo
e389c088eb
WIP: rebasing on master 2021-05-10 20:20:57 +02:00
Marin Postma
706643dfed
type setting struct 2021-05-10 17:30:09 +02:00
Marin Postma
b192cb9c1f
enable string syntax for the filters 2021-05-06 12:48:31 +02:00
Marin Postma
a717925caa
remove filters, rename facet_filters to filter 2021-05-04 18:20:56 +02:00
bors[bot]
8bc7dd8b03
Merge #143
143: Shared update store r=irevoire a=MarinPostma

This PR changes the updates process so that only one instance of an update store is shared among indexes.

This allows updates to always be processed sequentially without additional synchronization, and fixes the bug where all the first pending update for each index were reported as processing whereas only one was.

EDIT:

I ended having to rewrite the whole `UpdateStore` to allow updates being really queued and processed sequentially in the ordered they were added. For that purpose I created a `pending_queue` that orders the updates by a global update id.

To find the next `update_id` to use, both globally and for each index, I have created another database that contains the next id to use.

Finally, all updates that have been processed (with success or otherwise) are all stores in an `updates` database.

The layout for the keys of these databases are such that it is easy to iterate over the elements for a particular index, and greatly reduces the amount of code to do so, compared to the former implementation.

I have also simplified the locking mechanism for the update store, thanks to the StateLock data structure, that allow both an arbitrary number of readers and a single writer to concurrently access the state. The current state can be either Idle, Processing, or Snapshotting. When an update or snapshotting is ongoing, the process holds the state lock until it is done processing its task. When it is done, it sets bask the state to Idle.

I have made other small improvements here and there, and have let some other for work, such as:
- When creating an update file to hold a request's content, it would be preferable to first create a temporary file, and then atomically persist it when we have written to it. This would simplify the case when there is no data to be written to the file, since we wouldn't have to take care about cleaning after ourselves.
- The logic for content validation must be factored.
- Some more tests related to error handling in the process_pending_update function.
- The issue #159

close #114


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-04-27 18:41:55 +00:00
Marin Postma
bb79a15c04
reenable ranking rules route 2021-04-27 15:29:00 +02:00
Marin Postma
4fe2a13c71
rewrite update store 2021-04-27 15:20:52 +02:00
Marin Postma
ee675eadf1
fix stats 2021-04-27 15:10:55 +02:00
Marin Postma
33830d5ecf
fix snapshots 2021-04-27 15:09:55 +02:00
Clémentine Urquizar
f80ea24d2b
Add tests on every platform and fix clippy errors 2021-04-27 12:42:59 +02:00
Marin Postma
c2461e5066
review fixes 2021-04-26 10:20:46 +02:00
Marin Postma
881b099c8e
add tests 2021-04-20 21:23:34 +02:00
Marin Postma
c6bb36efa5
implement _formated 2021-04-20 21:23:28 +02:00
Marin Postma
6c470cf687
enable distinct-attribute setting route 2021-04-20 11:34:18 +02:00
Clémentine Urquizar
8eceba98d3
Handle star in attributes_to_retrieve 2021-04-19 18:20:19 +02:00
tamo
ec3a08ea0c
remove another unused legacy file 2021-04-15 14:44:43 +02:00
Alexey Shekhirin
f87afbc558
fix(http): commit date & SHA in startup message 2021-04-13 20:16:18 +03:00
Alexey Shekhirin
9eaf048a06
fix(http): use BTreeMap instead of HashMap to preserve stats order 2021-04-13 11:59:07 +03:00
Alexey Shekhirin
ae1655586c
fixes after review 2021-04-09 14:40:48 +03:00
Alexey Shekhirin
dd9eae8c26
feat(http): stats route 2021-04-09 14:34:42 +03:00
tamo
b1962c8e02
remove legacy files from meilisearch that have been replaced by a macro in routes/settings/mod.rs 2021-04-06 16:29:04 +02:00
tamo
40ef9a3c6a
push a first implementation of the stop_words 2021-04-06 16:29:04 +02:00
tamo
73973e2b9e
fix more settings routes 2021-04-01 15:50:45 +02:00
tamo
79c63049d7
update the settings routes 2021-04-01 11:52:26 +02:00
Irevoire
96cffeab1e
update all the response format to be ISO with meilisearch, see #64 2021-04-01 11:43:03 +02:00
Alexey Shekhirin
3d51db5929 fix(ci, http): commit_sha and commit_date in docker builds
chore(ci): cache dependencies in Docker build
2021-03-31 13:56:28 +03:00
Irevoire
7ecefe37da
fix root route 2021-03-19 11:34:54 +01:00
Irevoire
c8b05712fa
return 202 on settings update / reset 2021-03-17 14:44:32 +01:00
bors[bot]
f8ea081df5
Merge #98
98: replace body with json r=curquiza a=MarinPostma



Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-03-16 17:12:30 +00:00
tamo
233c1e304d
use json instead of body when crafting the request 2021-03-16 17:45:59 +01:00
tamo
a268d0e283
return a 200 on health check 2021-03-16 17:42:01 +01:00
mpostma
3c25ab0d50
replace body with json 2021-03-16 16:46:07 +01:00
bors[bot]
c6dbd81823
Merge #90
90: restore version route r=MarinPostma a=MarinPostma

close #74


Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-03-16 13:53:23 +00:00
bors[bot]
2138f54954
Merge #89
89: delete index returns 204 instead of 200 r=curquiza a=MarinPostma

 close #63

Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-03-16 13:01:32 +00:00
mpostma
f4cf96915a
remove guard on add documetn route 2021-03-16 12:04:32 +01:00
mpostma
6a742ee62c
restore version route 2021-03-15 19:11:27 +01:00
mpostma
58fab035bb
delete index returns 204 instead of 200 2021-03-15 18:44:33 +01:00
mpostma
dd324807f9
last review edits + fmt 2021-03-15 18:11:10 +01:00
mpostma
abbea59732
fix clippy warnings 2021-03-15 16:52:05 +01:00
mpostma
01479dcf99
rename name to uid in code 2021-03-15 14:43:47 +01:00
mpostma
55fadd7f87
change facetedAttributes to attributesForFaceting 2021-03-15 13:53:50 +01:00
mpostma
c4846dafca
implement update index 2021-03-12 14:48:43 +01:00
mpostma
66b64c1f80
correct error on settings delete unexisting index 2021-03-11 22:33:31 +01:00
mpostma
53cf500e36
uuid resolver hard state 2021-03-10 18:04:20 +01:00
mpostma
5ecf514d28
restructure project 2021-03-10 13:46:49 +01:00