249 Commits

Author SHA1 Message Date
tamo
6d837e3e07
the route to create a dump must return a 202 2021-05-11 17:34:34 +02:00
tamo
efca63f9ce
[WIP] rebase on main 2021-05-10 20:25:09 +02:00
tamo
c3552cecdf
WIP rebase on main 2021-05-10 20:24:18 +02:00
tamo
0275b36fb0
[WIP] rebase on main 2021-05-10 20:24:14 +02:00
tamo
e389c088eb
WIP: rebasing on master 2021-05-10 20:20:57 +02:00
Marin Postma
706643dfed
type setting struct 2021-05-10 17:30:09 +02:00
Marin Postma
b192cb9c1f
enable string syntax for the filters 2021-05-06 12:48:31 +02:00
Marin Postma
a717925caa
remove filters, rename facet_filters to filter 2021-05-04 18:20:56 +02:00
bors[bot]
8bc7dd8b03
Merge #143
143: Shared update store r=irevoire a=MarinPostma

This PR changes the updates process so that only one instance of an update store is shared among indexes.

This allows updates to always be processed sequentially without additional synchronization, and fixes the bug where all the first pending update for each index were reported as processing whereas only one was.

EDIT:

I ended having to rewrite the whole `UpdateStore` to allow updates being really queued and processed sequentially in the ordered they were added. For that purpose I created a `pending_queue` that orders the updates by a global update id.

To find the next `update_id` to use, both globally and for each index, I have created another database that contains the next id to use.

Finally, all updates that have been processed (with success or otherwise) are all stores in an `updates` database.

The layout for the keys of these databases are such that it is easy to iterate over the elements for a particular index, and greatly reduces the amount of code to do so, compared to the former implementation.

I have also simplified the locking mechanism for the update store, thanks to the StateLock data structure, that allow both an arbitrary number of readers and a single writer to concurrently access the state. The current state can be either Idle, Processing, or Snapshotting. When an update or snapshotting is ongoing, the process holds the state lock until it is done processing its task. When it is done, it sets bask the state to Idle.

I have made other small improvements here and there, and have let some other for work, such as:
- When creating an update file to hold a request's content, it would be preferable to first create a temporary file, and then atomically persist it when we have written to it. This would simplify the case when there is no data to be written to the file, since we wouldn't have to take care about cleaning after ourselves.
- The logic for content validation must be factored.
- Some more tests related to error handling in the process_pending_update function.
- The issue #159

close #114


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-04-27 18:41:55 +00:00
Marin Postma
bb79a15c04
reenable ranking rules route 2021-04-27 15:29:00 +02:00
Marin Postma
4fe2a13c71
rewrite update store 2021-04-27 15:20:52 +02:00
Marin Postma
ee675eadf1
fix stats 2021-04-27 15:10:55 +02:00
Marin Postma
33830d5ecf
fix snapshots 2021-04-27 15:09:55 +02:00
Clémentine Urquizar
f80ea24d2b
Add tests on every platform and fix clippy errors 2021-04-27 12:42:59 +02:00
Marin Postma
c2461e5066
review fixes 2021-04-26 10:20:46 +02:00
Marin Postma
881b099c8e
add tests 2021-04-20 21:23:34 +02:00
Marin Postma
c6bb36efa5
implement _formated 2021-04-20 21:23:28 +02:00
Marin Postma
6c470cf687
enable distinct-attribute setting route 2021-04-20 11:34:18 +02:00
Clémentine Urquizar
8eceba98d3
Handle star in attributes_to_retrieve 2021-04-19 18:20:19 +02:00
tamo
ec3a08ea0c
remove another unused legacy file 2021-04-15 14:44:43 +02:00
Alexey Shekhirin
f87afbc558
fix(http): commit date & SHA in startup message 2021-04-13 20:16:18 +03:00
Alexey Shekhirin
9eaf048a06
fix(http): use BTreeMap instead of HashMap to preserve stats order 2021-04-13 11:59:07 +03:00
Alexey Shekhirin
ae1655586c
fixes after review 2021-04-09 14:40:48 +03:00
Alexey Shekhirin
dd9eae8c26
feat(http): stats route 2021-04-09 14:34:42 +03:00
tamo
b1962c8e02
remove legacy files from meilisearch that have been replaced by a macro in routes/settings/mod.rs 2021-04-06 16:29:04 +02:00
tamo
40ef9a3c6a
push a first implementation of the stop_words 2021-04-06 16:29:04 +02:00
tamo
73973e2b9e
fix more settings routes 2021-04-01 15:50:45 +02:00
tamo
79c63049d7
update the settings routes 2021-04-01 11:52:26 +02:00
Irevoire
96cffeab1e
update all the response format to be ISO with meilisearch, see #64 2021-04-01 11:43:03 +02:00
Alexey Shekhirin
3d51db5929 fix(ci, http): commit_sha and commit_date in docker builds
chore(ci): cache dependencies in Docker build
2021-03-31 13:56:28 +03:00
Irevoire
7ecefe37da
fix root route 2021-03-19 11:34:54 +01:00
Irevoire
c8b05712fa
return 202 on settings update / reset 2021-03-17 14:44:32 +01:00
bors[bot]
f8ea081df5
Merge #98
98: replace body with json r=curquiza a=MarinPostma



Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-03-16 17:12:30 +00:00
tamo
233c1e304d
use json instead of body when crafting the request 2021-03-16 17:45:59 +01:00
tamo
a268d0e283
return a 200 on health check 2021-03-16 17:42:01 +01:00
mpostma
3c25ab0d50
replace body with json 2021-03-16 16:46:07 +01:00
bors[bot]
c6dbd81823
Merge #90
90: restore version route r=MarinPostma a=MarinPostma

close #74


Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-03-16 13:53:23 +00:00
bors[bot]
2138f54954
Merge #89
89: delete index returns 204 instead of 200 r=curquiza a=MarinPostma

 close #63

Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-03-16 13:01:32 +00:00
mpostma
f4cf96915a
remove guard on add documetn route 2021-03-16 12:04:32 +01:00
mpostma
6a742ee62c
restore version route 2021-03-15 19:11:27 +01:00
mpostma
58fab035bb
delete index returns 204 instead of 200 2021-03-15 18:44:33 +01:00
mpostma
dd324807f9
last review edits + fmt 2021-03-15 18:11:10 +01:00
mpostma
abbea59732
fix clippy warnings 2021-03-15 16:52:05 +01:00
mpostma
01479dcf99
rename name to uid in code 2021-03-15 14:43:47 +01:00
mpostma
55fadd7f87
change facetedAttributes to attributesForFaceting 2021-03-15 13:53:50 +01:00
mpostma
c4846dafca
implement update index 2021-03-12 14:48:43 +01:00
mpostma
66b64c1f80
correct error on settings delete unexisting index 2021-03-11 22:33:31 +01:00
mpostma
53cf500e36
uuid resolver hard state 2021-03-10 18:04:20 +01:00
mpostma
5ecf514d28
restructure project 2021-03-10 13:46:49 +01:00