Commit Graph

398 Commits

Author SHA1 Message Date
tamo
26dcb9e66d
bump milli version and fix a performance issue for large dumps 2021-05-10 20:25:12 +02:00
Marin Postma
956012da95
fix dump lock 2021-05-10 20:25:12 +02:00
tamo
24192fc550
fix tests 2021-05-10 20:25:12 +02:00
tamo
efca63f9ce
[WIP] rebase on main 2021-05-10 20:25:09 +02:00
tamo
c3552cecdf
WIP rebase on main 2021-05-10 20:24:18 +02:00
tamo
0f94ef8abc
WIP: dump 2021-05-10 20:24:18 +02:00
tamo
0275b36fb0
[WIP] rebase on main 2021-05-10 20:24:14 +02:00
tamo
1b5fc61eb6
[WIP] rebase on main 2021-05-10 20:23:12 +02:00
tamo
0fee81678e
[WIP] rebase on main 2021-05-10 20:22:18 +02:00
tamo
c4d898a265
split the dumps between v1 and v2 2021-05-10 20:20:57 +02:00
tamo
e389c088eb
WIP: rebasing on master 2021-05-10 20:20:57 +02:00
bors[bot]
ceb8d6e1c9
Merge #186
186: settings fix r=MarinPostma a=MarinPostma

add type checked settigns validation. For now it only transform the settings accepting wildcard


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-05-10 16:42:12 +00:00
Marin Postma
0cc79d414f
add test 2021-05-10 18:34:25 +02:00
Marin Postma
8d11b368d1
implement check 2021-05-10 18:22:41 +02:00
Marin Postma
706643dfed
type setting struct 2021-05-10 17:30:09 +02:00
bors[bot]
998d5ead34
Merge #182
182: remove facet setting r=MarinPostma a=MarinPostma

remove useless code


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-05-05 11:22:12 +00:00
Marin Postma
ec7eb7798f
remove facet setting 2021-05-04 22:36:31 +02:00
bors[bot]
88ae02f8d9
Merge #174
174: Upgrade Tokenizer r=MarinPostma a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-05-04 15:57:07 +00:00
Clémentine Urquizar
eb03a3ccb1
Upgrade Milli and Tokenizer 2021-05-04 17:56:19 +02:00
bors[bot]
77740829bd
Merge #177
177: bump milli r=MarinPostma a=MarinPostma



Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-05-04 13:49:37 +00:00
Marin Postma
928fb34eff
bump milli and fix tests 2021-05-04 15:10:22 +02:00
bors[bot]
1e6b40a24b
Merge #172
172: Fix cors authentication issue r=MarinPostma a=MarinPostma

The error was due to the middleware returning an error, instead of a response containing the error.

close #110


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-05-03 08:38:42 +00:00
Marin Postma
78217bcf18
Fix cors authentication issue 2021-04-29 16:28:12 +02:00
bors[bot]
53c88d9fa3
Merge #170
170: Improve CI r=MarinPostma a=curquiza

Checked with @Kerollmops to improve (a little bit) the CI execution time.

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-04-29 14:08:33 +00:00
bors[bot]
b14fdb1163
Merge #171
171: Update mini-dashboard with version 0.1.2 r=MarinPostma a=mdubus

Update of the mini-dashboard sha1 & assets-url, due to a new release

Co-authored-by: Morgane Dubus <morgane.d@meilisearch.com>
2021-04-29 13:48:54 +00:00
Morgane Dubus
3d5fba94c2 Update mini-dashboard with version 0.1.2 2021-04-29 15:22:41 +02:00
Clémentine Urquizar
3ee2b07918
Improve CI 2021-04-29 15:19:48 +02:00
bors[bot]
8bc7dd8b03
Merge #143
143: Shared update store r=irevoire a=MarinPostma

This PR changes the updates process so that only one instance of an update store is shared among indexes.

This allows updates to always be processed sequentially without additional synchronization, and fixes the bug where all the first pending update for each index were reported as processing whereas only one was.

EDIT:

I ended having to rewrite the whole `UpdateStore` to allow updates being really queued and processed sequentially in the ordered they were added. For that purpose I created a `pending_queue` that orders the updates by a global update id.

To find the next `update_id` to use, both globally and for each index, I have created another database that contains the next id to use.

Finally, all updates that have been processed (with success or otherwise) are all stores in an `updates` database.

The layout for the keys of these databases are such that it is easy to iterate over the elements for a particular index, and greatly reduces the amount of code to do so, compared to the former implementation.

I have also simplified the locking mechanism for the update store, thanks to the StateLock data structure, that allow both an arbitrary number of readers and a single writer to concurrently access the state. The current state can be either Idle, Processing, or Snapshotting. When an update or snapshotting is ongoing, the process holds the state lock until it is done processing its task. When it is done, it sets bask the state to Idle.

I have made other small improvements here and there, and have let some other for work, such as:
- When creating an update file to hold a request's content, it would be preferable to first create a temporary file, and then atomically persist it when we have written to it. This would simplify the case when there is no data to be written to the file, since we wouldn't have to take care about cleaning after ourselves.
- The logic for content validation must be factored.
- Some more tests related to error handling in the process_pending_update function.
- The issue #159

close #114


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-04-27 18:41:55 +00:00
Thomas Payet
e6fd1afc3d
Merge pull request #163 from meilisearch/curquiza-patch-1
Update README.md
2021-04-27 18:51:04 +02:00
Marin Postma
a961f0ce75
fix clippy warnings 2021-04-27 18:28:46 +02:00
Clémentine Urquizar
cea0c1f41d
Update README.md 2021-04-27 16:33:22 +02:00
Clémentine Urquizar
703d2026e4
Update README.md 2021-04-27 16:33:00 +02:00
bors[bot]
3d85b2d854
Merge #162
162: Re-enable ranking rules route r=MarinPostma a=MarinPostma

re-enable ranking rules setting route


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-04-27 13:55:40 +00:00
Marin Postma
bb79a15c04
reenable ranking rules route 2021-04-27 15:29:00 +02:00
Marin Postma
4fe2a13c71
rewrite update store 2021-04-27 15:20:52 +02:00
Marin Postma
51829ad85e
review fixes 2021-04-27 15:10:57 +02:00
Marin Postma
c78f351300
fix tests 2021-04-27 15:10:57 +02:00
Marin Postma
ee675eadf1
fix stats 2021-04-27 15:10:55 +02:00
Marin Postma
33830d5ecf
fix snapshots 2021-04-27 15:09:55 +02:00
Marin Postma
2b154524bb
fix filtered out pending update 2021-04-27 15:09:23 +02:00
Marin Postma
b626d02ffe
simplify index actor run loop 2021-04-27 15:09:22 +02:00
Marin Postma
9ce68d11a7
single update store instance 2021-04-27 15:09:21 +02:00
Marin Postma
5a38f13cae
multi_index udpate store 2021-04-27 15:07:13 +02:00
bors[bot]
7055384aeb
Merge #116
116: Add tests for every plateform + clippy r=MarinPostma a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-04-27 11:07:58 +00:00
Clémentine Urquizar
0c41adf868
Update CI 2021-04-27 12:43:00 +02:00
Clémentine Urquizar
1ba46f8f77
Disable clippy rule 2021-04-27 12:43:00 +02:00
Clémentine Urquizar
f80ea24d2b
Add tests on every platform and fix clippy errors 2021-04-27 12:42:59 +02:00
bors[bot]
d34d7cbc37
Merge #161
161: put mini-dashboard in out-dir r=MarinPostma a=MarinPostma

This PR puts the mini-dashboard during build in the `OUT_DIR` specified by cargo. This allow the mini-dashboard artifacts to be cleaned when `cargo clean` is ran, and not pollute the working directory with unwanted files.


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-04-27 07:40:23 +00:00
Marin Postma
5014f74649
put mini-dashboard in out-dir 2021-04-27 09:32:17 +02:00
bors[bot]
1f32f35d9e
Merge #160
160: Update version for the next release (alpha4) r=MarinPostma a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-04-26 19:09:08 +00:00