Commit Graph

379 Commits

Author SHA1 Message Date
bors[bot]
77740829bd
Merge #177
177: bump milli r=MarinPostma a=MarinPostma



Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-05-04 13:49:37 +00:00
Marin Postma
928fb34eff
bump milli and fix tests 2021-05-04 15:10:22 +02:00
bors[bot]
1e6b40a24b
Merge #172
172: Fix cors authentication issue r=MarinPostma a=MarinPostma

The error was due to the middleware returning an error, instead of a response containing the error.

close #110


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-05-03 08:38:42 +00:00
Marin Postma
78217bcf18
Fix cors authentication issue 2021-04-29 16:28:12 +02:00
bors[bot]
53c88d9fa3
Merge #170
170: Improve CI r=MarinPostma a=curquiza

Checked with @Kerollmops to improve (a little bit) the CI execution time.

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-04-29 14:08:33 +00:00
bors[bot]
b14fdb1163
Merge #171
171: Update mini-dashboard with version 0.1.2 r=MarinPostma a=mdubus

Update of the mini-dashboard sha1 & assets-url, due to a new release

Co-authored-by: Morgane Dubus <morgane.d@meilisearch.com>
2021-04-29 13:48:54 +00:00
Morgane Dubus
3d5fba94c2 Update mini-dashboard with version 0.1.2 2021-04-29 15:22:41 +02:00
Clémentine Urquizar
3ee2b07918
Improve CI 2021-04-29 15:19:48 +02:00
bors[bot]
8bc7dd8b03
Merge #143
143: Shared update store r=irevoire a=MarinPostma

This PR changes the updates process so that only one instance of an update store is shared among indexes.

This allows updates to always be processed sequentially without additional synchronization, and fixes the bug where all the first pending update for each index were reported as processing whereas only one was.

EDIT:

I ended having to rewrite the whole `UpdateStore` to allow updates being really queued and processed sequentially in the ordered they were added. For that purpose I created a `pending_queue` that orders the updates by a global update id.

To find the next `update_id` to use, both globally and for each index, I have created another database that contains the next id to use.

Finally, all updates that have been processed (with success or otherwise) are all stores in an `updates` database.

The layout for the keys of these databases are such that it is easy to iterate over the elements for a particular index, and greatly reduces the amount of code to do so, compared to the former implementation.

I have also simplified the locking mechanism for the update store, thanks to the StateLock data structure, that allow both an arbitrary number of readers and a single writer to concurrently access the state. The current state can be either Idle, Processing, or Snapshotting. When an update or snapshotting is ongoing, the process holds the state lock until it is done processing its task. When it is done, it sets bask the state to Idle.

I have made other small improvements here and there, and have let some other for work, such as:
- When creating an update file to hold a request's content, it would be preferable to first create a temporary file, and then atomically persist it when we have written to it. This would simplify the case when there is no data to be written to the file, since we wouldn't have to take care about cleaning after ourselves.
- The logic for content validation must be factored.
- Some more tests related to error handling in the process_pending_update function.
- The issue #159

close #114


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-04-27 18:41:55 +00:00
Thomas Payet
e6fd1afc3d
Merge pull request #163 from meilisearch/curquiza-patch-1
Update README.md
2021-04-27 18:51:04 +02:00
Marin Postma
a961f0ce75
fix clippy warnings 2021-04-27 18:28:46 +02:00
Clémentine Urquizar
cea0c1f41d
Update README.md 2021-04-27 16:33:22 +02:00
Clémentine Urquizar
703d2026e4
Update README.md 2021-04-27 16:33:00 +02:00
bors[bot]
3d85b2d854
Merge #162
162: Re-enable ranking rules route r=MarinPostma a=MarinPostma

re-enable ranking rules setting route


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-04-27 13:55:40 +00:00
Marin Postma
bb79a15c04
reenable ranking rules route 2021-04-27 15:29:00 +02:00
Marin Postma
4fe2a13c71
rewrite update store 2021-04-27 15:20:52 +02:00
Marin Postma
51829ad85e
review fixes 2021-04-27 15:10:57 +02:00
Marin Postma
c78f351300
fix tests 2021-04-27 15:10:57 +02:00
Marin Postma
ee675eadf1
fix stats 2021-04-27 15:10:55 +02:00
Marin Postma
33830d5ecf
fix snapshots 2021-04-27 15:09:55 +02:00
Marin Postma
2b154524bb
fix filtered out pending update 2021-04-27 15:09:23 +02:00
Marin Postma
b626d02ffe
simplify index actor run loop 2021-04-27 15:09:22 +02:00
Marin Postma
9ce68d11a7
single update store instance 2021-04-27 15:09:21 +02:00
Marin Postma
5a38f13cae
multi_index udpate store 2021-04-27 15:07:13 +02:00
bors[bot]
7055384aeb
Merge #116
116: Add tests for every plateform + clippy r=MarinPostma a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-04-27 11:07:58 +00:00
Clémentine Urquizar
0c41adf868
Update CI 2021-04-27 12:43:00 +02:00
Clémentine Urquizar
1ba46f8f77
Disable clippy rule 2021-04-27 12:43:00 +02:00
Clémentine Urquizar
f80ea24d2b
Add tests on every platform and fix clippy errors 2021-04-27 12:42:59 +02:00
bors[bot]
d34d7cbc37
Merge #161
161: put mini-dashboard in out-dir r=MarinPostma a=MarinPostma

This PR puts the mini-dashboard during build in the `OUT_DIR` specified by cargo. This allow the mini-dashboard artifacts to be cleaned when `cargo clean` is ran, and not pollute the working directory with unwanted files.


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-04-27 07:40:23 +00:00
Marin Postma
5014f74649
put mini-dashboard in out-dir 2021-04-27 09:32:17 +02:00
bors[bot]
1f32f35d9e
Merge #160
160: Update version for the next release (alpha4) r=MarinPostma a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-04-26 19:09:08 +00:00
Clémentine Urquizar
f3b6bf55a6
Update version for the next release (alpha4) 2021-04-26 19:05:16 +02:00
bors[bot]
9e6a7e3aa9
Merge #153
153: integrate mini dashboard r=MarinPostma a=MarinPostma

This PR integrate the [mini dashboard](https://github.com/meilisearch/mini-dashboard) to transplant.

It adds a build feature `mini-dashboard` to statically add the mini-dashboard to the MeiliSearch binary. The mini-dashboard build feature is enabled by default and can be disabled by building MeiliSearch with `cargo build --no-default-features`.

- [x] Fetch the mini-dashboard from the Github release
- [x] Check that the SHA1 on the downloaded payload matches the one in the metadata
- [x] Unpack the mini dashboard in `meilisearch-http/mini-dashboard`
- [x] serve the mini-dashboard if the `mini-dashboard` feature is enabled
- [x] Update CI to build MeiliSearch with mini-dashboard for releases

close #87

## Shasum check and build optimizations.

In order to make sure that the right bundle for the mini-dashboard is downloaded, its shasum is computed and compared to the one specified in the `Cargo.toml`. If the shasums match, them the shasum is written to the `.mini-dashboard.sha1` file for later comparison. On subsequent builds, the build script will check that both the mini-dashboard assets and the shasum file are found and that the shasum file content matches the one from the toml file. It will only preform a re-generation on the static dashboard files if it finds that either the dashboard is not present where it expects it to be, or if it finds out that it is outdated, by comparing the shasums.

## Notes

I had to rely on a [custom patch](https://github.com/MarinPostma/actix-web-static-files/tree/actix-web-4) of actix-web-static-files, to support actix-web 4 beta6. there is currently a [pr on the official repo](https://github.com/kilork/actix-web-static-files/pull/35) to support actix-web 4, but it most likely won't be merged until actix is stabilized.


Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-04-26 16:22:20 +00:00
Marin Postma
77481d7c76
update gitignore 2021-04-26 18:21:09 +02:00
Marin Postma
c2461e5066
review fixes 2021-04-26 10:20:46 +02:00
Marin Postma
e4bd1bc5ce
update actix-web-static-file rev 2021-04-22 11:42:41 +02:00
Marin Postma
90f57c1329
update CI & Dockerfile 2021-04-22 11:22:09 +02:00
Marin Postma
6af769af20
bump mini-dashboard 2021-04-22 10:45:05 +02:00
Marin Postma
6bcf20c70e
serve static site 2021-04-22 10:26:54 +02:00
Marin Postma
bb79695e44
load mini-dashboard assets 2021-04-22 10:26:54 +02:00
Marin Postma
ea5517bc8c
add mini-dashboard feature 2021-04-22 10:26:54 +02:00
bors[bot]
da08a1f25c
Merge #157
157: Use <em> tags instead of <mark> tags for highlighting r=MarinPostma a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-04-22 08:11:07 +00:00
Clémentine Urquizar
a72d2f66cd
use <em> tags instead of <mark> tags for highlighting 2021-04-21 19:14:55 +02:00
bors[bot]
e5df58bc04
Merge #150
150: add _formated field to search result r=MarinPostma a=MarinPostma

close #75 

Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-04-21 16:33:30 +00:00
bors[bot]
662ffc8fa5
Merge #155
155: Fix dockerfile r=MarinPostma a=curquiza

docker build and run works now :)

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-04-21 10:22:01 +00:00
Clémentine Urquizar
ce5e4743e6
Fix dockerfile 2021-04-21 11:00:04 +02:00
Marin Postma
dd2914873b
fix document fields order 2021-04-20 21:30:30 +02:00
Marin Postma
d9a29cae60
fix ignored displayed attributes 2021-04-20 21:23:35 +02:00
Marin Postma
7a737d2bd3
support wildcard 2021-04-20 21:23:35 +02:00
Marin Postma
881b099c8e
add tests 2021-04-20 21:23:34 +02:00