tamo
7d748fa384
integrate the new Settings in the dumps
2021-05-10 20:48:06 +02:00
tamo
d767990424
fix the import of the updates in the dump
2021-05-10 20:25:12 +02:00
tamo
ef438852cd
fix the v1
2021-05-10 20:25:12 +02:00
tamo
40ced3ff8d
first working version
2021-05-10 20:25:12 +02:00
tamo
5f5402a3ab
provide a way to access the internal content path of all processing State
2021-05-10 20:25:12 +02:00
tamo
26dcb9e66d
bump milli version and fix a performance issue for large dumps
2021-05-10 20:25:12 +02:00
Marin Postma
956012da95
fix dump lock
2021-05-10 20:25:12 +02:00
tamo
24192fc550
fix tests
2021-05-10 20:25:12 +02:00
tamo
efca63f9ce
[WIP] rebase on main
2021-05-10 20:25:09 +02:00
tamo
c3552cecdf
WIP rebase on main
2021-05-10 20:24:18 +02:00
tamo
0f94ef8abc
WIP: dump
2021-05-10 20:24:18 +02:00
tamo
0275b36fb0
[WIP] rebase on main
2021-05-10 20:24:14 +02:00
tamo
1b5fc61eb6
[WIP] rebase on main
2021-05-10 20:23:12 +02:00
tamo
0fee81678e
[WIP] rebase on main
2021-05-10 20:22:18 +02:00
tamo
c4d898a265
split the dumps between v1 and v2
2021-05-10 20:20:57 +02:00
tamo
e389c088eb
WIP: rebasing on master
2021-05-10 20:20:57 +02:00
Marin Postma
0cc79d414f
add test
2021-05-10 18:34:25 +02:00
Marin Postma
8d11b368d1
implement check
2021-05-10 18:22:41 +02:00
Marin Postma
706643dfed
type setting struct
2021-05-10 17:30:09 +02:00
Marin Postma
ec7eb7798f
remove facet setting
2021-05-04 22:36:31 +02:00
Clémentine Urquizar
eb03a3ccb1
Upgrade Milli and Tokenizer
2021-05-04 17:56:19 +02:00
Marin Postma
928fb34eff
bump milli and fix tests
2021-05-04 15:10:22 +02:00
bors[bot]
1e6b40a24b
Merge #172
...
172: Fix cors authentication issue r=MarinPostma a=MarinPostma
The error was due to the middleware returning an error, instead of a response containing the error.
close #110
Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-05-03 08:38:42 +00:00
Marin Postma
78217bcf18
Fix cors authentication issue
2021-04-29 16:28:12 +02:00
Morgane Dubus
3d5fba94c2
Update mini-dashboard with version 0.1.2
2021-04-29 15:22:41 +02:00
bors[bot]
8bc7dd8b03
Merge #143
...
143: Shared update store r=irevoire a=MarinPostma
This PR changes the updates process so that only one instance of an update store is shared among indexes.
This allows updates to always be processed sequentially without additional synchronization, and fixes the bug where all the first pending update for each index were reported as processing whereas only one was.
EDIT:
I ended having to rewrite the whole `UpdateStore` to allow updates being really queued and processed sequentially in the ordered they were added. For that purpose I created a `pending_queue` that orders the updates by a global update id.
To find the next `update_id` to use, both globally and for each index, I have created another database that contains the next id to use.
Finally, all updates that have been processed (with success or otherwise) are all stores in an `updates` database.
The layout for the keys of these databases are such that it is easy to iterate over the elements for a particular index, and greatly reduces the amount of code to do so, compared to the former implementation.
I have also simplified the locking mechanism for the update store, thanks to the StateLock data structure, that allow both an arbitrary number of readers and a single writer to concurrently access the state. The current state can be either Idle, Processing, or Snapshotting. When an update or snapshotting is ongoing, the process holds the state lock until it is done processing its task. When it is done, it sets bask the state to Idle.
I have made other small improvements here and there, and have let some other for work, such as:
- When creating an update file to hold a request's content, it would be preferable to first create a temporary file, and then atomically persist it when we have written to it. This would simplify the case when there is no data to be written to the file, since we wouldn't have to take care about cleaning after ourselves.
- The logic for content validation must be factored.
- Some more tests related to error handling in the process_pending_update function.
- The issue #159
close #114
Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2021-04-27 18:41:55 +00:00
Marin Postma
a961f0ce75
fix clippy warnings
2021-04-27 18:28:46 +02:00
Marin Postma
bb79a15c04
reenable ranking rules route
2021-04-27 15:29:00 +02:00
Marin Postma
4fe2a13c71
rewrite update store
2021-04-27 15:20:52 +02:00
Marin Postma
51829ad85e
review fixes
2021-04-27 15:10:57 +02:00
Marin Postma
c78f351300
fix tests
2021-04-27 15:10:57 +02:00
Marin Postma
ee675eadf1
fix stats
2021-04-27 15:10:55 +02:00
Marin Postma
33830d5ecf
fix snapshots
2021-04-27 15:09:55 +02:00
Marin Postma
2b154524bb
fix filtered out pending update
2021-04-27 15:09:23 +02:00
Marin Postma
b626d02ffe
simplify index actor run loop
2021-04-27 15:09:22 +02:00
Marin Postma
9ce68d11a7
single update store instance
2021-04-27 15:09:21 +02:00
Marin Postma
5a38f13cae
multi_index udpate store
2021-04-27 15:07:13 +02:00
Clémentine Urquizar
1ba46f8f77
Disable clippy rule
2021-04-27 12:43:00 +02:00
Clémentine Urquizar
f80ea24d2b
Add tests on every platform and fix clippy errors
2021-04-27 12:42:59 +02:00
Marin Postma
5014f74649
put mini-dashboard in out-dir
2021-04-27 09:32:17 +02:00
Clémentine Urquizar
f3b6bf55a6
Update version for the next release (alpha4)
2021-04-26 19:05:16 +02:00
Marin Postma
c2461e5066
review fixes
2021-04-26 10:20:46 +02:00
Marin Postma
e4bd1bc5ce
update actix-web-static-file rev
2021-04-22 11:42:41 +02:00
Marin Postma
90f57c1329
update CI & Dockerfile
2021-04-22 11:22:09 +02:00
Marin Postma
6af769af20
bump mini-dashboard
2021-04-22 10:45:05 +02:00
Marin Postma
6bcf20c70e
serve static site
2021-04-22 10:26:54 +02:00
Marin Postma
bb79695e44
load mini-dashboard assets
2021-04-22 10:26:54 +02:00
Marin Postma
ea5517bc8c
add mini-dashboard feature
2021-04-22 10:26:54 +02:00
Clémentine Urquizar
a72d2f66cd
use <em> tags instead of <mark> tags for highlighting
2021-04-21 19:14:55 +02:00
Marin Postma
dd2914873b
fix document fields order
2021-04-20 21:30:30 +02:00
Marin Postma
d9a29cae60
fix ignored displayed attributes
2021-04-20 21:23:35 +02:00
Marin Postma
7a737d2bd3
support wildcard
2021-04-20 21:23:35 +02:00
Marin Postma
881b099c8e
add tests
2021-04-20 21:23:34 +02:00
Marin Postma
c6bb36efa5
implement _formated
2021-04-20 21:23:28 +02:00
Marin Postma
526a05565e
add SearchHit structure
2021-04-20 21:22:48 +02:00
Clémentine Urquizar
b8e535579f
Update version for the next release (alpha3)
2021-04-20 16:11:07 +02:00
Marin Postma
f8c338e3a7
add test for dedicated distinct route
2021-04-20 15:49:17 +02:00
Marin Postma
6c470cf687
enable distinct-attribute setting route
2021-04-20 11:34:18 +02:00
Marin Postma
ec63e13896
bump actix
2021-04-20 11:29:32 +02:00
Marin Postma
1746132c7d
add test set/reset distinct attribute
2021-04-20 11:29:08 +02:00
mpostma
ec230c2835
enable distinct
2021-04-20 11:29:06 +02:00
Marin Postma
45665245dc
bump actix
2021-04-20 11:07:23 +02:00
bors[bot]
94c5c5843b
Merge #149
...
149: Handle star in attributes_to_retrieve r=MarinPostma a=curquiza
Closes #147
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-04-19 17:29:21 +00:00
Clémentine Urquizar
8eceba98d3
Handle star in attributes_to_retrieve
2021-04-19 18:20:19 +02:00
Clémentine Urquizar
2c380731b9
Update milli version to v0.1.1
2021-04-19 16:03:39 +02:00
bors[bot]
7ce74f95a2
Merge #146
...
146: Remove another unused legacy file r=MarinPostma a=irevoire
When doing #135 I missed an old useless file in the scr/routes directory
Co-authored-by: tamo <tamo@meilisearch.com>
2021-04-15 18:05:28 +00:00
tamo
ec3a08ea0c
remove another unused legacy file
2021-04-15 14:44:43 +02:00
Clémentine Urquizar
b0717b75d9
Update tokenizer to v0.2.1
2021-04-14 19:06:18 +02:00
Alexey Shekhirin
f87afbc558
fix(http): commit date & SHA in startup message
2021-04-13 20:16:18 +03:00
Alexey Shekhirin
9eaf048a06
fix(http): use BTreeMap instead of HashMap to preserve stats order
2021-04-13 11:59:07 +03:00
Alexey Shekhirin
adfdb99abc
feat(http): calculate updates' and uuids' dbs size
2021-04-09 15:59:12 +03:00
Alexey Shekhirin
ae1655586c
fixes after review
2021-04-09 14:40:48 +03:00
Alexey Shekhirin
698a1ea582
feat(http): store processing as RwLock<Option<Uuid>> in index_actor
2021-04-09 14:34:43 +03:00
Alexey Shekhirin
87412f63ef
feat(http): implement is_indexing for stats
2021-04-09 14:34:42 +03:00
Alexey Shekhirin
09d9a29176
test(http): server & index stats
2021-04-09 14:34:42 +03:00
Alexey Shekhirin
dd9eae8c26
feat(http): stats route
2021-04-09 14:34:42 +03:00
bors[bot]
a1d04fbff5
Merge #136
...
136: Rename update status "pending" into "enqueued" r=curquiza a=curquiza
Closes #107
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-04-08 16:46:12 +00:00
Alexey Shekhirin
51ba1bd7d3
fix(http, index): init analyzer with optional stop words
...
Next release
update tokenizer
2021-04-08 17:16:13 +03:00
bors[bot]
f881e8691e
Merge #135
...
135: Add stop words r=curquiza a=irevoire
closes #21
Co-authored-by: tamo <tamo@meilisearch.com>
2021-04-08 11:29:00 +00:00
Clémentine Urquizar
cb23775d18
Rename pending into enqueued
2021-04-07 19:46:36 +02:00
bors[bot]
0344cf5874
Merge #122
...
122: Update display r=MarinPostma a=curquiza
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-04-07 12:33:25 +00:00
tamo
dcd60a5b45
add more tests for the stop_words
2021-04-06 18:29:38 +02:00
tamo
b1962c8e02
remove legacy files from meilisearch that have been replaced by a macro in routes/settings/mod.rs
2021-04-06 16:29:04 +02:00
tamo
40ef9a3c6a
push a first implementation of the stop_words
2021-04-06 16:29:04 +02:00
Clémentine Urquizar
4ee6ce7871
Next release
2021-04-01 17:16:16 +02:00
bors[bot]
6cb8052d3d
Merge #104
...
104: Update all the response format (issue #64 ) r=MarinPostma a=irevoire
closes #64
Co-authored-by: Irevoire <tamo@meilisearch.com>
Co-authored-by: tamo <tamo@meilisearch.com>
2021-04-01 14:22:57 +00:00
tamo
73973e2b9e
fix more settings routes
2021-04-01 15:50:45 +02:00
Marin Postma
248e9b3808
Merge remote-tracking branch 'origin/main' into snapshots
2021-04-01 15:10:33 +02:00
tamo
79c63049d7
update the settings routes
2021-04-01 11:52:26 +02:00
Irevoire
96cffeab1e
update all the response format to be ISO with meilisearch, see #64
2021-04-01 11:43:03 +02:00
Marin Postma
d8af4a7202
ignore snapshot test ( #130 )
2021-03-31 20:07:52 +02:00
Alexey Shekhirin
3d51db5929
fix(ci, http): commit_sha and commit_date in docker builds
...
chore(ci): cache dependencies in Docker build
2021-03-31 13:56:28 +03:00
tamo
0f2143e7fd
remove the now useless dead_code flags
2021-03-26 14:15:12 +01:00
Clémentine Urquizar
b9f79c8df0
Update display
2021-03-26 12:12:55 +01:00
mpostma
d7c077cffb
atomic snapshot import
2021-03-25 14:48:51 +01:00
mpostma
7d6ec7f3d3
resolve merge
2021-03-25 14:21:05 +01:00
mpostma
f3dc853be3
Merge remote-tracking branch 'origin/main' into snapshots
2021-03-25 13:45:07 +01:00
mpostma
48507460b2
add snapshot tests
2021-03-25 12:02:10 +01:00
mpostma
d029464de8
fix snapshot path
2021-03-25 10:23:31 +01:00
mpostma
79d09705d8
perform snapshot on startup
2021-03-25 09:35:15 +01:00
