Commit Graph

575 Commits

Author SHA1 Message Date
Alexey Shekhirin
efa69875d9
refactor(http): use reset_sortable_fields 2021-09-07 15:04:44 +03:00
many
c0f9c891f5
Set max_memory value to unlimited during tests
because tests run several meilisearch in parallel,
we over estimate the value for max_memory making the tests on widows crash
2021-09-06 14:38:10 +02:00
Clémentine Urquizar
33514b28be
Merge pull request #1588 from meilisearch/test-new-indexer
Integrate the new indexer
2021-09-06 10:21:42 +02:00
Clément Renault
7e80337e5b
Bump milli to v0.12.0 2021-09-02 18:19:12 +02:00
Kerollmops
4cdf680a81
Make the MaxMemory use the default value when undefined 2021-09-02 18:19:11 +02:00
Kerollmops
0cd66c3a89
Bump the milli version 2021-09-02 18:19:00 +02:00
Clément Renault
b092a624ed
Introduce the MaxMemory struct that defaults to 2/3 of the available memory 2021-09-02 18:18:59 +02:00
Clémentine Urquizar
24e84d7ca1
Test new indexer 2021-09-02 18:11:20 +02:00
Clémentine Urquizar
90116155b4
Update version for the next release (v0.22.0) 2021-09-01 12:33:30 +02:00
bors[bot]
19c5c74291
Merge #1652 #1654 #1657
1652: Remove dependabot r=MarinPostma a=curquiza

Fixes #1649 

Dependabot for vulnerability and security updates is still activated.

1654: Add Script for Windows r=MarinPostma a=singh08prashant

fixes #1570 

changes:

1. added script for detecting windows os running git bash
2. appended `.exe` to `$release_file` for windows as listed [here](https://github.com/meilisearch/MeiliSearch/releases/)
3. removed global `$BINARY_NAME='meilisearch'` as windows require `.exe` file

1657: Bring vergen hotfix from `stable` to `main` r=MarinPostma a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: singh08prashant <singh08prashant@gmail.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
2021-08-31 14:31:42 +00:00
Kerollmops
9d0fa8112b
Remove unused Arc import 2021-08-31 14:50:36 +02:00
Kerollmops
b8c954eb3f
Bump the MeiliSearch version to v0.21.1 2021-08-30 17:41:25 +02:00
Kerollmops
a8c146fd13
Unwrap or unknown the commit hash 2021-08-30 17:41:24 +02:00
Kerollmops
1782753387
Bump vergen and remove unused build feature 2021-08-30 15:03:45 +02:00
Kerollmops
cf4a466b6b
Make sure that the order of the filterableAttributes is constant 2021-08-26 11:06:05 +02:00
Kerollmops
087e4626ce
Make sure that the order of the sortableAttributes is constant 2021-08-26 11:06:04 +02:00
Kerollmops
64462c842b
Test the search with sort time queries with POST and GET methods 2021-08-25 17:39:25 +02:00
Kerollmops
e0f73fe742
Introduce the sort search parameter 2021-08-25 17:39:25 +02:00
Kerollmops
ea4c831de0
Integrate the sortable-attributes into the settings 2021-08-25 17:39:25 +02:00
Kerollmops
51387b2c80
Introduce the new invalid sortable error codes 2021-08-25 17:29:30 +02:00
bors[bot]
2d8dd87cad
Merge #1623
1623: Use Setting enum r=Kerollmops a=shekhirin

Resolves https://github.com/meilisearch/MeiliSearch/issues/1620

Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
2021-08-25 14:58:40 +00:00
Alexey Shekhirin
d9dd2a038b
refactor(http): use Setting enum 2021-08-25 17:43:46 +03:00
Irevoire
672fcee8aa
remove sentry 2021-08-24 12:38:31 +02:00
Clémentine Urquizar
149f46c184
Fix pacic when decoding 2021-08-23 12:37:51 +02:00
Clémentine Urquizar
38fc876704
Update tokenizer and new milli version with new tags 2021-08-18 14:55:10 +02:00
Clémentine Urquizar
39d5a99095
Update milli and tokenizer version 2021-08-18 12:09:34 +02:00
bors[bot]
2beb306834
Merge #1577
1577: Update milli dependency: fix facet values bugs r=Kerollmops a=curquiza

Fixes #1576 

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-08-16 16:13:42 +00:00
Clémentine Urquizar
f3e595e2f0
Update milli dependency 2021-08-16 13:36:42 +02:00
Clémentine Urquizar
621529e9dc
Update telemetry link 2021-08-12 18:58:07 +02:00
Clémentine Urquizar
7531280764
Update tokenizer version to v0.2.4 2021-08-12 13:55:47 +02:00
bors[bot]
7e3b2ddff2
Merge #1554
1554: Fix dump v1 (attributesForFaceting, and criteria) r=curquiza a=MarinPostma

close #1553


Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-08-05 19:45:52 +00:00
bors[bot]
312d93961a
Merge #1556
1556: Update milli to v0.9.0 r=MarinPostma a=curquiza

Fixes #1552 

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-08-05 14:04:55 +00:00
mpostma
8f05d8d546 fix clippy warnings 2021-08-05 16:00:47 +02:00
mpostma
f5ddea481a reintroduce exactness 2021-08-05 15:59:39 +02:00
mpostma
29ca8271b3 test dumpv1 format regression 2021-08-05 15:59:39 +02:00
mpostma
3084537d1e restore attributes for faceting in dump v1 2021-08-05 15:59:39 +02:00
Clémentine Urquizar
992b082c6f
Fix docs link anchor 2021-08-05 13:28:32 +02:00
Clémentine Urquizar
31fe263356
Update milli to v0.9.0 2021-08-05 13:08:27 +02:00
bors[bot]
7a0b20c740
Merge #1532
1532: Start writing documentation for newcomers r=MarinPostma a=irevoire



Co-authored-by: Tamo <tamo@meilisearch.com>
2021-08-03 09:26:45 +00:00
bors[bot]
9810f6b695
Merge #1540
1540: Update milli to version 0.8.1 r=curquiza a=curquiza

Integrates this fix into MeiliSearch https://github.com/meilisearch/milli/pull/296

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-07-29 17:15:52 +00:00
bors[bot]
09c74c04a0
Merge #1539
1539: Use serdeval for validating json format. r=curquiza a=MarinPostma

uses [serdeval](https://github.com/MarinPostma/serdeval) to validate that the json payload is valid json, and in the correct format.

fix #1535


Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-07-29 17:05:13 +00:00
Clémentine Urquizar
1b5d918cb9
Fix rustfmt 2021-07-29 18:32:09 +02:00
Clémentine Urquizar
bf76d4a43c
Make clippy happy 2021-07-29 18:14:36 +02:00
mpostma
53b4b2fcbc Use serdeval for validating json format. 2021-07-29 18:02:54 +02:00
Clémentine Urquizar
9a8629a6a9
Update milli 2021-07-29 17:45:31 +02:00
Tamo
78308365ec
fix typos 2021-07-29 14:40:41 +02:00
Tamo
243233f652
import .git to docker to fix vergen 2021-07-28 19:12:40 +02:00
Clémentine Urquizar
80916baa21
Add FieldId in import 2021-07-28 12:25:13 +02:00
Clémentine Urquizar
1df8f041bd
Update meilisearch-http/src/index/search.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-07-28 12:10:25 +02:00
Clémentine Urquizar
6a6e2a8cd1
Update meilisearch-http/src/index/search.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-07-28 12:08:51 +02:00
Clémentine Urquizar
f9d337b320
Update meilisearch-http/src/index/search.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-07-28 12:08:36 +02:00
Clémentine Urquizar
feb069f604
Update meilisearch-http/src/index/search.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-07-28 12:08:28 +02:00
Clémentine Urquizar
7e0eed5772
Update meilisearch-http/src/index/search.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-07-28 12:08:24 +02:00
Clémentine Urquizar
9bdd040dd0
Update meilisearch-http/src/index/mod.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-07-28 12:08:19 +02:00
Clémentine Urquizar
e5dabf265a
Update milli version to v0.8.0 2021-07-28 10:52:47 +02:00
Tamo
1a1046a0ef
start writing some documentation for newcomers 2021-07-27 16:35:42 +02:00
Morgane Dubus
d3cd7e92d1
Update mini-dashboard version to v0.1.4 2021-07-27 11:44:20 +02:00
Tamo
f79b8287f5
update vergen 2021-07-26 15:25:30 +02:00
Tamo
5d4a0ac844
sentry was never sending anything 2021-07-21 11:50:54 +02:00
bors[bot]
0136b02e5b
Merge #1498
1498: Show the filterable and not the faceted attributes in the settings r=Kerollmops a=Kerollmops

Fixes #1497

Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-07-13 07:27:14 +00:00
Clément Renault
f49a01703a
Show the filterable and not the faceted attributes in the settings 2021-07-09 16:11:37 +02:00
mpostma
a71fa25ebe auth tests 2021-07-07 17:47:48 +02:00
mpostma
83725a1330 fix search permissions 2021-07-07 10:39:04 +02:00
mpostma
41e271974a add tests 2021-07-06 16:21:15 +02:00
mpostma
520d37983c implement index search methods 2021-07-06 11:54:09 +02:00
mpostma
066085f6f5 fix index deletion bug 2021-07-05 18:42:13 +02:00
mpostma
2f3a439566 fix tests 2021-07-05 16:31:52 +02:00
mpostma
9681ffca52 change index create http code 2021-07-05 16:31:51 +02:00
bors[bot]
fddc60f893
Merge #1471
1471: Bump milli to 0.7.2 r=irevoire a=irevoire



Co-authored-by: Tamo <tamo@meilisearch.com>
2021-07-05 13:29:38 +00:00
bors[bot]
0f024cc225
Merge #1478
1478: refactor routes r=irevoire a=MarinPostma

refactor the route directory, so the module tree follows the route structure


Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-07-05 12:55:39 +00:00
mpostma
575ec2a06f refactor routes 2021-07-05 14:33:48 +02:00
mpostma
bc85d30076
add test 2021-07-05 12:33:28 +02:00
mpostma
bc417726fc
fix update loop bug 2021-07-05 12:33:22 +02:00
Tamo
9949a2a930
bump milli to 0.7.2 2021-07-05 12:19:27 +02:00
bors[bot]
71e1cb472f
Merge #1457
1457: Hotfix highlight on emojis panic r=Kerollmops a=ManyTheFish

When the highlight bound is in the middle of a character
or if we are out of bounds, we highlight the complete matching word.

note: we should enhance the tokenizer and the Highlighter to match char indices.

Fix #1368

Co-authored-by: many <maxime@meilisearch.com>
2021-07-01 14:48:18 +00:00
many
38161ede33
Add test with special characters 2021-07-01 16:44:17 +02:00
Kerollmops
fa5f8f9531
Fix an issue with the update loop falsely breaking 2021-07-01 14:53:31 +02:00
many
acfe31151e
Hotfix panic for unicode characters
When the highlight bound is in the middle of a character
or if we are out of bounds, we highlight the complete matching word.

note: we should enhance the tokenizer and the Highlighter to match char indices.

Fix #1368
2021-07-01 14:49:22 +02:00
bors[bot]
5c9401ad94
Merge #1438
1438: Update milli to 0.7.1 r=curquiza a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-30 18:49:41 +00:00
bors[bot]
768987583a
Merge #1428
1428: Accept any content type as json r=curquiza a=irevoire



Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-30 18:29:57 +00:00
Clémentine Urquizar
4f0d3b065f
Update milli 2021-06-30 18:39:06 +02:00
Tamo
a95c44193d
Do not block when sending update notifications 2021-06-30 17:29:22 +02:00
Tamo
2830853665
accept any content type as json 2021-06-30 17:05:59 +02:00
Clémentine Urquizar
01de7f9e36
Update version 2021-06-30 15:59:59 +02:00
bors[bot]
eeca841a21
Merge #259
259: Run rustfmt one the whole project and add it to the CI r=curquiza a=irevoire

Since there is currently no other PR modifying the code, I think it's a good time to reformat everything and add rustfmt to the ci.

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-30 11:55:30 +00:00
bors[bot]
f1cc141f6c
Merge #258
258: Use rustls instead of openssl r=curquiza a=irevoire

I also removed all the `default-features` of reqwest since we are only using the JSON one.
Fix #255

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-29 14:42:25 +00:00
Tamo
29bf6a8d42
run rustfmt one the whole project and add it to the CI 2021-06-29 15:25:18 +02:00
Tamo
fe7640555d
fix the content-type 2021-06-29 13:16:56 +02:00
Tamo
ec809ca487
use rustls instead of openssl and remove all default-features of reqwest 2021-06-29 13:07:40 +02:00
Tamo
1dc99ea451
accept no content-type as json 2021-06-29 11:59:25 +02:00
bors[bot]
f12ace3fbf
Merge #256
256: Update heed and milli r=irevoire a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-29 08:49:22 +00:00
Clémentine Urquizar
c09e610bb5
Update heed and milli 2021-06-29 10:25:47 +02:00
bors[bot]
712abf4c5f
Merge #246
246: Stop logging the no space left on device error r=curquiza a=irevoire

closes #208
@qdequele what do you think of that?
Are there any other errors we need to ignore?

As you can see in the code, once we are in `Sentry` the error has already been converted to a `String` so the only thing we can do to see if we need to send the error or not is to match the `String` against our error message. 
If we have a lot of other logs we want to ignore I would suggest prefixing all the logs with something like:
```
User error: No space left on device
```
So in Sentry, we could just check if the log start by `User error:` and ignore all these errors at once

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-29 08:20:49 +00:00
bors[bot]
b0f399a51d
Merge #249
249: Use half of the computer threads for the indexing process by default r=Kerollmops a=irevoire

closes #241 
By default, we use only half of the CPU threads when indexing documents; this allows the user to use the search while indexing. Also, the machine will not appear unresponsive when indexing a large batch of documents.

On the special case where a user only has one core, we use it entirely 😄 

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-28 15:25:11 +00:00
Morgane Dubus
a26bb50d62
Update mini-dashboard to v.0.1.3 2021-06-28 15:13:52 +02:00
Tamo
a59f437ee3
use only half of the computer threads for the indexation by default 2021-06-28 14:35:50 +02:00
Tamo
d74c698adc
stop logging the no space left on device error 2021-06-28 13:59:48 +02:00
Tamo
c1c50f6714
unused borrow that must be used 2021-06-28 13:35:25 +02:00
marin postma
01b09c065b
change route to service<resource> 2021-06-24 19:02:28 +02:00
marin postma
3b601f615a
declare new authentication related errors 2021-06-24 16:53:20 +02:00
marin postma
fbd58f2eec
clippy 2021-06-24 16:36:22 +02:00
marin postma
79fc3bb84e
fmt 2021-06-24 16:36:22 +02:00
marin postma
8e4928c7ea
fix tests 2021-06-24 16:36:22 +02:00
marin postma
d078cbf39b
remove authentication middleware 2021-06-24 16:36:21 +02:00
marin postma
561596d8bc
update stats routes 2021-06-24 16:36:18 +02:00
marin postma
549b489c8a
update settings routes 2021-06-24 16:35:48 +02:00
marin postma
1e9f374ff8
update running route 2021-06-24 16:35:12 +02:00
marin postma
817fcfdd88
update keys route 2021-06-24 16:35:12 +02:00
marin postma
fab50256bc
update index routes 2021-06-24 16:35:04 +02:00
marin postma
b044608b25
update health route 2021-06-24 16:32:45 +02:00
marin postma
ce4fb8ce20
update dump route 2021-06-24 16:32:43 +02:00
marin postma
adf91d286b
update documents and search routes 2021-06-24 16:32:15 +02:00
marin postma
0c1c7a3dd9
implement authentication policies 2021-06-24 16:31:30 +02:00
marin postma
5b71751391
policies macros 2021-06-24 16:31:30 +02:00
marin postma
12f6709e1c
move authencation to extractor mod 2021-06-24 16:31:28 +02:00
marin postma
5229f1e220
experimental auth extractor 2021-06-24 16:30:15 +02:00
bors[bot]
b6ca7929eb
Merge #240
240: Rework error messages r=irevoire a=MarinPostma

Simplify the error messages, and make them more compliant with legacy Meilisearch.

Basically, stop composing the messages, and simply forward the message of inner errors.


Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-24 11:36:11 +00:00
Tamo
ad8d9a97d6
debug the body of every http request 2021-06-24 11:22:11 +02:00
Tamo
36f32f58d4
add the log_level variable to the cli and reduce the log level of milli and grenad 2021-06-24 11:20:52 +02:00
Tamo
b4fd4212ad
reduce the log level of some info! 2021-06-24 11:20:52 +02:00
marin postma
a1d34faaad
decompose error messages 2021-06-24 10:57:28 +02:00
bors[bot]
a2368db154
Merge #239
239: Bump milli to 0.6.0 r=MarinPostma a=MarinPostma

fix #231


Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-24 08:08:41 +00:00
marin postma
74bb748a4e
bump milli to 0.6.0 2021-06-23 18:40:19 +02:00
bors[bot]
8638c9ab77
Merge #232
232: Fix payload size limit r=MarinPostma a=MarinPostma

Fix #223

This was due to the fact that Payload ignores the limit payload size limit. I fixed it by implementing my own `Payload` extractor that checks that the size of the payload is not too large.

I also refactored the `create_app` a bit.

Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-23 16:06:08 +00:00
bors[bot]
b676b10cfe
Merge #238
238: Fix settings subroutes get r=MarinPostma a=MarinPostma

Fix #225 

Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-23 15:45:50 +00:00
marin postma
f68c257452
move flush in write_to_file function 2021-06-23 16:49:25 +02:00
marin postma
880fc069bd
remove dbg 2021-06-23 16:49:25 +02:00
marin postma
a838238a63
move payload to own module 2021-06-23 16:49:25 +02:00
marin postma
834995b130
clippy + fmt 2021-06-23 16:49:23 +02:00
marin postma
b000ae7614
remove file if write to update file fails 2021-06-23 16:48:33 +02:00
marin postma
f62779671b
change error message for payload size limit 2021-06-23 16:48:33 +02:00
marin postma
4b292c6e9b
add payload limit to app config 2021-06-23 16:48:33 +02:00
marin postma
1c13100948
implement custom payload 2021-06-23 16:48:31 +02:00
marin postma
71226feb74
refactor create_app macro 2021-06-23 16:47:15 +02:00
marin postma
b9b4feada8
add tests 2021-06-23 16:21:32 +02:00
bors[bot]
3175f09989
Merge #235
235: Fix dump not found error r=MarinPostma a=MarinPostma

fix #233


Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-23 14:21:07 +00:00
marin postma
322d6b8cfe
fix serialization bug in settings 2021-06-23 15:25:56 +02:00
marin postma
da36a6b5cd
fix not found error 2021-06-23 15:06:36 +02:00
bors[bot]
f2b2ca6d55
Merge #227
227: improve mini dashboard routing r=MarinPostma a=MarinPostma

The dependency we use to statically serve the mini-dashboard used globing to serve the mini-dashboard files. This caused all unfound routes to be caught by the "/" serving the dashboard assets. This fix makes it so that the assets have a dedicated route, and any unfound route is caught by the default service and return a 404.


Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-23 13:01:40 +00:00
Clémentine Urquizar
ec3140a29e
Fix clippy 2021-06-23 11:23:57 +02:00
Clémentine Urquizar
00b0a00fc5
Add exhaustiveFacetsCount 2021-06-23 11:05:30 +02:00
bors[bot]
adb970edcc
Merge #226
226: Make facetsDistribution name iso r=MarinPostma a=curquiza

Even if there is an English mistake in `facets_distribution` (because of the `s`) @gmourier asked me to keep the typo: the name of `facetsDistribution` might change completely in the future, he wants to avoid two breakings.

@gmourier can you confirm before we merge this PR?

Sorry I left this update in the code (I'm confused because no issues was open to update `facetsDistribution`), there might have been a confusion with `fieldsDistribution` that has been renamed into `fieldDistribution`. Sorry!

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-23 08:14:12 +00:00
Clémentine Urquizar
6d24a4744f
Roll back facetsDistribution 2021-06-23 10:04:01 +02:00
marin postma
b1a5ef0aab
improve mini dashboard routing 2021-06-22 21:49:05 +02:00
bors[bot]
7ec752ed1c
Merge #224
224: Update version for alpha 6 r=MarinPostma a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-22 18:20:09 +00:00
Clémentine Urquizar
0de696feaf
Update version for alpha 6 2021-06-22 18:40:51 +02:00
marin postma
3456a78552
refactor formatter
share the analyzer instance between the formatter and the
compute_matches function
2021-06-22 18:28:20 +02:00
marin postma
eb3d63691a
add tests 2021-06-22 18:12:53 +02:00
marin postma
c4ee937635
optimize fromat string 2021-06-22 18:12:53 +02:00
marin postma
f6d1fb7ac2
fmt 2021-06-22 18:12:53 +02:00