Commit Graph

545 Commits

Author SHA1 Message Date
Tamo
ea5ae2bae5
sort the imports 2021-10-29 17:25:54 +02:00
Tamo
72e3adc55e
display an instance-id instead of a user-id 2021-10-29 17:25:54 +02:00
Tamo
b250392e8d
remove the first - in the path to the db instance in the instance-id 2021-10-29 17:25:53 +02:00
Tamo
d8b0d68840
use a regex to count the number of filters instead of split + flatten 2021-10-29 17:25:53 +02:00
Tamo
a1ab02f9fb
remove some commented code 2021-10-29 17:25:53 +02:00
Tamo
bba64b32ca
async_traits is not needed anymore 2021-10-29 17:25:53 +02:00
Tamo
9abd2aa9d7
make the analytics interval a const 2021-10-29 17:25:53 +02:00
Tamo
ed750e8792
fix start_since_day 2021-10-29 17:25:53 +02:00
Tamo
37ca50832c
fix the sort position 2021-10-29 17:25:52 +02:00
Tamo
31c7a0105b
fix a bug on the batch documents function 2021-10-29 17:25:52 +02:00
Tamo
ddab9eafa1
fix a typo 2021-10-29 17:25:52 +02:00
Tamo
76a4f86e0c
rename user-id to instance-uid 2021-10-29 17:25:52 +02:00
Tamo
6b34318274
makes clippy happy 2021-10-29 17:25:52 +02:00
Tamo
5508c6c154
a bit of styling 2021-10-29 17:25:52 +02:00
Tamo
9a62ac0c94
send the analytics only once every hours 2021-10-29 17:25:52 +02:00
Tamo
01737ef847
remove all the debug prints 2021-10-29 17:25:51 +02:00
Tamo
10de92987a
compile write_user_id only when the analytics are enabled 2021-10-29 17:25:51 +02:00
Tamo
87a8bf5e96
write and load the user-id in the dumps 2021-10-29 17:25:51 +02:00
Tamo
ba14ea1243
plug the new batchers into the documents route 2021-10-29 17:25:51 +02:00
Tamo
9be90011c6
save the user-id in the config dir of the OS 2021-10-29 17:25:51 +02:00
Tamo
f9b14ca149
simplify the search batcher 2021-10-29 17:25:50 +02:00
Tamo
6591acfdfa
rename the documents batchers 2021-10-29 17:25:50 +02:00
Tamo
e64ba122e1
factorize the code between the two documents batcher 2021-10-29 17:25:50 +02:00
Tamo
a9523146a3
simplify the into_events methods 2021-10-29 17:25:50 +02:00
Tamo
392ee86714
implement the documents batcher 2021-10-29 17:25:50 +02:00
Tamo
1d73f484f0
update the primary key when creating a new index 2021-10-29 17:25:50 +02:00
Tamo
cfcd3ae048
move the version to context.app 2021-10-29 17:25:50 +02:00
Tamo
5395041dcb
fix the stats and stop sending events when no request happened 2021-10-29 17:25:49 +02:00
Tamo
40eabd50d1
integrate the search batcher in the search route 2021-10-29 17:25:49 +02:00
Tamo
35ffd0ec3a
integrate the search batcher in the tick 2021-10-29 17:25:49 +02:00
Tamo
d3d76bf97a
wip create a search batcher 2021-10-29 17:25:49 +02:00
Tamo
595ae42e94
update the name of the Launched event 2021-10-29 17:25:49 +02:00
Tamo
0667d940f9
update the name of nb_cores in the identify 2021-10-29 17:25:49 +02:00
Irevoire
75d1272325
log the dump creation 2021-10-29 17:25:49 +02:00
Irevoire
8e2d6cf87d
add the content type to all the route 2021-10-29 17:25:48 +02:00
Irevoire
9e1bba40f7
do not print anything if no user id was found 2021-10-29 17:25:48 +02:00
Irevoire
f7bb499c28
send the first identify + launched for the first time events right away instead of batching them 2021-10-29 17:25:48 +02:00
Irevoire
b33b1ef3dd
update the way of getting and saving the user-id to the file system 2021-10-29 17:25:48 +02:00
Irevoire
30aeda7a1a
update the identify call to the latest spec version 2021-10-29 17:25:48 +02:00
Irevoire
22d9d660cc
log all the required settings route 2021-10-29 17:25:48 +02:00
Irevoire
7524bfc07f
log the all settings updated route 2021-10-29 17:25:48 +02:00
Tamo
bda7472880
log the documetns updated route 2021-10-29 17:25:48 +02:00
Tamo
1ed05c6c07
log documents added 2021-10-29 17:25:47 +02:00
Tamo
0b3e0a59cb
log index updated 2021-10-29 17:25:47 +02:00
Tamo
0616f68eb0
implements part of the search 2021-10-29 17:25:47 +02:00
Tamo
6b8e5a4c92
log the index created route 2021-10-29 17:25:47 +02:00
Tamo
d72c887422
makes the analytics available for all the routes 2021-10-29 17:25:47 +02:00
Tamo
664d09e86a
makes the analytics works with the option and the feature 2021-10-29 17:25:47 +02:00
Tamo
e226b1a87f
rewrite the main analytics module and the information sent in the tick 2021-10-29 17:25:42 +02:00
many
7464720426
Fix some errors 2021-10-28 10:47:59 +02:00
many
61c15b69fb
Change malformed_payload error 2021-10-27 11:13:12 +02:00
many
b4038597ba
Keep persisting tmp files in database directory and put non-persisting tmp files in default tmp dir 2021-10-18 14:16:35 +02:00
Paul Sanders
c9e1d054c7 Fix csv content-type error 2021-10-12 13:38:48 -04:00
Tamo
66dbd3cd34
makes clippy happy 2021-10-06 17:39:04 +02:00
Tamo
9a1e44dc78
Apply suggestion
- remove the payload_error_handler in favor of a PayloadError::from

- merge the two match branch into one

- makes the accepted content type a const instead of recalculating it for every error
2021-10-06 17:15:47 +02:00
Tamo
c6d107a05f
makes the content-type mandatory for every routes 2021-10-06 17:15:47 +02:00
bors[bot]
2190764162
Merge #1768
1768: Fix auth error r=irevoire a=MarinPostma

fix a small auth error, that set the invalid token error token to "hello". This was invilisble to the user because the invalid token is not returned.

thank you hawk-eye `@irevoire` 

Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-10-05 15:16:14 +00:00
mpostma
3b91764587 fix auth error 2021-10-05 09:09:40 +02:00
Anirudh Rowjee
bffab21b10 Changes
1. Removed redundant scope registration
2021-10-04 14:47:05 +05:30
Anirudh Rowjee
151f691609 Fixes #1750
This commit is a fix to issue #1750.
As a part of the changes to solve this issue, the following changes have
been made -
1. Route registration for static assets has been modified
2. the `mut` keyword on the `scope` has been removed.
2021-10-02 15:24:04 +05:30
mpostma
4eb3817b03
missing payload error 2021-09-30 16:58:13 +02:00
mpostma
18cb514073
invalid content type error 2021-09-30 16:58:13 +02:00
mpostma
137272b8de
empty content type error 2021-09-30 16:58:13 +02:00
mpostma
fe5df6d06f fix payload content type guards 2021-09-29 19:04:47 +02:00
bors[bot]
5fad37aebd
Merge #1711
1711: MeiliSearch refactor introducing OBKV format r=MarinPostma a=MarinPostma

This PR refactor some multiple components of meilisearch, and introduce the obkv document format to meilisearch

- [x] Split meilisearch-http and meilisearch-lib
- [x] Replace `IndexActor` and `UuidResolver` with `IndexResolver`
- [x] Remove mentions to Actor
- [x] Remove Actor traits to simplify code
- [x] Integrate obkv document format
- [x] Remove `Data`
- [x] Restore all route
- [x] Replace `Box<dyn error>` with `anyhow::Error`
- [x] Introduce update file store
- [x] Update file store error handling
- [x] Fix dumps
- [x] Fix snapshots
- [x] Fix tests
- [x] Update module documentation
- [x] add csv suppport (feat `@ManyTheFish` #1729 )
- [x] add jsonl support
- [x] integrate geosearch (feat `@irevoire` #1725) 

partially implements #1691 and #1690. The error handling is very basic now, I will finish it in the next pr.

Some unit tests have been disabled, I will re-enable them ASAP, but they need a bit more work.

close #1531 


P.S: sorry for this monstrous PR :'(

Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: many <maxime@meilisearch.com>
2021-09-29 14:38:55 +00:00
mpostma
8fa6502b16 review changes 2021-09-29 14:17:41 +02:00
mpostma
1f537e1b60 jsonl support 2021-09-29 11:28:02 +02:00
mpostma
5bac65f8b8 add missing content type errors 2021-09-29 09:55:35 +02:00
mpostma
911630000f split csv and json document routes 2021-09-29 00:12:25 +02:00
mpostma
102c46f88b clippy + fmt 2021-09-28 22:22:59 +02:00
mpostma
3503fbf7fe re-export milli from meilisearch_lib 2021-09-28 22:08:03 +02:00
mpostma
7a27cbcc78 rename RegisterUpdate to store::Update 2021-09-28 20:20:13 +02:00
mpostma
56766cffc3 remove module level doc 2021-09-28 18:58:56 +02:00
mpostma
692c676625 fix tests 2021-09-28 18:57:36 +02:00
Tamo
ddfd7def35 add a TODO while waiting for the tests to be fixed 2021-09-28 18:17:56 +02:00
Tamo
539a57026d
fix the sort error messages 2021-09-28 14:50:26 +02:00
Tamo
c1376a9f2a
add the geosearch to Meilisearch 2021-09-28 14:50:26 +02:00
mpostma
90018755c5 restore snapshots 2021-09-27 16:48:03 +02:00
mpostma
b9d189bf12 restore document deletion routes 2021-09-24 15:21:07 +02:00
mpostma
c32012c44a restore settings updates 2021-09-24 14:55:57 +02:00
mpostma
dfce44fa3b rename data to meilisearch 2021-09-24 12:03:16 +02:00
mpostma
5353be74c3 refactor index actor 2021-09-22 15:07:04 +02:00
mpostma
60518449fc split meilisearch-http and meilisearch-lib 2021-09-21 13:23:22 +02:00
mpostma
09d4e37044 split data and api keys 2021-09-20 15:31:03 +02:00
mpostma
e14640e530 refactor meilisearch 2021-09-20 14:54:20 +02:00
Felix Yan
a57943b77e
Use tikv-jemallocator instead of jemallocator
`jemallocator` has been abandoned for nearly two years, and `rustc`
itself moved to use `tikv-jemallocator` instead:
3965773ae7

Let's switch to a better maintained version.
2021-09-14 18:30:24 +03:00
bors[bot]
6fafdb7711
Merge #1651 #1676 #1684
1651: Use reset_sortable_fields r=Kerollmops a=shekhirin

Resolves https://github.com/meilisearch/MeiliSearch/issues/1635

1676: Add curl binary to final stage image r=curquiza a=ook

Reference: #1673 

Changes: * add `curl` binary to final docker Melisearch image.

For metrics, docker funny layer management makes this add a  shrink from 319MB to 315MB:

```
☁  MeiliSearch [feature/1673-add-curl-to-docker-image]  docker image ls
REPOSITORY                                                         TAG                  IMAGE ID       CREATED         SIZE
getmeili/meilisearch                                               0.22.0_ook_1673      938e239ad989   2 hours ago     315MB
getmeili/meilisearch                                               latest               258fa3aa1230   6 days ago      319MB
```

1684: bump dependencies r=MarinPostma a=MarinPostma

Bump meilisearch dependencies.

We still depend on custom patch that have been upgraded along the way.

Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
Co-authored-by: Thomas Lecavelier <thomas@followanalytics.com>
Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-09-13 13:20:29 +00:00
mpostma
0f7625e29a bump dependencies 2021-09-13 15:17:08 +02:00
bors[bot]
c101b2a5cb
Merge #1686
1686: Bump milli r=curquiza a=irevoire

 fixes #1685, #1678, #1671, #1677 and #1680

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-09-08 16:31:02 +00:00
bors[bot]
971c361e0f
Merge #1682
1682: Change the format of custom ranking rules when importing old dumps r=curquiza a=Kerollmops

This PR changes the format of the custom ranking rules from `asc(price)` to `title:asc` as the format changed between v0.21 and v0.22. The dumps are now correctly importing the custom ranking rules.

This PR also change the previous default ranking rules (without sort) to the new default ranking rules (with the new sort).

Co-authored-by: Kerollmops <clement@meilisearch.com>
2021-09-08 16:20:10 +00:00
Kerollmops
be50b2bec6
Change the format of custom ranking rules when importing v2 dumps 2021-09-08 17:56:21 +02:00
Tamo
49c918defa
bump milli 2021-09-08 17:41:47 +02:00
many
169e739634
Remove useless indexer options 2021-09-08 13:40:05 +02:00
many
08138c7c23
Use set indexer options instead of create a default one 2021-09-08 13:40:00 +02:00
Kerollmops
331d28102f
Change the format of custom ranking rules when importing v1 dumps 2021-09-07 17:16:40 +02:00
Alexey Shekhirin
efa69875d9
refactor(http): use reset_sortable_fields 2021-09-07 15:04:44 +03:00
many
c0f9c891f5
Set max_memory value to unlimited during tests
because tests run several meilisearch in parallel,
we over estimate the value for max_memory making the tests on widows crash
2021-09-06 14:38:10 +02:00
Kerollmops
4cdf680a81
Make the MaxMemory use the default value when undefined 2021-09-02 18:19:11 +02:00
Kerollmops
0cd66c3a89
Bump the milli version 2021-09-02 18:19:00 +02:00
Clément Renault
b092a624ed
Introduce the MaxMemory struct that defaults to 2/3 of the available memory 2021-09-02 18:18:59 +02:00