Commit Graph

528 Commits

Author SHA1 Message Date
Tamo
87a8bf5e96
write and load the user-id in the dumps 2021-10-29 17:25:51 +02:00
Tamo
ba14ea1243
plug the new batchers into the documents route 2021-10-29 17:25:51 +02:00
Tamo
9be90011c6
save the user-id in the config dir of the OS 2021-10-29 17:25:51 +02:00
Tamo
f9b14ca149
simplify the search batcher 2021-10-29 17:25:50 +02:00
Tamo
6591acfdfa
rename the documents batchers 2021-10-29 17:25:50 +02:00
Tamo
e64ba122e1
factorize the code between the two documents batcher 2021-10-29 17:25:50 +02:00
Tamo
a9523146a3
simplify the into_events methods 2021-10-29 17:25:50 +02:00
Tamo
392ee86714
implement the documents batcher 2021-10-29 17:25:50 +02:00
Tamo
1d73f484f0
update the primary key when creating a new index 2021-10-29 17:25:50 +02:00
Tamo
cfcd3ae048
move the version to context.app 2021-10-29 17:25:50 +02:00
Tamo
5395041dcb
fix the stats and stop sending events when no request happened 2021-10-29 17:25:49 +02:00
Tamo
40eabd50d1
integrate the search batcher in the search route 2021-10-29 17:25:49 +02:00
Tamo
35ffd0ec3a
integrate the search batcher in the tick 2021-10-29 17:25:49 +02:00
Tamo
d3d76bf97a
wip create a search batcher 2021-10-29 17:25:49 +02:00
Tamo
595ae42e94
update the name of the Launched event 2021-10-29 17:25:49 +02:00
Tamo
0667d940f9
update the name of nb_cores in the identify 2021-10-29 17:25:49 +02:00
Irevoire
75d1272325
log the dump creation 2021-10-29 17:25:49 +02:00
Irevoire
8e2d6cf87d
add the content type to all the route 2021-10-29 17:25:48 +02:00
Irevoire
9e1bba40f7
do not print anything if no user id was found 2021-10-29 17:25:48 +02:00
Irevoire
f7bb499c28
send the first identify + launched for the first time events right away instead of batching them 2021-10-29 17:25:48 +02:00
Irevoire
b33b1ef3dd
update the way of getting and saving the user-id to the file system 2021-10-29 17:25:48 +02:00
Irevoire
30aeda7a1a
update the identify call to the latest spec version 2021-10-29 17:25:48 +02:00
Irevoire
22d9d660cc
log all the required settings route 2021-10-29 17:25:48 +02:00
Irevoire
7524bfc07f
log the all settings updated route 2021-10-29 17:25:48 +02:00
Tamo
bda7472880
log the documetns updated route 2021-10-29 17:25:48 +02:00
Tamo
1ed05c6c07
log documents added 2021-10-29 17:25:47 +02:00
Tamo
0b3e0a59cb
log index updated 2021-10-29 17:25:47 +02:00
Tamo
0616f68eb0
implements part of the search 2021-10-29 17:25:47 +02:00
Tamo
6b8e5a4c92
log the index created route 2021-10-29 17:25:47 +02:00
Tamo
d72c887422
makes the analytics available for all the routes 2021-10-29 17:25:47 +02:00
Tamo
664d09e86a
makes the analytics works with the option and the feature 2021-10-29 17:25:47 +02:00
Tamo
e226b1a87f
rewrite the main analytics module and the information sent in the tick 2021-10-29 17:25:42 +02:00
many
7464720426
Fix some errors 2021-10-28 10:47:59 +02:00
many
61c15b69fb
Change malformed_payload error 2021-10-27 11:13:12 +02:00
many
b4038597ba
Keep persisting tmp files in database directory and put non-persisting tmp files in default tmp dir 2021-10-18 14:16:35 +02:00
Paul Sanders
c9e1d054c7 Fix csv content-type error 2021-10-12 13:38:48 -04:00
Tamo
66dbd3cd34
makes clippy happy 2021-10-06 17:39:04 +02:00
Tamo
9a1e44dc78
Apply suggestion
- remove the payload_error_handler in favor of a PayloadError::from

- merge the two match branch into one

- makes the accepted content type a const instead of recalculating it for every error
2021-10-06 17:15:47 +02:00
Tamo
c6d107a05f
makes the content-type mandatory for every routes 2021-10-06 17:15:47 +02:00
bors[bot]
2190764162
Merge #1768
1768: Fix auth error r=irevoire a=MarinPostma

fix a small auth error, that set the invalid token error token to "hello". This was invilisble to the user because the invalid token is not returned.

thank you hawk-eye `@irevoire` 

Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-10-05 15:16:14 +00:00
mpostma
3b91764587 fix auth error 2021-10-05 09:09:40 +02:00
Anirudh Rowjee
bffab21b10 Changes
1. Removed redundant scope registration
2021-10-04 14:47:05 +05:30
Anirudh Rowjee
151f691609 Fixes #1750
This commit is a fix to issue #1750.
As a part of the changes to solve this issue, the following changes have
been made -
1. Route registration for static assets has been modified
2. the `mut` keyword on the `scope` has been removed.
2021-10-02 15:24:04 +05:30
mpostma
4eb3817b03
missing payload error 2021-09-30 16:58:13 +02:00
mpostma
18cb514073
invalid content type error 2021-09-30 16:58:13 +02:00
mpostma
137272b8de
empty content type error 2021-09-30 16:58:13 +02:00
mpostma
fe5df6d06f fix payload content type guards 2021-09-29 19:04:47 +02:00
bors[bot]
5fad37aebd
Merge #1711
1711: MeiliSearch refactor introducing OBKV format r=MarinPostma a=MarinPostma

This PR refactor some multiple components of meilisearch, and introduce the obkv document format to meilisearch

- [x] Split meilisearch-http and meilisearch-lib
- [x] Replace `IndexActor` and `UuidResolver` with `IndexResolver`
- [x] Remove mentions to Actor
- [x] Remove Actor traits to simplify code
- [x] Integrate obkv document format
- [x] Remove `Data`
- [x] Restore all route
- [x] Replace `Box<dyn error>` with `anyhow::Error`
- [x] Introduce update file store
- [x] Update file store error handling
- [x] Fix dumps
- [x] Fix snapshots
- [x] Fix tests
- [x] Update module documentation
- [x] add csv suppport (feat `@ManyTheFish` #1729 )
- [x] add jsonl support
- [x] integrate geosearch (feat `@irevoire` #1725) 

partially implements #1691 and #1690. The error handling is very basic now, I will finish it in the next pr.

Some unit tests have been disabled, I will re-enable them ASAP, but they need a bit more work.

close #1531 


P.S: sorry for this monstrous PR :'(

Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: many <maxime@meilisearch.com>
2021-09-29 14:38:55 +00:00
mpostma
8fa6502b16 review changes 2021-09-29 14:17:41 +02:00
mpostma
1f537e1b60 jsonl support 2021-09-29 11:28:02 +02:00
mpostma
5bac65f8b8 add missing content type errors 2021-09-29 09:55:35 +02:00
mpostma
911630000f split csv and json document routes 2021-09-29 00:12:25 +02:00
mpostma
102c46f88b clippy + fmt 2021-09-28 22:22:59 +02:00
mpostma
3503fbf7fe re-export milli from meilisearch_lib 2021-09-28 22:08:03 +02:00
mpostma
7a27cbcc78 rename RegisterUpdate to store::Update 2021-09-28 20:20:13 +02:00
mpostma
56766cffc3 remove module level doc 2021-09-28 18:58:56 +02:00
mpostma
692c676625 fix tests 2021-09-28 18:57:36 +02:00
Tamo
ddfd7def35 add a TODO while waiting for the tests to be fixed 2021-09-28 18:17:56 +02:00
Tamo
539a57026d
fix the sort error messages 2021-09-28 14:50:26 +02:00
Tamo
c1376a9f2a
add the geosearch to Meilisearch 2021-09-28 14:50:26 +02:00
mpostma
90018755c5 restore snapshots 2021-09-27 16:48:03 +02:00
mpostma
b9d189bf12 restore document deletion routes 2021-09-24 15:21:07 +02:00
mpostma
c32012c44a restore settings updates 2021-09-24 14:55:57 +02:00
mpostma
dfce44fa3b rename data to meilisearch 2021-09-24 12:03:16 +02:00
mpostma
5353be74c3 refactor index actor 2021-09-22 15:07:04 +02:00
mpostma
60518449fc split meilisearch-http and meilisearch-lib 2021-09-21 13:23:22 +02:00
mpostma
09d4e37044 split data and api keys 2021-09-20 15:31:03 +02:00
mpostma
e14640e530 refactor meilisearch 2021-09-20 14:54:20 +02:00
Felix Yan
a57943b77e
Use tikv-jemallocator instead of jemallocator
`jemallocator` has been abandoned for nearly two years, and `rustc`
itself moved to use `tikv-jemallocator` instead:
3965773ae7

Let's switch to a better maintained version.
2021-09-14 18:30:24 +03:00
bors[bot]
6fafdb7711
Merge #1651 #1676 #1684
1651: Use reset_sortable_fields r=Kerollmops a=shekhirin

Resolves https://github.com/meilisearch/MeiliSearch/issues/1635

1676: Add curl binary to final stage image r=curquiza a=ook

Reference: #1673 

Changes: * add `curl` binary to final docker Melisearch image.

For metrics, docker funny layer management makes this add a  shrink from 319MB to 315MB:

```
☁  MeiliSearch [feature/1673-add-curl-to-docker-image]  docker image ls
REPOSITORY                                                         TAG                  IMAGE ID       CREATED         SIZE
getmeili/meilisearch                                               0.22.0_ook_1673      938e239ad989   2 hours ago     315MB
getmeili/meilisearch                                               latest               258fa3aa1230   6 days ago      319MB
```

1684: bump dependencies r=MarinPostma a=MarinPostma

Bump meilisearch dependencies.

We still depend on custom patch that have been upgraded along the way.

Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
Co-authored-by: Thomas Lecavelier <thomas@followanalytics.com>
Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-09-13 13:20:29 +00:00
mpostma
0f7625e29a bump dependencies 2021-09-13 15:17:08 +02:00
bors[bot]
c101b2a5cb
Merge #1686
1686: Bump milli r=curquiza a=irevoire

 fixes #1685, #1678, #1671, #1677 and #1680

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-09-08 16:31:02 +00:00
bors[bot]
971c361e0f
Merge #1682
1682: Change the format of custom ranking rules when importing old dumps r=curquiza a=Kerollmops

This PR changes the format of the custom ranking rules from `asc(price)` to `title:asc` as the format changed between v0.21 and v0.22. The dumps are now correctly importing the custom ranking rules.

This PR also change the previous default ranking rules (without sort) to the new default ranking rules (with the new sort).

Co-authored-by: Kerollmops <clement@meilisearch.com>
2021-09-08 16:20:10 +00:00
Kerollmops
be50b2bec6
Change the format of custom ranking rules when importing v2 dumps 2021-09-08 17:56:21 +02:00
Tamo
49c918defa
bump milli 2021-09-08 17:41:47 +02:00
many
169e739634
Remove useless indexer options 2021-09-08 13:40:05 +02:00
many
08138c7c23
Use set indexer options instead of create a default one 2021-09-08 13:40:00 +02:00
Kerollmops
331d28102f
Change the format of custom ranking rules when importing v1 dumps 2021-09-07 17:16:40 +02:00
Alexey Shekhirin
efa69875d9
refactor(http): use reset_sortable_fields 2021-09-07 15:04:44 +03:00
many
c0f9c891f5
Set max_memory value to unlimited during tests
because tests run several meilisearch in parallel,
we over estimate the value for max_memory making the tests on widows crash
2021-09-06 14:38:10 +02:00
Kerollmops
4cdf680a81
Make the MaxMemory use the default value when undefined 2021-09-02 18:19:11 +02:00
Kerollmops
0cd66c3a89
Bump the milli version 2021-09-02 18:19:00 +02:00
Clément Renault
b092a624ed
Introduce the MaxMemory struct that defaults to 2/3 of the available memory 2021-09-02 18:18:59 +02:00
Clémentine Urquizar
24e84d7ca1
Test new indexer 2021-09-02 18:11:20 +02:00
bors[bot]
19c5c74291
Merge #1652 #1654 #1657
1652: Remove dependabot r=MarinPostma a=curquiza

Fixes #1649 

Dependabot for vulnerability and security updates is still activated.

1654: Add Script for Windows r=MarinPostma a=singh08prashant

fixes #1570 

changes:

1. added script for detecting windows os running git bash
2. appended `.exe` to `$release_file` for windows as listed [here](https://github.com/meilisearch/MeiliSearch/releases/)
3. removed global `$BINARY_NAME='meilisearch'` as windows require `.exe` file

1657: Bring vergen hotfix from `stable` to `main` r=MarinPostma a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: singh08prashant <singh08prashant@gmail.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
2021-08-31 14:31:42 +00:00
Kerollmops
9d0fa8112b
Remove unused Arc import 2021-08-31 14:50:36 +02:00
Kerollmops
a8c146fd13
Unwrap or unknown the commit hash 2021-08-30 17:41:24 +02:00
Kerollmops
cf4a466b6b
Make sure that the order of the filterableAttributes is constant 2021-08-26 11:06:05 +02:00
Kerollmops
087e4626ce
Make sure that the order of the sortableAttributes is constant 2021-08-26 11:06:04 +02:00
Kerollmops
e0f73fe742
Introduce the sort search parameter 2021-08-25 17:39:25 +02:00
Kerollmops
ea4c831de0
Integrate the sortable-attributes into the settings 2021-08-25 17:39:25 +02:00
Kerollmops
51387b2c80
Introduce the new invalid sortable error codes 2021-08-25 17:29:30 +02:00
bors[bot]
2d8dd87cad
Merge #1623
1623: Use Setting enum r=Kerollmops a=shekhirin

Resolves https://github.com/meilisearch/MeiliSearch/issues/1620

Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
2021-08-25 14:58:40 +00:00
Alexey Shekhirin
d9dd2a038b
refactor(http): use Setting enum 2021-08-25 17:43:46 +03:00
Irevoire
672fcee8aa
remove sentry 2021-08-24 12:38:31 +02:00
Clémentine Urquizar
621529e9dc
Update telemetry link 2021-08-12 18:58:07 +02:00
bors[bot]
7e3b2ddff2
Merge #1554
1554: Fix dump v1 (attributesForFaceting, and criteria) r=curquiza a=MarinPostma

close #1553


Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-08-05 19:45:52 +00:00
mpostma
8f05d8d546 fix clippy warnings 2021-08-05 16:00:47 +02:00
mpostma
f5ddea481a reintroduce exactness 2021-08-05 15:59:39 +02:00
mpostma
29ca8271b3 test dumpv1 format regression 2021-08-05 15:59:39 +02:00