searchengines-web/MeiliSearch
searchengines-web/MeiliSearch
1
0
Fork 0
mirror of https://github.com/meilisearch/MeiliSearch synced 2024-12-24 13:40:31 +01:00
Code Issues Releases Wiki Activity
3,410 Commits 176 Branches 395 Tags 188 MiB
Commit Graph

717 Commits

Author SHA1 Message Date
many
6bbc1b4316
Remove error flattening in task serialization 2021-11-04 17:40:28 +01:00
many
3c696da274
Update tests 2021-11-04 17:40:28 +01:00
bors[bot]
d9d6dee550
Merge #1873 
1873: Change lacking errors r=ManyTheFish a=ManyTheFish



Co-authored-by: many <maxime@meilisearch.com>
 2021-11-04 14:21:52 +00:00
many
b59145385e
Fix PR comments 2021-11-04 14:57:27 +01:00
bors[bot]
3f4e0ec971
Merge #1875 #1876 
1875: Fix search post event and disk size analytics r=irevoire a=gmourier

- Branch POST search on the post_search aggregator
- Use largest disk `total_space` instead of `available_space` 

1876: Update SEGMENT_API_KEY r=irevoire a=gmourier

Branch it on our Segment production stack

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
 2021-11-04 10:16:13 +00:00
Guillaume Mourier
6d6725b3b8 Update SEGMENT_API_KEY 2021-11-04 08:10:12 +01:00
Guillaume Mourier
6660be2cb7 Branch POST /search on the dedicated analytics aggregator 2021-11-04 08:03:48 +01:00
Guillaume Mourier
847fcb570b Use total_space of the largest disk instead of available_space 2021-11-04 08:03:11 +01:00
many
30a094cbb2
Change lacking errors 2021-11-03 14:33:33 +01:00
Tamo
904bae98f8
send the analytics even when the search fail 2021-11-02 12:38:01 +01:00
marin postma
519093ea65
fix bad rebase 2021-10-29 17:32:49 +02:00
Tamo
bd49d1c4b5
fix one small bug 2021-10-29 17:25:56 +02:00
marin postma
2665c0099d
clippy + fmt 2021-10-29 17:25:56 +02:00
marin postma
d65f055030
pass anaytics into Arc instead of static ref 2021-10-29 17:25:55 +02:00
Tamo
66d87761b7
align the parameters in the launche resume 2021-10-29 17:25:55 +02:00
Tamo
ba69ad672a
fix the timing issue 2021-10-29 17:25:55 +02:00
Tamo
7934e3956b
replace all mutexes by channel 2021-10-29 17:25:55 +02:00
Guillaume Mourier
68fe93b7db
add ranking_rules marker before sort_position 2021-10-29 17:25:55 +02:00
Tamo
efd0ea9e1e
makes clippy happier 2021-10-29 17:25:55 +02:00
Tamo
6ef73eb226
fix all the single settings route and add the searchable attributes Updated event 2021-10-29 17:25:55 +02:00
Tamo
fc2f23d36c
move the start_since_days to teh root of the identify 2021-10-29 17:25:54 +02:00
Tamo
7c39fab453
move the user-agent out of the context in every request 2021-10-29 17:25:54 +02:00
Tamo
c5164c01c0
set the total of sortable attributes and filterable-attributes to 0 when not set 2021-10-29 17:25:54 +02:00
Tamo
351ad32d77
fix the index_creation boolean 2021-10-29 17:25:54 +02:00
Tamo
3ad8311bdd
split the analytics in a module 2021-10-29 17:25:54 +02:00
Tamo
ea5ae2bae5
sort the imports 2021-10-29 17:25:54 +02:00
Tamo
72e3adc55e
display an instance-id instead of a user-id 2021-10-29 17:25:54 +02:00
Tamo
b250392e8d
remove the first - in the path to the db instance in the instance-id 2021-10-29 17:25:53 +02:00
Tamo
d8b0d68840
use a regex to count the number of filters instead of split + flatten 2021-10-29 17:25:53 +02:00
Tamo
c4737749ab
bump segment to be able to display a user 2021-10-29 17:25:53 +02:00
Tamo
a1ab02f9fb
remove some commented code 2021-10-29 17:25:53 +02:00
Tamo
bba64b32ca
async_traits is not needed anymore 2021-10-29 17:25:53 +02:00
Tamo
9abd2aa9d7
make the analytics interval a const 2021-10-29 17:25:53 +02:00
Tamo
de35a9a605
use an official release of segment 2021-10-29 17:25:53 +02:00
Tamo
ed750e8792
fix start_since_day 2021-10-29 17:25:53 +02:00
Tamo
37ca50832c
fix the sort position 2021-10-29 17:25:52 +02:00
Tamo
31c7a0105b
fix a bug on the batch documents function 2021-10-29 17:25:52 +02:00
Tamo
ddab9eafa1
fix a typo 2021-10-29 17:25:52 +02:00
Tamo
76a4f86e0c
rename user-id to instance-uid 2021-10-29 17:25:52 +02:00
Tamo
6b34318274
makes clippy happy 2021-10-29 17:25:52 +02:00
Tamo
5508c6c154
a bit of styling 2021-10-29 17:25:52 +02:00
Tamo
9a62ac0c94
send the analytics only once every hours 2021-10-29 17:25:52 +02:00
Tamo
01737ef847
remove all the debug prints 2021-10-29 17:25:51 +02:00
Tamo
10de92987a
compile write_user_id only when the analytics are enabled 2021-10-29 17:25:51 +02:00
Tamo
87a8bf5e96
write and load the user-id in the dumps 2021-10-29 17:25:51 +02:00
Tamo
ba14ea1243
plug the new batchers into the documents route 2021-10-29 17:25:51 +02:00
Tamo
9be90011c6
save the user-id in the config dir of the OS 2021-10-29 17:25:51 +02:00
Tamo
f9b14ca149
simplify the search batcher 2021-10-29 17:25:50 +02:00
Tamo
6591acfdfa
rename the documents batchers 2021-10-29 17:25:50 +02:00
Tamo
e64ba122e1
factorize the code between the two documents batcher 2021-10-29 17:25:50 +02:00
Tamo
a9523146a3
simplify the into_events methods 2021-10-29 17:25:50 +02:00
Tamo
392ee86714
implement the documents batcher 2021-10-29 17:25:50 +02:00
Tamo
1d73f484f0
update the primary key when creating a new index 2021-10-29 17:25:50 +02:00
Tamo
cfcd3ae048
move the version to context.app 2021-10-29 17:25:50 +02:00
Tamo
5395041dcb
fix the stats and stop sending events when no request happened 2021-10-29 17:25:49 +02:00
Tamo
40eabd50d1
integrate the search batcher in the search route 2021-10-29 17:25:49 +02:00
Tamo
35ffd0ec3a
integrate the search batcher in the tick 2021-10-29 17:25:49 +02:00
Tamo
d3d76bf97a
wip create a search batcher 2021-10-29 17:25:49 +02:00
Tamo
595ae42e94
update the name of the Launched event 2021-10-29 17:25:49 +02:00
Tamo
0667d940f9
update the name of nb_cores in the identify 2021-10-29 17:25:49 +02:00
Irevoire
75d1272325
log the dump creation 2021-10-29 17:25:49 +02:00
Irevoire
8e2d6cf87d
add the content type to all the route 2021-10-29 17:25:48 +02:00
Irevoire
9e1bba40f7
do not print anything if no user id was found 2021-10-29 17:25:48 +02:00
Irevoire
f7bb499c28
send the first identify + launched for the first time events right away instead of batching them 2021-10-29 17:25:48 +02:00
Irevoire
b33b1ef3dd
update the way of getting and saving the user-id to the file system 2021-10-29 17:25:48 +02:00
Irevoire
30aeda7a1a
update the identify call to the latest spec version 2021-10-29 17:25:48 +02:00
Irevoire
22d9d660cc
log all the required settings route 2021-10-29 17:25:48 +02:00
Irevoire
7524bfc07f
log the all settings updated route 2021-10-29 17:25:48 +02:00
Tamo
bda7472880
log the documetns updated route 2021-10-29 17:25:48 +02:00
Tamo
1ed05c6c07
log documents added 2021-10-29 17:25:47 +02:00
Tamo
0b3e0a59cb
log index updated 2021-10-29 17:25:47 +02:00
Tamo
0616f68eb0
implements part of the search 2021-10-29 17:25:47 +02:00
Tamo
6b8e5a4c92
log the index created route 2021-10-29 17:25:47 +02:00
Tamo
d72c887422
makes the analytics available for all the routes 2021-10-29 17:25:47 +02:00
Tamo
664d09e86a
makes the analytics works with the option and the feature 2021-10-29 17:25:47 +02:00
Tamo
e226b1a87f
rewrite the main analytics module and the information sent in the tick 2021-10-29 17:25:42 +02:00
bors[bot]
cf67964133
Merge #1848 
1848: Error format and Definition r=MarinPostma a=ManyTheFish



Co-authored-by: many <maxime@meilisearch.com>
 2021-10-28 14:15:35 +00:00
many
cbaca2b579
Fix PR comments 2021-10-28 15:42:42 +02:00
Clémentine Urquizar
a76d9b15c9
Update version for the next release (v0.24.0) 2021-10-28 12:24:49 +02:00
many
59636fa688
Pimp error where no document is provided 2021-10-28 12:13:51 +02:00
many
ff0908d3fa
Ignore errors tests that show unrelated bugs 2021-10-28 11:41:59 +02:00
many
21f35762ca
Fix content type test 2021-10-28 10:57:11 +02:00
many
7464720426
Fix some errors 2021-10-28 10:47:59 +02:00
CuriousCorrelation
6641e7aa50 Add tests for mini-dashboard status and assets 2021-10-27 17:57:25 +05:30
many
61c15b69fb
Change malformed_payload error 2021-10-27 11:13:12 +02:00
many
8ec0c4c913
Add bad_request error tests 2021-10-27 11:13:12 +02:00
many
b4038597ba
Keep persisting tmp files in database directory and put non-persisting tmp files in default tmp dir 2021-10-18 14:16:35 +02:00
Clémentine Urquizar
b602a0836a
Merge branch 'main' into stable 2021-10-14 13:43:21 +02:00
Paul Sanders
c9e1d054c7 Fix csv content-type error 2021-10-12 13:38:48 -04:00
Clémentine Urquizar
0f342ac46e
Update MeiliSearch version 2021-10-12 16:43:31 +02:00
Clémentine Urquizar
3edbc74430
Merge branch 'main' into stable 2021-10-11 18:30:10 +02:00
Tom Parker-Shemilt
6c46fbbc57 Remove memmap dependency 2021-10-10 22:33:40 +01:00
Tamo
66dbd3cd34
makes clippy happy 2021-10-06 17:39:04 +02:00
Tamo
9a1e44dc78
Apply suggestion 
- remove the payload_error_handler in favor of a PayloadError::from

- merge the two match branch into one

- makes the accepted content type a const instead of recalculating it for every error
 2021-10-06 17:15:47 +02:00
Tamo
37b267ffb3
duplicate the post document tests with the put verb 2021-10-06 17:15:47 +02:00
Tamo
dfa199f98f
add content-type tests 
fix typo

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
 2021-10-06 17:15:47 +02:00
Tamo
c6d107a05f
makes the content-type mandatory for every routes 2021-10-06 17:15:47 +02:00
bors[bot]
2190764162
Merge #1768 
1768: Fix auth error r=irevoire a=MarinPostma

fix a small auth error, that set the invalid token error token to "hello". This was invilisble to the user because the invalid token is not returned.

thank you hawk-eye `@irevoire` 

Co-authored-by: mpostma <postma.marin@protonmail.com>
 2021-10-05 15:16:14 +00:00
mpostma
3b91764587 fix auth error 2021-10-05 09:09:40 +02:00
Anirudh Rowjee
bffab21b10 Changes 
1. Removed redundant scope registration
 2021-10-04 14:47:05 +05:30
Anirudh Rowjee
151f691609 Fixes #1750 
This commit is a fix to issue #1750.
As a part of the changes to solve this issue, the following changes have
been made -
1. Route registration for static assets has been modified
2. the `mut` keyword on the `scope` has been removed.
 2021-10-02 15:24:04 +05:30
mpostma
4eb3817b03
missing payload error 2021-09-30 16:58:13 +02:00
mpostma
18cb514073
invalid content type error 2021-09-30 16:58:13 +02:00
mpostma
137272b8de
empty content type error 2021-09-30 16:58:13 +02:00
Kerollmops
c388dca5ec
Check that invalid updates do not create an index with a primary key 2021-09-30 15:46:04 +02:00
bors[bot]
05d5de47cb
Merge #1737 
1737: Update version for the next release (v0.23.0) r=irevoire a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
 2021-09-29 22:06:46 +00:00
mpostma
fe5df6d06f fix payload content type guards 2021-09-29 19:04:47 +02:00
Clémentine Urquizar
b17dae9ac0
Update version for the next release (v0.23.0) 2021-09-29 18:40:35 +02:00
bors[bot]
5fad37aebd
Merge #1711 
1711: MeiliSearch refactor introducing OBKV format r=MarinPostma a=MarinPostma

This PR refactor some multiple components of meilisearch, and introduce the obkv document format to meilisearch

- [x] Split meilisearch-http and meilisearch-lib
- [x] Replace `IndexActor` and `UuidResolver` with `IndexResolver`
- [x] Remove mentions to Actor
- [x] Remove Actor traits to simplify code
- [x] Integrate obkv document format
- [x] Remove `Data`
- [x] Restore all route
- [x] Replace `Box<dyn error>` with `anyhow::Error`
- [x] Introduce update file store
- [x] Update file store error handling
- [x] Fix dumps
- [x] Fix snapshots
- [x] Fix tests
- [x] Update module documentation
- [x] add csv suppport (feat `@ManyTheFish` #1729 )
- [x] add jsonl support
- [x] integrate geosearch (feat `@irevoire` #1725) 

partially implements #1691 and #1690. The error handling is very basic now, I will finish it in the next pr.

Some unit tests have been disabled, I will re-enable them ASAP, but they need a bit more work.

close #1531 


P.S: sorry for this monstrous PR :'(

Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: many <maxime@meilisearch.com>
 2021-09-29 14:38:55 +00:00
mpostma
8fa6502b16 review changes 2021-09-29 14:17:41 +02:00
mpostma
1f537e1b60 jsonl support 2021-09-29 11:28:02 +02:00
mpostma
5bac65f8b8 add missing content type errors 2021-09-29 09:55:35 +02:00
mpostma
911630000f split csv and json document routes 2021-09-29 00:12:25 +02:00
mpostma
102c46f88b clippy + fmt 2021-09-28 22:22:59 +02:00
mpostma
5fa9bc67d7 remove unused dependencies 2021-09-28 22:16:18 +02:00
mpostma
3503fbf7fe re-export milli from meilisearch_lib 2021-09-28 22:08:03 +02:00
mpostma
7a27cbcc78 rename RegisterUpdate to store::Update 2021-09-28 20:20:13 +02:00
mpostma
56766cffc3 remove module level doc 2021-09-28 18:58:56 +02:00
mpostma
692c676625 fix tests 2021-09-28 18:57:36 +02:00
Tamo
ddfd7def35 add a TODO while waiting for the tests to be fixed 2021-09-28 18:17:56 +02:00
Tamo
539a57026d
fix the sort error messages 2021-09-28 14:50:26 +02:00
Tamo
654f49ccec
[WIP] put milli on branch main 2021-09-28 14:50:26 +02:00
Tamo
c1376a9f2a
add the geosearch to Meilisearch 2021-09-28 14:50:26 +02:00
mpostma
90018755c5 restore snapshots 2021-09-27 16:48:03 +02:00
mpostma
b9d189bf12 restore document deletion routes 2021-09-24 15:21:07 +02:00
mpostma
c32012c44a restore settings updates 2021-09-24 14:55:57 +02:00
mpostma
dfce44fa3b rename data to meilisearch 2021-09-24 12:03:16 +02:00
mpostma
5353be74c3 refactor index actor 2021-09-22 15:07:04 +02:00
mpostma
60518449fc split meilisearch-http and meilisearch-lib 2021-09-21 13:23:22 +02:00
mpostma
09d4e37044 split data and api keys 2021-09-20 15:31:03 +02:00
mpostma
e14640e530 refactor meilisearch 2021-09-20 14:54:20 +02:00
Felix Yan
a57943b77e
Use tikv-jemallocator instead of jemallocator 
`jemallocator` has been abandoned for nearly two years, and `rustc`
itself moved to use `tikv-jemallocator` instead:
3965773ae7

Let's switch to a better maintained version.
 2021-09-14 18:30:24 +03:00
bors[bot]
6fafdb7711
Merge #1651 #1676 #1684 
1651: Use reset_sortable_fields r=Kerollmops a=shekhirin

Resolves https://github.com/meilisearch/MeiliSearch/issues/1635

1676: Add curl binary to final stage image r=curquiza a=ook

Reference: #1673 

Changes: * add `curl` binary to final docker Melisearch image.

For metrics, docker funny layer management makes this add a  shrink from 319MB to 315MB:

```
☁  MeiliSearch [feature/1673-add-curl-to-docker-image] ⚡ docker image ls
REPOSITORY                                                         TAG                  IMAGE ID       CREATED         SIZE
getmeili/meilisearch                                               0.22.0_ook_1673      938e239ad989   2 hours ago     315MB
getmeili/meilisearch                                               latest               258fa3aa1230   6 days ago      319MB
```

1684: bump dependencies r=MarinPostma a=MarinPostma

Bump meilisearch dependencies.

We still depend on custom patch that have been upgraded along the way.

Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
Co-authored-by: Thomas Lecavelier <thomas@followanalytics.com>
Co-authored-by: mpostma <postma.marin@protonmail.com>
 2021-09-13 13:20:29 +00:00
mpostma
0f7625e29a bump dependencies 2021-09-13 15:17:08 +02:00
Clémentine Urquizar
6d2f7af642
Bump milli: fix some crashes 2021-09-13 12:14:54 +02:00
bors[bot]
c101b2a5cb
Merge #1686 
1686: Bump milli r=curquiza a=irevoire

 fixes #1685, #1678, #1671, #1677 and #1680

Co-authored-by: Tamo <tamo@meilisearch.com>
 2021-09-08 16:31:02 +00:00
bors[bot]
971c361e0f
Merge #1682 
1682: Change the format of custom ranking rules when importing old dumps r=curquiza a=Kerollmops

This PR changes the format of the custom ranking rules from `asc(price)` to `title:asc` as the format changed between v0.21 and v0.22. The dumps are now correctly importing the custom ranking rules.

This PR also change the previous default ranking rules (without sort) to the new default ranking rules (with the new sort).

Co-authored-by: Kerollmops <clement@meilisearch.com>
 2021-09-08 16:20:10 +00:00
Kerollmops
be50b2bec6
Change the format of custom ranking rules when importing v2 dumps 2021-09-08 17:56:21 +02:00
Tamo
49c918defa
bump milli 2021-09-08 17:41:47 +02:00
many
169e739634
Remove useless indexer options 2021-09-08 13:40:05 +02:00
many
08138c7c23
Use set indexer options instead of create a default one 2021-09-08 13:40:00 +02:00
Kerollmops
331d28102f
Change the format of custom ranking rules when importing v1 dumps 2021-09-07 17:16:40 +02:00
Alexey Shekhirin
efa69875d9
refactor(http): use reset_sortable_fields 2021-09-07 15:04:44 +03:00
many
c0f9c891f5
Set max_memory value to unlimited during tests 
because tests run several meilisearch in parallel,
we over estimate the value for max_memory making the tests on widows crash
 2021-09-06 14:38:10 +02:00
Clémentine Urquizar
33514b28be
Merge pull request #1588 from meilisearch/test-new-indexer 
Integrate the new indexer
 2021-09-06 10:21:42 +02:00
Clément Renault
7e80337e5b
Bump milli to v0.12.0 2021-09-02 18:19:12 +02:00
Kerollmops
4cdf680a81
Make the MaxMemory use the default value when undefined 2021-09-02 18:19:11 +02:00
Kerollmops
0cd66c3a89
Bump the milli version 2021-09-02 18:19:00 +02:00
Clément Renault
b092a624ed
Introduce the MaxMemory struct that defaults to 2/3 of the available memory 2021-09-02 18:18:59 +02:00
Clémentine Urquizar
24e84d7ca1
Test new indexer 2021-09-02 18:11:20 +02:00