Commit Graph

664 Commits

Author SHA1 Message Date
Tamo
cfcd3ae048
move the version to context.app 2021-10-29 17:25:50 +02:00
Tamo
5395041dcb
fix the stats and stop sending events when no request happened 2021-10-29 17:25:49 +02:00
Tamo
40eabd50d1
integrate the search batcher in the search route 2021-10-29 17:25:49 +02:00
Tamo
35ffd0ec3a
integrate the search batcher in the tick 2021-10-29 17:25:49 +02:00
Tamo
d3d76bf97a
wip create a search batcher 2021-10-29 17:25:49 +02:00
Tamo
595ae42e94
update the name of the Launched event 2021-10-29 17:25:49 +02:00
Tamo
0667d940f9
update the name of nb_cores in the identify 2021-10-29 17:25:49 +02:00
Irevoire
75d1272325
log the dump creation 2021-10-29 17:25:49 +02:00
Irevoire
8e2d6cf87d
add the content type to all the route 2021-10-29 17:25:48 +02:00
Irevoire
9e1bba40f7
do not print anything if no user id was found 2021-10-29 17:25:48 +02:00
Irevoire
f7bb499c28
send the first identify + launched for the first time events right away instead of batching them 2021-10-29 17:25:48 +02:00
Irevoire
b33b1ef3dd
update the way of getting and saving the user-id to the file system 2021-10-29 17:25:48 +02:00
Irevoire
30aeda7a1a
update the identify call to the latest spec version 2021-10-29 17:25:48 +02:00
Irevoire
22d9d660cc
log all the required settings route 2021-10-29 17:25:48 +02:00
Irevoire
7524bfc07f
log the all settings updated route 2021-10-29 17:25:48 +02:00
Tamo
bda7472880
log the documetns updated route 2021-10-29 17:25:48 +02:00
Tamo
1ed05c6c07
log documents added 2021-10-29 17:25:47 +02:00
Tamo
0b3e0a59cb
log index updated 2021-10-29 17:25:47 +02:00
Tamo
0616f68eb0
implements part of the search 2021-10-29 17:25:47 +02:00
Tamo
6b8e5a4c92
log the index created route 2021-10-29 17:25:47 +02:00
Tamo
d72c887422
makes the analytics available for all the routes 2021-10-29 17:25:47 +02:00
Tamo
664d09e86a
makes the analytics works with the option and the feature 2021-10-29 17:25:47 +02:00
Tamo
e226b1a87f
rewrite the main analytics module and the information sent in the tick 2021-10-29 17:25:42 +02:00
bors[bot]
cf67964133
Merge #1848
1848: Error format and Definition r=MarinPostma a=ManyTheFish



Co-authored-by: many <maxime@meilisearch.com>
2021-10-28 14:15:35 +00:00
many
cbaca2b579
Fix PR comments 2021-10-28 15:42:42 +02:00
Clémentine Urquizar
a76d9b15c9
Update version for the next release (v0.24.0) 2021-10-28 12:24:49 +02:00
many
59636fa688
Pimp error where no document is provided 2021-10-28 12:13:51 +02:00
many
ff0908d3fa
Ignore errors tests that show unrelated bugs 2021-10-28 11:41:59 +02:00
many
21f35762ca
Fix content type test 2021-10-28 10:57:11 +02:00
many
7464720426
Fix some errors 2021-10-28 10:47:59 +02:00
CuriousCorrelation
6641e7aa50 Add tests for mini-dashboard status and assets 2021-10-27 17:57:25 +05:30
many
61c15b69fb
Change malformed_payload error 2021-10-27 11:13:12 +02:00
many
8ec0c4c913
Add bad_request error tests 2021-10-27 11:13:12 +02:00
many
b4038597ba
Keep persisting tmp files in database directory and put non-persisting tmp files in default tmp dir 2021-10-18 14:16:35 +02:00
Clémentine Urquizar
b602a0836a
Merge branch 'main' into stable 2021-10-14 13:43:21 +02:00
Paul Sanders
c9e1d054c7 Fix csv content-type error 2021-10-12 13:38:48 -04:00
Clémentine Urquizar
0f342ac46e
Update MeiliSearch version 2021-10-12 16:43:31 +02:00
Clémentine Urquizar
3edbc74430
Merge branch 'main' into stable 2021-10-11 18:30:10 +02:00
Tom Parker-Shemilt
6c46fbbc57 Remove memmap dependency 2021-10-10 22:33:40 +01:00
Tamo
66dbd3cd34
makes clippy happy 2021-10-06 17:39:04 +02:00
Tamo
9a1e44dc78
Apply suggestion
- remove the payload_error_handler in favor of a PayloadError::from

- merge the two match branch into one

- makes the accepted content type a const instead of recalculating it for every error
2021-10-06 17:15:47 +02:00
Tamo
37b267ffb3
duplicate the post document tests with the put verb 2021-10-06 17:15:47 +02:00
Tamo
dfa199f98f
add content-type tests
fix typo

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-06 17:15:47 +02:00
Tamo
c6d107a05f
makes the content-type mandatory for every routes 2021-10-06 17:15:47 +02:00
bors[bot]
2190764162
Merge #1768
1768: Fix auth error r=irevoire a=MarinPostma

fix a small auth error, that set the invalid token error token to "hello". This was invilisble to the user because the invalid token is not returned.

thank you hawk-eye `@irevoire` 

Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-10-05 15:16:14 +00:00
mpostma
3b91764587 fix auth error 2021-10-05 09:09:40 +02:00
Anirudh Rowjee
bffab21b10 Changes
1. Removed redundant scope registration
2021-10-04 14:47:05 +05:30
Anirudh Rowjee
151f691609 Fixes #1750
This commit is a fix to issue #1750.
As a part of the changes to solve this issue, the following changes have
been made -
1. Route registration for static assets has been modified
2. the `mut` keyword on the `scope` has been removed.
2021-10-02 15:24:04 +05:30
mpostma
4eb3817b03
missing payload error 2021-09-30 16:58:13 +02:00
mpostma
18cb514073
invalid content type error 2021-09-30 16:58:13 +02:00
mpostma
137272b8de
empty content type error 2021-09-30 16:58:13 +02:00
Kerollmops
c388dca5ec
Check that invalid updates do not create an index with a primary key 2021-09-30 15:46:04 +02:00
bors[bot]
05d5de47cb
Merge #1737
1737: Update version for the next release (v0.23.0) r=irevoire a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-09-29 22:06:46 +00:00
mpostma
fe5df6d06f fix payload content type guards 2021-09-29 19:04:47 +02:00
Clémentine Urquizar
b17dae9ac0
Update version for the next release (v0.23.0) 2021-09-29 18:40:35 +02:00
bors[bot]
5fad37aebd
Merge #1711
1711: MeiliSearch refactor introducing OBKV format r=MarinPostma a=MarinPostma

This PR refactor some multiple components of meilisearch, and introduce the obkv document format to meilisearch

- [x] Split meilisearch-http and meilisearch-lib
- [x] Replace `IndexActor` and `UuidResolver` with `IndexResolver`
- [x] Remove mentions to Actor
- [x] Remove Actor traits to simplify code
- [x] Integrate obkv document format
- [x] Remove `Data`
- [x] Restore all route
- [x] Replace `Box<dyn error>` with `anyhow::Error`
- [x] Introduce update file store
- [x] Update file store error handling
- [x] Fix dumps
- [x] Fix snapshots
- [x] Fix tests
- [x] Update module documentation
- [x] add csv suppport (feat `@ManyTheFish` #1729 )
- [x] add jsonl support
- [x] integrate geosearch (feat `@irevoire` #1725) 

partially implements #1691 and #1690. The error handling is very basic now, I will finish it in the next pr.

Some unit tests have been disabled, I will re-enable them ASAP, but they need a bit more work.

close #1531 


P.S: sorry for this monstrous PR :'(

Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: many <maxime@meilisearch.com>
2021-09-29 14:38:55 +00:00
mpostma
8fa6502b16 review changes 2021-09-29 14:17:41 +02:00
mpostma
1f537e1b60 jsonl support 2021-09-29 11:28:02 +02:00
mpostma
5bac65f8b8 add missing content type errors 2021-09-29 09:55:35 +02:00
mpostma
911630000f split csv and json document routes 2021-09-29 00:12:25 +02:00
mpostma
102c46f88b clippy + fmt 2021-09-28 22:22:59 +02:00
mpostma
5fa9bc67d7 remove unused dependencies 2021-09-28 22:16:18 +02:00
mpostma
3503fbf7fe re-export milli from meilisearch_lib 2021-09-28 22:08:03 +02:00
mpostma
7a27cbcc78 rename RegisterUpdate to store::Update 2021-09-28 20:20:13 +02:00
mpostma
56766cffc3 remove module level doc 2021-09-28 18:58:56 +02:00
mpostma
692c676625 fix tests 2021-09-28 18:57:36 +02:00
Tamo
ddfd7def35 add a TODO while waiting for the tests to be fixed 2021-09-28 18:17:56 +02:00
Tamo
539a57026d
fix the sort error messages 2021-09-28 14:50:26 +02:00
Tamo
654f49ccec
[WIP] put milli on branch main 2021-09-28 14:50:26 +02:00
Tamo
c1376a9f2a
add the geosearch to Meilisearch 2021-09-28 14:50:26 +02:00
mpostma
90018755c5 restore snapshots 2021-09-27 16:48:03 +02:00
mpostma
b9d189bf12 restore document deletion routes 2021-09-24 15:21:07 +02:00
mpostma
c32012c44a restore settings updates 2021-09-24 14:55:57 +02:00
mpostma
dfce44fa3b rename data to meilisearch 2021-09-24 12:03:16 +02:00
mpostma
5353be74c3 refactor index actor 2021-09-22 15:07:04 +02:00
mpostma
60518449fc split meilisearch-http and meilisearch-lib 2021-09-21 13:23:22 +02:00
mpostma
09d4e37044 split data and api keys 2021-09-20 15:31:03 +02:00
mpostma
e14640e530 refactor meilisearch 2021-09-20 14:54:20 +02:00
Felix Yan
a57943b77e
Use tikv-jemallocator instead of jemallocator
`jemallocator` has been abandoned for nearly two years, and `rustc`
itself moved to use `tikv-jemallocator` instead:
3965773ae7

Let's switch to a better maintained version.
2021-09-14 18:30:24 +03:00
bors[bot]
6fafdb7711
Merge #1651 #1676 #1684
1651: Use reset_sortable_fields r=Kerollmops a=shekhirin

Resolves https://github.com/meilisearch/MeiliSearch/issues/1635

1676: Add curl binary to final stage image r=curquiza a=ook

Reference: #1673 

Changes: * add `curl` binary to final docker Melisearch image.

For metrics, docker funny layer management makes this add a  shrink from 319MB to 315MB:

```
☁  MeiliSearch [feature/1673-add-curl-to-docker-image]  docker image ls
REPOSITORY                                                         TAG                  IMAGE ID       CREATED         SIZE
getmeili/meilisearch                                               0.22.0_ook_1673      938e239ad989   2 hours ago     315MB
getmeili/meilisearch                                               latest               258fa3aa1230   6 days ago      319MB
```

1684: bump dependencies r=MarinPostma a=MarinPostma

Bump meilisearch dependencies.

We still depend on custom patch that have been upgraded along the way.

Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
Co-authored-by: Thomas Lecavelier <thomas@followanalytics.com>
Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-09-13 13:20:29 +00:00
mpostma
0f7625e29a bump dependencies 2021-09-13 15:17:08 +02:00
Clémentine Urquizar
6d2f7af642
Bump milli: fix some crashes 2021-09-13 12:14:54 +02:00
bors[bot]
c101b2a5cb
Merge #1686
1686: Bump milli r=curquiza a=irevoire

 fixes #1685, #1678, #1671, #1677 and #1680

Co-authored-by: Tamo <tamo@meilisearch.com>
2021-09-08 16:31:02 +00:00
bors[bot]
971c361e0f
Merge #1682
1682: Change the format of custom ranking rules when importing old dumps r=curquiza a=Kerollmops

This PR changes the format of the custom ranking rules from `asc(price)` to `title:asc` as the format changed between v0.21 and v0.22. The dumps are now correctly importing the custom ranking rules.

This PR also change the previous default ranking rules (without sort) to the new default ranking rules (with the new sort).

Co-authored-by: Kerollmops <clement@meilisearch.com>
2021-09-08 16:20:10 +00:00
Kerollmops
be50b2bec6
Change the format of custom ranking rules when importing v2 dumps 2021-09-08 17:56:21 +02:00
Tamo
49c918defa
bump milli 2021-09-08 17:41:47 +02:00
many
169e739634
Remove useless indexer options 2021-09-08 13:40:05 +02:00
many
08138c7c23
Use set indexer options instead of create a default one 2021-09-08 13:40:00 +02:00
Kerollmops
331d28102f
Change the format of custom ranking rules when importing v1 dumps 2021-09-07 17:16:40 +02:00
Alexey Shekhirin
efa69875d9
refactor(http): use reset_sortable_fields 2021-09-07 15:04:44 +03:00
many
c0f9c891f5
Set max_memory value to unlimited during tests
because tests run several meilisearch in parallel,
we over estimate the value for max_memory making the tests on widows crash
2021-09-06 14:38:10 +02:00
Clémentine Urquizar
33514b28be
Merge pull request #1588 from meilisearch/test-new-indexer
Integrate the new indexer
2021-09-06 10:21:42 +02:00
Clément Renault
7e80337e5b
Bump milli to v0.12.0 2021-09-02 18:19:12 +02:00
Kerollmops
4cdf680a81
Make the MaxMemory use the default value when undefined 2021-09-02 18:19:11 +02:00
Kerollmops
0cd66c3a89
Bump the milli version 2021-09-02 18:19:00 +02:00
Clément Renault
b092a624ed
Introduce the MaxMemory struct that defaults to 2/3 of the available memory 2021-09-02 18:18:59 +02:00
Clémentine Urquizar
24e84d7ca1
Test new indexer 2021-09-02 18:11:20 +02:00
Clémentine Urquizar
90116155b4
Update version for the next release (v0.22.0) 2021-09-01 12:33:30 +02:00
bors[bot]
19c5c74291
Merge #1652 #1654 #1657
1652: Remove dependabot r=MarinPostma a=curquiza

Fixes #1649 

Dependabot for vulnerability and security updates is still activated.

1654: Add Script for Windows r=MarinPostma a=singh08prashant

fixes #1570 

changes:

1. added script for detecting windows os running git bash
2. appended `.exe` to `$release_file` for windows as listed [here](https://github.com/meilisearch/MeiliSearch/releases/)
3. removed global `$BINARY_NAME='meilisearch'` as windows require `.exe` file

1657: Bring vergen hotfix from `stable` to `main` r=MarinPostma a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: singh08prashant <singh08prashant@gmail.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
2021-08-31 14:31:42 +00:00
Kerollmops
9d0fa8112b
Remove unused Arc import 2021-08-31 14:50:36 +02:00
Kerollmops
b8c954eb3f
Bump the MeiliSearch version to v0.21.1 2021-08-30 17:41:25 +02:00
Kerollmops
a8c146fd13
Unwrap or unknown the commit hash 2021-08-30 17:41:24 +02:00
Kerollmops
1782753387
Bump vergen and remove unused build feature 2021-08-30 15:03:45 +02:00
Kerollmops
cf4a466b6b
Make sure that the order of the filterableAttributes is constant 2021-08-26 11:06:05 +02:00
Kerollmops
087e4626ce
Make sure that the order of the sortableAttributes is constant 2021-08-26 11:06:04 +02:00
Kerollmops
64462c842b
Test the search with sort time queries with POST and GET methods 2021-08-25 17:39:25 +02:00
Kerollmops
e0f73fe742
Introduce the sort search parameter 2021-08-25 17:39:25 +02:00
Kerollmops
ea4c831de0
Integrate the sortable-attributes into the settings 2021-08-25 17:39:25 +02:00
Kerollmops
51387b2c80
Introduce the new invalid sortable error codes 2021-08-25 17:29:30 +02:00
bors[bot]
2d8dd87cad
Merge #1623
1623: Use Setting enum r=Kerollmops a=shekhirin

Resolves https://github.com/meilisearch/MeiliSearch/issues/1620

Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
2021-08-25 14:58:40 +00:00
Alexey Shekhirin
d9dd2a038b
refactor(http): use Setting enum 2021-08-25 17:43:46 +03:00
Irevoire
672fcee8aa
remove sentry 2021-08-24 12:38:31 +02:00
Clémentine Urquizar
149f46c184
Fix pacic when decoding 2021-08-23 12:37:51 +02:00
Clémentine Urquizar
38fc876704
Update tokenizer and new milli version with new tags 2021-08-18 14:55:10 +02:00
Clémentine Urquizar
39d5a99095
Update milli and tokenizer version 2021-08-18 12:09:34 +02:00
bors[bot]
2beb306834
Merge #1577
1577: Update milli dependency: fix facet values bugs r=Kerollmops a=curquiza

Fixes #1576 

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-08-16 16:13:42 +00:00
Clémentine Urquizar
f3e595e2f0
Update milli dependency 2021-08-16 13:36:42 +02:00
Clémentine Urquizar
621529e9dc
Update telemetry link 2021-08-12 18:58:07 +02:00
Clémentine Urquizar
7531280764
Update tokenizer version to v0.2.4 2021-08-12 13:55:47 +02:00
bors[bot]
7e3b2ddff2
Merge #1554
1554: Fix dump v1 (attributesForFaceting, and criteria) r=curquiza a=MarinPostma

close #1553


Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-08-05 19:45:52 +00:00
bors[bot]
312d93961a
Merge #1556
1556: Update milli to v0.9.0 r=MarinPostma a=curquiza

Fixes #1552 

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-08-05 14:04:55 +00:00
mpostma
8f05d8d546 fix clippy warnings 2021-08-05 16:00:47 +02:00
mpostma
f5ddea481a reintroduce exactness 2021-08-05 15:59:39 +02:00
mpostma
29ca8271b3 test dumpv1 format regression 2021-08-05 15:59:39 +02:00
mpostma
3084537d1e restore attributes for faceting in dump v1 2021-08-05 15:59:39 +02:00
Clémentine Urquizar
992b082c6f
Fix docs link anchor 2021-08-05 13:28:32 +02:00
Clémentine Urquizar
31fe263356
Update milli to v0.9.0 2021-08-05 13:08:27 +02:00
bors[bot]
7a0b20c740
Merge #1532
1532: Start writing documentation for newcomers r=MarinPostma a=irevoire



Co-authored-by: Tamo <tamo@meilisearch.com>
2021-08-03 09:26:45 +00:00
bors[bot]
9810f6b695
Merge #1540
1540: Update milli to version 0.8.1 r=curquiza a=curquiza

Integrates this fix into MeiliSearch https://github.com/meilisearch/milli/pull/296

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-07-29 17:15:52 +00:00
bors[bot]
09c74c04a0
Merge #1539
1539: Use serdeval for validating json format. r=curquiza a=MarinPostma

uses [serdeval](https://github.com/MarinPostma/serdeval) to validate that the json payload is valid json, and in the correct format.

fix #1535


Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-07-29 17:05:13 +00:00
Clémentine Urquizar
1b5d918cb9
Fix rustfmt 2021-07-29 18:32:09 +02:00
Clémentine Urquizar
bf76d4a43c
Make clippy happy 2021-07-29 18:14:36 +02:00
mpostma
53b4b2fcbc Use serdeval for validating json format. 2021-07-29 18:02:54 +02:00
Clémentine Urquizar
9a8629a6a9
Update milli 2021-07-29 17:45:31 +02:00
Tamo
78308365ec
fix typos 2021-07-29 14:40:41 +02:00
Tamo
243233f652
import .git to docker to fix vergen 2021-07-28 19:12:40 +02:00
Clémentine Urquizar
80916baa21
Add FieldId in import 2021-07-28 12:25:13 +02:00
Clémentine Urquizar
1df8f041bd
Update meilisearch-http/src/index/search.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-07-28 12:10:25 +02:00
Clémentine Urquizar
6a6e2a8cd1
Update meilisearch-http/src/index/search.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-07-28 12:08:51 +02:00
Clémentine Urquizar
f9d337b320
Update meilisearch-http/src/index/search.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-07-28 12:08:36 +02:00
Clémentine Urquizar
feb069f604
Update meilisearch-http/src/index/search.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-07-28 12:08:28 +02:00
Clémentine Urquizar
7e0eed5772
Update meilisearch-http/src/index/search.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-07-28 12:08:24 +02:00
Clémentine Urquizar
9bdd040dd0
Update meilisearch-http/src/index/mod.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-07-28 12:08:19 +02:00
Clémentine Urquizar
e5dabf265a
Update milli version to v0.8.0 2021-07-28 10:52:47 +02:00
Tamo
1a1046a0ef
start writing some documentation for newcomers 2021-07-27 16:35:42 +02:00
Morgane Dubus
d3cd7e92d1
Update mini-dashboard version to v0.1.4 2021-07-27 11:44:20 +02:00
Tamo
f79b8287f5
update vergen 2021-07-26 15:25:30 +02:00
Tamo
5d4a0ac844
sentry was never sending anything 2021-07-21 11:50:54 +02:00
bors[bot]
0136b02e5b
Merge #1498
1498: Show the filterable and not the faceted attributes in the settings r=Kerollmops a=Kerollmops

Fixes #1497

Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-07-13 07:27:14 +00:00
Clément Renault
f49a01703a
Show the filterable and not the faceted attributes in the settings 2021-07-09 16:11:37 +02:00