Commit Graph

7550 Commits

Author SHA1 Message Date
Tamo
f586028f9a
fix the searchable fields bug when a field is nested
Update milli/src/index.rs

Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-05-16 17:24:36 +02:00
bors[bot]
a65a2bea1e
Merge #2396
2396: Fix dump bug r=curquiza a=MarinPostma

Only check db version file if we aren't loading a dump or a snapshot, because we can assume that if we are loading a dump or a snapshot, then we can safely work with the database. The db version file will be (over)written just after that.

This was introduced by #2356.
fix #2389


Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-05-16 13:59:20 +00:00
ad hoc
5670b4d012
fix dump import error 2022-05-16 14:33:33 +02:00
bors[bot]
e1e85267fd
Merge #526
526: remove useless comment r=irevoire a=MarinPostma



Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-05-16 10:01:43 +00:00
bors[bot]
b9866d8df2
Merge #2339
2339: deny warnings in CI r=irevoire a=MarinPostma

Add `RUSTFLAGS= -D warnings` to the CI so all warnings are treated as hard errors.

Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-05-16 09:58:30 +00:00
bors[bot]
b9b9cba154
Merge #2383
2383: v0.27.0: bring `stable` into `main` r=Kerollmops a=curquiza

Bring `stable` into `main`

Co-authored-by: ad hoc <postma.marin@protonmail.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Paul Sanders <psanders1@gmail.com>
Co-authored-by: Irevoire <tamo@meilisearch.com>
Co-authored-by: Morgane Dubus <30866152+mdubus@users.noreply.github.com>
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
2022-05-16 08:35:25 +00:00
Clémentine Urquizar
cd2239eb2d
Merge branch 'release-v0.27.0' into stable 2022-05-09 10:51:32 +02:00
bors[bot]
51809eb260
Merge #525
525: Simplify the error creation with thiserror r=irevoire a=irevoire

I introduced [`thiserror`](https://docs.rs/thiserror/latest/thiserror/) to implements all the `Display` trait and most of the `impl From<xxx> for yyy` in way less lines.
And then I introduced a cute macro to implements the `impl<X, Y, Z> From<X> for Z where Y: From<X>, Z: From<X>` more easily.

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-05-04 15:47:32 +00:00
Tamo
484a9ddb27
Simplify the error creation with thiserror and a smol friendly macro 2022-05-04 17:24:00 +02:00
bors[bot]
65e6aa0de2
Merge #523
523: Improve geosearch error messages r=irevoire a=irevoire

Improve the geosearch error messages (#488).
And try to parse the string as specified in https://github.com/meilisearch/meilisearch/issues/2354

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-05-04 13:36:11 +00:00
ad hoc
348af6cfbf
deny-rust-warnings 2022-05-04 15:20:45 +02:00
bors[bot]
f3b9f7b867
Merge #527
527: Remove the wip section part of the contributing file r=curquiza a=Kerollmops

Everything was good in the _Development Workflow_ section so I removed the _WIP Section_ part, now this PR fixes https://github.com/meilisearch/milli/issues/513.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-05-04 13:11:30 +00:00
Kerollmops
48cdfddebf
Remove the wip section part of the contributing file 2022-05-04 14:44:51 +02:00
Tamo
c55368ddd4
apply code suggestion
Co-authored-by: Kerollmops <kero@meilisearch.com>
2022-05-04 14:11:03 +02:00
bors[bot]
60ccb3fa4c
Merge #524
524: Add benchmark on nested fields r=irevoire a=irevoire

fixes #500

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-05-04 11:56:18 +00:00
bors[bot]
5337bdb9c5
Merge #2366
2366: Bump milli to v0.26.4 r=curquiza a=curquiza

Fixes the milli related issues
- https://github.com/meilisearch/meilisearch/issues/2352
- https://github.com/meilisearch/meilisearch/issues/2358
- https://github.com/meilisearch/meilisearch/issues/2338

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-05-04 10:30:42 +00:00
bors[bot]
a350cc1186
Merge #2355
2355: feat(http): add analytics on typo tolerance setting r=curquiza a=MarinPostma

Add analytics for the typo settings.

Also make settings analytics return null for settings that are not set, instead of a default value, as seen with `@gmourier` 

Co-authored-by: ad hoc <postma.marin@protonmail.com>
Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
2022-05-04 10:15:15 +00:00
Kerollmops
5c4c38c79c
Fix the tests about the nested fields 2022-05-04 12:10:52 +02:00
ad hoc
b94eabe48c
apply clippy fixes 2022-05-04 11:33:43 +02:00
Clémentine Urquizar
c46f3587de
Bump milli to v0.26.4 2022-05-04 11:25:36 +02:00
ad hoc
5ad5d56f7e
remove useless comment 2022-05-04 10:43:54 +02:00
bors[bot]
0c2c8af44e
Merge #520
520: fix mistake in Settings initialization r=irevoire a=MarinPostma

fix settings not being correctly initialized and add a test to make sure that they are in the future.

fix https://github.com/meilisearch/meilisearch/issues/2358


Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-05-03 15:32:18 +00:00
bors[bot]
2fe9a02b1c
Merge #522
522: Do not generate keys that are too long for LMDB r=Kerollmops a=Kerollmops

This PR fixes https://github.com/meilisearch/meilisearch/issues/2338 by making sure that we do not generate keys that are too long for LMDB especially when we are creating our prefix and proximity pairs keys.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-05-03 11:54:10 +00:00
Kerollmops
211c8763b9
Make sure that we do not generate too long keys 2022-05-03 10:03:15 +02:00
Kerollmops
7e47031bdc
Add a test for long keys in LMDB 2022-05-03 10:03:13 +02:00
Tamo
f820c9804d
add one nested benchmark 2022-05-02 19:35:57 +02:00
Tamo
3cb1f6d0a1
improve geosearch error messages 2022-05-02 19:20:47 +02:00
ad hoc
34f75d9792
settings analytics return null when no set 2022-04-29 16:38:21 +02:00
bors[bot]
3c5f7dbf7e
Merge #2356
2356: fix dumps bug r=MarinPostma a=MarinPostma

move the db version check after the dump import
fix #2353


Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-04-29 14:33:58 +00:00
ad hoc
1ee3d6ae33
fix mistake in Settings initialization 2022-04-29 16:24:25 +02:00
bors[bot]
312515dd6b
Merge #507
507: deny warnings in CI r=Kerollmops a=MarinPostma

Add `RUSTFLAGS= -D warnings` to the CI so all warnings are treated as hard errors.

Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-04-28 15:16:35 +00:00
ad hoc
3eb3f0269e
deny warnings in CI 2022-04-28 15:35:12 +02:00
bors[bot]
9db86aac51
Merge #518
518: Return facets even when there is no value associated to it r=Kerollmops a=Kerollmops

This PR is related to https://github.com/meilisearch/meilisearch/issues/2352 and should fix the issue when Meilisearch is up-to-date with this PR.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-04-28 09:04:36 +00:00
Guillaume Mourier
3d0a4a3d18 fix(http): fix event name for typo tolerance settings update 2022-04-27 14:49:21 +02:00
ad hoc
6025372565
fix(lib): Check db presence after dumps 2022-04-27 10:41:09 +02:00
ad hoc
3d10af0333
feat(http): add analytics on typo tolerance setting 2022-04-26 18:29:32 +02:00
bors[bot]
2aae19dc52
Merge #517
517: Make nightly CI run every week r=Kerollmops a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-04-26 16:22:25 +00:00
Kerollmops
a4d343aade
Add a test to check for the returned facet distribution 2022-04-26 18:12:58 +02:00
bors[bot]
c2bd94c871
Merge #511
511: Update version in every workspace r=curquiza a=curquiza

Checked with `@Kerollmops` 

- Update the version into every workspace (the current version is v0.27.0, but I forgot to update it for the previous release)
- add `publish = false` except in `milli` workspace.


Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-04-26 16:06:47 +00:00
Kerollmops
7d1c2d97bf
Return facets even when there is no values associated to it 2022-04-26 17:59:53 +02:00
bors[bot]
d388ea0f9d
Merge #506
506: fix cargo warnings r=Kerollmops a=MarinPostma

fix cargo warnings


Co-authored-by: ad hoc <postma.marin@protonmail.com>
2022-04-26 15:45:20 +00:00
Clémentine Urquizar
ec89030483
Update bors toml 2022-04-26 17:36:04 +02:00
ad hoc
5c29258e8e
fix cargo warnings 2022-04-26 17:33:11 +02:00
bors[bot]
2fdf520271
Merge #514
514: Stop flattening every field r=Kerollmops a=irevoire

When we need to flatten a document:
* The primary key contains a `.`.
* Some fields need to be flattened

Instead of flattening the whole object and thus creating a lot of allocations with the `serde_json_flatten_crate`, we instead generate a minimal sub-object containing only the fields that need to be flattened.
That should create fewer allocations and thus index faster.

---------

```
group                                                             indexing_main_e1e362fa                 indexing_stop-flattening-every-field_40d1bd6b
-----                                                             ----------------------                 ---------------------------------------------
indexing/Indexing geo_point                                       1.99      23.7±0.23s        ? ?/sec    1.00      11.9±0.21s        ? ?/sec
indexing/Indexing movies in three batches                         1.00      18.2±0.24s        ? ?/sec    1.01      18.3±0.29s        ? ?/sec
indexing/Indexing movies with default settings                    1.00      17.5±0.09s        ? ?/sec    1.01      17.7±0.26s        ? ?/sec
indexing/Indexing songs in three batches with default settings    1.00      64.8±0.47s        ? ?/sec    1.00      65.1±0.49s        ? ?/sec
indexing/Indexing songs with default settings                     1.00      54.9±0.99s        ? ?/sec    1.01      55.7±1.34s        ? ?/sec
indexing/Indexing songs without any facets                        1.00      50.6±0.62s        ? ?/sec    1.01      50.9±1.05s        ? ?/sec
indexing/Indexing songs without faceted numbers                   1.00      54.0±1.14s        ? ?/sec    1.01      54.7±1.13s        ? ?/sec
indexing/Indexing wiki                                            1.00     996.2±8.54s        ? ?/sec    1.02   1021.1±30.63s        ? ?/sec
indexing/Indexing wiki in three batches                           1.00    1136.8±9.72s        ? ?/sec    1.00    1138.6±6.59s        ? ?/sec
```

So basically everything slowed down a liiiiiittle bit except the dataset with a nested field which got twice faster

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-04-26 11:50:33 +00:00
Tamo
f19d2dc548
Only flatten the required fields
apply review comments

Co-authored-by: Kerollmops <kero@meilisearch.com>
2022-04-26 12:33:46 +02:00
bors[bot]
5adeac8047
Merge #516
516: Fix the indexing fuzzer r=irevoire a=irevoire



Co-authored-by: Tamo <tamo@meilisearch.com>
2022-04-26 08:35:03 +00:00
Clémentine Urquizar
7cb7643565
Make nightly CI run every week
Update CI

Fix CI
2022-04-25 18:52:27 +02:00
Clémentine Urquizar
d138b3c704
Update version 2022-04-25 18:43:46 +02:00
Tamo
fa6f495662
fix the indexing fuzzer 2022-04-25 18:32:06 +02:00
bors[bot]
8cc86d5a8d
Merge #515
515: Improve the README r=curquiza a=Kerollmops

This PR closes #512 by adding more content to the README. We listed all of the subcrates of the repository, changed the descriptions of the subcrates, and added a simple example usage in the README.

Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
2022-04-25 16:15:12 +00:00