Commit Graph

9998 Commits

Author SHA1 Message Date
Clément Renault
02335ee72d
Introduce the biggest-value-sizes command on the infos binary 2020-08-21 14:44:42 +02:00
Clément Renault
1e3e756c19
Introduce the words-frequencies command on the infos binary 2020-08-21 14:44:42 +02:00
Kerollmops
6a230fe803
Move the contains_documents logic to a function 2020-08-21 14:44:42 +02:00
Kerollmops
e55a569629
Compress much more the documents database 2020-08-21 14:44:42 +02:00
mpostma
445a6c9ea2 update options name 2020-08-21 14:42:20 +02:00
bors[bot]
67b7d60cb0
Merge #920
920: fix bug and add tests r=MarinPostma a=LegendreM

- add tests about updates
- fix select bug

fix #896

Co-authored-by: many <maxime@meilisearch.com>
2020-08-19 07:56:27 +00:00
many
94b3e8e56e
fix bug and add tests
- add tests about updates
- fix select bug

fix #896
2020-08-19 09:51:57 +02:00
bors[bot]
89b5ae63fc
Merge #915
915: fix unwrap bug r=Kerollmops a=MarinPostma

fix #912.

Co-authored-by: mpostma <postma.marin@protonmail.com>
2020-08-18 12:50:10 +00:00
Kerollmops
962bad3cea
Introduce an infos binary to fetch stats 2020-08-17 19:41:49 +02:00
mpostma
2a79dc9ded log error on unwrap error 2020-08-17 16:32:40 +02:00
Clément Renault
8806fcd545
Introduce a better query and document lexer 2020-08-16 14:36:54 +02:00
Clément Renault
1e358e3ae8
Introduce the AstarBagIter that iterates through best paths 2020-08-15 16:24:06 +02:00
mpostma
5ed62dbf76 fix unwrap bug 2020-08-14 12:16:48 +02:00
bors[bot]
cb267b68ed
Merge #910
910: Fix typo in error message r=MarinPostma a=curquiza

Thanks to @ppamorim for reporting the typos to me!

Co-authored-by: Clementine Urquizar <clementine@meilisearch.com>
2020-08-13 15:43:58 +00:00
Clementine Urquizar
6539be6c46 Fix typo in error message 2020-08-13 17:13:19 +02:00
Clément Renault
7dc594ba4d
Introduce the Search builder struct 2020-08-13 14:27:51 +02:00
bors[bot]
a23bdb31a3
Merge #829
829: implement snapshoting r=MarinPostma a=LegendreM

related to #551.

This pull request permit user to create periodically a snapshot of MeiliSearch database via a command line and launch meiliSearch from a snapshot with another command

## Documentation

### schedule a snapshot
`--snapshot-path <DIRECTORY_PATH>`:
this will periodically create a snapshot `<DB_NAME>.tar.gz` in the specified directory

### change period between 2 snapshot creation
`--snapshot-interval-sec <GAP_IN_SEC>`
choose the time gap between 2 snapshot

### start meilisearch from a snapshot
`--load-from-snapshot <FILE_PATH>`
this will use the snapshot stored at `<FILE_PATH>` to initialize MeiliSearch database,

`--ignore-snapshot-if-db-exists` if set and if a db already exists,
this will skip snapshot importation and continue process with actual db instead of quitting process by returning an Error

`--ignore-missing-snapshot` if set and if no snapshot exists at provided path,
this will skip snapshot importation and continue process with actual db instead of quitting process by returning an Error

Co-authored-by: many <maxime@meilisearch.com>
2020-08-12 16:37:31 +00:00
many
9014290875
implement snapshot 2020-08-12 17:46:28 +02:00
bors[bot]
1903302a74
Merge #906
906: Facet distribution correct case r=LegendreM a=MarinPostma

~

Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: marin <postma.marin@protonmail.com>
2020-08-12 09:04:36 +00:00
Clément Renault
bfb46cbfbe
Introduce the Crtierion enum 2020-08-12 10:43:02 +02:00
mpostma
75c3cb4bb6 fix compile error 2020-08-12 10:31:11 +02:00
marin
bfd0f806f8 requested changed
Co-authored-by: Clément Renault <renault.cle@gmail.com>
2020-08-12 10:31:11 +02:00
mpostma
afab8a7846 clean facet result types 2020-08-12 10:31:11 +02:00
mpostma
afacdbc7a0 update tests for facets distribution case 2020-08-12 10:31:11 +02:00
mpostma
18a50b4dac fix facet distribution case 2020-08-12 10:31:10 +02:00
Clément Renault
6d04a285dc
Retrieve and display the distances of the words found 2020-08-11 15:18:02 +02:00
Clément Renault
1bd37d213a
Lowercase quoted words 2020-08-10 14:49:09 +02:00
Clément Renault
883a8109c8
Show both database and documents database sizes 2020-08-10 14:37:18 +02:00
Clément Renault
a4e0f3f724
Remove the useless TransitiveArc from the serve binary 2020-08-10 14:06:27 +02:00
Clément Renault
edc06a97d6
Remove the useless stats binary 2020-08-10 13:55:02 +02:00
Clément Renault
ae77fe5a69
Introduce an option to specify the maximum database size 2020-08-10 13:53:53 +02:00
Clément Renault
394844062f
Move the documents MTBL database inside the Index 2020-08-10 13:47:19 +02:00
Clément Renault
ecd2b2f217
Make the final merge done in parallel 2020-08-07 15:44:04 +02:00
Clément Renault
91282c8b6a
Move the documents into another file 2020-08-07 13:11:31 +02:00
Clément Renault
fae694a102
Put the documents into an MTBL database 2020-08-07 12:14:40 +02:00
Clément Renault
d5a356902a
Update oxidized-mtbl 2020-08-07 12:14:03 +02:00
Clément Renault
405a71d3a4
Accept csv from stdin 2020-08-06 13:38:21 +02:00
Clément Renault
d3b1096510
Compute the word attribute postings lists on each threads 2020-08-06 11:50:27 +02:00
Clément Renault
8d734941af
Clean up some lines 2020-08-06 10:20:26 +02:00
Clément Renault
a4e3c7c37c
Force the Papa parse delimiter 2020-08-05 14:11:46 +02:00
Clément Renault
6508d497ce
Replace the regex highlighting by a simple algorithm 2020-08-05 13:52:27 +02:00
Clément Renault
4873abe145
Introduce option flags to toggle the indexing engine 2020-08-05 12:10:41 +02:00
Clément Renault
bd4b18541c
Introduce a new indexer which uses an MTBL sorter 2020-08-04 15:44:37 +02:00
Clément Renault
3f21760d56
Update README.md 2020-08-04 15:40:37 +02:00
Clément Renault
bc3a0ac6a3
Display the milli logo and update the description 2020-08-04 15:40:02 +02:00
bors[bot]
fb69769991
Merge #889
889: Fix clippy warnings r=MarinPostma a=TaKO8Ki

Good day!

Since `cargo clippy` showed two warnings like the following, I've fixed them. This is a small PR.

```sh
warning: use of `ok_or` followed by a function call
   --> meilisearch-core/src/database.rs:185:18
    |
185 |                 .ok_or(Error::VersionMismatch("bad VERSION file".to_string()))?;
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `ok_or_else(|| Error::VersionMismatch("bad VERSION file".to_string()))`
    |
    = note: `#[warn(clippy::or_fun_call)]` on by default
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call

warning: useless use of `format!`
   --> meilisearch-core/src/database.rs:208:59
    |
208 |                         return Err(Error::VersionMismatch(format!("<0.12.0")));
    |                                                           ^^^^^^^^^^^^^^^^^^ help: consider using `.to_string()`: `"<0.12.0".to_string()`
    |
    = note: `#[warn(clippy::useless_format)]` on by default
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#useless_format

warning: 2 warnings emitted
```

Co-authored-by: Takayuki Maeda <41065217+TaKO8Ki@users.noreply.github.com>
2020-07-29 11:40:08 +00:00
Takayuki Maeda
750e7382c6 fix clippy warnings 2020-07-29 11:32:34 +09:00
bors[bot]
2464cc7a6d
Merge #888
888: Remove schema mention in error message r=MarinPostma a=curquiza

We avoid mentioning the schema since MeiliSearch is schemaless for the user 🙂

Co-authored-by: Clementine Urquizar <clementine@meilisearch.com>
2020-07-28 15:20:59 +00:00
Clementine Urquizar
f078cbac4d Remove schema mention in error message 2020-07-28 15:18:05 +02:00
bors[bot]
aa545e5386
Merge #638 #828 #865
638: Update requitites for source build(rust version) r=MarinPostma a=djKooks

Hello,
I just found that compile via source has been failed by issue here:
```
error[E0658]: the `#[non_exhaustive]` attribute is an experimental feature
  --> /Users/kwangin.jung/.cargo/registry/src/github.com-1ecc6299db9ec823/whoami-0.8.1/src/lib.rs:40:1
   |
40 | #[non_exhaustive]
   | ^^^^^^^^^^^^^^^^^
   |
   = note: for more information, see https://github.com/rust-lang/rust/issues/44109

error[E0658]: the `#[non_exhaustive]` attribute is an experimental feature
   --> /Users/kwangin.jung/.cargo/registry/src/github.com-1ecc6299db9ec823/whoami-0.8.1/src/lib.rs:102:1
    |
102 | #[non_exhaustive]
    | ^^^^^^^^^^^^^^^^^
    |
    = note: for more information, see https://github.com/rust-lang/rust/issues/44109
```
Seems `#[non_exhaustive]` is a new feature on Rust 1.40.0, so added as pre-requitites.


828: Cleanup readme r=MarinPostma a=tpayet

Closes #613 

865: Update movie dataset with genre field r=MarinPostma a=bidoubiwa

Updated the movie dataset by adding  the `genre` field to each movies where the genre could be fetched.
The `genre` was fetch for each movie by making a search request on the bigger movie dataset (200mb) using MeilISearch. 

I make this proposition to make testing and trying  more accessible. 

```json
{
  "id": "323661",
  "title": "Mune: Guardian of the Moon",
  "poster": "https://image.tmdb.org/t/p/w1280/4vzqow7mVUahqA4hHoe2UpQOxy.jpg",
  "overview": "When a faun named Mune becomes the Guardian of the Moon, little did he had unprepared experience with the Moon and an accident that could put both the Moon and the Sun in danger, including a corrupt titan named Necross who wants the Sun for himself and placing the balance of night and day in great peril. Now with the help of a wax-child named Glim and the warrior, Sohone who also became the Sun Guardian, they go out on an exciting journey to get the Sun back and restore the Moon to their rightful place in the sky.",
  "release_date": 1423094400,
  "genre": [
    "Animation",
    "Family",
    "Adventure",
    "Fantasy",
    "Comedy"
  ]
}
{
  "id": "306",
  "title": "Beverly Hills Cop III",
  "poster": "https://image.tmdb.org/t/p/w1280/tw9gAhqQcBFX0X0XfVbWqUsmzoU.jpg",
  "overview": "Back in sunny southern California and on the trail of two murderers, Axel Foley again teams up with LA cop Billy Rosewood. Soon, they discover that an amusement park is being used as a front for a massive counterfeiting ring – and it's run by the same gang that shot Billy's boss.",
  "release_date": 769741200,
  "genre": [
    "Action",
    "Comedy",
    "Crime"
  ]
}
```

Co-authored-by: kwangin.jung <inylove82@gmail.com>
Co-authored-by: Thomas Payet <thomas@meilisearch.com>
Co-authored-by: Charlotte Vermandel <charlottevermandel@gmail.com>
2020-07-24 09:45:01 +00:00