Commit Graph

8601 Commits

Author SHA1 Message Date
Clément Renault
d19f394630
Make the indexer support gzipped CSV as input 2020-08-21 18:10:24 +02:00
Clément Renault
ff479c865d
Replace pipe by ringtail to improve stdin read performances 2020-08-21 17:45:52 +02:00
Clément Renault
ada30c2789
Introducing more arguments to specify the different compression algorithms 2020-08-21 16:41:26 +02:00
Clément Renault
02335ee72d
Introduce the biggest-value-sizes command on the infos binary 2020-08-21 14:44:42 +02:00
Clément Renault
1e3e756c19
Introduce the words-frequencies command on the infos binary 2020-08-21 14:44:42 +02:00
Kerollmops
6a230fe803
Move the contains_documents logic to a function 2020-08-21 14:44:42 +02:00
Kerollmops
e55a569629
Compress much more the documents database 2020-08-21 14:44:42 +02:00
mpostma
445a6c9ea2 update options name 2020-08-21 14:42:20 +02:00
bors[bot]
67b7d60cb0
Merge #920
920: fix bug and add tests r=MarinPostma a=LegendreM

- add tests about updates
- fix select bug

fix #896

Co-authored-by: many <maxime@meilisearch.com>
2020-08-19 07:56:27 +00:00
many
94b3e8e56e
fix bug and add tests
- add tests about updates
- fix select bug

fix #896
2020-08-19 09:51:57 +02:00
bors[bot]
89b5ae63fc
Merge #915
915: fix unwrap bug r=Kerollmops a=MarinPostma

fix #912.

Co-authored-by: mpostma <postma.marin@protonmail.com>
2020-08-18 12:50:10 +00:00
Kerollmops
962bad3cea
Introduce an infos binary to fetch stats 2020-08-17 19:41:49 +02:00
mpostma
2a79dc9ded log error on unwrap error 2020-08-17 16:32:40 +02:00
Clément Renault
8806fcd545
Introduce a better query and document lexer 2020-08-16 14:36:54 +02:00
Clément Renault
1e358e3ae8
Introduce the AstarBagIter that iterates through best paths 2020-08-15 16:24:06 +02:00
mpostma
5ed62dbf76 fix unwrap bug 2020-08-14 12:16:48 +02:00
bors[bot]
cb267b68ed
Merge #910
910: Fix typo in error message r=MarinPostma a=curquiza

Thanks to @ppamorim for reporting the typos to me!

Co-authored-by: Clementine Urquizar <clementine@meilisearch.com>
2020-08-13 15:43:58 +00:00
Clementine Urquizar
6539be6c46 Fix typo in error message 2020-08-13 17:13:19 +02:00
Clément Renault
7dc594ba4d
Introduce the Search builder struct 2020-08-13 14:27:51 +02:00
bors[bot]
a23bdb31a3
Merge #829
829: implement snapshoting r=MarinPostma a=LegendreM

related to #551.

This pull request permit user to create periodically a snapshot of MeiliSearch database via a command line and launch meiliSearch from a snapshot with another command

## Documentation

### schedule a snapshot
`--snapshot-path <DIRECTORY_PATH>`:
this will periodically create a snapshot `<DB_NAME>.tar.gz` in the specified directory

### change period between 2 snapshot creation
`--snapshot-interval-sec <GAP_IN_SEC>`
choose the time gap between 2 snapshot

### start meilisearch from a snapshot
`--load-from-snapshot <FILE_PATH>`
this will use the snapshot stored at `<FILE_PATH>` to initialize MeiliSearch database,

`--ignore-snapshot-if-db-exists` if set and if a db already exists,
this will skip snapshot importation and continue process with actual db instead of quitting process by returning an Error

`--ignore-missing-snapshot` if set and if no snapshot exists at provided path,
this will skip snapshot importation and continue process with actual db instead of quitting process by returning an Error

Co-authored-by: many <maxime@meilisearch.com>
2020-08-12 16:37:31 +00:00
many
9014290875
implement snapshot 2020-08-12 17:46:28 +02:00
bors[bot]
1903302a74
Merge #906
906: Facet distribution correct case r=LegendreM a=MarinPostma

~

Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: marin <postma.marin@protonmail.com>
2020-08-12 09:04:36 +00:00
Clément Renault
bfb46cbfbe
Introduce the Crtierion enum 2020-08-12 10:43:02 +02:00
mpostma
75c3cb4bb6 fix compile error 2020-08-12 10:31:11 +02:00
marin
bfd0f806f8 requested changed
Co-authored-by: Clément Renault <renault.cle@gmail.com>
2020-08-12 10:31:11 +02:00
mpostma
afab8a7846 clean facet result types 2020-08-12 10:31:11 +02:00
mpostma
afacdbc7a0 update tests for facets distribution case 2020-08-12 10:31:11 +02:00
mpostma
18a50b4dac fix facet distribution case 2020-08-12 10:31:10 +02:00
Clément Renault
6d04a285dc
Retrieve and display the distances of the words found 2020-08-11 15:18:02 +02:00
Clément Renault
1bd37d213a
Lowercase quoted words 2020-08-10 14:49:09 +02:00
Clément Renault
883a8109c8
Show both database and documents database sizes 2020-08-10 14:37:18 +02:00
Clément Renault
a4e0f3f724
Remove the useless TransitiveArc from the serve binary 2020-08-10 14:06:27 +02:00
Clément Renault
edc06a97d6
Remove the useless stats binary 2020-08-10 13:55:02 +02:00
Clément Renault
ae77fe5a69
Introduce an option to specify the maximum database size 2020-08-10 13:53:53 +02:00
Clément Renault
394844062f
Move the documents MTBL database inside the Index 2020-08-10 13:47:19 +02:00
Clément Renault
ecd2b2f217
Make the final merge done in parallel 2020-08-07 15:44:04 +02:00
Clément Renault
91282c8b6a
Move the documents into another file 2020-08-07 13:11:31 +02:00
Clément Renault
fae694a102
Put the documents into an MTBL database 2020-08-07 12:14:40 +02:00
Clément Renault
d5a356902a
Update oxidized-mtbl 2020-08-07 12:14:03 +02:00
Clément Renault
405a71d3a4
Accept csv from stdin 2020-08-06 13:38:21 +02:00
Clément Renault
d3b1096510
Compute the word attribute postings lists on each threads 2020-08-06 11:50:27 +02:00
Clément Renault
8d734941af
Clean up some lines 2020-08-06 10:20:26 +02:00
Clément Renault
a4e3c7c37c
Force the Papa parse delimiter 2020-08-05 14:11:46 +02:00
Clément Renault
6508d497ce
Replace the regex highlighting by a simple algorithm 2020-08-05 13:52:27 +02:00
Clément Renault
4873abe145
Introduce option flags to toggle the indexing engine 2020-08-05 12:10:41 +02:00
Clément Renault
bd4b18541c
Introduce a new indexer which uses an MTBL sorter 2020-08-04 15:44:37 +02:00
Clément Renault
3f21760d56
Update README.md 2020-08-04 15:40:37 +02:00
Clément Renault
bc3a0ac6a3
Display the milli logo and update the description 2020-08-04 15:40:02 +02:00
bors[bot]
fb69769991
Merge #889
889: Fix clippy warnings r=MarinPostma a=TaKO8Ki

Good day!

Since `cargo clippy` showed two warnings like the following, I've fixed them. This is a small PR.

```sh
warning: use of `ok_or` followed by a function call
   --> meilisearch-core/src/database.rs:185:18
    |
185 |                 .ok_or(Error::VersionMismatch("bad VERSION file".to_string()))?;
    |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ help: try this: `ok_or_else(|| Error::VersionMismatch("bad VERSION file".to_string()))`
    |
    = note: `#[warn(clippy::or_fun_call)]` on by default
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#or_fun_call

warning: useless use of `format!`
   --> meilisearch-core/src/database.rs:208:59
    |
208 |                         return Err(Error::VersionMismatch(format!("<0.12.0")));
    |                                                           ^^^^^^^^^^^^^^^^^^ help: consider using `.to_string()`: `"<0.12.0".to_string()`
    |
    = note: `#[warn(clippy::useless_format)]` on by default
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#useless_format

warning: 2 warnings emitted
```

Co-authored-by: Takayuki Maeda <41065217+TaKO8Ki@users.noreply.github.com>
2020-07-29 11:40:08 +00:00
Takayuki Maeda
750e7382c6 fix clippy warnings 2020-07-29 11:32:34 +09:00