Kerollmops
68856e5e2f
Disable the default snappy compression for the http-ui crate
2021-09-08 14:17:32 +02:00
Clémentine Urquizar
eb7b9d9dbf
Update version for the next release (v0.13.0)
2021-09-08 10:59:30 +02:00
bors[bot]
5cbe879325
Merge #308
...
308: Implement a better parallel indexer r=Kerollmops a=ManyTheFish
Rewrite the indexer:
- enhance memory consumption control
- optimize parallelism using rayon and crossbeam channel
- factorize the different parts and make new DB implementation easier
- optimize and fix prefix databases
Co-authored-by: many <maxime@meilisearch.com>
2021-09-02 15:03:52 +00:00
Clémentine Urquizar
285849e3a6
Update version for the next release (v0.12.0)
2021-09-02 10:08:41 +02:00
many
1d314328f0
Plug new indexer
2021-09-01 16:48:36 +02:00
Tamo
d106eb5b90
add the sortable attributes to http-ui and fix the tests
2021-08-30 16:25:10 +02:00
Kerollmops
af65485ba7
Reexport the grenad CompressionType from milli
2021-08-24 18:15:31 +02:00
Kerollmops
2f20257070
Update milli to the v0.11.0
2021-08-24 18:10:11 +02:00
Clément Renault
89d0758713
Revert "Revert "Sort at query time""
2021-08-24 11:55:16 +02:00
Clémentine Urquizar
88f6c18665
Update version for the next release (v0.10.2)
2021-08-23 11:33:30 +02:00
Clémentine Urquizar
922f9fd4d5
Revert "Sort at query time"
2021-08-20 18:09:17 +02:00
bors[bot]
41fc0dcb62
Merge #309
...
309: Sort at query time r=Kerollmops a=Kerollmops
This PR:
- Makes the `Asc/Desc` criteria work with strings too, it first returns documents ordered by numbers then by strings, and finally the documents that can't be ordered. Note that it is lexicographically ordered and not ordered by character, which means that it doesn't know about wide and short characters i.e. `a`, `丹`, `▲`.
- Changes the syntax for the `Asc/Desc` criterion by now using a colon to separate the name and the order i.e. `title:asc`, `price:desc`.
- Add the `Sort` criterion at the third position in the ranking rules by default.
- Add the `sort_criteria` method to the `Search` builder struct to let the users define the `Asc/Desc` sortable attributes they want to use at query time. Note that we need to check that the fields are registered in the sortable attributes before performing the search.
- Introduce a new `InvalidSortableAttribute` user error that is raised when the sort criteria declared at query time are not part of the sortable attributes.
- `@ManyTheFish` introduced integration tests for the dynamic Sort criterion.
Fixes #305 .
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: many <maxime@meilisearch.com>
2021-08-18 16:55:32 +00:00
bors[bot]
198c416bd8
Merge #312
...
312: Update milli version to v0.10.1 r=Kerollmops a=curquiza
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-08-18 12:08:04 +00:00
Clémentine Urquizar
6cb9c3b81f
Update milli version to v0.10.1
2021-08-18 13:46:27 +02:00
Clémentine Urquizar
42cf847a63
Update tokenizer version to v0.2.5
2021-08-18 13:37:41 +02:00
Kerollmops
5b88df508e
Use the new Asc/Desc syntax everywhere
2021-08-17 14:15:22 +02:00
Clémentine Urquizar
fcc520e49a
Update version for the next release (v0.10.0)
2021-08-16 12:00:28 +02:00
Clémentine Urquizar
7f26c75610
Update milli to v0.9.0
2021-08-04 16:04:55 +02:00
Kerollmops
341c244965
Bump milli to v0.8.1
2021-07-29 15:56:36 +02:00
Clémentine Urquizar
6a141694da
Update version for the next release (v0.8.0)
2021-07-27 16:38:42 +02:00
bors[bot]
cc54c41e30
Merge #283
...
283: Use the AlwaysFreePages flag when opening an index r=irevoire a=Kerollmops
We introduced a new flag in our fork of LMDB, this `AlwaysFreePages` flag forces LMDB to always free the single pages it uses before writing to the disk instead of keeping them in a linked list.
Declaring this flag reduces the memory print (leak) we have on memory after indexing a lot of documents.
Fixes #279 .
Co-authored-by: Kerollmops <clement@meilisearch.com>
2021-07-05 16:59:16 +00:00
Irevoire
4562b278a8
remove a warning and add a log
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-07-05 17:46:02 +02:00
Tamo
a57e522a67
introduce a die route let the program exit itself alone
2021-07-05 17:38:10 +02:00
Kerollmops
91c5d0c042
Use the AlwaysFreePages flag when opening an index
2021-07-05 16:36:13 +02:00
Kerollmops
a6b4069172
Bump to v0.7.2
2021-07-05 10:54:53 +02:00
Clémentine Urquizar
b489515f4d
Update milli version to v0.7.1
2021-06-30 13:52:46 +02:00
Clément Renault
80c6aaf1fd
Bump milli to 0.7.0
2021-06-28 18:31:56 +02:00
Clément Renault
bdc5599b73
Bump heed to use the git repo with v0.12.0
2021-06-28 18:26:20 +02:00
Kerollmops
98285b4b18
Bump milli to 0.6.0
2021-06-23 17:30:26 +02:00
Clémentine Urquizar
9885fb4159
Update version for the next release (v0.5.1)
2021-06-23 14:05:20 +02:00
bors[bot]
634201244c
Merge #250 #251
...
250: Add the limit field to http-ui r=Kerollmops a=irevoire
251: Fix the limit r=Kerollmops a=irevoire
There was no check on the limit and thus if a user specified a very large number this line could cause a panic.
Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-22 13:00:52 +00:00
Tamo
81643e6d70
add the limit field to http-ui
2021-06-22 14:47:23 +02:00
Tamo
77eb37934f
add jemalloc to http-ui and the benchmarks
2021-06-22 14:17:56 +02:00
Clémentine Urquizar
320670f8fe
Update version for the next release (v0.5.0)
2021-06-21 15:59:17 +02:00
Clémentine Urquizar
35fcc351a0
Update version for the next release (v0.4.2)
2021-06-20 17:37:24 +02:00
Kerollmops
ccd6f13793
Update version to the next release (0.4.1)
2021-06-17 15:01:20 +02:00
Tamo
9716fb3b36
format the whole project
2021-06-16 18:33:33 +02:00
Clémentine Urquizar
f5ff3e8e19
Update version for the next release (v0.4.0)
2021-06-16 14:01:05 +02:00
Kerollmops
78fe4259a9
Fix the http-ui crate
2021-06-14 18:06:23 +02:00
Clémentine Urquizar
7d5395c12b
Update Tokenizer version to v0.2.3
2021-06-10 17:00:04 +02:00
Clémentine Urquizar
dc64e139b9
Update version for the next release (v0.3.1)
2021-06-09 14:39:21 +02:00
Kerollmops
103dddba2f
Move the UpdateStore into the http-ui crate
2021-06-08 17:59:51 +02:00
Clémentine Urquizar
3b2b3aeea9
Update Cargo.toml for next release v0.3.0
2021-06-03 12:24:27 +02:00
Kerollmops
3b1cd4c4b4
Rename the FacetCondition into FilterCondition
2021-06-02 16:24:58 +02:00
Kerollmops
c10469ddb6
Patch the http-ui crate to support filterable fields
2021-06-02 16:24:58 +02:00
Tamo
608c5bad24
fix http-ui
2021-06-01 16:24:46 +02:00
Kerollmops
5012cc3a32
Fix the http-ui crate to support split facet databases
2021-05-25 11:31:06 +02:00
Clémentine Urquizar
1e11578ef0
Update version for the next release (v0.2.1)
2021-05-05 14:57:34 +02:00
Clémentine Urquizar
a8680887d8
Upgrade Milli version (v0.2.0)
2021-05-03 14:50:47 +02:00
Clémentine Urquizar
34e02aba42
Upgrade Tokenizer version (v0.2.2)
2021-05-03 10:55:55 +02:00
Kerollmops
e65bad16cc
Compute the words prefixes at the end of an update
2021-04-27 14:39:52 +02:00
Clément Renault
7aa5753ed2
Make the attribute positions range bounds to be fixed
2021-04-27 14:35:43 +02:00
Kerollmops
e8cc7f9cee
Expose a route in the http-ui to update the WordsLevelPositions
2021-04-27 14:25:34 +02:00
tamo
7fa3a1d23e
makes clippy happy http-ui
2021-04-21 12:36:45 +02:00
Clémentine Urquizar
127d3d028e
Update version for the next release (v0.1.1)
2021-04-19 14:48:13 +02:00
Alexey Shekhirin
33860bc3b7
test(update, settings): set & reset synonyms
...
fixes after review
more fixes after review
2021-04-18 11:24:17 +03:00
Alexey Shekhirin
e39aabbfe6
feat(search, update): synonyms
2021-04-18 11:24:17 +03:00
Clémentine Urquizar
2c5c79d68e
Update Tokenizer version to v0.2.1
2021-04-14 18:54:04 +02:00
Alexey Shekhirin
3af8fa194c
test(http): combine settings assert_(ser|de)_tokens into 1 test
2021-04-10 12:13:59 +03:00
Alexey Shekhirin
84c1dda39d
test(http): setting enum serialize/deserialize
2021-04-08 17:03:40 +03:00
Alexey Shekhirin
dc636d190d
refactor(http, update): introduce setting enum
2021-04-08 17:03:40 +03:00
tamo
a2f46029c7
implement a first version of the stop_words
...
The front must provide a BTreeSet containing the stop words
The stop_words are set at None if an empty Set is provided
add the stop-words in the http-ui interface
Use maplit in the test
and remove all the useless drop(rtxn) at the end of all tests
2021-04-01 13:57:55 +02:00
tamo
62a8f1d707
bump the version of the tokenizer
2021-04-01 13:49:22 +02:00
tamo
13ce0ebb87
stop requestings the facets if the user has hidden them
2021-03-31 16:27:32 +02:00
tamo
bcc131e866
add a button to display or hide the facets
2021-03-31 16:18:53 +02:00
tamo
73dcdb27f6
select a specific release of the tokenizer instead of using the latests git commit
2021-03-25 15:00:18 +01:00
Kerollmops
5af63c74e0
Speed-up the MatchingWords highlighting struct
2021-03-03 15:45:03 +01:00
Clément Renault
c62d2f56d8
Expose an http route for the WordsPrefixes update
2021-02-21 12:16:53 +01:00
Kerollmops
519b1cb5c9
Update dependencies
2021-02-21 10:26:04 +01:00
Clément Renault
fecf3d6fc1
Move the command lines helpers into different crates
2021-02-14 18:55:15 +01:00
Clément Renault
e8639517da
Change the project to become a workspace with milli as a default-member
2021-02-12 16:15:09 +01:00
mpostma
8f43698a60
fix httpui
2021-02-01 19:49:51 +01:00
mpostma
3b60432687
Use update_id in UpdateBuilder
...
Add `the update_id` to the to the updates. The rationale is the
following:
- It allows for better tracability of the update events, thus improved
debugging and logging.
- The enigne is now aware of what he's already processed, and can return
it if asked. It may not make sense now, but in the future, the update
store may not work the same way, and this information about the state
of the engine will be desirable (distributed environement).
2021-02-01 19:46:34 +01:00
Clément Renault
65b821b192
Rename the Index facets method into facets_distribution
2021-01-27 14:15:33 +01:00
Kerollmops
61dbcfa44a
Bump the roaring to 0.6.4
2021-01-26 14:38:43 +01:00
Clément Renault
b0c31500fc
Simplify the front page
2021-01-26 14:14:38 +01:00
Kerollmops
7be275b692
Add the count to the facet distribution
2021-01-26 14:14:37 +01:00
Kerollmops
33945a3115
Introduce a new facet filters query field
2021-01-26 14:09:08 +01:00
Clément Renault
a8e3269ad6
Introduce a basic front to display facets
2021-01-26 14:06:28 +01:00
Clément Renault
2cd8675734
Show facet values even for empty queries
2021-01-26 14:06:28 +01:00
Kerollmops
aa129dd7e8
Display the number of candidates instead of the returned document count
2021-01-26 14:06:28 +01:00
Kerollmops
510df4729c
Append the facet value to the facet query on click
2021-01-26 14:06:28 +01:00
Kerollmops
d25a859985
Display the facet values on the HTML debug page
2021-01-26 14:06:28 +01:00
Kerollmops
3b64735058
Introduce a struct to compute facets values
2021-01-26 14:06:27 +01:00
mpostma
87a56d2bc9
Fix settings bug
...
replace ids with str in settings
This allows for better maintainability of the settings code, since
updating the searchable attributes is now straightforward.
criterion use string
fix reindexing fieldid remaping
add tests for primary_key compute
fix tests
fix http-ui
fixup! add tests for primary_key compute
code improvements settings
update deps
fixup! code improvements settings
fixup! refactor settings updates and fix bug
fixup! Fix settings bug
fixup! Fix settings bug
fixup! Fix settings bug
Update src/update/index_documents/transform.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
fixup! Fix settings bug
2021-01-26 13:53:08 +01:00
Clément Renault
44c0dd0762
Fix an fst Set related warning
2021-01-13 11:03:03 +01:00
mpostma
4f7f7538f7
highlight with new tokenizer
2021-01-11 21:59:37 +01:00
mpostma
1ae761311e
integrate with meilisearch tokenizer
2021-01-07 16:14:27 +01:00
Kerollmops
a576c7ae4b
Display the update meta result content on the update page
2020-12-22 13:42:43 +01:00
Kerollmops
6c7db3d956
Display the time it took to process an update
2020-12-22 13:42:43 +01:00
Kerollmops
9fcbc83ebc
Accept and mirror compression of documents additions
2020-12-22 13:42:42 +01:00
mpostma
49a016b53d
create update handler trait
...
fix type inference error
2020-12-22 12:59:15 +01:00
Kerollmops
77e951e933
Use the byte-unit crate to ease library usage
2020-12-20 12:00:37 +01:00
Clément Renault
5a23417499
Compress updates content using gzip
2020-12-17 10:59:58 +01:00
Clément Renault
e7f2ab9138
Bump grenad to fix an indexing bug
2020-12-05 16:39:15 +01:00
Clément Renault
0959e1501f
Introduce the FacetRevRange Iterator struct
2020-12-04 12:02:23 +01:00
Clément Renault
61b383f422
Introduce the criteria update setting
2020-12-04 12:02:22 +01:00
Clément Renault
58a1f9081c
Allow users to abort pending updates, one by one or all at once
2020-12-01 14:51:05 +01:00
Clément Renault
ba4ba685f9
Make the facet levels maps to previous level groups and don't split them
2020-11-28 12:43:43 +01:00
Clément Renault
fc686aaca7
Use the De Morgan law to simplify the NOT operation
2020-11-23 16:43:56 +01:00
