Kerollmops
481b0bf277
Warn for when a facet key is too large for LMDB
2021-06-22 10:57:46 +02:00
Clémentine Urquizar
daef43f504
Rename FieldsDistribution into FieldDistribution
2021-06-21 15:57:41 +02:00
Tamo
d08cfda796
convert the field_distribution to a BTreeMap and avoid counting twice the same documents
2021-06-17 18:31:54 +02:00
Tamo
969adaefdf
rename fields_distribution in field_distribution
2021-06-17 15:16:20 +02:00
Tamo
9716fb3b36
format the whole project
2021-06-16 18:33:33 +02:00
Kerollmops
713acc408b
Introduce the primary key to the Settings builder structure
2021-06-16 11:03:36 +02:00
Kerollmops
28c004aa2c
Prefer using constant for the database names
2021-06-15 11:13:04 +02:00
Kerollmops
312c2d1d8e
Use the Error enum everywhere in the project
2021-06-14 16:58:38 +02:00
Kerollmops
d2b1ecc885
Remove a lot of serialization unreachable errors
2021-06-14 16:48:51 +02:00
Kerollmops
65b1d09d55
Move the obkv merging functions into the merge_function module
2021-06-14 16:48:51 +02:00
Kerollmops
ab727e428b
Remove the docid_word_positions_merge method that must never be called
2021-06-14 16:48:51 +02:00
Kerollmops
93a8633f18
Remove the documents_merge method that must never be called
2021-06-14 16:48:51 +02:00
Kerollmops
cfc7314bd1
Prefer using an explicit merge function name
2021-06-14 16:48:50 +02:00
Kerollmops
93978ec38a
Serializing a RoaringBitmap into a Vec cannot fail
2021-06-14 16:48:50 +02:00
Kerollmops
ff9414a6ba
Use the out of the compute_primary_key_pair function
2021-06-14 16:48:50 +02:00
bors[bot]
39ed133f9f
Merge #193
...
193: Fix primary key behavior r=Kerollmops a=MarinPostma
this pr:
- Adds early returns on empty document additions, avoiding error messages to be returned when adding no documents and no primary key was set.
- Changes the primary key inference logic to match that of legacy meilisearch.
close #194
Co-authored-by: Marin Postma <postma.marin@protonmail.com>
Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-03 10:24:21 +00:00
marin postma
57898d8a90
fix silent deserialize error
2021-06-03 10:42:55 +02:00
Many
8e6d1ff0dc
Update milli/src/update/index_documents/store.rs
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-06-01 17:04:02 +02:00
many
4ddf008be2
add field id word count database
2021-05-31 16:27:28 +02:00
Kerollmops
1c0a5cd136
Resolve code modification suggestions
2021-05-31 15:22:50 +02:00
Clément Renault
3a4a150ef0
Fix the tests and remaining warnings
2021-05-25 11:31:06 +02:00
Clément Renault
bd7b285bae
Split the update side to use the number and the strings facet databases
2021-05-25 11:30:00 +02:00
Marin Postma
eeb0c70ea2
meilisearch compatible primary key inference
2021-05-06 22:42:32 +02:00
Marin Postma
313c362461
early return on empty document addition
2021-05-06 18:14:16 +02:00
Alexey Shekhirin
f8d0f5265f
fix(update): fields distribution after documents merge
2021-05-04 22:12:20 +03:00
Alexey Shekhirin
d81c0e8bba
feat(update): disable autogenerate_docids by default
2021-04-30 21:41:34 +03:00
Marin Postma
e8e32e0ba1
make document addition number visible
2021-04-29 20:05:07 +02:00
Kerollmops
e65bad16cc
Compute the words prefixes at the end of an update
2021-04-27 14:39:52 +02:00
Clément Renault
7aa5753ed2
Make the attribute positions range bounds to be fixed
2021-04-27 14:35:43 +02:00
Kerollmops
bd1a371c62
Compute the WordsLevelPositions only once
2021-04-27 14:25:34 +02:00
Kerollmops
3069bf4f4a
Fix and improve the words-level-positions computation
2021-04-27 14:25:34 +02:00
Kerollmops
3a25137ee4
Expose and use the WordsLevelPositions update
2021-04-27 14:25:34 +02:00
Kerollmops
9242f2f1d4
Store the first word positions levels
2021-04-27 14:25:34 +02:00
Kerollmops
c9b2d3ae1a
Warn instead of returning an error when a conversion fails
2021-04-20 10:23:31 +02:00
Kerollmops
51767725b2
Simplify integer and float functions trait bounds
2021-04-20 10:23:31 +02:00
Alexey Shekhirin
2658c5c545
feat(index): update fields distribution in clear & delete operations
...
fixes after review
bump the version of the tokenizer
implement a first version of the stop_words
The front must provide a BTreeSet containing the stop words
The stop_words are set at None if an empty Set is provided
add the stop-words in the http-ui interface
Use maplit in the test
and remove all the useless drop(rtxn) at the end of all tests
Integrate the stop_words in the querytree
remove the stop_words from the querytree except if it was a prefix or a typo
more fixes after review
2021-04-01 19:12:35 +03:00
Alexey Shekhirin
27c7ab6e00
feat(index): store fields distribution in index
2021-04-01 18:35:19 +03:00
tamo
a2f46029c7
implement a first version of the stop_words
...
The front must provide a BTreeSet containing the stop words
The stop_words are set at None if an empty Set is provided
add the stop-words in the http-ui interface
Use maplit in the test
and remove all the useless drop(rtxn) at the end of all tests
2021-04-01 13:57:55 +02:00
Alexey Shekhirin
9205b640a4
feat(index): introduce fields_ids_distribution
2021-03-31 18:44:47 +03:00
mpostma
615fe095e1
update index updated at on index writes
2021-03-15 14:05:47 +01:00
Kerollmops
f376c6a728
Make sure we retrieve the docid word positions
2021-03-03 15:45:03 +01:00
many
246286f0eb
take hard separator into account
2021-03-03 15:45:03 +01:00
mpostma
e08b6b3ec7
add primary key to fields_id_map when not present
2021-03-01 16:10:16 +01:00
Kerollmops
a4a48be923
Run the words prefixes update inside of the indexing documents update
2021-02-17 11:22:26 +01:00
Clément Renault
62eee9c69e
Introduce the sorter_into_lmdb_database helper function
2021-02-17 11:12:39 +01:00
Clément Renault
f365de636f
Compute and write the word-prefix-docids database
2021-02-17 11:12:38 +01:00
Clément Renault
89ce4e74fe
Do not change the primary key type when we serialize documents
2021-02-15 21:24:36 +01:00
Clément Renault
e8639517da
Change the project to become a workspace with milli as a default-member
2021-02-12 16:15:09 +01:00
