Clément Renault
081278dfd6
Use the facet string levels when computing the facet distribution
2021-07-21 16:59:39 +02:00
Clément Renault
5676b204dd
Fix the facet string levels codecs
2021-07-21 16:59:38 +02:00
Kerollmops
8c86348119
Indexing the facet strings levels
2021-07-21 16:59:38 +02:00
Kerollmops
a7ae552ba7
Fix the FacetStringLevelZeroRange range when unbounded
2021-07-21 16:59:38 +02:00
Kerollmops
757b2b502a
Remove the FacetValueStringCodec
2021-07-21 16:59:38 +02:00
Kerollmops
adfd4da24c
Introduce the FacetStringIter iterator
2021-07-21 16:59:38 +02:00
Kerollmops
a79661c6dc
Introduce a lot of facet string helper iterators
2021-07-21 16:59:38 +02:00
Kerollmops
851f979039
Describe the way we want to group the facet strings
2021-07-21 16:59:38 +02:00
Kerollmops
f858f64b1f
Move the facet number iterators into their own module
2021-07-21 16:59:37 +02:00
Kerollmops
9f8095c069
Make sure that we don't keep a reference on the LMDB key when using put_current
2021-07-21 10:35:35 +02:00
Kerollmops
a9553af635
Add a test to check that we can index more that 256 fields
2021-07-06 11:58:03 +02:00
Kerollmops
838ed1cd32
Use an u16 field id instead of one byte
2021-07-06 11:58:03 +02:00
Kerollmops
91c5d0c042
Use the AlwaysFreePages flag when opening an index
2021-07-05 16:36:13 +02:00
Kerollmops
a6b4069172
Bump to v0.7.2
2021-07-05 10:54:53 +02:00
many
9f62149b94
Fix matching lenghth in matching_words
2021-07-01 19:03:28 +02:00
Clémentine Urquizar
3c149d8a43
Update tokenizer version to v0.2.3
2021-06-30 18:41:35 +02:00
bors[bot]
b4dcdbf00d
Merge #269 #271
...
269: Fix bug when inserting previously deleted documents r=Kerollmops a=Kerollmops
This PR fixes #268 .
The issue was in the `ExternalDocumentsIds` implementation in the specific case that an external document id was in the soft map marked as deleted.
The bug was due to a wrong assumption on my side about how the FST unions were returning the `IndexedValue`s, I thought the values returned in an array were in the same order as the FSTs given to the `OpBuilder` but in fact, [the `IndexedValue`'s `index` field was here to indicate from which FST the values were coming from](https://docs.rs/fst/0.4.7/fst/map/struct.IndexedValue.html ).
271: Remove the roaring operation functions warnings r=Kerollmops a=Kerollmops
In this PR we are just replacing the usages of the roaring operations function by the new operators. This removes a lot of warnings.
Co-authored-by: Kerollmops <clement@meilisearch.com>
2021-06-30 12:34:55 +00:00
Kerollmops
32b7bd366f
Remove the roaring operation functions warnings
2021-06-30 14:12:56 +02:00
Kerollmops
c92ef54466
Add a test for when we insert a previously deleted document
2021-06-30 14:00:01 +02:00
Kerollmops
28782ff99d
Fix ExternalDocumentsIds struct when inserting previously deleted ids
2021-06-30 14:00:01 +02:00
Clémentine Urquizar
b489515f4d
Update milli version to v0.7.1
2021-06-30 13:52:46 +02:00
Kerollmops
54889813ce
Implement some debug functions on the ExternalDocumentsIds struct
2021-06-30 11:29:41 +02:00
Kerollmops
4bce66d5ff
Make the Index::delete_* method private
2021-06-30 10:07:31 +02:00
Irevoire
6044b80362
Update milli/src/search/matching_words.rs
...
Co-authored-by: Clément Renault <renault.cle@gmail.com>
2021-06-30 00:35:26 +02:00
Tamo
be75e738b1
add more tests
2021-06-29 16:24:58 +02:00
Tamo
56fceb1928
re-implement the Damerau-Levenshtein used for the highlighting
2021-06-29 15:36:03 +02:00
Clément Renault
80c6aaf1fd
Bump milli to 0.7.0
2021-06-28 18:31:56 +02:00
Clément Renault
bdc5599b73
Bump heed to use the git repo with v0.12.0
2021-06-28 18:26:20 +02:00
Clément Renault
0013236e5d
Fix the LMDB and heed invalid interactions.
...
It is undefined behavior to keep a reference to the database while
modifying it, we were keeping references in the database and also
feeding the heed put_current methods with keys referenced inside
the database itself.
https://github.com/Kerollmops/heed/pull/108
2021-06-28 16:19:02 +02:00
Kerollmops
9e5f9a8a10
Add a test for the words level positions generation bug
2021-06-28 16:08:31 +02:00
Kerollmops
98285b4b18
Bump milli to 0.6.0
2021-06-23 17:30:26 +02:00
Kerollmops
4fc8f06791
Rename faceted_fields into filterable_fields
2021-06-23 17:26:54 +02:00
Kerollmops
c31cadb54f
Do not consider the searchable field as filterable
2021-06-23 17:26:54 +02:00
bors[bot]
2ab24c4f49
Merge #256
...
256: Update version for the next release (v0.5.1) r=Kerollmops a=curquiza
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-23 12:29:57 +00:00
Clémentine Urquizar
9885fb4159
Update version for the next release (v0.5.1)
2021-06-23 14:05:20 +02:00
Kerollmops
a6218a20ae
Introduce a new InvalidFacetsDistribution user error
2021-06-23 13:56:19 +02:00
Kerollmops
2364777838
Return an error for when a field distribution cannot be done
2021-06-23 11:50:49 +02:00
Kerollmops
aeaac743ff
Replace an if let some by a match
2021-06-23 11:33:30 +02:00
Tamo
8d2a0b43ff
run the formatter on the whole project a second time
2021-06-22 15:36:22 +02:00
Tamo
3d90b03d7b
fix the limit
...
There was no check on the limit and thus, if a user especified a very large number this line could causes a panic
2021-06-22 14:52:13 +02:00
bors[bot]
5b6adc6d96
Merge #245
...
245: Warn for when a key is too large for LMDB r=Kerollmops a=Kerollmops
Closes #191 , and resolves #140 .
Co-authored-by: Kerollmops <clement@meilisearch.com>
2021-06-22 12:10:52 +00:00
Kerollmops
51dbb2e06d
Warn for when a key is too large for LMDB
2021-06-22 11:51:36 +02:00
Kerollmops
aecbd14761
Improve the error message for InvalidDocumentId
2021-06-22 11:31:58 +02:00
Kerollmops
0cca2ea24f
Return a MissingDocumentId when a document doesn't have one
2021-06-22 11:22:33 +02:00
Kerollmops
481b0bf277
Warn for when a facet key is too large for LMDB
2021-06-22 10:57:46 +02:00
bors[bot]
b073fd49ea
Merge #244
...
244: Update version for the next release (v0.5.0) r=Kerollmops a=curquiza
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-21 14:27:10 +00:00
Clémentine Urquizar
320670f8fe
Update version for the next release (v0.5.0)
2021-06-21 15:59:17 +02:00
Clémentine Urquizar
daef43f504
Rename FieldsDistribution into FieldDistribution
2021-06-21 15:57:41 +02:00
Clémentine Urquizar
35fcc351a0
Update version for the next release (v0.4.2)
2021-06-20 17:37:24 +02:00
bors[bot]
5b19dd23d9
Merge #240
...
240: Field distribution r=Kerollmops a=irevoire
closes #199
closes #198
Co-authored-by: Tamo <tamo@meilisearch.com>
2021-06-19 10:14:25 +00:00
Tamo
d08cfda796
convert the field_distribution to a BTreeMap and avoid counting twice the same documents
2021-06-17 18:31:54 +02:00
bors[bot]
a9e552ab18
Merge #238
...
238: Integration tests on filters and distinct r=Kerollmops a=ManyTheFish
Fix #216
Fix #120
Co-authored-by: many <maxime@meilisearch.com>
2021-06-17 15:00:51 +00:00
many
6cb1102bdb
Fix PR comments
2021-06-17 15:19:03 +02:00
Tamo
969adaefdf
rename fields_distribution in field_distribution
2021-06-17 15:16:20 +02:00
Kerollmops
ccd6f13793
Update version to the next release (0.4.1)
2021-06-17 15:01:20 +02:00
many
f496cd320d
Add distinct integration tests
2021-06-17 14:33:18 +02:00
many
9f4184208e
Add test on filters
2021-06-17 13:56:09 +02:00
marin
70bee7d405
re-export remaining error types
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-06-17 11:49:03 +02:00
marin postma
abbebad669
change sub errors visibility
2021-06-17 11:44:01 +02:00
Tamo
9716fb3b36
format the whole project
2021-06-16 18:33:33 +02:00
Clémentine Urquizar
f5ff3e8e19
Update version for the next release (v0.4.0)
2021-06-16 14:01:05 +02:00
many
ce0315a10f
Close write transaction in test
2021-06-16 11:03:37 +02:00
Kerollmops
7ac441e473
Fix small typos
2021-06-16 11:03:37 +02:00
Kerollmops
adf0c389c5
Rename FilterParsing into InvalidFilter
2021-06-16 11:03:36 +02:00
Kerollmops
8cfe3e1ec0
Rename DatabaseSizeReached into MaxDatabaseSizeReached
2021-06-16 11:03:36 +02:00
Kerollmops
4eda438f6f
Add a new Error for when a user use a non-filtered attribute in a filter
2021-06-16 11:03:36 +02:00
Kerollmops
713acc408b
Introduce the primary key to the Settings builder structure
2021-06-16 11:03:36 +02:00
Kerollmops
a7d6930905
Replace the panicking expect by tracked Errors
2021-06-15 11:51:32 +02:00
Kerollmops
f0e804afd5
Rename the FieldIdMapMissingEntry from_db_name field into process
2021-06-15 11:13:04 +02:00
Kerollmops
28c004aa2c
Prefer using constant for the database names
2021-06-15 11:13:04 +02:00
Kerollmops
312c2d1d8e
Use the Error enum everywhere in the project
2021-06-14 16:58:38 +02:00
Kerollmops
ca78cb5aca
Introduce more variants to the error module enums
2021-06-14 16:58:38 +02:00
Kerollmops
456541e921
Implement the Display trait on the Error type
2021-06-14 16:48:51 +02:00
Kerollmops
44c353fafd
Introduce some way to construct an Error
2021-06-14 16:48:51 +02:00
Kerollmops
23fcf7920e
Introduce a basic version of the InternalError struct
2021-06-14 16:48:51 +02:00
Kerollmops
d2b1ecc885
Remove a lot of serialization unreachable errors
2021-06-14 16:48:51 +02:00
Kerollmops
65b1d09d55
Move the obkv merging functions into the merge_function module
2021-06-14 16:48:51 +02:00
Kerollmops
ab727e428b
Remove the docid_word_positions_merge method that must never be called
2021-06-14 16:48:51 +02:00
Kerollmops
93a8633f18
Remove the documents_merge method that must never be called
2021-06-14 16:48:51 +02:00
Kerollmops
cfc7314bd1
Prefer using an explicit merge function name
2021-06-14 16:48:50 +02:00
Kerollmops
93978ec38a
Serializing a RoaringBitmap into a Vec cannot fail
2021-06-14 16:48:50 +02:00
Kerollmops
ff9414a6ba
Use the out of the compute_primary_key_pair function
2021-06-14 16:48:50 +02:00
Many
f4cab080a6
Update milli/src/search/query_tree.rs
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-06-10 11:30:51 +02:00
Many
36715f571c
Update milli/src/search/criteria/proximity.rs
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-06-10 11:30:33 +02:00
many
e923a3ed6a
Replace Consecutive by Phrase in query tree
...
Replace Consecutive by Phrase in query tree in order to remove theorical bugs,
due of the Consecutive enum type.
2021-06-10 11:16:16 +02:00
Clémentine Urquizar
dc64e139b9
Update version for the next release (v0.3.1)
2021-06-09 14:39:21 +02:00
bors[bot]
afb4133bd2
Merge #212 #222 #223
...
212: Introduce integration test on criteria r=Kerollmops a=ManyTheFish
- add pre-ranked dataset
- test each criterion 1 by 1
- test all criteria in several order
222: Move the `UpdateStore` into the http-ui crate r=Kerollmops a=Kerollmops
We no more need to have the `UpdateStore` inside of the mill crate as this is the job of the caller to stack the updates and sequentially give them to milli.
223: Update dataset links r=Kerollmops a=curquiza
Co-authored-by: many <maxime@meilisearch.com>
Co-authored-by: Many <legendre.maxime.isn@gmail.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-09 08:47:19 +00:00
bors[bot]
6faa87302c
Merge #220
...
220: Make hard separators split phrase query r=Kerollmops a=ManyTheFish
hard separators will now split a phrase query as two sequential phrases (double-quoted strings):
the query `"Radioactive (Imagine Dragons)"` would be considered equivalent to `"Radioactive" "Imagine Dragons"` which as the little disadvantage of not keeping the order of the two (or more) separate phrases.
Fix #208
Co-authored-by: many <maxime@meilisearch.com>
Co-authored-by: Many <legendre.maxime.isn@gmail.com>
2021-06-09 08:22:58 +00:00
Many
f4ff30e99d
Update milli/tests/search/mod.rs
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-06-09 10:12:24 +02:00
Many
ab696f6a23
Update milli/tests/search/query_criteria.rs
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-06-09 10:12:17 +02:00
Kerollmops
0bf4f3f48a
Modify a test to check that criteria additions change the fields ids map
2021-06-08 18:14:34 +02:00
Kerollmops
82df524e09
Make sure that we register the field when setting criteria
2021-06-08 18:14:33 +02:00
Kerollmops
103dddba2f
Move the UpdateStore into the http-ui crate
2021-06-08 17:59:51 +02:00
Many
faf148d297
Update milli/src/search/query_tree.rs
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-06-08 17:52:37 +02:00
Kerollmops
133ab98260
Use the index primary key when deleting documents
2021-06-08 17:33:29 +02:00
many
b489d699ce
Make hard separators split phrase query
...
hard separators will now split a phrase query as double double-quotes
Fix #208
2021-06-08 17:29:38 +02:00
Many
afb09c914d
Update milli/tests/search/query_criteria.rs
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-06-08 16:53:56 +02:00
many
b64cd2a3e3
Resolve PR comments
2021-06-08 14:14:34 +02:00
many
1fcc5f73ac
Factorize tests using macro_rules
2021-06-08 12:33:02 +02:00
many
10882bcbce
Introduce integration test on criteria
2021-06-03 14:44:53 +02:00
