feat(index): update fields distribution in clear & delete operations

fixes after review

bump the version of the tokenizer

implement a first version of the stop_words

The front must provide a BTreeSet containing the stop words
The stop_words are set at None if an empty Set is provided
add the stop-words in the http-ui interface

Use maplit in the test
and remove all the useless drop(rtxn) at the end of all tests

Integrate the stop_words in the querytree

remove the stop_words from the querytree except if it was a prefix or a typo

more fixes after review
This commit is contained in:
Alexey Shekhirin 2021-04-01 10:07:16 +03:00
parent 27c7ab6e00
commit 2658c5c545
No known key found for this signature in database
GPG key ID: AF9A26AA133B5B98
7 changed files with 128 additions and 34 deletions

View file

@ -19,6 +19,11 @@ impl<'a> ExternalDocumentsIds<'a> {
}
}
/// Returns `true` if hard and soft external documents lists are empty.
pub fn is_empty(&self) -> bool {
self.hard.is_empty() && self.soft.is_empty()
}
pub fn get<A: AsRef<[u8]>>(&self, external_id: A) -> Option<u32> {
let external_id = external_id.as_ref();
match self.soft.get(external_id).or_else(|| self.hard.get(external_id)) {