1184: normalize synonyms during indexation r=MarinPostma a=LegendreM
fix#1135#964
Normalizes the synonyms before indexing them, so they are not case sensitive anymore. Then normalization also involves deunicoding is some cases, such as accents, so `été` and `ete` are considered equivalent in a search for synonyms.
Co-authored-by: many <maxime@meilisearch.com>
Co-authored-by: Many <legendre.maxime.isn@gmail.com>
1174: Limit query words number r=MarinPostma a=MarinPostma
This pr adds a limit to the number of words taken into account in a search query. Using query string that are too long leads to huge performance hits and ressources consumtion, that occasionally crashes the machine. The limit has been hard set to 10, and tests have been added to make sure that it is taken into account.
close#941
Co-authored-by: mpostma <postma.marin@protonmail.com>
1167: Update dumps ci r=LegendreM a=MarinPostma
Now that the dump test are re-entrant, they can be run from a multithreaded context, whereas they used to be ran from a single threaded context, in a separate CI task.
Co-authored-by: mpostma <postma.marin@protonmail.com>
1100: [fix] Remove some clippy warnings r=MarinPostma a=woshilapin
fix#1099
I'm also wondering if I should add `-- --deny warnings` to the modified line in `test.yml`.
Co-authored-by: Jean SIMARD <woshilapin@tuziwo.info>
849: Update nbHits count with filtered documents r=MarinPostma a=balajisivaraman
Closes#764close#1039
After discussing with @MarinPostma on Slack, this is my first attempt at implementing this for the basic flow that will go through `bucket_sort_with_distinct`.
A few thoughts here:
- For getting the count of filtered documents alone, I originally thought of using `filter_map.values().filter(|&&v| !v).count()`. In a few cases, this was the same as what I have now implemented. But I realised I couldn't do something similar for `distinct`. So for being consistent, I have implemented both in a similar fashion.
- I also needed the `contains_key` check to ensure we're not counting the same document ID twice.
@MarinPostma also mentioned that this will be an approximation since the sort is lazy. In the test example that I've updated, the actual filtered count will be just 19 (for `male` records), but due to the `limit` in play, it returns 32 (filtering out 11 records overall).
Please let me know if this is the kind of fix we are looking for, and I can implement it in the placeholder search also.
Co-authored-by: Balaji Sivaraman <balaji@balajisivaraman.com>
1089: Fix clear bug r=Kerollmops a=MarinPostma
close#1088
The placeholder data was not cleared on when deleting all documents.
Co-authored-by: mpostma <postma.marin@protonmail.com>
