mirror of
https://github.com/meilisearch/MeiliSearch
synced 2024-12-23 13:10:06 +01:00
935f18efcf
(Some of) these specialized tokenizations include huge dictionaries that currently account for 90% (!) of the meilisearch binary size. This commit adds chinese, hebrew, japanese, and thai feature flags that are propagated via milli down to the charabia crate. To keep it backward compatible, they are enabled by default. Related to meilisearch/milli#632