bors[bot] b985b96e4e
Merge #3530
3530: Fix highlighter bug r=Kerollmops a=ManyTheFish

# Pull Request

There was a highlighting issue on CJK's character, we were highlighting too many characters and these additional characters were duplicated after the highlight tag.

## Related issue
Fixes #3517 
Fixes #3526 

## What does this PR do?
- add a test showcasing the bug
- fix the bug by activating the char_map creation of the tokenizer during the highlighting process


Co-authored-by: ManyTheFish <many@meilisearch.com>
2023-02-23 10:59:43 +00:00
..
2022-12-21 11:27:15 +08:00
2023-01-24 13:20:20 +01:00
2023-02-23 10:59:43 +00:00
2023-01-19 15:48:20 +01:00
2023-01-19 15:48:20 +01:00
2023-01-19 16:10:05 +01:00
2023-01-19 16:10:05 +01:00