Commit Graph

3086 Commits

Author SHA1 Message Date
mpostma
7ccbbb7a75
update changelog 2021-01-06 11:54:06 +01:00
mpostma
948c89c26f
bump meilisearch 2021-01-06 11:41:44 +01:00
bors[bot]
768791440a
Merge #1167
1167: Update dumps ci r=LegendreM a=MarinPostma

Now that the dump test are re-entrant, they can be run from a multithreaded context, whereas they used to be ran from a single threaded context, in a separate CI task.

Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-01-06 09:42:59 +00:00
bors[bot]
08a8dc0d0d
Merge #1091
1091: New tokenizer r=LegendreM a=MarinPostma

Integration of the new tokenizer to meilisearch.

- Tokenize and normalizes the query string for better search results
- Language sensitive tokenization and normalization during indexation
- better support for Chinese thanks to jieba (when Chinese characters are detected)

To do in a later PR:
- Use a common tokenization instance
- use tokenization for synonyms

close #624

Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: many <maxime@meilisearch.com>
2021-01-06 08:47:53 +00:00
mpostma
0675ecdd73
remove specific task for dump in ci 2021-01-05 21:55:14 +01:00
mpostma
08c160c178
un-ignore dump tests 2021-01-05 21:54:14 +01:00
many
677627586c
fix test set
fix dump tests
2021-01-05 21:37:05 +01:00
mpostma
0731971300
fix style 2021-01-05 15:21:06 +01:00
mpostma
c290719984
remove byte offset in index_seq 2021-01-05 15:21:06 +01:00
mpostma
2a145e288c
fix style 2021-01-05 15:21:06 +01:00
many
aeb676e757
skip indexation while token is not a word 2021-01-05 15:21:06 +01:00
many
2852349e68
update tokenizer version 2021-01-05 15:21:06 +01:00
many
0447594e02
add search test on chinese scripts 2021-01-05 15:21:05 +01:00
many
748a8240dd
fix highlight shifting bug 2021-01-05 15:21:05 +01:00
mpostma
808be4678a
fix style 2021-01-05 15:21:05 +01:00
mpostma
398577f116
bump tokenizer 2021-01-05 15:21:05 +01:00
mpostma
8e64a24d19
fix suggestions 2021-01-05 15:21:05 +01:00
mpostma
8b149c9aa3
update tokenizer dep to release 2021-01-05 15:21:05 +01:00
mpostma
a7c88c7951
restore synonyms tests 2021-01-05 15:21:05 +01:00
mpostma
db64e19b8d
all tests pass 2021-01-05 15:21:05 +01:00
mpostma
b574960755
fix split_query_string 2021-01-05 15:21:05 +01:00
mpostma
c6434f609c
fix indexing length 2021-01-05 15:21:05 +01:00
mpostma
206308c1aa
replace hashset with fst::Set 2021-01-05 15:21:05 +01:00
mpostma
6527d3e492
better separator handling 2021-01-05 15:21:05 +01:00
mpostma
e616b1e356
hard separator offset 2021-01-05 15:21:05 +01:00
mpostma
8843062604
fix indexer tests 2021-01-05 15:21:05 +01:00
mpostma
5e00842087
integration with new tokenizer wip 2021-01-05 15:21:05 +01:00
mpostma
8a4d05b7bb
remove meilisearch tokenizer 2021-01-05 15:21:05 +01:00
bors[bot]
061832af7f
Merge #1163
1163: remove benches r=LegendreM a=MarinPostma

remove unused benches, that did not compile either


Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-01-05 13:27:42 +00:00
bors[bot]
9dd818ed7b
Merge #1165
1165: Bumps r=MarinPostma a=MarinPostma



Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-01-05 12:55:50 +00:00
mpostma
0e04c90abe
remove benches 2021-01-05 10:54:19 +01:00
mpostma
b07e21ab3c temp 2021-01-05 00:21:42 +01:00
mpostma
83ea088bf7
fix incompatible deps 2021-01-04 18:33:22 +01:00
mpostma
48eb78b14d
bump deps 2021-01-04 16:56:28 +01:00
bors[bot]
e3d1314bd8
Merge #1147
1147: Increasing payload default size r=LegendreM a=sanders41

References issue #1137

Increasing the default payload size from 10mb to 100mb.

Co-authored-by: Paul Sanders <psanders1@gmail.com>
2021-01-04 12:47:06 +00:00
mpostma
b4d447b5cb temp 2021-01-01 16:59:49 +01:00
bors[bot]
a05aef5c14
Merge #1151
1151: Fixing a comment typo r=MarinPostma a=sanders41

Fixed a typo in a code comment.

Co-authored-by: Paul Sanders <psanders1@gmail.com>
2020-12-31 15:18:40 +00:00
Paul Sanders
3de5161dd8 Fixing a comment typo 2020-12-31 07:32:27 -05:00
mpostma
d1e9ded76f
setting builder takes ownership 2020-12-31 00:50:30 +01:00
mpostma
12ee7b9b13 impl get all updates 2020-12-30 19:17:13 +01:00
mpostma
d9dc2036a7 support error & return document count on addition 2020-12-30 18:44:33 +01:00
mpostma
54861335a0 retrieve update status 2020-12-30 18:16:07 +01:00
Paul Sanders
8e0d8f4533 Increasing payload default size 2020-12-29 16:55:35 -05:00
mpostma
0cd9e62fc6 search first iteration 2020-12-24 12:58:34 +01:00
mpostma
02ef1d41d7 route document add json 2020-12-23 16:12:37 +01:00
mpostma
1a38bfd31f data add documents 2020-12-23 13:52:28 +01:00
mpostma
0d7c4beecd reimplement Data 2020-12-22 17:53:13 +01:00
mpostma
55e1552957 update queue refactor, first iteration 2020-12-22 17:13:50 +01:00
mpostma
7c9eaaeadb clean code, and fix errors 2020-12-22 14:02:41 +01:00
bors[bot]
d12ef576fc
Merge #1142
1142: Update interface.html r=Kerollmops a=curquiza

😇

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2020-12-21 10:58:35 +00:00