bors[bot]
08a8dc0d0d
Merge #1091
...
1091: New tokenizer r=LegendreM a=MarinPostma
Integration of the new tokenizer to meilisearch.
- Tokenize and normalizes the query string for better search results
- Language sensitive tokenization and normalization during indexation
- better support for Chinese thanks to jieba (when Chinese characters are detected)
To do in a later PR:
- Use a common tokenization instance
- use tokenization for synonyms
close #624
Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: many <maxime@meilisearch.com>
2021-01-06 08:47:53 +00:00
mpostma
0675ecdd73
remove specific task for dump in ci
2021-01-05 21:55:14 +01:00
mpostma
08c160c178
un-ignore dump tests
2021-01-05 21:54:14 +01:00
many
677627586c
fix test set
...
fix dump tests
2021-01-05 21:37:05 +01:00
mpostma
0731971300
fix style
2021-01-05 15:21:06 +01:00
mpostma
c290719984
remove byte offset in index_seq
2021-01-05 15:21:06 +01:00
mpostma
2a145e288c
fix style
2021-01-05 15:21:06 +01:00
many
aeb676e757
skip indexation while token is not a word
2021-01-05 15:21:06 +01:00
many
2852349e68
update tokenizer version
2021-01-05 15:21:06 +01:00
many
0447594e02
add search test on chinese scripts
2021-01-05 15:21:05 +01:00
many
748a8240dd
fix highlight shifting bug
2021-01-05 15:21:05 +01:00
mpostma
808be4678a
fix style
2021-01-05 15:21:05 +01:00
mpostma
398577f116
bump tokenizer
2021-01-05 15:21:05 +01:00
mpostma
8e64a24d19
fix suggestions
2021-01-05 15:21:05 +01:00
mpostma
8b149c9aa3
update tokenizer dep to release
2021-01-05 15:21:05 +01:00
mpostma
a7c88c7951
restore synonyms tests
2021-01-05 15:21:05 +01:00
mpostma
db64e19b8d
all tests pass
2021-01-05 15:21:05 +01:00
mpostma
b574960755
fix split_query_string
2021-01-05 15:21:05 +01:00
mpostma
c6434f609c
fix indexing length
2021-01-05 15:21:05 +01:00
mpostma
206308c1aa
replace hashset with fst::Set
2021-01-05 15:21:05 +01:00
mpostma
6527d3e492
better separator handling
2021-01-05 15:21:05 +01:00
mpostma
e616b1e356
hard separator offset
2021-01-05 15:21:05 +01:00
mpostma
8843062604
fix indexer tests
2021-01-05 15:21:05 +01:00
mpostma
5e00842087
integration with new tokenizer wip
2021-01-05 15:21:05 +01:00
mpostma
8a4d05b7bb
remove meilisearch tokenizer
2021-01-05 15:21:05 +01:00
bors[bot]
061832af7f
Merge #1163
...
1163: remove benches r=LegendreM a=MarinPostma
remove unused benches, that did not compile either
Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-01-05 13:27:42 +00:00
bors[bot]
9dd818ed7b
Merge #1165
...
1165: Bumps r=MarinPostma a=MarinPostma
Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-01-05 12:55:50 +00:00
mpostma
0e04c90abe
remove benches
2021-01-05 10:54:19 +01:00
mpostma
83ea088bf7
fix incompatible deps
2021-01-04 18:33:22 +01:00
mpostma
48eb78b14d
bump deps
2021-01-04 16:56:28 +01:00
bors[bot]
e3d1314bd8
Merge #1147
...
1147: Increasing payload default size r=LegendreM a=sanders41
References issue #1137
Increasing the default payload size from 10mb to 100mb.
Co-authored-by: Paul Sanders <psanders1@gmail.com>
2021-01-04 12:47:06 +00:00
bors[bot]
a05aef5c14
Merge #1151
...
1151: Fixing a comment typo r=MarinPostma a=sanders41
Fixed a typo in a code comment.
Co-authored-by: Paul Sanders <psanders1@gmail.com>
2020-12-31 15:18:40 +00:00
Paul Sanders
3de5161dd8
Fixing a comment typo
2020-12-31 07:32:27 -05:00
Paul Sanders
8e0d8f4533
Increasing payload default size
2020-12-29 16:55:35 -05:00
bors[bot]
d12ef576fc
Merge #1142
...
1142: Update interface.html r=Kerollmops a=curquiza
😇
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2020-12-21 10:58:35 +00:00
Clémentine Urquizar
a05eea3a11
Update interface.html
2020-12-21 10:15:19 +01:00
bors[bot]
446b2e7058
Merge #1128
...
1128: Settings consistency r=MarinPostma a=MarinPostma
- close #1124 , fix #761
- fix some clippy warnings
- makes dump process reentrant
Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: marin <postma.marin@protonmail.com>
2020-12-16 14:12:09 +00:00
marin
e06f3808c0
requested changes
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
Update meilisearch-http/src/routes/setting.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
Update meilisearch-schema/src/schema.rs
Update meilisearch-schema/src/schema.rs
2020-12-16 15:08:36 +01:00
mpostma
6d79107b14
make dumps reentrant
2020-12-15 13:05:01 +01:00
mpostma
5fe0e06342
fix clippy warnings
2020-12-15 12:42:19 +01:00
mpostma
6eb7843858
fix tests
2020-12-15 12:05:17 +01:00
mpostma
2904ca7f57
update codebase with shcema refactor
2020-12-15 12:04:51 +01:00
mpostma
54686b0505
refactor schema
2020-12-15 12:04:33 +01:00
bors[bot]
861c6fec06
Merge #1126
...
1126: Bumps r=MarinPostma a=MarinPostma
bump various meilisearch dependencies
Co-authored-by: mpostma <postma.marin@protonmail.com>
2020-12-14 19:03:59 +00:00
bors[bot]
eec954ede1
Merge #1134
...
1134: Add Roadmap to README r=MarinPostma a=curquiza
Co-authored-by: Clementine Urquizar <clementine@meilisearch.com>
2020-12-14 14:59:38 +00:00
Clementine Urquizar
aa99c1ba55
Add Roadmap in README
2020-12-14 15:38:47 +01:00
bors[bot]
dec0e2545d
Merge #1131
...
1131: fix attributes to retrieve bug r=Kerollmops a=MarinPostma
fix bug when using empty `attributeToRetrieve`
Co-authored-by: mpostma <postma.marin@protonmail.com>
2020-12-10 22:36:42 +00:00
mpostma
90cf4b9462
test attributesToRetrieve
2020-12-10 16:15:12 +01:00
mpostma
2bd5d2474e
fix attributes to retrieve bug
2020-12-10 15:58:24 +01:00
mpostma
a6e08a83a7
bump whoami
2020-12-09 13:44:35 +01:00
