Commit Graph

357 Commits

Author SHA1 Message Date
mpostma 7652fc1a04
bump meiliseach 2021-03-22 16:03:19 +01:00
mpostma 9992c36ced
Merge branch 'stable'
fix conflict with master
2021-03-16 16:59:39 +01:00
Jean SIMARD d1f34f926e
[ci] Add all target to check 2021-03-02 20:48:57 +01:00
many 1df0fdf3e2
fix synonyms normalization
Synonyms needs to be indexed in ascendant order,
and the new normalization step for synonyms potentially changes this order
which break the indexation process
because "Harry Potter" > "HP"  but "harry potter" < "hp"
2021-02-03 15:21:06 +01:00
mpostma 8760beed1c
bump meilisearch 2021-02-02 14:23:33 +01:00
bors[bot] c984fa1071
Merge #1176
1176: fix race condition in  document addition r=Kerollmops a=MarinPostma

As described in #1160, there was a race condition when updating settings and adding documents simultaneously. This was due to the schema being updated and document addition being processed in two different transactions. This PR moves the schema update logic for the primary key in the same transaction as the document addition, while maintaining the input checks for the validity of the primary key in the http route, in order not to break the error reporting for the document addition route.

close #1160.

Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: marin <postma.marin@protonmail.com>
2021-02-02 09:26:32 +00:00
bors[bot] 81e9fd8933
Merge #1184
1184: normalize synonyms during indexation r=MarinPostma a=LegendreM

fix #1135 #964

Normalizes the synonyms before indexing them, so they are not case sensitive anymore. Then normalization also involves deunicoding is some cases, such as accents, so `été` and `ete` are considered equivalent in a search for synonyms.

Co-authored-by: many <maxime@meilisearch.com>
Co-authored-by: Many <legendre.maxime.isn@gmail.com>
2021-02-01 14:12:57 +00:00
Many 940f83698c
Update meilisearch-core/src/update/settings_update.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-02-01 12:06:48 +01:00
bors[bot] f37a420a04
Merge #1174
1174: Limit query words number r=MarinPostma a=MarinPostma

This pr adds a limit to the number of words taken into account in a search query. Using query string that are too long leads to huge performance hits and ressources consumtion, that occasionally crashes the machine. The limit has been hard set to 10, and tests have been added to make sure that it is taken into account.

close #941

Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-01-28 17:38:34 +00:00
many eeccdce33a
update tokenizer to v0.1.3 2021-01-28 10:33:44 +01:00
marin 1d910dbb42
Update meilisearch-core/src/update/documents_addition.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-01-15 00:55:31 +01:00
many 7a7cb9bcbf
update dependencies 2021-01-13 15:48:53 +01:00
many 9b47bbc1ac
bump meilisearch 2021-01-13 15:37:15 +01:00
mpostma 430a5f902b
fix race condition in document addition 2021-01-13 13:17:52 +01:00
Many bc0d53e819
Update meilisearch-core/src/update/settings_update.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-01-13 13:17:19 +01:00
many 06b2a587af
normalize synonyms during indexation 2021-01-12 13:53:32 +01:00
mpostma 81f343a46a
add word limit to search queries 2021-01-08 16:23:23 +01:00
mpostma 948c89c26f
bump meilisearch 2021-01-06 11:41:44 +01:00
many 677627586c
fix test set
fix dump tests
2021-01-05 21:37:05 +01:00
mpostma 0731971300
fix style 2021-01-05 15:21:06 +01:00
mpostma c290719984
remove byte offset in index_seq 2021-01-05 15:21:06 +01:00
mpostma 2a145e288c
fix style 2021-01-05 15:21:06 +01:00
many aeb676e757
skip indexation while token is not a word 2021-01-05 15:21:06 +01:00
many 2852349e68
update tokenizer version 2021-01-05 15:21:06 +01:00
many 748a8240dd
fix highlight shifting bug 2021-01-05 15:21:05 +01:00
mpostma 808be4678a
fix style 2021-01-05 15:21:05 +01:00
mpostma 398577f116
bump tokenizer 2021-01-05 15:21:05 +01:00
mpostma 8e64a24d19
fix suggestions 2021-01-05 15:21:05 +01:00
mpostma 8b149c9aa3
update tokenizer dep to release 2021-01-05 15:21:05 +01:00
mpostma a7c88c7951
restore synonyms tests 2021-01-05 15:21:05 +01:00
mpostma db64e19b8d
all tests pass 2021-01-05 15:21:05 +01:00
mpostma b574960755
fix split_query_string 2021-01-05 15:21:05 +01:00
mpostma c6434f609c
fix indexing length 2021-01-05 15:21:05 +01:00
mpostma 206308c1aa
replace hashset with fst::Set 2021-01-05 15:21:05 +01:00
mpostma 6527d3e492
better separator handling 2021-01-05 15:21:05 +01:00
mpostma e616b1e356
hard separator offset 2021-01-05 15:21:05 +01:00
mpostma 8843062604
fix indexer tests 2021-01-05 15:21:05 +01:00
mpostma 5e00842087
integration with new tokenizer wip 2021-01-05 15:21:05 +01:00
mpostma 8a4d05b7bb
remove meilisearch tokenizer 2021-01-05 15:21:05 +01:00
bors[bot] 061832af7f
Merge #1163
1163: remove benches r=LegendreM a=MarinPostma

remove unused benches, that did not compile either


Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-01-05 13:27:42 +00:00
mpostma 0e04c90abe
remove benches 2021-01-05 10:54:19 +01:00
mpostma 48eb78b14d
bump deps 2021-01-04 16:56:28 +01:00
mpostma 5fe0e06342 fix clippy warnings 2020-12-15 12:42:19 +01:00
mpostma 2904ca7f57 update codebase with shcema refactor 2020-12-15 12:04:51 +01:00
mpostma d45c794a9e bump rustyline 2020-12-09 12:41:36 +01:00
mpostma c9dd7e10b9 bump ordered floats 2020-12-09 12:40:24 +01:00
mpostma 56ad400c49 update heed 2020-12-09 11:27:38 +01:00
mpostma e2b0402cf5 bump regex 2020-12-09 10:28:22 +01:00
mpostma 0c7fffeaf6 update env-logger 2020-12-09 10:25:17 +01:00
mpostma 5f8dc21dd2 bump once-cell 2020-12-09 10:22:14 +01:00