Commit Graph

299 Commits

Author SHA1 Message Date
Marin Postma
4fe885408b
fix arm 2021-03-29 17:19:31 +02:00
mpostma
9992c36ced
Merge branch 'stable'
fix conflict with master
2021-03-16 16:59:39 +01:00
Jean SIMARD
d1f34f926e
[ci] Add all target to check 2021-03-02 20:48:57 +01:00
many
1df0fdf3e2
fix synonyms normalization
Synonyms needs to be indexed in ascendant order,
and the new normalization step for synonyms potentially changes this order
which break the indexation process
because "Harry Potter" > "HP"  but "harry potter" < "hp"
2021-02-03 15:21:06 +01:00
bors[bot]
c984fa1071
Merge #1176
1176: fix race condition in  document addition r=Kerollmops a=MarinPostma

As described in #1160, there was a race condition when updating settings and adding documents simultaneously. This was due to the schema being updated and document addition being processed in two different transactions. This PR moves the schema update logic for the primary key in the same transaction as the document addition, while maintaining the input checks for the validity of the primary key in the http route, in order not to break the error reporting for the document addition route.

close #1160.

Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: marin <postma.marin@protonmail.com>
2021-02-02 09:26:32 +00:00
bors[bot]
81e9fd8933
Merge #1184
1184: normalize synonyms during indexation r=MarinPostma a=LegendreM

fix #1135 #964

Normalizes the synonyms before indexing them, so they are not case sensitive anymore. Then normalization also involves deunicoding is some cases, such as accents, so `été` and `ete` are considered equivalent in a search for synonyms.

Co-authored-by: many <maxime@meilisearch.com>
Co-authored-by: Many <legendre.maxime.isn@gmail.com>
2021-02-01 14:12:57 +00:00
Many
940f83698c
Update meilisearch-core/src/update/settings_update.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-02-01 12:06:48 +01:00
marin
1d910dbb42
Update meilisearch-core/src/update/documents_addition.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
2021-01-15 00:55:31 +01:00
mpostma
430a5f902b
fix race condition in document addition 2021-01-13 13:17:52 +01:00
Many
bc0d53e819
Update meilisearch-core/src/update/settings_update.rs
Co-authored-by: marin <postma.marin@protonmail.com>
2021-01-13 13:17:19 +01:00
many
06b2a587af
normalize synonyms during indexation 2021-01-12 13:53:32 +01:00
mpostma
81f343a46a
add word limit to search queries 2021-01-08 16:23:23 +01:00
many
677627586c
fix test set
fix dump tests
2021-01-05 21:37:05 +01:00
mpostma
0731971300
fix style 2021-01-05 15:21:06 +01:00
mpostma
c290719984
remove byte offset in index_seq 2021-01-05 15:21:06 +01:00
mpostma
2a145e288c
fix style 2021-01-05 15:21:06 +01:00
many
aeb676e757
skip indexation while token is not a word 2021-01-05 15:21:06 +01:00
many
2852349e68
update tokenizer version 2021-01-05 15:21:06 +01:00
many
748a8240dd
fix highlight shifting bug 2021-01-05 15:21:05 +01:00
mpostma
808be4678a
fix style 2021-01-05 15:21:05 +01:00
mpostma
8e64a24d19
fix suggestions 2021-01-05 15:21:05 +01:00
mpostma
a7c88c7951
restore synonyms tests 2021-01-05 15:21:05 +01:00
mpostma
db64e19b8d
all tests pass 2021-01-05 15:21:05 +01:00
mpostma
b574960755
fix split_query_string 2021-01-05 15:21:05 +01:00
mpostma
c6434f609c
fix indexing length 2021-01-05 15:21:05 +01:00
mpostma
206308c1aa
replace hashset with fst::Set 2021-01-05 15:21:05 +01:00
mpostma
6527d3e492
better separator handling 2021-01-05 15:21:05 +01:00
mpostma
e616b1e356
hard separator offset 2021-01-05 15:21:05 +01:00
mpostma
8843062604
fix indexer tests 2021-01-05 15:21:05 +01:00
mpostma
5e00842087
integration with new tokenizer wip 2021-01-05 15:21:05 +01:00
mpostma
8a4d05b7bb
remove meilisearch tokenizer 2021-01-05 15:21:05 +01:00
mpostma
5fe0e06342 fix clippy warnings 2020-12-15 12:42:19 +01:00
mpostma
2904ca7f57 update codebase with shcema refactor 2020-12-15 12:04:51 +01:00
mpostma
56ad400c49 update heed 2020-12-09 11:27:38 +01:00
bors[bot]
f564a9ce51
Merge #849
849: Update nbHits count with filtered documents r=MarinPostma a=balajisivaraman

Closes #764 
close #1039

After discussing with @MarinPostma on Slack, this is my first attempt at implementing this for the basic flow that will go through `bucket_sort_with_distinct`.

A few thoughts here: 

- For getting the count of filtered documents alone, I originally thought of using `filter_map.values().filter(|&&v| !v).count()`. In a few cases, this was the same as what I have now implemented. But I realised I couldn't do something similar for `distinct`. So for being consistent, I have implemented both in a similar fashion.
- I also needed the `contains_key` check to ensure we're not counting the same document ID twice.

@MarinPostma also mentioned that this will be an approximation since the sort is lazy. In the test example that I've updated, the actual filtered count will be just 19 (for `male` records), but due to the `limit` in play, it returns 32 (filtering out 11 records overall).

Please let me know if this is the kind of fix we are looking for, and I can implement it in the placeholder search also.

Co-authored-by: Balaji Sivaraman <balaji@balajisivaraman.com>
2020-11-26 09:53:13 +00:00
Balaji Sivaraman
75e22fc7f5 feat(search): update nbHits count with filtered docs for placeholder search 2020-11-19 21:02:47 +05:30
Balaji Sivaraman
43df4a56c4 feat(search): update nbHits count with filtered docs for core flow 2020-11-19 19:35:37 +05:30
mpostma
3a0861694d fix clear document bug 2020-11-19 14:04:07 +01:00
mpostma
a8ab15d65d Revert "Merge #1001"
This reverts commit 690eab4a25, reversing
changes made to 086020e543.

update changelog
2020-11-02 15:10:09 +01:00
mpostma
190b78b7be Revert "Merge #1037"
This reverts commit 257f9fb2b2, reversing
changes made to 9bae7a35bf.
2020-10-27 17:27:47 +01:00
mpostma
d35a104ad3 requested changes 2020-10-27 11:53:24 +01:00
mpostma
33c7c5a7e3 remove del_synonyms function 2020-10-26 21:33:39 +01:00
mpostma
f9ab85adbe deunicase synonyms 2020-10-26 17:47:55 +01:00
many
50f0fbb05c
remove useless function after health route refacto #1026 2020-10-20 16:21:46 +02:00
mpostma
dc2e5ceed2 fix bug 2020-10-16 14:16:12 +02:00
many
1639a7338d add test to reproduce #891 bug report
fix bug
2020-10-16 13:35:11 +02:00
mpostma
ac7226bb27 fix deserializer 2020-10-16 13:02:44 +02:00
bors[bot]
f359b64d59
Merge #946
946: Sort displayedAttributes field r=MarinPostma a=gorogoroumaru

Fix #943

displayedAttributes use the HashSet struct which is an unsorted structure, so I changed the implementation from HashSet into BTreeSet.

Co-authored-by: gorogoroumaru <zokutyou2@gmail.com>
2020-10-13 14:37:47 +00:00
qdequele
704defea78 fix clippy 2020-10-13 10:01:57 +02:00
gorogoroumaru
f4d918d22a
Merge branch 'master' into issue943 2020-10-02 21:01:31 +09:00