Commit Graph

176 Commits

Author SHA1 Message Date
Quentin de Quelen
dc6907e748
rebase from master 2020-02-11 15:28:00 +01:00
qdequele
a5b0e468ee
fix for review 2020-02-11 15:28:00 +01:00
qdequele
50a9825a0f
fix some uses cases on settings 2020-02-11 15:27:59 +01:00
Quentin de Quelen
585bba43a0
set new attributes indexed if needed 2020-02-11 15:27:58 +01:00
qdequele
9c0497c419
change the way settings are show in updates 2020-02-11 15:27:58 +01:00
qdequele
f77f38dfa0
fix update system 2020-02-11 15:27:57 +01:00
qdequele
58fe87067b
finish settings 2020-02-11 15:27:57 +01:00
Quentin de Quelen
dbba310770
squash me 2020-02-11 15:27:57 +01:00
Quentin de Quelen
6deb481589
definitely remove attributes_ranked on settings; auto create it with ranking_rules 2020-02-11 15:27:57 +01:00
Quentin de Quelen
036977bfe4
add the possibility to totally clear the schema 2020-02-11 15:27:57 +01:00
Quentin de Quelen
7a6f583b1f
fix issue on ranking rules 2020-02-11 15:27:56 +01:00
Quentin de Quelen
e078eafb1f
clean unused functions 2020-02-11 15:27:56 +01:00
Quentin de Quelen
6f534540a6
fix error on stop words fst 2020-02-11 15:27:56 +01:00
qdequele
38d57d213f
expose api for new settings 2020-02-11 15:27:56 +01:00
qdequele
ae0a11e422
fix schema & fix tests 2020-02-11 15:27:55 +01:00
qdequele
a35eb16a2a
store the schema after each document updates 2020-02-11 15:27:54 +01:00
qdequele
4f0ead625b
adapt meilisearch-http to the new schemaless option 2020-02-11 15:27:54 +01:00
qdequele
21d122a870
rewrite indexed_pos -> field_id for hightligths 2020-02-11 15:27:54 +01:00
qdequele
130fb74928
introduce a new schemaless way 2020-02-11 15:27:54 +01:00
qdequele
bbe1845f66
squash-me 2020-02-11 15:27:54 +01:00
qdequele
2ee90a891c
introduce a new settings update system 2020-02-11 15:27:54 +01:00
qdequele
110adcae85
Remove the schema; fix #422 2020-02-11 15:27:53 +01:00
qdequele
91c6539baf
Rewrite the stop-words endpoint; fix #417 2020-02-11 15:27:53 +01:00
qdequele
f0590d3301
Change documents routes; fix #416 2020-02-11 15:27:53 +01:00
Clément Renault
7c0d8f073b
Support compaction with multi database 2020-01-24 17:38:14 +01:00
Clément Renault
a2bc689b92
Fix the tests a little bit 2020-01-22 18:12:56 +01:00
Clément Renault
a9adbda2cd
Make the engine support non-exact multi-words synonyms 2020-01-22 18:11:58 +01:00
Clément Renault
0b9fe2c072
Introduce the new Query Tree creation supporting more operations 2020-01-22 17:46:46 +01:00
Clément Renault
789e05304c
Replace prints by debug logs 2020-01-21 11:05:34 +01:00
Clément Renault
7604387701
Clean up the dependencies 2020-01-21 11:04:25 +01:00
Clément Renault
daffcaf4c6
Make the docids OR operation method conditional 2020-01-19 12:29:06 +01:00
Clément Renault
ff1ec599e0
Try a better version of sdset 2020-01-19 12:01:24 +01:00
Clément Renault
e44d498c94
Display more debug info for prefix tolerant fetches 2020-01-19 11:07:32 +01:00
Clément Renault
c334d6b7fe
Avoid sorting sorted sequences, prefer using set operations 2020-01-19 10:58:01 +01:00
Clément Renault
5465e401bb
Catch query tree related errors 2020-01-17 10:41:27 +01:00
Clément Renault
9cc3c56c9c
Fix the prefix system 2020-01-16 18:41:27 +01:00
Clément Renault
d7a7560220
Use an union instead of a sort for prefix fetching 2020-01-16 17:09:27 +01:00
Clément Renault
70a529d197
Reduce the number of args of update functions 2020-01-16 16:29:50 +01:00
Clément Renault
be31a14326
Make the clear all operation clear caches 2020-01-16 16:19:04 +01:00
Clément Renault
96139da0d2
Reintroduce the distinct search system 2020-01-16 15:55:55 +01:00
Clément Renault
74fa9ee4df
Introduce a better higlighting system 2020-01-16 14:56:16 +01:00
Clément Renault
00336c5154
Reintroduce a basic highlight display 2020-01-16 14:24:45 +01:00
Clément Renault
3912d1ec4b
Improve query parsing and interpretation 2020-01-16 14:11:17 +01:00
Clément Renault
70d4f47f37
Differentiate short words as prefix or exact matches 2020-01-16 12:01:51 +01:00
Clément Renault
9809ded23d
Implement synonym fetching 2020-01-16 11:38:23 +01:00
Clément Renault
5f9a3546e0
Use an union instead of a sort for OR ops 2020-01-15 15:14:24 +01:00
Clément Renault
db625a08f7
Update lock file 2020-01-15 12:25:14 +01:00
Clément Renault
44fec1b6c9
Cache prefixes of a length of 2 2020-01-14 18:17:52 +01:00
Clément Renault
54dacb362d
Use different algorithms for different documents ratios 2020-01-14 17:51:08 +01:00
Clément Renault
6edb460bea
Try with an exponential search 2020-01-14 16:52:24 +01:00
Clément Renault
40dab80dfa
Change the way we filter the documents 2020-01-14 14:18:01 +01:00
Clément Renault
681711fced
Fix query ids to be usize 2020-01-14 13:12:42 +01:00
Clément Renault
21c1473e0c
Introduce the distance data 2020-01-14 11:38:04 +01:00
Clément Renault
8acbdcbbad
wip: Make the new query tree work with the criteria 2020-01-13 14:36:06 +01:00
Clément Renault
da8abebfa2
Introduce the query words mapping along with the query tree 2020-01-13 13:29:47 +01:00
Clément Renault
4f7a7ea0bb
Faster intersection group by 2020-01-09 16:30:03 +01:00
Clément Renault
d6c9ba8f08
Store the postings lists 2020-01-09 15:04:53 +01:00
Clément Renault
ec8916bf54
Change the debug outputs 2020-01-09 12:05:39 +01:00
Clément Renault
81c573ec92
Add the raw document IDs to the postings lists 2020-01-08 15:30:43 +01:00
Clément Renault
9420edadf4
Introduce the Postings type to decorrelate the DocumentIds 2020-01-08 14:48:23 +01:00
Clément Renault
d724a7659e
Introduce a query tree context struct 2020-01-08 13:37:22 +01:00
Clément Renault
887c212b49
Add more logs about the docids construction 2020-01-08 13:22:42 +01:00
Clément Renault
07937ed6d7
Use the prefix caches 2020-01-08 13:14:07 +01:00
Clément Renault
a262c67ec3
limit the search in the FST 2020-01-08 13:06:12 +01:00
Clément Renault
13ca30c4d8
WIP: Made the query tree traversing support prefix search 2020-01-08 12:02:58 +01:00
Clément Renault
fbcec2975d wip: Impl a basic tree traversing 2020-01-07 18:24:13 +01:00
Clément Renault
6e1f4af833 wip: Create a tree from query but need to show synonyms 2020-01-07 18:24:13 +01:00
Clément Renault
856c5c4214 Fix group offset computing 2019-12-31 14:24:10 +01:00
Clément Renault
670e80c151 Use the cached postings lists in the query system 2019-12-31 13:32:36 +01:00
Clément Renault
eed07c724f Add more logging for postings lists fetching by word 2019-12-31 13:32:36 +01:00
Clément Renault
99d35fb940 Introduce a first version of a number of candidates reducer
It works by ignoring the postings lists associated to documents that the previous words did not returned
2019-12-31 13:32:36 +01:00
Clément Renault
106b886873
Cache the prefix postings lists 2019-12-30 18:01:32 +01:00
Clément Renault
928876b553
Introduce the postings lists caching stores
Currently not used
2019-12-30 18:01:27 +01:00
Clément Renault
58836d89aa
Rename the PrefixCache into PrefixDocumentsCache 2019-12-30 15:42:09 +01:00
Clément Renault
1a5a104f13
Display proximity evaluation number of calls 2019-12-30 15:42:09 +01:00
Clément Renault
064cfa4755
Add more debug, where are those 100ms 2019-12-30 15:42:08 +01:00
Clément Renault
ed6172aa94
Add a time measurement of the criterion loop 2019-12-30 15:42:08 +01:00
Clément Renault
8c140f6bcd
Increase the disk usage limit 2019-12-30 15:42:08 +01:00
Clément Renault
1e1f0fcaf5
Introduce a basic cache system for first letters 2019-12-30 15:42:08 +01:00
Clément Renault
d21352a109
Change the time measurement of the FST 2019-12-30 15:42:08 +01:00
Clément Renault
4be11f961b
Use an ugly trick to avoid cloning the FST 2019-12-30 15:42:07 +01:00
Clément Renault
1163f390b3
Restrict FST search to the first letter of the word 2019-12-30 15:42:07 +01:00
Clément Renault
691e2a3c1d
Fix a blocking channel, appearing like a deadlock 2019-12-30 15:28:28 +01:00
Clément Renault
04bb49989f
Add more debug timings 2019-12-20 14:18:48 +01:00
Clément Renault
d12ff15ee3
Set the indexes info in the create_index function 2019-12-19 10:38:56 +01:00
Clément Renault
40c0b14d1c
Reintroduce searchable attributes and reordering 2019-12-13 14:38:25 +01:00
Clément Renault
a4dd033ccf
Rename raw_matches into bare_matches 2019-12-13 14:38:25 +01:00
Clément Renault
48e8778881
Clean up the modules declarations 2019-12-13 14:38:25 +01:00
Clément Renault
4be23efe66
Remove the AttrCount type
Could probably be reintroduced later
2019-12-13 14:38:25 +01:00
Clément Renault
7d67750865
Reintroduce exacteness for one word document field 2019-12-13 14:38:25 +01:00
Clément Renault
746e6e170c
Make the test pass again 2019-12-13 14:38:24 +01:00
Clément Renault
d93e35cace
Introduce ContextMut and Context structs 2019-12-13 14:38:24 +01:00
Clément Renault
d75339a271
Prefer summing the attribute 2019-12-13 14:38:24 +01:00
Clément Renault
86ee0cbd6e
Introduce bucket_sort_with_distinct function 2019-12-13 14:38:24 +01:00
Clément Renault
248ccfc0d8
Update the criteria to the new ones 2019-12-13 14:38:24 +01:00
Clément Renault
ea148575cf
Remove the raw_query functions 2019-12-13 14:38:23 +01:00
Clément Renault
efc2be0b7b
Bump the sdset dependency to 0.3.6 2019-12-13 14:38:23 +01:00
Clément Renault
8d71112dcb
Rewrite the phrase query postings lists
This simplified the multiword_rewrite_matches function a little bit.
2019-12-13 14:38:23 +01:00
Clément Renault
dd03a6256a
Debug pre filtered number of documents 2019-12-13 14:38:23 +01:00
Clément Renault
9c03bb3428
First probably working phrase query doc filtering 2019-12-13 14:38:23 +01:00