Commit Graph

140 Commits

Author SHA1 Message Date
Clément Renault
6534a9ec1d
Clean up many warning messages 2019-10-08 17:31:07 +02:00
Clément Renault
0a5ad4db06
Move the push update functions to their related modules 2019-10-08 17:24:11 +02:00
Clément Renault
6ee0d72c7b
Expose the synonyms operation updates on the Index 2019-10-08 17:18:22 +02:00
Clément Renault
ba32ce21d0
Introduce synonyms deletions updates 2019-10-08 17:16:48 +02:00
Clément Renault
0e224efa46
Introduce synonyms additions updates 2019-10-08 17:06:56 +02:00
Clément Renault
175461c13a
Port all tests to the TempDatabase struct 2019-10-08 16:16:30 +02:00
Clément Renault
c514692233
Introduce the TempDatabase in the QueryBuilder tests 2019-10-08 15:22:36 +02:00
Clément Renault
d8d0442d63
Fix many indexing and searching related bugs 2019-10-08 14:56:14 +02:00
Clément Renault
2236ebbd42
Introduce an example file to test indexing and searching csv documents 2019-10-08 14:48:48 +02:00
Clément Renault
0bfba3e4ba
Introduce a query_builder method on Index 2019-10-07 17:55:46 +02:00
Clément Renault
a57a64823e
Make possible to create an index and add a schema later on 2019-10-07 17:48:26 +02:00
Clément Renault
aa05459e4f
Introduce a background thread that manage updates to do 2019-10-07 16:16:04 +02:00
Clément Renault
0615c5c52d
Consume updates in the order of insertion 2019-10-07 15:00:28 +02:00
Clément Renault
487411340a
Prefix all the store names to avoid colliding with main stores 2019-10-07 10:56:55 +02:00
Clément Renault
5139dc7f3e
Let the caller commit/abort the operation 2019-10-07 10:52:45 +02:00
Clément Renault
88d0d3931c Store the schema in the main index 2019-10-04 17:49:13 +02:00
Clément Renault
df2ef8d2e1 Introduce update_task, popping an update and pushing the result of it 2019-10-04 17:49:13 +02:00
Clément Renault
29229b2137
Remove the update from the database when popped out 2019-10-04 17:16:34 +02:00
Clément Renault
851cc38216
Introduce the Database struct to manage indexes 2019-10-04 16:49:17 +02:00
Clément Renault
effbbc7370
Load the indexes at startup 2019-10-04 13:26:33 +02:00
Clément Renault
08e3f23408
Add the meilidb-schema/tokenizer projects 2019-10-04 10:29:44 +02:00
Clément Renault
62a0aefe44
Make the project be a workspace 2019-10-04 10:26:32 +02:00
qdequele
3fe346101b
chore: change logs in query_builder from info! to trace! 2019-09-24 13:35:46 +02:00
Clément Renault
69e0bae75e
chore: Bump dependencies 2019-09-18 14:42:23 +02:00
qdequele
59ae6458dc chore: bump dependencies 2019-09-17 18:50:44 +02:00
Clément Renault
c8ee21f227
feat: Replace the rayon::scope by always checking time 2019-09-01 18:52:38 +02:00
Clément Renault
9c5ec110e5
feat: Introduce a way to enable or disable query timeouts 2019-08-28 13:24:34 +02:00
Clément Renault
67302d09f3
feat: Multiword rewrite while there is time 2019-08-19 11:12:23 +02:00
Clément Renault
7dc9ea78fa
feat: Make the automaton DFA construction lazy 2019-08-19 11:12:23 +02:00
Clément Renault
0ee56314fb
feat: Try to simplify Store trait bound with a rayon scope 2019-08-19 11:10:54 +02:00
Clément Renault
b7b60b5fe5
feat: Introduce a new thread to avoid waiting on doc indexes fetchs 2019-08-16 16:35:19 +02:00
Clément Renault
d9c9fafd78
feat: Fetch doc indexes while there is time 2019-08-16 15:01:25 +02:00
Clément Renault
bb0a79c577
feat: Process automatons in the order they were sort 2019-08-16 12:25:35 +02:00
Clément Renault
81d44a0854
feat: Order automatons by importance 2019-08-16 12:19:34 +02:00
Clément Renault
bf3c2c3725 feat: Move the multi-word rewriting algorithm into its own function 2019-08-16 11:25:42 +02:00
Clément Renault
89df496f0c feat: Separate highlights from matches to make the code easier to follow 2019-08-16 11:25:42 +02:00
Clément Renault
9959f2e952 feat: Move the RawDocument type to its own module 2019-08-16 11:25:42 +02:00
Clément Renault
795557c046 feat: Remove query splitting from the automaton generation 2019-08-16 11:25:42 +02:00
Clément Renault
225a3bf184 test: Produce tests that work with the new cumulative word index system 2019-08-16 11:25:42 +02:00
Clément Renault
e65d7418b7 feat: Remove the query index from the Automaton type 2019-08-16 11:25:42 +02:00
Clément Renault
f478bbf826 feat: Introduce the QueryEnhancer in the query synonym system 2019-08-16 11:25:42 +02:00
Clément Renault
5e691c2140 feat: Introduce the QueryEnhancer type 2019-08-16 11:25:42 +02:00
Clément Renault
9175e4686b
feat: Collect TmpMatches only on tests, producing data useful for tests 2019-07-01 14:55:47 +02:00
Clément Renault
e8afca614c
chore: Little clean ups of meilidb-core 2019-07-01 14:34:06 +02:00
Clément Renault
6b6db2f8e6
feat: Introduce the Highlight type to simplify the data oriented design 2019-07-01 12:06:16 +02:00
Clément Renault
b7ed22bc59
feat: Introduce on the fly attributes reordering with meilidb-core 2019-07-01 12:03:31 +02:00
Clément Renault
f5d52396f5
feat: Support query words splits 2019-06-28 18:04:35 +02:00
Clément Renault
9cc154da05
chore: Rewrite tests to use iterators and be easily testable 2019-06-28 18:04:35 +02:00
Clément Renault
5aa49d232c
feat: Rewrite Automaton generation related code 2019-06-28 18:04:35 +02:00
Clément Renault
9f320590d3
feat: Support query words concatenation 2019-06-27 10:14:17 +02:00
Clément Renault
0a5d4eb7ed
feat: Normalize synonym strings and query strings to search for synonyms 2019-06-26 10:45:51 +02:00
Clément Renault
3dcbc737f3
feat: Make synonyms be not considered like exact matches 2019-06-26 10:45:51 +02:00
Clément Renault
43f11e929d
fix: Do not trigger a synonym when its not the last word and is a prefix 2019-06-26 10:45:51 +02:00
Clément Renault
8f2a551cca
feat: Trigger synonym replacement only when the last word is tipped 2019-06-26 10:45:50 +02:00
Clément Renault
8f044c6853
fix: Only create non-prefix DFA when generating synonyms alternatives 2019-06-26 10:45:50 +02:00
Clément Renault
0633f16b4d
feat: Make multi-word support multi-word synonyms 2019-06-26 10:45:50 +02:00
Clément Renault
59fafb8b30
feat: Support one word has multi-word alternatives 2019-06-26 10:45:50 +02:00
Clément Renault
d2bd99cc2a
fix: Append DocIndexes when building InMemorySetStore from an Iterator 2019-06-26 10:45:50 +02:00
Clément Renault
62930ecc4e
feat: Deduplicate automatons when synonyms produce duplicated ones 2019-06-26 10:45:49 +02:00
Clément Renault
6cb57aa8a4
feat: Unique word has multi-word synonyms basically work 2019-06-26 10:45:49 +02:00
Clément Renault
9861c3878e
tests: Add more tests about synonyms 2019-06-26 10:45:49 +02:00
Clément Renault
707d7b062b
feat: Made query handle synonyms via the Store 2019-06-26 10:45:49 +02:00
Clément Renault
18736bdcd0
feat: Introduce the synonyms concept to the Store trait 2019-06-26 10:45:49 +02:00
Clément Renault
e8b2e86007
feat: Introduce a basic way to handle synonyms 2019-06-26 10:45:48 +02:00
Clément Renault
102fb506db
chore: Do a little clippy pass 2019-05-22 11:00:58 +02:00
Clément Renault
fa099555c0
feat: Use safest SetBuf constructor instead of new_unchecked 2019-05-21 18:15:48 +02:00
Clément Renault
94f9587db1
feat: Implement Debug on RawDocument for more convenience 2019-05-20 11:21:41 +02:00
Clément Renault
6df8f62022
test: Add more test to some criteria 2019-05-20 11:21:40 +02:00
Clément Renault
8c71473498
feat: Introduce the Criterion::name to allow better debugging 2019-05-20 11:21:40 +02:00
Clément Renault
1f2abce7c3
feat: Introduce the DocumentsDeletion type 2019-05-15 15:42:11 +02:00
Clément Renault
e67ada8823
feat: Introduce the DocumentsAddition type 2019-05-15 15:42:11 +02:00
Clément Renault
42e39f6eb5
feat: Introduce a simplified version of the Store trait 2019-05-15 15:42:11 +02:00
Clément Renault
6eb25687f8
feat: Handle word doc-indexes sled tree errors 2019-05-15 15:42:10 +02:00
Clément Renault
737db5668b
chore: Remove the WriteToBytes trait 2019-05-15 15:42:10 +02:00
Clément Renault
f16e0333e4
chore: Remove the SharedData/Cursor types 2019-05-15 15:42:09 +02:00
Clément Renault
27ffcaabe9
chore: Remove the DocIndexes type 2019-05-15 15:42:09 +02:00
Clément Renault
db031a5b95
chore: Remove the DocIds type 2019-05-15 15:42:09 +02:00
Clément Renault
3dc057ca9c
feat: Introduce the new Index system 2019-05-15 15:42:08 +02:00
Clément Renault
645bab7748
feat: Index documents using the Serializer struct 2019-04-29 14:32:03 +02:00
Clément Renault
187e6740bd
feat: Allow users to construct query builders from database indexes 2019-04-29 14:32:02 +02:00
Clément Renault
4b40d5b0d4
feat: Introduce the Index struct 2019-04-29 14:32:02 +02:00
Clément Renault
f7eced03fd
chore: Using a fork of the fst library that support Arc<[u8]> 2019-04-29 14:32:01 +02:00
Clément Renault
95dfbd1fe0
feat: Introduce the meilidb-data schema module 2019-04-29 14:32:00 +02:00
Clément Renault
acede0f3e8
fix: Correctly assert the DocIndex memory size 2019-04-29 14:31:59 +02:00
Clément Renault
5d5bcf7011
feat: Remove the FilterFunc alias type 2019-04-29 14:31:37 +02:00
Clément Renault
19e67dcf0b
feat: Move query splitting into the tokenizer workspace 2019-04-29 14:31:37 +02:00
Clément Renault
1897da5348
feat: Move tokenizer things into the meilidb-tokenizer workspace 2019-04-29 14:31:37 +02:00
Clément Renault
bc227bef21
chore: Add a nightly feature to meilidb-core 2019-04-29 14:31:36 +02:00
Clément Renault
d0786b4156
chore: Move the SortByAttr into meilidb 2019-04-29 14:31:36 +02:00
Clément Renault
14790eeae3
chore: Move index related things to the meilidb-core workspace member 2019-04-29 14:31:35 +02:00