199 Commits

Author	SHA1	Message	Date
Clément Renault	4dff8a215e	Reduce the ram consumption when re-indexing all the documents	2019-10-29 17:46:23 +01:00
Clément Renault	e9dce3ce81	Add a test to ensure that the indexer support stop words	2019-10-29 16:18:06 +01:00
Clément Renault	ff7dde7522	Make the RawIndexer support stop words	2019-10-29 16:18:06 +01:00
Clément Renault	a226fd23c3	Introduce the stop words deletion update type	2019-10-29 16:18:06 +01:00
Clément Renault	776673ebae	Introduce the stop words addition update type	2019-10-29 15:24:09 +01:00
Clément Renault	8a17fcdda5	Introduce a function to get all updates results	2019-10-29 11:37:40 +01:00
Clément Renault	ac12a4b9c9	Make documents additions accept only the last duplicate document	2019-10-28 20:40:33 +01:00
Clément Renault	a43b37dfc1	Send channel notification when clearing documents	2019-10-28 17:58:22 +01:00
Clément Renault	c08dcac1d4	Abort the update transaction before calling the update callback	2019-10-28 17:55:43 +01:00
Clément Renault	9a57cab3ee	Fix the exactness criterion algorithm	2019-10-26 18:34:40 +02:00
Clément Renault	4111b99a6d	Only highlight query words areas not the whole words	2019-10-26 15:56:34 +02:00
Clément Renault	cb5c77e536	Make sure that automatons group with more automatons are better	2019-10-24 15:18:53 +02:00
Clément Renault	047eba3ff3	Introduce a clear all documents update	2019-10-23 16:39:10 +02:00
Clément Renault	eb67195840	Commit an update only when it is Ok	2019-10-23 15:52:40 +02:00
Clément Renault	7d9cf8d713	Clean up the fetch algorithm	2019-10-23 12:06:21 +02:00
Clément Renault	03eb7898e7	Introduce a basic working version of phrase query for splitting words	2019-10-23 11:40:13 +02:00
Clément Renault	858bf359b8	Make possible to convert recursive object into strings	2019-10-22 16:02:02 +02:00
Clément Renault	0f30a221fa	Introduce the reindex_all_documents indexing function	2019-10-22 14:07:27 +02:00
Clément Renault	78535b3e33	Move to heed 0.1.0	2019-10-21 12:05:53 +02:00
Clément Renault	9dce41ed6b	Cargo clippy pass	2019-10-18 13:30:06 +02:00
Clément Renault	ca26a0f2e4	Cargo fmt pass	2019-10-18 13:30:06 +02:00
Clément Renault	2ef51f7df9	Introduce a function to copy and compact a database env	2019-10-18 12:56:56 +02:00
Clément Renault	526202ec8b	Make the levenshtein algorithm consider transpositions to cost 1	2019-10-17 18:07:15 +02:00
Clément Renault	dd74af4c70	Use an RoTxn to retrieve the number of documents	2019-10-17 17:30:54 +02:00
Clément Renault	d941c512db	Rework the QueryBuilder to make it easier to construct and use	2019-10-17 14:45:21 +02:00
Clément Renault	2ea3e9b081	Improve the automaton producer quality by changing the production order	2019-10-17 13:19:08 +02:00
Clément Renault	da71821204	Make the example take the fetch-timeout-ms argument into account	2019-10-17 13:19:08 +02:00
Clément Renault	1cf6afad9a	Fix updates serialization to use serde_json instead of bincode	2019-10-17 12:31:46 +02:00
Clément Renault	1667e1b32f	Move to zerocopy-lmdb	2019-10-16 17:12:55 +02:00
Clément Renault	5e8d432614	Introduce an Iterator to visit all documents attributes counts	2019-10-15 15:27:18 +02:00
Clément Renault	3278d22279	Introduce an Index mathod to retrieve the currently processed update	2019-10-15 14:54:52 +02:00
Clément Renault	0de37819b4	Simplify the document fields counts deletion	2019-10-15 11:17:23 +02:00
Clément Renault	9ff92c5d15	Update the exact criterion to use the documents fields counts	2019-10-14 18:48:54 +02:00
Clément Renault	e629f51af4	Use the documents_fileds_count store in the QueryBuilder	2019-10-14 18:48:32 +02:00
Clément Renault	b377003192	Compute and store the number of words in documents fields	2019-10-14 14:07:10 +02:00
Clément Renault	a7e40a78c1	Introduce the DocumentsFieldsCounts store	2019-10-14 14:06:34 +02:00
Clément Renault	9cdda8c46a	Make the RawIndexer index_text method return the number of words	2019-10-14 13:56:52 +02:00
Clément Renault	710ab2386c	Get ride of rust messagepack (rmp)	2019-10-11 16:17:37 +02:00
Clément Renault	02575a2ef6	Introduce customs updates	2019-10-11 15:33:35 +02:00
Clément Renault	da6ab2753e	Rename Update/Type SchemaUpdate into Schema	2019-10-11 13:49:17 +02:00
Clément Renault	12b80e08be	Do not commit updates, let the user do	2019-10-11 11:29:47 +02:00
Clément Renault	9dca18f966	Fix the DocumentsIdsIter and do not iter on an Option	2019-10-10 18:32:22 +02:00
Clément Renault	9eb27811b1	Remove the useless generic documents_deletion parameter	2019-10-10 16:16:53 +02:00
Clément Renault	485480560a	Add method to create a query builder along with criterion	2019-10-10 15:32:08 +02:00
Clément Renault	e09d3b654d	Rename main_store into common_store	2019-10-10 15:22:23 +02:00
Clément Renault	19c22a8c5e	Create two open and create index functions	2019-10-10 13:48:30 +02:00
Clément Renault	7b26bd88c0	Improve the DocumentsIdsIter internal	2019-10-10 13:40:18 +02:00
Clément Renault	d1e59be46b	Introduce the DocumentsIds iterator	2019-10-10 10:35:57 +02:00
Clément Renault	46c19dfc5a	Expose the UpdateType	2019-10-10 10:24:41 +02:00
Clément Renault	d8fdad1455	Make the timeout QueryBuilder setting optional to and pass the tests	2019-10-09 17:59:31 +02:00
Clément Renault	f56636e1e9	Merge branch 'moving-to-lmdb'	2019-10-09 17:23:48 +02:00
Clément Renault	8f63ec39da	Unrestrict static lifetime of Criterion names	2019-10-09 16:15:31 +02:00
Clément Renault	5a1c1aeb02	Reintroduce the sort-by-attr criterion	2019-10-09 16:08:30 +02:00
Clément Renault	6ec575f8de	Use a buffered sync channel to avoid blocking the update system	2019-10-09 15:49:35 +02:00
Clément Renault	683b6afbfb	Introduce a way to filter documents with a basic syntax	2019-10-09 14:20:37 +02:00
Clément Renault	663714bb6d	Make the example return documents field in a consistent order	2019-10-09 13:48:33 +02:00
Clément Renault	bb35ca0d40	Reintroduce the distinct and filtering of documents	2019-10-09 13:44:18 +02:00
Clément Renault	5f3072e67e	Support a basic update callback system	2019-10-09 11:45:19 +02:00
Clément Renault	2a4707d51e	Expose a function to be able to now the status of an update	2019-10-08 17:35:47 +02:00
Clément Renault	6534a9ec1d	Clean up many warning messages	2019-10-08 17:31:07 +02:00
Clément Renault	0a5ad4db06	Move the push update functions to their related modules	2019-10-08 17:24:11 +02:00
Clément Renault	6ee0d72c7b	Expose the synonyms operation updates on the Index	2019-10-08 17:18:22 +02:00
Clément Renault	ba32ce21d0	Introduce synonyms deletions updates	2019-10-08 17:16:48 +02:00
Clément Renault	0e224efa46	Introduce synonyms additions updates	2019-10-08 17:06:56 +02:00
Clément Renault	175461c13a	Port all tests to the TempDatabase struct	2019-10-08 16:16:30 +02:00
Clément Renault	c514692233	Introduce the TempDatabase in the QueryBuilder tests	2019-10-08 15:22:36 +02:00
Clément Renault	d8d0442d63	Fix many indexing and searching related bugs	2019-10-08 14:56:14 +02:00
Clément Renault	2236ebbd42	Introduce an example file to test indexing and searching csv documents	2019-10-08 14:48:48 +02:00
Clément Renault	0bfba3e4ba	Introduce a query_builder method on Index	2019-10-07 17:55:46 +02:00
Clément Renault	a57a64823e	Make possible to create an index and add a schema later on	2019-10-07 17:48:26 +02:00
Clément Renault	aa05459e4f	Introduce a background thread that manage updates to do	2019-10-07 16:16:04 +02:00
Clément Renault	0615c5c52d	Consume updates in the order of insertion	2019-10-07 15:00:28 +02:00
Clément Renault	487411340a	Prefix all the store names to avoid colliding with main stores	2019-10-07 10:56:55 +02:00
Clément Renault	5139dc7f3e	Let the caller commit/abort the operation	2019-10-07 10:52:45 +02:00
Clément Renault	88d0d3931c	Store the schema in the main index	2019-10-04 17:49:13 +02:00
Clément Renault	df2ef8d2e1	Introduce update_task, popping an update and pushing the result of it	2019-10-04 17:49:13 +02:00
Clément Renault	29229b2137	Remove the update from the database when popped out	2019-10-04 17:16:34 +02:00
Clément Renault	851cc38216	Introduce the Database struct to manage indexes	2019-10-04 16:49:17 +02:00
Clément Renault	effbbc7370	Load the indexes at startup	2019-10-04 13:26:33 +02:00
Clément Renault	08e3f23408	Add the meilidb-schema/tokenizer projects	2019-10-04 10:29:44 +02:00
Clément Renault	62a0aefe44	Make the project be a workspace	2019-10-04 10:26:32 +02:00
qdequele	3fe346101b	chore: change logs in query_builder from info! to trace!	2019-09-24 13:35:46 +02:00
Clément Renault	69e0bae75e	chore: Bump dependencies	2019-09-18 14:42:23 +02:00
qdequele	59ae6458dc	chore: bump dependencies	2019-09-17 18:50:44 +02:00
Clément Renault	c8ee21f227	feat: Replace the rayon::scope by always checking time	2019-09-01 18:52:38 +02:00
Clément Renault	9c5ec110e5	feat: Introduce a way to enable or disable query timeouts	2019-08-28 13:24:34 +02:00
Clément Renault	67302d09f3	feat: Multiword rewrite while there is time	2019-08-19 11:12:23 +02:00
Clément Renault	7dc9ea78fa	feat: Make the automaton DFA construction lazy	2019-08-19 11:12:23 +02:00
Clément Renault	0ee56314fb	feat: Try to simplify Store trait bound with a rayon scope	2019-08-19 11:10:54 +02:00
Clément Renault	b7b60b5fe5	feat: Introduce a new thread to avoid waiting on doc indexes fetchs	2019-08-16 16:35:19 +02:00
Clément Renault	d9c9fafd78	feat: Fetch doc indexes while there is time	2019-08-16 15:01:25 +02:00
Clément Renault	bb0a79c577	feat: Process automatons in the order they were sort	2019-08-16 12:25:35 +02:00
Clément Renault	81d44a0854	feat: Order automatons by importance	2019-08-16 12:19:34 +02:00
Clément Renault	bf3c2c3725	feat: Move the multi-word rewriting algorithm into its own function	2019-08-16 11:25:42 +02:00
Clément Renault	89df496f0c	feat: Separate highlights from matches to make the code easier to follow	2019-08-16 11:25:42 +02:00
Clément Renault	9959f2e952	feat: Move the RawDocument type to its own module	2019-08-16 11:25:42 +02:00
Clément Renault	795557c046	feat: Remove query splitting from the automaton generation	2019-08-16 11:25:42 +02:00
Clément Renault	225a3bf184	test: Produce tests that work with the new cumulative word index system	2019-08-16 11:25:42 +02:00
Clément Renault	e65d7418b7	feat: Remove the query index from the Automaton type	2019-08-16 11:25:42 +02:00
Clément Renault	f478bbf826	feat: Introduce the QueryEnhancer in the query synonym system	2019-08-16 11:25:42 +02:00