208 Commits

Author	SHA1	Message	Date
Many	36715f571c	Update milli/src/search/criteria/proximity.rs ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2021-06-10 11:30:33 +02:00
many	e923a3ed6a	Replace Consecutive by Phrase in query tree ... Replace Consecutive by Phrase in query tree in order to remove theorical bugs, due of the Consecutive enum type.	2021-06-10 11:16:16 +02:00
many	26a9974667	Make asc/desc criterion return resting documents ... Fix #161.2	2021-06-02 17:41:48 +02:00
many	e857ca4d7d	Fix PR comments	2021-06-01 18:06:46 +02:00
many	c701f8bf36	Use field id word count database in exactness criterion	2021-05-31 16:27:28 +02:00
bors[bot]	2f5e61bacb	Merge #184 ... 184: Transfer numbers and strings facets into the appropriate facet databases r=Kerollmops a=Kerollmops This pull request is related to https://github.com/meilisearch/milli/issues/152 and changes the layout of the facets values, numbers and strings are now in dedicated databases and the user no more needs to define the type of the fields. No more conversion between the two types is done, numbers (floats and integers converted to f64) go to the facet float database and strings go to the strings facet database. There is one related issue that I found regarding CSVs, the values in a CSV are always considered to be strings, [meilisearch/specifications#28](d916b57d74/text/0028-indexing-csv.md) fixes this issue by allowing the user to define the fields types using `:` in the "CSV Formatting Rules" section. All previous tests on facets have been modified to pass again and I have also done hand-driven tests with the 115m songs dataset. Everything seems to be good! Fixes #192. Co-authored-by: Clément Renault <clement@meilisearch.com> Co-authored-by: Kerollmops <clement@meilisearch.com>	2021-05-31 13:32:58 +00:00
many	a5e98cf46d	Fix plane sweep algorithm	2021-05-25 18:21:55 +02:00
Clément Renault	3a4a150ef0	Fix the tests and remaining warnings	2021-05-25 11:31:06 +02:00
Clément Renault	02c655ff1a	Refine the facet distribution to use both databases	2021-05-25 11:30:00 +02:00
Clément Renault	bd7b285bae	Split the update side to use the number and the strings facet databases	2021-05-25 11:30:00 +02:00
Clément Renault	597144b0b9	Use both number and string facet databases in the distinct system	2021-05-25 11:29:59 +02:00
many	a3944a7083	Introduce a filtered_candidates field	2021-05-11 11:37:40 +02:00
many	efba662ca6	Fix clippy warnings in cirteria	2021-05-10 10:27:18 +02:00
many	e923d51b8f	Make bucket candidates optionals	2021-05-10 10:27:04 +02:00
Many	44b6843de7	Fix pull request reviews ... Update milli/src/fields_ids_map.rs Update milli/src/search/criteria/exactness.rs Update milli/src/search/criteria/mod.rs	2021-05-06 14:31:03 +02:00
many	c1ce4e4ca9	Introduce mocked ExactAttribute step in exactness criterion	2021-05-06 14:28:31 +02:00
many	a3f8686fbf	Introduce exactness criterion	2021-05-06 14:28:30 +02:00
many	ee09e50e7f	Remove excluded document in criteria iterations ... - pass excluded document to criteria to remove them in higher levels of the bucket-sort - merge already returned document with excluded documents to avoid duplicas Related to #125 and #112 Fix #170	2021-04-29 12:09:38 +02:00
many	31607bf9cd	Add a threshold on proximity when choosing between linear/set algorithm	2021-04-28 14:57:22 +02:00
many	3b7e6afb55	Make some refacto and add documentation	2021-04-28 13:53:27 +02:00
Many	0add4d735c	Update milli/src/search/criteria/attribute.rs ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2021-04-27 17:40:34 +02:00
Many	3794ffc952	Update milli/src/search/criteria/attribute.rs ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2021-04-27 17:39:23 +02:00
Many	329bd4a1bb	Update milli/src/search/criteria/attribute.rs ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2021-04-27 17:39:03 +02:00
Many	3b1358b62f	Update milli/src/search/criteria/attribute.rs ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2021-04-27 17:32:19 +02:00
Many	c862b1bc6b	Update milli/src/search/criteria/attribute.rs ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2021-04-27 17:32:10 +02:00
Many	e92d137676	Update milli/src/search/criteria/attribute.rs ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2021-04-27 17:31:42 +02:00
Many	b3d6c6a9a0	Update milli/src/search/criteria/attribute.rs ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2021-04-27 17:31:13 +02:00
Many	498c2b298c	Update milli/src/search/criteria/attribute.rs ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2021-04-27 17:30:02 +02:00
Many	0e4e6dfada	Update milli/src/search/criteria/proximity.rs ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2021-04-27 17:29:52 +02:00
Many	47d780b8ce	Update milli/src/search/criteria/mod.rs ... Co-authored-by: Irevoire <tamo@meilisearch.com>	2021-04-27 14:39:53 +02:00
Many	0daa0e170a	Fix PR comments ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2021-04-27 14:39:53 +02:00
many	71740805a7	Fix forgotten typo tests	2021-04-27 14:39:53 +02:00
many	e77291a6f3	Optimize Atrribute criterion on big requests	2021-04-27 14:39:53 +02:00
many	716c8e22b0	Add style and comments	2021-04-27 14:39:52 +02:00
many	f853790016	Use the LCM of 10 first numbers to compute attribute rank	2021-04-27 14:39:52 +02:00
many	2b036449be	Fix the return of equal candidates in different pages	2021-04-27 14:39:52 +02:00
many	0efa011e09	Make a small code clean-up	2021-04-27 14:39:52 +02:00
many	17c8c6f945	Make set algorithm return None when nothing can be returned	2021-04-27 14:39:52 +02:00
many	b3e2280bb9	Debug attribute criterion ... * debug folding when initializing iterators	2021-04-27 14:39:52 +02:00
many	1eee0029a8	Make attribute criterion typo/prefix tolerant	2021-04-27 14:39:52 +02:00
many	59f58c15f7	Implement attribute criterion ... * Implement WordLevelIterator * Implement QueryLevelIterator * Implement set algorithm based on iterators Not tested + Some TODO to fix	2021-04-27 14:39:52 +02:00
Clément Renault	361193099f	Reduce the amount of branches when query tree flattened	2021-04-27 14:39:52 +02:00
many	ab92c814c3	Fix attributes score	2021-04-27 14:35:43 +02:00
Clément Renault	658f316511	Introduce the Initial Criterion	2021-04-27 14:35:43 +02:00
many	75e7b1e3da	Implement test Context methods	2021-04-27 14:25:34 +02:00
many	4ff67ec2ee	Implement attribute criterion for small amounts of candidates	2021-04-27 14:25:34 +02:00
Kerollmops	0f4c0beffd	Introduce the Attribute criterion	2021-04-27 14:25:34 +02:00
tamo	f8dee1b402	[makes clippy happy] search/criteria/proximity.rs	2021-04-21 12:36:45 +02:00
Kerollmops	51767725b2	Simplify integer and float functions trait bounds	2021-04-20 10:23:31 +02:00
Marin Postma	75464a1baa	review fixes	2021-04-15 16:25:56 +02:00
Marin Postma	45c45e11dd	implement distinct attribute ... distinct can return error facet distinct on numbers return distinct error review fixes make get_facet_value more generic fixes	2021-04-15 16:25:55 +02:00
Alexey Shekhirin	1e3f05db8f	use fixed number of candidates as a threshold	2021-03-30 11:57:10 +03:00
Alexey Shekhirin	a776ec9718	fix division	2021-03-29 19:16:58 +03:00
Alexey Shekhirin	522e79f2e0	feat(search, criteria): introduce a percentage threshold to the asc/desc	2021-03-29 19:08:31 +03:00
Kerollmops	54b97ed8e1	Update the fetcher comments	2021-03-10 10:56:26 +01:00
Kerollmops	d301859bbd	Introduce a special word_derivations function for Proximity	2021-03-10 10:42:53 +01:00
Kerollmops	facfb4b615	Fix the bucket candidates	2021-03-10 10:42:53 +01:00
Kerollmops	42fd7dea78	Remove the useless typo cache	2021-03-10 10:42:53 +01:00
many	62a70c300d	Optimize words criterion	2021-03-10 10:42:53 +01:00
Kerollmops	d781a6164a	Rewrite some code with idiomatic Rust	2021-03-08 16:27:52 +01:00
Clément Renault	b18ec00a7a	Add a logging_timer macro to te criterion next methods	2021-03-08 16:12:06 +01:00
Kerollmops	82a0f678fb	Introduce a cache on the docid_word_positions database method	2021-03-08 16:12:03 +01:00
Clément Renault	5fcaedb880	Introduce a WordDerivationsCache struct	2021-03-08 16:00:53 +01:00
many	2606c92ef9	use plain sweep in proximity criterion	2021-03-08 15:58:39 +01:00
many	ae47bb3594	Introduce plane_sweep function in proximity criterion	2021-03-08 15:58:38 +01:00
Clément Renault	3c76b3548d	Rework the Asc/Desc criteria to be facet iterator based	2021-03-08 13:32:25 +01:00
Clément Renault	a58d2b6137	Print the Asc/Desc criterion field name in the debug prints	2021-03-08 13:32:25 +01:00
Kerollmops	9b6b35d9b7	Clean up some comments	2021-03-03 18:19:10 +01:00
Kerollmops	2cc4a467a6	Change the criterion output that cannot fail	2021-03-03 18:18:33 +01:00
Kerollmops	1fc25148da	Remove useless where clauses for the criteria	2021-03-03 18:09:19 +01:00
Kerollmops	5c5e51095c	Fix the Asc/Desc criteria to alsways return the QueryTree when available	2021-03-03 15:45:03 +01:00
many	cdaa96df63	optimize proximity criterion	2021-03-03 15:45:03 +01:00
Kerollmops	f118d7e067	build criteria from settings	2021-03-03 15:45:03 +01:00
Kerollmops	025835c5b2	Fix the criteria to avoid always returning a placeholder	2021-03-03 15:45:03 +01:00
Kerollmops	36c1f93ceb	Do an union of the bucket candidates	2021-03-03 15:45:03 +01:00
many	b0e0c5eba0	remove option of bucket_candidates	2021-03-03 15:45:03 +01:00
Kerollmops	daf126a638	Introduce the final Fetcher criterion	2021-03-03 15:45:03 +01:00
many	7ac09d7b7c	remove option of bucket_candidates	2021-03-03 15:45:03 +01:00
Kerollmops	5af63c74e0	Speed-up the MatchingWords highlighting struct	2021-03-03 15:45:03 +01:00
Kerollmops	4510bbccca	Add a lot of debug	2021-03-03 15:43:44 +01:00
Kerollmops	ae4a237e58	Fix the maximum_proximity function	2021-03-03 15:43:44 +01:00
Kerollmops	9bc9b36645	Introduce the Proximity criterion	2021-03-03 15:43:44 +01:00
many	3d731cc861	remove option on bucket_candidates	2021-03-03 15:43:44 +01:00
Clément Renault	14f9f85c4b	Introduce the AscDesc criterion	2021-03-03 15:43:44 +01:00
many	b5b7ec0162	implement initial state for words criterion	2021-03-03 15:43:44 +01:00
Kerollmops	3415812b06	Imrpove the intersection speed in the words criterion	2021-03-03 15:43:43 +01:00
Clément Renault	ef381e17bb	Compute the candidates for each sub query tree	2021-03-03 15:43:43 +01:00
Kerollmops	e174ccbd8e	Use the words criterion in the search module	2021-03-03 15:43:43 +01:00
Clément Renault	1e47f9b3ff	Introduce the Words criterion	2021-03-03 15:43:43 +01:00
many	2d068bd45b	implement Context trait for criteria	2021-03-03 15:43:43 +01:00
many	d92ad5640a	remove option on bucket_candidates	2021-03-03 15:43:43 +01:00
many	fb7e6df790	add tests on typo criterion	2021-03-03 15:43:43 +01:00
Kerollmops	c5a32fd4fa	Fix the typo criterion	2021-03-03 15:43:42 +01:00
many	a273c46559	clean warnings	2021-03-03 15:43:42 +01:00
many	9e093d5ff3	add cache on alterate_query_tree function	2021-03-03 15:43:42 +01:00
many	41fc51ebcf	optimize alterate_query_tree when number_typos is zero	2021-03-03 15:43:42 +01:00
many	4da6e1ea9c	add cache in typo criterion	2021-03-03 15:43:42 +01:00
Kerollmops	67c71130df	Reduce the number of calls to alterate_query_tree	2021-03-03 15:43:42 +01:00
many	9ccaea2afc	simplify criterion context	2021-03-03 15:43:42 +01:00
Clément Renault	229130ed25	Correctly compute the bucket candidates for the Typo criterion	2021-03-03 15:43:42 +01:00
Clément Renault	5344abc008	Introduce the CriterionResult return type	2021-03-03 15:43:41 +01:00
many	86bcecf840	change variable's name from distance to proximity	2021-03-03 15:43:41 +01:00
many	4128bdc859	reduce match possibilities in docids fetchers	2021-03-03 15:43:41 +01:00
many	907482c8ac	clean docids fetchers	2021-03-03 15:43:41 +01:00
many	774a255f2e	use prefix cache in criteria	2021-03-03 15:43:41 +01:00
many	98e69e63d2	implement Context trait for criteria	2021-03-03 15:43:41 +01:00
Clément Renault	ad20d72a39	Introduce the Typo criterion	2021-03-03 15:43:41 +01:00
Clément Renault	f0ddea821c	Introduce the Typo criterion	2021-03-03 15:43:41 +01:00