64 commits

Author	SHA1	Message	Date
Clément Renault	0dbf1a16ff	Make clippy happy	2023-11-23 14:11:38 +01:00
Louis Dureuil	825257da76	Use more efficient method for deletion in benchmarks	2023-11-09 16:13:15 +01:00
Louis Dureuil	620fee35f9	Fix benches	2023-11-06 11:56:46 +01:00
Louis Dureuil	2263dff02b	Stop using removed delete pipelines almost everywhere	2023-10-30 11:41:22 +01:00
ManyTheFish	1ba8a40d61	Remove formating benchmark because they can't be isoloated easily anymore	2023-04-06 15:10:16 +02:00
ManyTheFish	37489fd495	Return an internal error in the case of matching word is invalid	2023-03-01 19:05:16 +01:00
Clément Renault	1d507c84b2	Fix the formatting	2023-01-17 18:25:55 +01:00
Clément Renault	1b78231e18	Make clippy happy	2023-01-17 18:25:54 +01:00
Loïc Lecrenier	02fd06ea0b	Integrate deserr	2023-01-11 13:56:47 +01:00
Loïc Lecrenier	80588daae5	Fix compilation error in formatting benches	2022-11-28 10:27:15 +01:00
Kerollmops	6603437cb1	Introduce an indexation abortion function when indexing documents	2022-10-17 17:28:03 +02:00
ManyTheFish	5391e3842c	replace optional_words by term_matching_strategy	2022-08-22 17:47:19 +02:00
ManyTheFish	f9029727e0	Fix benchmarks	2022-08-22 14:55:53 +02:00
Irevoire	4aae07d5f5	expose the size methods	2022-08-17 17:07:38 +02:00
Loïc Lecrenier	fb2b6c0c28	Use mimalloc for benchmarks on all platforms	2022-08-10 16:56:42 +02:00
Loïc Lecrenier	8f73251012	Use mimalloc for benchmarks on macOS	2022-08-10 13:30:56 +02:00
bors[bot]	941af58239	Merge #561 ... 561: Enriched documents batch reader r=curquiza a=Kerollmops ~This PR is based on #555 and must be rebased on main after it has been merged to ease the review.~ This PR contains the work in #555 and can be merged on main as soon as reviewed and approved. - [x] Create an `EnrichedDocumentsBatchReader` that contains the external documents id. - [x] Extract the primary key name and make it accessible in the `EnrichedDocumentsBatchReader`. - [x] Use the external id from the `EnrichedDocumentsBatchReader` in the `Transform::read_documents`. - [x] Remove the `update_primary_key` from the _transform.rs_ file. - [x] Really generate the auto-generated documents ids. - [x] Insert the (auto-generated) document ids in the document while processing it in `Transform::read_documents`. Co-authored-by: Kerollmops <clement@meilisearch.com>	2022-07-21 07:08:50 +00:00
Loïc Lecrenier	8270e2b768	Fix name of "release_date" facet in movies benchmarks	2022-07-18 10:34:12 +02:00
Kerollmops	448114cc1c	Fix the benchmarks with the new indexation API	2022-07-12 15:22:09 +02:00
Kerollmops	a892a4a79c	Introduce a function to extend from a JSON array of objects	2022-07-12 15:14:06 +02:00
Kerollmops	ea852200bb	Fix the format used for a geo deleting benchmark	2022-07-12 14:55:52 +02:00
Kerollmops	399eec5c01	Fix the indexation tests	2022-07-12 14:55:51 +02:00
Kerollmops	fcfc4caf8c	Move the Object type in the lib.rs file and use it everywhere	2022-07-12 14:55:51 +02:00
Kerollmops	a97d4d63b9	Fix the benchmarks	2022-07-12 14:55:50 +02:00
Loïc Lecrenier	aae03356cb	Use BufReader to read datasets in benchmarks	2022-07-06 18:20:15 +02:00
bors[bot]	eeba196053	Merge #572 ... 572: Add reindexing benchmarks r=Kerollmops a=irevoire With #557 coming, we should add benchmarks that measure our impact on the reindexing process. Co-authored-by: Tamo <tamo@meilisearch.com>	2022-07-05 14:43:01 +00:00
Loïc Lecrenier	9bc7627e27	Fix deserialisation of NDJson documents in benchmarks	2022-07-05 15:51:06 +02:00
Tamo	2700d8dc67	Add reindexing benchmarks	2022-07-05 14:46:46 +02:00
ManyTheFish	2652310f2a	Change delete benchmark names	2022-06-16 10:32:58 +02:00
ManyTheFish	adbb0ff318	Add deletion benchmarks	2022-06-16 10:17:58 +02:00
ManyTheFish	727d663f28	Update benchmarks	2022-06-02 18:07:10 +02:00
ManyTheFish	4dd7b20c32	Update benchmarks	2022-06-02 17:33:25 +02:00
Tamo	f820c9804d	add one nested benchmark	2022-05-02 19:35:57 +02:00
bors[bot]	ea4bb9402f	Merge #483 ... 483: Enhance matching words r=Kerollmops a=ManyTheFish # Summary Enhance milli word-matcher making it handle match computing and cropping. # Implementation ## Computing best matches for cropping Before we were considering that the first match of the attribute was the best one, this was accurate when only one word was searched but was missing the target when more than one word was searched. Now we are searching for the best matches interval to crop around, the chosen interval is the one: 1) that have the highest count of unique matches > for example, if we have a query `split the world`, then the interval `the split the split the` has 5 matches but only 2 unique matches (1 for `split` and 1 for `the`) where the interval `split of the world` has 3 matches and 3 unique matches. So the interval `split of the world` is considered better. 2) that have the minimum distance between matches > for example, if we have a query `split the world`, then the interval `split of the world` has a distance of 3 (2 between `split` and `the`, and 1 between `the` and `world`) where the interval `split the world` has a distance of 2. So the interval `split the world` is considered better. 3) that have the highest count of ordered matches > for example, if we have a query `split the world`, then the interval `the world split` has 2 ordered words where the interval `split the world` has 3. So the interval `split the world` is considered better. ## Cropping around the best matches interval Before we were cropping around the interval without checking the context. Now we are cropping around words in the same context as matching words. This means that we will keep words that are farther from the matching words but are in the same phrase, than words that are nearer but separated by a dot. > For instance, for the matching word `Split` the text: `Natalie risk her future. Split The World is a book written by Emily Henry. I never read it.` will be cropped like: `…. Split The World is a book written by Emily Henry. …` and not like: `Natalie risk her future. Split The World is a book …` Co-authored-by: ManyTheFish <many@meilisearch.com>	2022-04-19 11:42:32 +00:00
ManyTheFish	5809d3ae0d	Add first benchmarks on formatting	2022-04-12 16:31:58 +02:00
Irevoire	4f3ce6d9cd	nested fields	2022-04-07 16:58:46 +02:00
Clément Renault	acfc96525c	Apply GitHub suggestions	2022-02-23 16:20:29 +01:00
Clément Renault	a820aa11e6	Add a new movies benchmark to test multi batch indexing	2022-02-23 16:20:29 +01:00
Kerollmops	8d2e3e4aba	Add a new wiki benchmark to test multi batch indexing	2022-02-23 16:20:29 +01:00
Kerollmops	ab5247dc64	Add a new songs benchmark to test multi batch indexing	2022-02-23 16:20:28 +01:00
Marin Postma	0c84a40298	document batch support ... reusable transform rework update api add indexer config fix tests review changes Co-authored-by: Clément Renault <clement@meilisearch.com> fmt	2022-01-19 12:40:20 +01:00
Clément Renault	94011bb9a8	Fix the benchmarks to work with optional filters	2021-12-09 12:14:16 +01:00
Marin Postma	6eb47ab792	remove update_id in UpdateBuilder	2021-11-16 13:07:04 +01:00
Tamo	6831c23449	merge with main	2021-11-06 16:34:30 +01:00
Tamo	5c01e9bf7c	fix the benchmarks	2021-11-06 16:03:49 +01:00
marin postma	baddd80069	implement review suggestions	2021-10-25 18:29:12 +02:00
Tamo	ed7fd855af	fix the wiki indexing benchmark	2021-10-11 14:26:36 +02:00
many	d2427f18e5	Enhance CSV document parsing	2021-09-29 10:25:33 +02:00
Tamo	176160d32f	fix all benchmarks and add the compile time checking of the benhcmarks in the ci	2021-09-22 12:10:21 +02:00
Tamo	5e683ba472	add benchmarks for the geosearch	2021-09-20 10:44:37 +02:00