112 Commits

Author	SHA1	Message	Date
bors[bot]	5cbe879325	Merge #308 ... 308: Implement a better parallel indexer r=Kerollmops a=ManyTheFish Rewrite the indexer: - enhance memory consumption control - optimize parallelism using rayon and crossbeam channel - factorize the different parts and make new DB implementation easier - optimize and fix prefix databases Co-authored-by: many <maxime@meilisearch.com>	2021-09-02 15:03:52 +00:00
Clémentine Urquizar	285849e3a6	Update version for the next release (v0.12.0)	2021-09-02 10:08:41 +02:00
many	5c962c03dd	Fix and optimize word_prefix_pair_proximity_docids database	2021-09-01 16:48:40 +02:00
Kerollmops	2f20257070	Update milli to the v0.11.0	2021-08-24 18:10:11 +02:00
Clémentine Urquizar	88f6c18665	Update version for the next release (v0.10.2)	2021-08-23 11:33:30 +02:00
Clémentine Urquizar	6cb9c3b81f	Update milli version to v0.10.1	2021-08-18 13:46:27 +02:00
Clémentine Urquizar	fcc520e49a	Update version for the next release (v0.10.0)	2021-08-16 12:00:28 +02:00
Clémentine Urquizar	7f26c75610	Update milli to v0.9.0	2021-08-04 16:04:55 +02:00
Kerollmops	341c244965	Bump milli to v0.8.1	2021-07-29 15:56:36 +02:00
Clémentine Urquizar	6a141694da	Update version for the next release (v0.8.0)	2021-07-27 16:38:42 +02:00
Clément Renault	0227254a65	Return the original string values for the inverted facet index database	2021-07-21 16:59:39 +02:00
Kerollmops	0a78107525	Fix the infos crate to make it read u16 field ids	2021-07-06 11:58:03 +02:00
Kerollmops	91c5d0c042	Use the AlwaysFreePages flag when opening an index	2021-07-05 16:36:13 +02:00
Kerollmops	a6b4069172	Bump to v0.7.2	2021-07-05 10:54:53 +02:00
Clémentine Urquizar	b489515f4d	Update milli version to v0.7.1	2021-06-30 13:52:46 +02:00
Clément Renault	80c6aaf1fd	Bump milli to 0.7.0	2021-06-28 18:31:56 +02:00
Clément Renault	bdc5599b73	Bump heed to use the git repo with v0.12.0	2021-06-28 18:26:20 +02:00
Kerollmops	98285b4b18	Bump milli to 0.6.0	2021-06-23 17:30:26 +02:00
Clémentine Urquizar	9885fb4159	Update version for the next release (v0.5.1)	2021-06-23 14:05:20 +02:00
Tamo	d53df8a002	enable the jemallocator dependencies only when we are running on linux	2021-06-22 14:04:16 +02:00
Clémentine Urquizar	320670f8fe	Update version for the next release (v0.5.0)	2021-06-21 15:59:17 +02:00
Clémentine Urquizar	35fcc351a0	Update version for the next release (v0.4.2)	2021-06-20 17:37:24 +02:00
Kerollmops	ccd6f13793	Update version to the next release (0.4.1)	2021-06-17 15:01:20 +02:00
Tamo	9716fb3b36	format the whole project	2021-06-16 18:33:33 +02:00
Clémentine Urquizar	f5ff3e8e19	Update version for the next release (v0.4.0)	2021-06-16 14:01:05 +02:00
Kerollmops	28c004aa2c	Prefer using constant for the database names	2021-06-15 11:13:04 +02:00
Clémentine Urquizar	dc64e139b9	Update version for the next release (v0.3.1)	2021-06-09 14:39:21 +02:00
Clémentine Urquizar	3b2b3aeea9	Update Cargo.toml for next release v0.3.0	2021-06-03 12:24:27 +02:00
many	e857ca4d7d	Fix PR comments	2021-06-01 18:06:46 +02:00
many	b8e6db0feb	Add database in infos crate	2021-05-31 16:29:27 +02:00
Kerollmops	28bd9e183e	Fix the infos crate to support split facet databases	2021-05-25 11:31:06 +02:00
Clémentine Urquizar	1e11578ef0	Update version for the next release (v0.2.1)	2021-05-05 14:57:34 +02:00
Clémentine Urquizar	a8680887d8	Upgrade Milli version (v0.2.0)	2021-05-03 14:50:47 +02:00
Many	0daa0e170a	Fix PR comments ... Co-authored-by: Clément Renault <clement@meilisearch.com>	2021-04-27 14:39:53 +02:00
many	0d7d3ce802	Update roaring package	2021-04-27 14:39:53 +02:00
Kerollmops	7ff4a2a708	Display the number of entries in the infos crate	2021-04-27 14:39:52 +02:00
Kerollmops	1aad66bdaa	Compute stats about the word prefix level positions database in the infos crate	2021-04-27 14:39:52 +02:00
Kerollmops	e65bad16cc	Compute the words prefixes at the end of an update	2021-04-27 14:39:52 +02:00
Kerollmops	89ee2cf576	Introduce the TreeLevel struct	2021-04-27 14:25:35 +02:00
Kerollmops	8bd4f5d93e	Compute the biggest values of the words_level_positions_docids	2021-04-27 14:25:34 +02:00
Kerollmops	3069bf4f4a	Fix and improve the words-level-positions computation	2021-04-27 14:25:34 +02:00
Kerollmops	6b1b42b928	Introduce an infos wordsLevelPositionsDocids subcommand	2021-04-27 14:25:34 +02:00
Kerollmops	b0a417f342	Introduce the word_level_position_docids Index database	2021-04-27 14:25:34 +02:00
Kerollmops	51767725b2	Simplify integer and float functions trait bounds	2021-04-20 10:23:31 +02:00
Clémentine Urquizar	127d3d028e	Update version for the next release (v0.1.1)	2021-04-19 14:48:13 +02:00
Clément Renault	18844d60b5	Simplify the output of database sizes in the infos crate	2021-03-08 18:47:33 +01:00
Clément Renault	3d02b19fbd	Introduce the docids-words-positions subcommand to the infos crate	2021-03-08 18:47:33 +01:00
Kerollmops	bd63da0a0e	Add missing databases to the infos subcommand	2021-03-08 18:47:33 +01:00
Kerollmops	07784c8990	Tune the words prefixes threshold to compute for 1/1000 instead	2021-03-03 15:51:28 +01:00
Clément Renault	1eb7ce5cdb	Improve the export-documents infos command by accepting internal ids	2021-03-01 19:48:01 +01:00
Clément Renault	4884b324e6	Remove the useless external ids patch method in the infos crate	2021-03-01 19:48:01 +01:00
Clément Renault	78bede1ffb	Fix error displaying of the workspace members	2021-03-01 19:48:01 +01:00
Clément Renault	45330a5e47	Avoid creating a default empty database in the infos crate	2021-03-01 19:48:00 +01:00
Kerollmops	519b1cb5c9	Update dependencies	2021-02-21 10:26:04 +01:00
Kerollmops	aa4d9882d2	Introduce the new words-prefixes-docids infos subcomand	2021-02-17 11:22:27 +01:00
Kerollmops	49aee6d02c	Fix the database-stats infos subcommand	2021-02-17 11:22:27 +01:00
Kerollmops	7a0f86a04f	Introduce an infos command to extract the words prefixes fst	2021-02-17 11:22:27 +01:00
Kerollmops	8788485924	Take the prefix databases into account in the infos subcommand	2021-02-17 11:22:26 +01:00
Kerollmops	9b03b0a1b2	Introduce the word prefix pair proximity docids database	2021-02-17 11:12:38 +01:00
Clément Renault	5e7b26791b	Take the words-prefixes into account while computing the biggest values	2021-02-17 10:45:17 +01:00
Clément Renault	b3a21d5a50	Introduce the getters and setters for the words prefixes FST	2021-02-17 10:45:17 +01:00
Clément Renault	fecf3d6fc1	Move the command lines helpers into different crates	2021-02-14 18:55:15 +01:00