Commit Graph

179 Commits

Author SHA1 Message Date
Kerollmops
12358476da
Use the log crate instead of stderr 2020-07-12 10:55:09 +02:00
Kerollmops
2c62eeea3c
Rename the project milli 2020-07-12 00:16:41 +02:00
Kerollmops
f6eae91c7d
Pretty print the new dashboard numbers 2020-07-11 14:17:37 +02:00
Kerollmops
11c7fef80a
Implement a memory dumper
It moves the in memory HashMaps used when indexing to a disk based MTBL file
2020-07-07 16:48:49 +02:00
Kerollmops
7178b6c2c4
First basic version using MTBL again 2020-07-07 11:32:33 +02:00
Kerollmops
2a3b03138b
Use heed 0.8.1 with the RwIter append method 2020-07-05 19:50:28 +02:00
Kerollmops
46ced5c828
Introduce the RwIter append heed API 2020-07-04 12:34:10 +02:00
Kerollmops
2ae3f40971
Make the indexer ignore certain words
This is a preparation for making the indexing fully parallel by making the
indexer only be aware of certain words for each threads to avoid postings lists
conflicts for each words
2020-07-01 17:49:46 +02:00
Kerollmops
f98b615bf3
Replace the LRU by an Arc cache 2020-06-29 20:48:57 +02:00
Kerollmops
07abebfc46
Introduce a (too big) LRU cache 2020-06-29 18:15:03 +02:00
Kerollmops
5f0088594b
Index by writing directly into LMDB 2020-06-29 13:54:47 +02:00
Kerollmops
1628a31efa
Cache the unions of the derived words positions 2020-06-20 15:38:10 +02:00
Kerollmops
115e0142d9
Add a feature flags to enable the export of stats 2020-06-20 13:25:42 +02:00
Kerollmops
d6705d5529
Introduce the criterion dependency to bench the engine 2020-06-19 18:32:25 +02:00
Kerollmops
55a8941922
Optimize things 2020-06-19 17:48:17 +02:00
Kerollmops
a8cda248b4
Introduce a customized A* algorithm.
This custom algo lazily compute the intersections between words, to avoid too much set operations and database reads
2020-06-14 12:51:57 +02:00
Kerollmops
0a83a86e65
Fix multiple bugs 2020-06-11 11:55:03 +02:00
Kerollmops
13977d9338
squash-me 2020-06-09 23:06:59 +02:00
Kerollmops
dfdaceb410
Introduce a first basic working positions-based engine 2020-06-05 20:13:19 +02:00
Kerollmops
3a23dc242e
More efficiently merge MTBLs, more than two at a time 2020-06-04 16:17:24 +02:00
Kerollmops
dff68a339a
Use OnceCell to cache levenshtein builders 2020-05-31 19:27:11 +02:00
Kerollmops
a26553c90a
Reintroduce a simple HTTP server 2020-05-31 17:48:13 +02:00
Kerollmops
ba9527abc0
Support typos with a levenshtein automata 2020-05-31 17:01:11 +02:00
Kerollmops
6c726df9b9
Support multiple space seperated words 2020-05-31 16:09:34 +02:00
Kerollmops
24587148fd
Introduce MTBL parallel merging before LMDB writing 2020-05-31 14:22:57 +02:00
Kerollmops
3a998cf39c
Far better usage of rayon to fold indexed data 2020-05-31 14:22:57 +02:00
Kerollmops
1237306ca8
Introduce a thread that write to heed 2020-05-31 14:22:57 +02:00
Kerollmops
a81f201fad
Inroduce the use of RocksDB instead of sled (RAM) 2020-05-31 14:22:06 +02:00
Kerollmops
91ba938953
Initial commit 2020-05-31 14:22:06 +02:00