212 Commits

Author SHA1 Message Date
Clément Renault
f4b04dfb72
feat: Prefer doing DatabaseView updates atomically 2018-12-29 20:52:00 +01:00
Clément Renault
8412c14b5b
feat: Schema can be toml de/serialized 2018-12-28 19:24:50 +01:00
Clément Renault
b27f632e14
feat: Make the Attribute and WordArea errors recoverable 2018-12-28 16:15:22 +01:00
Clément Renault
fa238f21ef
feat: Move Database to its own module 2018-12-27 11:21:47 +01:00
Clément Renault
444a4c1af7
feat: Make the schema consider document ids 2018-12-27 11:21:47 +01:00
Clément Renault
b32c96cdc9
feat: Introduce a WordArea struct
Useful to highlight matching areas in the original text.
2018-12-24 15:58:46 +01:00
Clément Renault
4ebae7784c
feat: Create a strong DocumentId type
Forcing it to be something internal will permit to avoid possible miss comparisons to be done with other types.
2018-12-24 12:42:24 +01:00
Clément Renault
4a310c31ab
feat: Allow query distinct a range of documents 2018-12-18 14:00:30 +01:00
Clément Renault
d0ee5f12a0
feat: Introduce the BufferedDistinctMap struct 2018-12-18 14:00:30 +01:00
Clément Renault
7aafba67ed
feat: Allow querying a range of documents 2018-12-18 14:00:29 +01:00
Clément Renault
89c4af0cc2
chore: Clean up dependencies and avoid custom ones 2018-12-17 16:02:20 +01:00
Clément Renault
cdce016f7c
feat: Improve the Schema performances 2018-12-15 21:17:55 +01:00
Clément Renault
e15d5e6d93
feat: Improve the query distinct performances 2018-12-13 11:54:47 +01:00
Clément Renault
d3a67476e0
feat: Rename registers methods of DistinctMap 2018-12-13 11:54:09 +01:00
Clément Renault
5a2d8a2395
feat: Reexport the internal rocksdb 2018-12-13 11:52:34 +01:00
Clément Renault
df2b6a3e74
feat: Consider the last query word be a prefix
if the last word is not followed by a space.
2018-12-11 17:43:27 +01:00
Clément Renault
b5df87a403
feat: Introduce the Criteria type 2018-12-10 20:17:57 +01:00
Clément Renault
d615f89c56
feat: Introduce the SortBy help structure
This structure help ranking documents using stored attributes.
2018-12-10 20:17:52 +01:00
Clément Renault
23cce69dc5
feat: Introduce the CriteriaBuilder 2018-12-10 20:15:02 +01:00
Clément Renault
671bd3374f
fix: Break the proper loop when bucket sorting 2018-12-10 15:14:15 +01:00
Clément Renault
d8cfac914a
feat: Implement a better automaton builder
This new implementation only allows the last word of a query string to be a prefix.
2018-12-10 12:16:24 +01:00
Clément Renault
916b46c839
feat: Allow dumping the database, useful for full snapshot 2018-12-10 11:09:09 +01:00
Clément Renault
6cb1bfd815
feat: Use sdset Set primitives everywhere possible 2018-12-09 14:18:23 +01:00
Clément Renault
06ba82cfa8
feat: Database holds a DatabaseView and sync update ingestions 2018-12-09 13:42:56 +01:00
Clément Renault
0e825e05bb
feat: Make the DatabaseView become Sync + Send 2018-12-07 17:59:03 +01:00
Clément Renault
bec463a61a
fix: Update the DatabaseView to retrieve the index at creation
remove this computation from the QueryBuilder
2018-12-07 16:29:39 +01:00
Clément Renault
7c98771068
fix: Change the way the iterator upper bound is computed 2018-12-07 15:35:59 +01:00
Clément Renault
4626c77eac
fix: Change the way the BTreeMap store entries 2018-12-07 15:12:15 +01:00
Clément Renault
9342290afe
feat: Simplify the steps to query the database 2018-12-07 14:41:06 +01:00
Clément Renault
8bee31078d
chore: Update the module hierarchy 2018-12-07 12:22:51 +01:00
Clément Renault
2c3d71dd8f
fix: Improve the bucket sort algorithm 2018-12-07 11:53:17 +01:00
Clément Renault
731ed11153
feat: Index and store/serialize attributes while creating the update 2018-12-07 11:32:27 +01:00
Clément Renault
442834c28f
chore: Remove the MapSerializer temporarily 2018-12-06 17:23:01 +01:00
Clément Renault
b2cec98805
feat: Implemented a basic deserialiazation 2018-12-06 17:22:54 +01:00
Clément Renault
6c7a6c5219
feat: Implement DoubleEnded/ExactSizeIterator on DocumentIter 2018-12-06 17:22:25 +01:00
Clément Renault
6f8dbbde9a
feat: Implement the create/open/ingest_file Database methods 2018-12-06 17:22:20 +01:00
Clément Renault
42b0cf68eb
feat: Move the database Deserializer in its own module 2018-12-06 17:22:19 +01:00
Clément Renault
a5b80c72ae
feat: Move the DatabaseView in its own module 2018-12-06 17:22:19 +01:00
Clément Renault
8563c25d62
feat: Move DocumentKey/Attr to their own module 2018-12-06 17:22:19 +01:00
Clément Renault
3d0b334942
feat: Introduce the DocumentKey and DocumentKeyAttr structs 2018-12-06 17:22:07 +01:00
Clément Renault
700d625c93
feat: Introduce the Database and DatabaseView 2018-12-06 17:20:14 +01:00
Clément Renault
e559f7130c
feat: Replace the fnv hashmap by the hashbrown one 2018-12-02 13:11:02 +01:00
Clément Renault
51cc69342f
feat: Compacting "data-index" to avoid calling the merge operator 2018-12-02 13:06:07 +01:00
Clément Renault
c8aee7ed7a
fix: Make the merge operator work 2018-12-02 12:00:49 +01:00
Clément Renault
9a67db0989
test: Add some tests to DocIndexes 2018-12-02 12:00:49 +01:00
Clément Renault
5829d08bc0
chore: Avoid using the external library Itertools 2018-12-02 12:00:49 +01:00
Clément Renault
2719f1ad3b
feat: Introduce the PositiveUpdate 2018-12-02 12:00:49 +01:00
Clément Renault
8b2a8da8fa
feat: Remove the PositiveBlobBuilder 2018-12-02 12:00:49 +01:00
Clément Renault
ec5d17e8c2
feat: Implement the PositiveUpdate 2018-12-02 12:00:48 +01:00
Clément Renault
af791db23d
feat: Use the blob::OpBuilder to merge "data-index" blobs 2018-12-02 12:00:48 +01:00
Clément Renault
612a8d9d44
feat: Make the OpBuilder work only for PositiveBlob 2018-12-02 12:00:48 +01:00
Clément Renault
9b58ffe2d9
feat: Introduce the QueryBuilder struct 2018-12-02 12:00:37 +01:00
Clément Renault
b636e5fe57
feat: Introduce the "data-index" entry with merge compaction 2018-12-02 12:00:30 +01:00
Clément Renault
0e856db4e6
feat: Implement De/Serialize on Blob 2018-12-02 12:00:30 +01:00
Clément Renault
a43a772e9a
feat: Introduce typed keys constructors 2018-12-02 12:00:29 +01:00
Clément Renault
66dac923bf
feat: Introduce a working key-value based database 2018-12-02 12:00:29 +01:00
Clément Renault
86f23d2695
feat: Save the schema in the key-value store 2018-12-02 12:00:29 +01:00
Clément Renault
8df068af3c
feat: Introduce the Schema 2018-12-02 12:00:29 +01:00
Clément Renault
7c1a17520d
feat: Introduce the index module 2018-12-02 12:00:29 +01:00
Clément Renault
b3249d515d
feat: Introduce an Index system based on RocksDB 2018-12-02 12:00:29 +01:00
Clément Renault
cc52d5dda5
feat: Working on ops for Positive and Negative blobs 2018-12-02 11:59:48 +01:00
Clément Renault
34b43d4002
test: Add some more tests 2018-12-02 11:59:48 +01:00
Clément Renault
e68e6056c3 doc: Add a deep dive in Pentium 2018-10-21 18:21:04 +02:00
Clément Renault
26dcfe1e54 fix: Remove stop-words from the serve examples 2018-10-21 16:42:19 +02:00
Clément Renault
3d9488bd30 feat: Keep a stable order of documents
by ordering the deepest bucket by document id.
2018-10-18 17:29:27 +02:00
Clément Renault
86cb01b243 fix: Allow documents to bypass attribute distinction
In situations where the attribute is not present.
2018-10-18 15:08:04 +02:00
Clément Renault
98899d3ea0 fix: Change the tokenizer to accept quotes 2018-10-17 17:00:49 +02:00
Clément Renault
af91bfa11f fix: Clamp the document range requested 2018-10-17 16:59:39 +02:00
Clément Renault
37c709c9a9 feat: Introduce a way to distinct documents 2018-10-17 13:35:34 +02:00
Clément Renault
3acac1458f feat: Allow querying ranges in the results list 2018-10-11 16:09:28 +02:00
Clément Renault
8cd07462aa feat: Introduce the Criterion trait 2018-10-11 14:04:41 +02:00
Clément Renault
c56c35b45b feat: Introduce the Criteria struct 2018-10-10 16:57:21 +02:00
Clément Renault
7a668dde98 chore: Make the repo use examples and keep the library 2018-10-09 18:23:35 +02:00
Clément Renault
2944368897 doc: Update the README for the new CLI 2018-10-04 13:55:29 +02:00
Clément Renault
474028145d feat: Make a CLI that handle compile-time features 2018-10-03 16:21:33 +02:00
Clément Renault
6fa164dc56 chore: Make the project a workspace 2018-08-25 17:00:05 +02:00
Clément Renault
b9a4be10c6 feat: Implement the excat match ranking rule 2018-08-25 17:00:05 +02:00
Clément Renault
e4b1bb5d26 chore: Clean up ranking functions internal names 2018-08-25 17:00:05 +02:00
Kerollmops
23134fee02 feat: Fix the ranking algorithm to sort only the needed documents 2018-08-25 17:00:05 +02:00
Kerollmops
0ce08c8790 feat: Make querying always return limited results 2018-08-25 17:00:05 +02:00
Kerollmops
8fab80048c test: Add a raptor-search bench 2018-08-25 17:00:05 +02:00
Kerollmops
d210e5d8db feat: Introduce basic ranking rules 2018-08-25 17:00:05 +02:00
Clément Renault
c037fb85a8 Revert "feat: Implement a custom capped BTreeMap"
This reverts commit 1bcebeed83b7364fadd8530ad43788de9eb49d4b.
2018-08-25 17:00:05 +02:00
Clément Renault
3a194bfcc7 feat: Define a DocIndex struct 2018-08-25 17:00:05 +02:00
Kerollmops
a2958250c5 feat: Implement a custom capped BTreeMap 2018-08-25 17:00:05 +02:00
Kerollmops
bb08a9842c feat: Use BTreeMap instead of a custom algo 2018-08-25 17:00:05 +02:00
Kerollmops
22c26a5dd0 feat: Make the parsing more generic over json 2018-08-25 17:00:05 +02:00
Kerollmops
1e14df00d1 clean: Remove statics and use Arcs 2018-08-25 17:00:05 +02:00
Kerollmops
0df4f6f182 chore: Move raptor-http into it's own crate 2018-08-25 17:00:05 +02:00
Kerollmops
9c6663ee36 chore: Move raptor-bin into it's own crate 2018-08-25 17:00:05 +02:00
Kerollmops
70b079d79d feat: Add useful derivable traits to IndexedValues types 2018-08-25 17:00:05 +02:00
Kerollmops
33801f1d23 chore: Move Streams to map file 2018-08-25 17:00:05 +02:00
Kerollmops
593758c6d2 chore: Remove useless Fst prefixes 2018-08-25 17:00:05 +02:00
Kerollmops
5c76cd61f5 clean: Remove useless read_to_vec function 2018-08-25 17:00:05 +02:00
Kerollmops
1476aa3dba feat(search): Accept multiple words and do a simple union 2018-08-25 17:00:05 +02:00
Kerollmops
758baeb8e1 feat: Display not utf-8 error results 2018-08-25 17:00:05 +02:00
Kerollmops
0f9ada1c4e fix: Return a byte slice instead of a possibly wrong utf-8 str 2018-08-25 17:00:05 +02:00
Kerollmops
6d57a8af05 feat(search): Add a StreamWithState Streamer 2018-08-25 17:00:05 +02:00
Kerollmops
7fba62fc22 feat: Remove lazy_static's sync overhead 2018-08-25 17:00:05 +02:00
Kerollmops
2ffb140963 feat: Counter the lazyness of static variable loading 2018-08-25 17:00:05 +02:00
Kerollmops
2cb26e327c feat(search): Use an external levenstein automaton implementation 2018-08-25 17:00:05 +02:00
Kerollmops
80c05a9b50 feat(http): Avoid duplicating the Response body Vec 2018-08-25 17:00:05 +02:00
Clément Renault
a20405f786 dump: Make the data less prone of memory indirections 2018-05-05 10:50:36 +02:00
Clément Renault
55bda1cd71 search: levenshtein2 for word bigger than 8 chars 2018-04-22 23:09:38 +02:00
Kerollmops
b729c76bce
http: Make the map static 2018-05-12 15:11:31 +02:00
Kerollmops
c1513bf139
map: Update the map creation and loading 2018-05-12 15:11:30 +02:00
Kerollmops
96d2fbcd3d
map: Allow using the Levenshtein algorithm to search 2018-05-12 15:11:30 +02:00
Kerollmops
0581b296bb
http: Change the query param key name 2018-05-12 15:11:29 +02:00
Kerollmops
f851227ed4
map: Increase the SmallVec size 2018-05-12 15:11:29 +02:00
Kerollmops
c371fb56b7
map: Filter words a little bit 2018-05-12 15:11:28 +02:00
Kerollmops
2fa7178ed1
cli: Make work to index json lines 2018-05-12 15:11:28 +02:00
Kerollmops
820f38bd8a
chore: Initial commit 2018-05-12 15:11:27 +02:00