ManyTheFish
5e83bac448
Fix PR comments
2024-02-26 15:40:15 +01:00
ManyTheFish
a493a50825
Fix clippy
2024-02-22 14:53:33 +01:00
ManyTheFish
9d1f489a37
Fix facet incremental indexing
2024-02-21 18:42:16 +01:00
Louis Dureuil
7877788510
fix logs
2024-02-12 09:12:54 +01:00
ManyTheFish
be1b054b05
Compute chunk size based on the input data size ant the number of indexing threads
2024-02-08 17:28:37 +01:00
Clément Renault
9f9ad4cc05
Fix Clippy warnings
2024-01-16 15:27:24 +01:00
Clément Renault
d32eb11329
Move to the v0.20.0-alpha.9 of heed
2023-11-27 11:52:22 +01:00
Clément Renault
0d4482625a
Make the changes to use heed v0.20-alpha.6
2023-11-23 11:43:58 +01:00
Louis Dureuil
59f88c14b3
Simplify facet update after removing Index::faceted_documents_ids
2023-10-30 11:39:29 +01:00
Louis Dureuil
14832cb324
Remove Index::faceted_documents_ids
2023-10-30 11:37:32 +01:00
Louis Dureuil
04ec293024
Facet Incremental update
2023-10-30 11:37:30 +01:00
Tamo
d772073dfa
use a bufreader everytime there is a grenad<file>
2023-10-10 15:00:30 +02:00
Clément Renault
cfd1b2cc97
Fix the clippy warnings
2023-04-25 16:40:32 +02:00
Kerollmops
fbec48f56e
Merge remote-tracking branch 'milli/main' into bring-v1-changes
2023-02-06 16:48:10 +01:00
Louis Dureuil
20f05efb3c
clippy: needless_lifetimes
2023-01-31 11:12:59 +01:00
Louis Dureuil
3296cf7ae6
clippy: remove needless lifetimes
2023-01-31 09:32:40 +01:00
Clément Renault
1b78231e18
Make clippy happy
2023-01-17 18:25:54 +01:00
Loïc Lecrenier
f2cf981641
Add more tests and allow disabling of soft-deletion outside of tests
...
Also allow disabling soft-deletion in the IndexDocumentsConfig
2022-12-05 10:51:01 +01:00
Loïc Lecrenier
9dd4b33a9a
Fix bulk facet indexing bug
2022-11-30 14:27:36 +01:00
Loïc Lecrenier
0caadedd3b
Make clippy happy
2022-11-17 12:17:53 +01:00
Loïc Lecrenier
d95d02cb8a
Fix Facet Indexing bugs
...
1. Handle keys with variable length correctly
This fixes https://github.com/meilisearch/meilisearch/issues/3042 and
is easily reproducible with the updated fuzz tests, which now generate
keys with variable lengths.
2. Prevent adding facets to the database if their encoded value does
not satisfy `valid_lmdb_key`.
This fixes an indexing failure when a document had a filterable
attribute containing a value whose length is higher than ~500 bytes.
2022-11-17 11:29:42 +01:00
unvalley
abf1cf9cd5
Fix clippy errors
2022-11-04 09:27:46 +09:00
unvalley
d53a80b408
Fix clippy error
2022-10-28 23:41:35 +09:00
unvalley
f4ec1abb9b
Fix all clippy error after conflicts
2022-10-27 23:58:13 +09:00
Loïc Lecrenier
14ca8048a8
Add some documentation on how to run the facet db fuzzer
2022-10-26 13:48:01 +02:00
Loïc Lecrenier
2295e0e3ce
Use real delete function in facet indexing fuzz tests
...
By deleting multiple docids at once instead of one-by-one
2022-10-26 13:47:46 +02:00
Loïc Lecrenier
a034a1e628
Move StrRefCodec and ByteSliceRefCodec to their own files
2022-10-26 13:47:46 +02:00
Loïc Lecrenier
1165ba2171
Make facet deletion incremental
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
86d9f50b9c
Fix bugs in incremental facet indexing with variable parameters
...
e.g. add one facet value incrementally with a group_size = X and then
add another one with group_size = Y
It is not actually possible to do so with the public API of milli,
but I wanted to make sure the algorithm worked well in those cases
anyway.
The bugs were found by fuzzing the code with fuzzcheck, which I've added
to milli as a conditional dev-dependency. But it can be removed later.
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
985a94adfc
cargo fmt
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
27454e9828
Document and refine facet indexing algorithms
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
b2f01ad204
Refactor facet database tests
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
9026867d17
Give same interface to bulk and incremental facet indexing types
...
+ cargo fmt, oops, sorry for the bad history :(
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
330c9eb1b2
Rename facet codecs and refine FacetsUpdate API
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
485a72306d
Refactor facet-related codecs
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
079ed4a992
Add more snapshots
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
a7201ece04
cargo fmt
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
36296bbb20
Add facet incremental indexing snapshot tests + fix bug
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
07ff92c663
Add more snapshots from facet tests
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
61252248fb
Fix some facet indexing bugs
2022-10-26 13:47:04 +02:00
Loïc Lecrenier
d30c89e345
Fix compile error+warnings in new tests
2022-10-26 13:46:46 +02:00
Loïc Lecrenier
e8a156d682
Reorganise facets database indexing code
2022-10-26 13:46:46 +02:00
