Loïc Lecrenier
fc9f3f31e7
Change DocumentsBatchReader to access cursor and index at same time
...
Otherwise it is not possible to iterate over all documents while
using the fields index at the same time.
2022-07-18 16:08:14 +02:00
Loïc Lecrenier
ab1571cdec
Simplify Transform::read_documents, enabled by enriched documents reader
2022-07-18 12:45:47 +02:00
Many the fish
e261ef64d7
Update milli/src/search/matches/mod.rs
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-07-18 10:18:51 +02:00
Many the fish
1da4ab5918
Update milli/src/search/matches/mod.rs
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-07-18 10:18:03 +02:00
Kerollmops
448114cc1c
Fix the benchmarks with the new indexation API
2022-07-12 15:22:09 +02:00
Kerollmops
25e768f31c
Fix another issue with the nested primary key selector
2022-07-12 15:14:07 +02:00
Kerollmops
192793ee38
Add some tests to check for the nested documents ids
2022-07-12 15:14:07 +02:00
Kerollmops
a892a4a79c
Introduce a function to extend from a JSON array of objects
2022-07-12 15:14:06 +02:00
Kerollmops
dc61105554
Fix the nested document id fetching function
2022-07-12 15:14:06 +02:00
Kerollmops
2eec290424
Check the validity of the latitute and longitude numbers
2022-07-12 15:14:06 +02:00
Kerollmops
5d149d631f
Remove tests for a function that no more exists
2022-07-12 15:14:06 +02:00
Kerollmops
0bbcc7b180
Expose the DocumentId struct to be sure to inject the generated ids
2022-07-12 15:14:06 +02:00
Kerollmops
d1a4da9812
Generate a real UUIDv4 when ids are auto-generated
2022-07-12 15:14:06 +02:00
Kerollmops
c8ebf0de47
Rename the validate function as an enriching function
2022-07-12 15:14:06 +02:00
Kerollmops
905af2a2e9
Use the primary key and external id in the transform
2022-07-12 15:14:05 +02:00
Kerollmops
742543091e
Constify the default primary key name
2022-07-12 14:55:52 +02:00
Kerollmops
5f1bfb73ee
Extract the primary key name and make it accessible
2022-07-12 14:55:52 +02:00
Kerollmops
6a0a0ae94f
Make the Transform read from an EnrichedDocumentsBatchReader
2022-07-12 14:55:52 +02:00
Kerollmops
dc3f092d07
Do not leak an internal grenad Error
2022-07-12 14:55:52 +02:00
Kerollmops
8ebf5eed0d
Make the nested primary key work
2022-07-12 14:55:52 +02:00
Kerollmops
19eb3b4708
Make sur that we do not accept floats as documents ids
2022-07-12 14:55:52 +02:00
Kerollmops
2ceeb51c37
Support the auto-generated ids when validating documents
2022-07-12 14:55:51 +02:00
Kerollmops
399eec5c01
Fix the indexation tests
2022-07-12 14:55:51 +02:00
Kerollmops
fcfc4caf8c
Move the Object type in the lib.rs file and use it everywhere
2022-07-12 14:55:51 +02:00
Kerollmops
0146175fe6
Introduce the validate_documents_batch function
2022-07-12 14:55:51 +02:00
Kerollmops
cefffde9af
Improve the .gitignore of the fuzz crate
2022-07-12 14:55:51 +02:00
Kerollmops
bdc4263883
Introduce the validate_documents_batch function
2022-07-12 14:55:51 +02:00
Kerollmops
6d0498df24
Fix the fuzz tests
2022-07-12 14:52:56 +02:00
Kerollmops
e8297ad27e
Fix the tests for the new DocumentsBatchBuilder/Reader
2022-07-12 14:52:56 +02:00
Kerollmops
419ce3966c
Rework the DocumentsBatchBuilder/Reader to use grenad
2022-07-12 14:52:55 +02:00
Kerollmops
eb63af1f10
Update grenad to 0.4.2
2022-07-12 14:52:55 +02:00
Kerollmops
048e174efb
Do not allocate when parsing CSV headers
2022-07-12 14:52:55 +02:00
ManyTheFish
5d79617a56
Chores: Enhance smart-crop code comments
2022-07-07 16:28:09 +02:00
bors[bot]
ebddfdb9a3
Merge #578
...
578: Bump uuid to 1.1.2 r=ManyTheFish a=Kerollmops
Just to [align the version with Meilisearch](https://github.com/meilisearch/meilisearch/pull/2584 ).
Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-07-05 14:56:08 +00:00
Kerollmops
1bfdcfc84f
Bump uuid to 1.1.2
2022-07-05 16:23:36 +02:00
Tamo
250be9fe6c
put the threshold back to 10k
2022-07-05 15:57:44 +02:00
Tamo
b61efd09fc
Makes the internal soft deleted error a UserError
2022-07-05 15:34:45 +02:00
Tamo
eaf28b0628
Apply review suggestions
...
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-07-05 15:30:33 +02:00
Tamo
3b309f654a
Fasten the document deletion
...
When a document deletion occurs, instead of deleting the document we mark it as deleted
in the new “soft deleted” bitmap. It is then removed from the search, and all the other
endpoints.
2022-07-05 15:30:33 +02:00
Tamo
446439e8be
bump charabia
2022-07-05 12:19:30 +02:00
Dmytro Gordon
3ff03a3f5f
Fix not equal filter when field contains both number and strings
2022-06-27 15:55:17 +03:00
Kerollmops
cc48992e79
Bump the milli version to 0.31.1
2022-06-22 17:05:51 +02:00
Kerollmops
238692a8e7
Introduce the copy_to_path method on the Index
2022-06-22 16:49:47 +02:00
bors[bot]
290a40b7a5
Merge #564
...
564: Rename the limitedTo parameter into maxTotalHits r=curquiza a=Kerollmops
This PR is related to https://github.com/meilisearch/meilisearch/issues/2542 , it renames the `limitedTo` parameter into `maxTotalHits`.
Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-22 13:48:33 +00:00
bors[bot]
d546f6f40e
Merge #563
...
563: Improve the `estimatedNbHits` when a `distinctAttribute` is specified r=irevoire a=Kerollmops
This PR is related to https://github.com/meilisearch/meilisearch/issues/2532 but it doesn't fix it entirely. It improves it by computing the excluded documents (the ones with an already-seen distinct value) before stopping the loop, I think it was a mistake and should always have been this way.
The reason it doesn't fix the issue is that Meilisearch is lazy, just to be sure not to compute too many things and answer by taking too much time. When we deduplicate the documents by their distinct value we must do it along the water, everytime we see a new document we check that its distinct value of it doesn't collide with an already returned document.
The reason we can see the correct result when enough documents are fetched is that we were lucky to see all of the different distinct values possible in the dataset and all of the deduplication was done, no document can be returned.
If we wanted to implement that to have a correct `extimatedNbHits` every time we should have done a pass on the whole set of possible distinct values for the distinct attribute and do a big intersection, this could cost a lot of CPU cycles.
Co-authored-by: Kerollmops <clement@meilisearch.com>
2022-06-22 12:39:44 +00:00
Kerollmops
f5c3b951bc
Bump the milli version to 0.31.0
2022-06-22 12:08:16 +02:00
Kerollmops
d7c248042b
Rename the limitedTo parameter into maxTotalHits
2022-06-22 12:00:48 +02:00
Kerollmops
d2f84a9d9e
Improve the estimatedNbHits when distinct is enabled
2022-06-22 11:39:21 +02:00
bors[bot]
4f547eff02
Merge #560
...
560: Update version for next release (v0.30.0) r=curquiza a=curquiza
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-06-20 12:37:01 +00:00
Clémentine Urquizar
31f749b5d8
Update version for next release (v0.30.0)
2022-06-20 12:09:57 +02:00
