Alexey Shekhirin
c2517e7d5f
fix(facet): string fields sorting
2021-09-03 11:58:26 +03:00
bors[bot]
5cbe879325
Merge #308
...
308: Implement a better parallel indexer r=Kerollmops a=ManyTheFish
Rewrite the indexer:
- enhance memory consumption control
- optimize parallelism using rayon and crossbeam channel
- factorize the different parts and make new DB implementation easier
- optimize and fix prefix databases
Co-authored-by: many <maxime@meilisearch.com>
2021-09-02 15:03:52 +00:00
many
741a4444a9
Remove log in chunk generator
2021-09-02 16:57:46 +02:00
many
7f7fafb857
Make document_chunk_size settable from update builder
2021-09-02 15:25:39 +02:00
many
db0c681bae
Fix Pr comments
2021-09-02 15:17:52 +02:00
many
4860fd4529
Ignore empty facet values
2021-09-01 16:48:40 +02:00
many
b3a22f31f6
Fix memory consuption in word pair proximity extractor
2021-09-01 16:48:40 +02:00
many
9452fabfb2
Optimize cbo roaring bitmaps merge
2021-09-01 16:48:40 +02:00
many
8f702828ca
Ignore errors comming from crossbeam channel senders
2021-09-01 16:48:40 +02:00
many
e09eec37bc
Handle distance addition with hard separators
2021-09-01 16:48:40 +02:00
many
fc7cc770d4
Add logging timers
2021-09-01 16:48:40 +02:00
many
a2f59a28f7
Remove unwrap sending errors in channel
2021-09-01 16:48:40 +02:00
many
5c962c03dd
Fix and optimize word_prefix_pair_proximity_docids database
2021-09-01 16:48:40 +02:00
many
2d1727697d
Take stop word in account
2021-09-01 16:48:40 +02:00
many
823da19745
Fix test and use progress callback
2021-09-01 16:48:39 +02:00
many
1d314328f0
Plug new indexer
2021-09-01 16:48:36 +02:00
many
3aaf1d62f3
Publish grenad CompressionType type in milli
2021-09-01 16:42:08 +02:00
Alexey Shekhirin
0e379558a1
fix(search): get sortable_fields only if criteria present
2021-08-31 21:35:41 +03:00
bors[bot]
d6bba0663a
Merge #334
...
334: Wrap long values into BStr for warn logs r=Kerollmops a=shekhirin
Resolves https://github.com/meilisearch/milli/issues/263
Co-authored-by: Alexey Shekhirin <a.shekhirin@gmail.com>
2021-08-31 17:38:54 +00:00
Alexey Shekhirin
0b02eb456c
chore(update): wrap long values into BStr for warn logs
2021-08-31 20:28:16 +03:00
Kerollmops
f230ae6fd5
Introduce the reset_sortable_fields Settings method
2021-08-25 17:44:16 +02:00
Kerollmops
af65485ba7
Reexport the grenad CompressionType from milli
2021-08-24 18:15:31 +02:00
Clément Renault
89d0758713
Revert "Revert "Sort at query time""
2021-08-24 11:55:16 +02:00
Clément Renault
c084f7f731
Fix the facet string docids filterable deletion bug
2021-08-23 10:50:39 +02:00
Clémentine Urquizar
922f9fd4d5
Revert "Sort at query time"
2021-08-20 18:09:17 +02:00
many
d1df0d20f9
Add integration test of SortBy criterion
2021-08-18 16:21:51 +02:00
Kerollmops
1b7f6ea1e7
Return a new error when the sort criteria is not sortable
2021-08-18 15:04:07 +02:00
Kerollmops
71602e0f1b
Add the sortable fields into the settings and in the index
2021-08-18 15:04:07 +02:00
Kerollmops
407f53872a
Add a sort_criteria method to the Search builder struct
2021-08-18 15:04:07 +02:00
Kerollmops
687cd2e205
Introduce the new Sort criterion and AscDesc enum
2021-08-18 15:04:07 +02:00
Kerollmops
5b88df508e
Use the new Asc/Desc syntax everywhere
2021-08-17 14:15:22 +02:00
Kerollmops
fcedff95e8
Change the Asc/Desc criterion syntax to use a colon (:)
2021-08-17 14:03:21 +02:00
Kerollmops
e9ada44509
AscDesc criterion returns documents ordered by numbers then by strings
2021-08-17 13:21:31 +02:00
Kerollmops
110bf6b778
Make the FacetStringIter work in both, ascending and descending orders
2021-08-17 11:18:40 +02:00
Kerollmops
22ebd2658f
Introduce the EitherString/RevRange private aliases
2021-08-17 10:47:15 +02:00
Kerollmops
7a5889bc5a
Introduce the highest_reverse_iter private method
2021-08-17 10:45:26 +02:00
Kerollmops
ad0d311f8a
Introduce the FacetStringLevelZeroRevRange struct
2021-08-17 10:44:43 +02:00
Kerollmops
6214c38da9
Introduce the FacetStringGroupRevRange struct
2021-08-17 10:44:27 +02:00
Kerollmops
1c604de158
Introduce the highest_iter private method on the FacetStringIter struct
2021-08-17 10:41:11 +02:00
Kerollmops
64df159057
Introduce the new_reducing constructor on the FacetStringIter struct
2021-08-17 10:35:06 +02:00
Kerollmops
01a4052828
Move the FacetStringIter creation logic into a private new method
2021-08-17 10:29:43 +02:00
many
7dbefae1e3
Make facet string iterator non reducing
2021-08-12 17:23:39 +02:00
many
8fdf860c17
Remove max values by facet limit for facet distribution
2021-08-12 11:29:20 +02:00
bors[bot]
89b9b61840
Merge #300
...
300: Fix prefix level position docids database r=curquiza a=ManyTheFish
The prefix search was inverted when we generated the DB.
Instead of searching if word had a prefix in prefix fst,
we were searching if the word was a prefix of a prefix contained in the prefix fst.
The indexer, now, iterate over prefix contained in the fst
and search them by prefix in the word-level-position-docids database,
aggregating matches in a sorter.
Fix #299
Co-authored-by: many <maxime@meilisearch.com>
2021-08-04 16:52:09 +00:00
many
cdeb07f0fd
Fix prefix level position docids database
...
The prefix search was inverted when we generated the DB.
Instead of searching if word had a prefix in prefix fst,
we were searching if the word was a prefix of a prefix contained in the prefix fst.
The indexer, now, iterate over prefix contained in the fst
and search them by prefix in the word-level-position-docids database,
aggregating matches in a sorter.
Fix #299
2021-08-04 14:11:49 +02:00
Kerollmops
90514e03d1
Fix invalid faceted documents ids buffer size
2021-07-29 15:49:23 +02:00
bors[bot]
200e98c211
Merge #293
...
293: Make sure that the relevancy is not impacted by other settings r=Kerollmops a=Kerollmops
Fix https://github.com/meilisearch/meilisearch/issues/1505 .
fix https://github.com/meilisearch/MeiliSearch/issues/1529
Co-authored-by: Kerollmops <clement@meilisearch.com>
2021-07-27 16:04:52 +00:00
Kerollmops
dc2b63abdf
Introduce an empty FilterCondition variant to support unknown fields
2021-07-27 16:34:04 +02:00
Kerollmops
b12738cfe9
Use the right DB prefixes to store the faceted fields
2021-07-22 19:18:22 +02:00
Kerollmops
7aa6cc9b04
Do not insert fields in the map when changing the settings
2021-07-22 18:40:12 +02:00
