38 Commits

Author SHA1 Message Date
Clément Renault
4d90e3d2ec
Make Cargo and Clippy happy 2025-04-01 11:26:34 +02:00
Tamo
009c36a4d0 Add support for the progress API of arroy 2025-03-13 19:00:43 +01:00
Louis Dureuil
e2d372823a
Disable the cache by default and make it experimental 2025-03-13 17:22:51 +01:00
Tamo
5ef7767429 Let arroy uses all the memory available instead of 50% of the 70% 2025-03-13 15:06:03 +01:00
Tamo
ef9d9f8481
set the memory in arroy 2025-03-13 11:29:00 +01:00
ManyTheFish
6d52c6e711 Merge branch 'main' into granular-filterable-attributes 2025-03-11 10:05:58 +01:00
ManyTheFish
abef655849 Revert metadata creation when computing facet search and distinct 2025-03-10 15:45:59 +01:00
ManyTheFish
b88aa9cc76 Rely on FieldIdMapWithMetadata in facet search and filters 2025-03-05 18:22:12 +01:00
meili-bors[bot]
3fd86e8d76
Merge #5371
5371: Composite embedders r=irevoire a=dureuill

# Pull Request

## Related issue
Fixes #5343 

## What does this PR do?
- Implement [public usage](https://www.notion.so/meilisearch/Composite-embedder-usage-14a4b06b651f81859dc3df21e8cd02a0)
- Refactor the way we check if a parameter is mandatory/allowed/disallowed for a given source
- Take the "nesting context" into account for computer if a parameter is mandatory/allowed/disallowed
- Add tests checking all parameters with all sources, and made sure the results didn't change compared with v1.13

## Dumpless Upgrade

- This adds a new value for an existing parameter => compatible without change
- This adds new optional parameters => compatible without change

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-03-05 17:18:11 +00:00
ManyTheFish
659855c88e Refactor Settings Indexing process
**Changes:**
The transform structure is now relying on FieldIdMapWithMetadata and AttributePatterns to prepare
the obkv documents during a settings reindexing.
The InnerIndexSettingsDiff and InnerIndexSettings structs are now relying on FieldIdMapWithMetadata, FilterableAttributesRule and AttributePatterns to define the field and the databases that should be reindexed.
The faceted_fields_ids, localized_searchable_fields_ids and localized_faceted_fields_ids have been removed in favor of the FieldIdMapWithMetadata.
We are now relying on the FieldIdMapWithMetadata to retain vectors_fids from the facets and the searchables.

The searchable database computing is now relying on the FieldIdMapWithMetadata to know if a field is searchable and retrieve the locales.

The facet database computing is now relying on the FieldIdMapWithMetadata to compute the facet databases, the facet-search and retrieve the locales.

The facet level database computing is now relying on the FieldIdMapWithMetadata and the facet level database are cleared depending on the settings differences (clear_facet_levels_based_on_settings_diff).

The vector point extraction uses the FieldIdMapWithMetadata instead of FieldsIdsMapWithMetadata.

**Impact:**
- Dump import
- Settings update
2025-03-03 10:32:02 +01:00
meili-bors[bot]
c63c25a9a2
Merge #5355
5355: Support fetching the pooling method from the model configuration r=Kerollmops a=dureuill

# Pull Request

## Related issue
Fixes #5354 

## What does this PR do?
- Fetches the pooling configuration from the model repository
- Use a pooling method that depends on the pooling configuration of that model.
- Allow overriding the pooling method with a new huggingFace embedder parameter `pooling`
  - for backward-compatibility with Meilisearch v1.13
  - for compatibility with embedders that exhibit the same behavior as Meilisearch v1.13
- Handle the default value of that new parameter
   - for compatibility, when importing a db/a dump, it should be set to `forceMean`
   - when (re)set from the settings for an embedder, it should be set to `useModel`


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-02-27 14:55:13 +00:00
ManyTheFish
9f3663e768
Implement Incremental document database stats computing 2025-02-26 17:01:35 +01:00
Louis Dureuil
e374b095a2
Fix tests 2025-02-24 14:11:26 +01:00
Louis Dureuil
7b4ce468a6
Allow overriding pooling method 2025-02-18 17:12:23 +01:00
Kerollmops
aa2327591e
Add more mixing updates and replacements tests 2025-02-03 10:34:07 +01:00
Kerollmops
60470bb647
Fix the tests to use the new replace/update documents 2025-02-03 10:34:07 +01:00
Louis Dureuil
f0ec8cbffe
Add currently failing test 2025-01-14 10:23:15 +01:00
Clément Renault
68333424c6
Remove a useless script test 2025-01-08 15:59:43 +01:00
Gnosnay
525e67ba93 Fix the format and linter error 2024-12-28 20:35:55 +08:00
Gnosnay
44eb153619 Replace hardcoded string with constants 2024-12-28 20:35:55 +08:00
Tamo
df9b68f8ed
inital implementation of the progress 2024-12-11 16:25:01 +01:00
Clément Renault
da650f834e
Plug the NoPanicThreadPool in the tests and benchmarks 2024-11-27 17:04:49 +01:00
ManyTheFish
d66dc363ed Test and implement settings opt-out 2024-11-25 18:23:22 +01:00
Clément Renault
8380ddbdcd
Fix progress of into_changes 2024-11-20 15:10:09 +01:00
Clément Renault
567bd4538b
Fxi the into_changes stop processing 2024-11-20 14:58:25 +01:00
Louis Dureuil
61feca1f41
More tests pass 2024-11-20 13:16:43 +01:00
Clément Renault
5f8a82d6f5
Improve test 2024-11-20 13:16:39 +01:00
Clément Renault
fe04e51a49
One more 2024-11-20 13:16:38 +01:00
Clément Renault
01b27e40ad
Fix a bit of the placeholder search tests 2024-11-20 13:16:38 +01:00
Louis Dureuil
8076d98544
Fix stats_should_not_return_deleted_documents 2024-11-20 13:16:37 +01:00
Louis Dureuil
9e951baad5
One more test passing 2024-11-20 13:16:37 +01:00
Louis Dureuil
6641c3f59b
Remove all autogenerated tests 2024-11-20 13:16:34 +01:00
Clément Renault
32d0e50a75
Fix all the benchmark compilation errors 2024-11-20 13:16:32 +01:00
Clément Renault
aba8a0e9e0
Fix some tests but not all of them 2024-11-20 13:16:31 +01:00
Clément Renault
670aff5553
Remove useless Transform methods 2024-11-20 13:16:08 +01:00
ManyTheFish
10feeb88f2 Merge branch 'main' into indexer-edition-2024 2024-11-06 15:19:18 +01:00
Tamo
cf6ad1ae5e Merge branch 'main' into tmp-release-v1.11.0 2024-11-04 16:14:44 +01:00
Clément Renault
9c1e54a2c8
Move crates under a sub folder to clean up the code 2024-10-21 08:18:43 +02:00