92 Commits

Author SHA1 Message Date
Clément Renault
7df5715d39
Merge pull request #5406 from meilisearch/bump-heed
Bump heed to v0.22 and arroy to v0.6
2025-03-13 16:52:45 +01:00
meili-bors[bot]
a12b06d99d
Merge #5369
5369: exhaustive facet search r=ManyTheFish a=ManyTheFish

Fixes #5403

This PR adds an `exhaustiveFacetCount` field to the `/facet-search` API allowing the end-user to have a better facet count when having a distinct attribute set in the index settings.

 # Usage

`POST /index/:index_uid/facet-search`
**Body:**
```json
{
  "facetQuery": "blob",
  "facetName": "genres",
  "q": "",
  "exhaustiveFacetCount": true
}
```

# Prototype Docker images

```sh
$ docker pull getmeili/meilisearch:prototype-exhaustive-facet-search-00
```

Co-authored-by: ManyTheFish <many@meilisearch.com>
2025-03-13 10:36:04 +00:00
Kerollmops
3bc62f0549
WIP: Still need to introduce a Env::copy_to_path method 2025-03-13 11:07:39 +01:00
meili-bors[bot]
e2d0ce52ba
Merge #5384
5384: Get multiple documents by ids r=irevoire a=dureuill

# Pull Request

## Related issue
Fixes #5345 

## What does this PR do?
- Implements [public usage](https://www.notion.so/meilisearch/Get-documents-by-ID-1994b06b651f805ba273e1c6b75ce4d8)
- Slightly refactor error messages for the `/similar` route

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-03-12 17:26:49 +00:00
Louis Dureuil
60ff1b19a8
Searching for a document that does not exist no longer raises an error 2025-03-12 11:50:39 +01:00
ManyTheFish
eb3ff325d1 Add an exhaustiveFacetCount field to the facet-search API 2025-03-12 11:22:59 +01:00
meili-bors[bot]
a2a86ef4e2
Merge #5254
5254: Granular Filterable attribute settings r=ManyTheFish a=ManyTheFish

# Related
**Issue:** https://github.com/meilisearch/meilisearch/issues/5163
**PRD:** https://meilisearch.notion.site/API-usage-Settings-to-opt-out-indexing-features-filterableAttributes-1764b06b651f80aba8bdf359b2df3ca8

# Summary
Change the `filterableAttributes` settings to let the user choose which facet feature he wants to activate or not.
Deactivating a feature will avoid some database computation in the indexing process and save time and disk size.

# Example

`PATCH /indexes/:index_uid/settings`

```json
{
  "filterableAttributes": [
    {
      "patterns": [
        "cattos",
        "doggos.age"
      ],
      "features": {
        "facetSearch": false,
        "filter": {
          "equality": true,
          "comparison": false
        }
      }
    }
  ]
}
```

# Impact on the codebase
- Settings API:
  - `/settings`
  - `/settings/filterable-attributes`
  - OpenAPI 
  - may impact the LocalizedAttributesRules due to the AttributePatterns factorization
- Database:
  - Filterable attributes format changed
  - Faceted field_ids are no more stored in the database
  - FieldIdsMap has no more unexisting fields
- Search:
  - Search using filters
  - Facet search
  - `Attributes` ranking rule
  - Distinct attribute
  - Facet distribution
- Settings reindexing:
  - searchable
  - facet
  - vector
  - geo
- Document indexing:
  - searchable
  - facet
  - vector
  - geo
- Dump import

# Note for the reviewers
The changes are huge and have been split in different commits with a dedicated explanation, I suggest reviewing the commit 1by1

Co-authored-by: ManyTheFish <many@meilisearch.com>
2025-03-12 09:00:43 +00:00
ManyTheFish
6d52c6e711 Merge branch 'main' into granular-filterable-attributes 2025-03-11 10:05:58 +01:00
Louis Dureuil
54ee81bb09
Make composite embedders experimental 2025-03-10 14:22:47 +01:00
Louis Dureuil
f292fc9ac0
Add ids parameter to GET documents and POST documents/fetch 2025-03-10 11:11:48 +01:00
meili-bors[bot]
3fd86e8d76
Merge #5371
5371: Composite embedders r=irevoire a=dureuill

# Pull Request

## Related issue
Fixes #5343 

## What does this PR do?
- Implement [public usage](https://www.notion.so/meilisearch/Composite-embedder-usage-14a4b06b651f81859dc3df21e8cd02a0)
- Refactor the way we check if a parameter is mandatory/allowed/disallowed for a given source
- Take the "nesting context" into account for computer if a parameter is mandatory/allowed/disallowed
- Add tests checking all parameters with all sources, and made sure the results didn't change compared with v1.13

## Dumpless Upgrade

- This adds a new value for an existing parameter => compatible without change
- This adds new optional parameters => compatible without change

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-03-05 17:18:11 +00:00
ManyTheFish
967033579d Refactor search and facet-search
**Changes:**
The search filters are now using the FilterableAttributesFeatures from the FilterableAttributesRules to know if a field is filterable.
Moreover, the FilterableAttributesFeatures is more precise and an error will be returned if an operator is used on a field that doesn't have the related feature.
The facet-search is now checking if the feature is allowed in the FilterableAttributesFeatures and an error will be returned if the field doesn't have the related feature.

**Impact:**
- facet-search is now relying on AttributePatterns to match the locales
- search using filters is now relying on FilterableAttributesFeatures
- distinct attribute is now relying on FilterableAttributesRules
2025-03-03 10:25:32 +01:00
ManyTheFish
0200c65ebf Change the filterableAttributes setting API
**Changes:**
The filterableAttributes type has been changed from a `BTreeSet<String>` to a `Vec<FilterableAttributesRule>`,
Which is a list of rules defining patterns to match the documents' fields and a set of feature to apply on the matching fields.
The rule order given by the user is now an important information, the features applied on a filterable field will be chosen based on the rule order as we do for the LocalizedAttributesRules.
This means that the list will not be reordered anymore and will keep the user defined order,
moreover, if there are any duplicates, they will not be de-duplicated anymore.

**Impact:**
- Settings API
- the database format of the filterable attributes changed
- may impact the LocalizedAttributesRules due to the AttributePatterns factorization
- OpenAPI generator
2025-03-03 10:22:02 +01:00
Kerollmops
dfce20be21
Rename callTrace into progressTrace 2025-02-25 10:09:03 +01:00
Louis Dureuil
b85180fedb
Error types 2025-02-24 13:58:26 +01:00
Kerollmops
1d99c8465c
Hide the batch stats to make insta pass 2025-02-20 10:16:54 +01:00
Kerollmops
05cc8c650c
Expose the write channel congestion in the batches 2025-02-19 15:47:54 +01:00
Kerollmops
3ff1de0a21
Expose the call trace in the batch stats 2025-02-19 11:24:11 +01:00
Tamo
fa00b42c93 fix the missing batch in the dumps in meilisearch and meilitools 2025-02-11 11:14:49 +01:00
Kerollmops
7d0d8f4445
Make the feature experimental 2025-02-10 16:11:32 +01:00
Kerollmops
df40533741
Expose a route to get the update file content of a task 2025-02-10 14:05:32 +01:00
Tamo
7f82d33597
update the version file atomically 2025-02-06 18:23:28 +01:00
Tamo
8c5856007c
flush+sync the version file just in case 2025-02-06 18:04:43 +01:00
Tamo
628119e31e
fix the dumpless upgrade potential corruption when upgrading from the v1.12 2025-02-05 18:08:50 +01:00
Louis Dureuil
9996533364
Make search types serialize and deserialize so that reading from a proxy is possible 2025-02-05 15:03:16 +01:00
Louis Dureuil
b30e5a7a35
Add new permissions 2025-02-05 15:03:16 +01:00
Louis Dureuil
6d79cb23ba
New error codes 2025-02-05 15:03:16 +01:00
Louis Dureuil
73474e7af0
Network types 2025-02-05 15:03:15 +01:00
Louis Dureuil
7ae6dda03f
Add new experimental feature 2025-02-05 15:01:04 +01:00
Tamo
58f90b70c7
store the enqueued at to eases the batch deletion 2025-01-28 16:53:50 +01:00
manojks1999
528d9d6d8b Removed CouldNotUpgrade from error file 2025-01-26 21:04:57 +05:30
Tamo
fd5649091d
add the upgradeTo field in the details 2025-01-23 16:51:23 +01:00
Tamo
b9e9fc376a
add the version in the index-scheduler 2025-01-23 16:51:22 +01:00
Tamo
b132d70413
fix the details in all cases 2025-01-23 16:51:21 +01:00
Tamo
0cc25c7e4c
add a large test importing a data.ms from the v1.12.0 2025-01-23 16:51:18 +01:00
Tamo
102681e384
starts adding tests and fix the starts of meilisearch 2025-01-23 16:51:18 +01:00
Tamo
3ef7a478cd
move the version check to the task queue 2025-01-23 16:48:32 +01:00
Tamo
d3654906bf
Add the new tasks with most of the job done 2025-01-23 16:48:32 +01:00
Louis Dureuil
87ea080c10
Fully remove vector store feature 2025-01-14 13:55:34 +01:00
Louis Dureuil
de6cd3ac01
Consistent error codes 2025-01-14 13:55:34 +01:00
Louis Dureuil
03097e65e8
Always display embedders setting 2025-01-14 13:55:34 +01:00
Louis Dureuil
29eeb84ce3
Add --experimental-disable-vector-store CLI flag 2025-01-14 13:53:53 +01:00
Clément Renault
091f989b72
Upgrade incompatible dependencies 2025-01-08 15:58:03 +01:00
Clément Renault
48a9ad4c17
Fix insta to 1.39 2025-01-08 15:18:08 +01:00
Clément Renault
b997039a91
Upgrade compatible dependencies 2025-01-08 13:52:14 +01:00
Tamo
a8ef6f08e0
Update crates/meilisearch-types/src/settings.rs
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-01-07 16:31:39 +01:00
Tamo
ae5a04e85c
apply review comments 2025-01-07 16:30:14 +01:00
Tamo
8ebfc9fa92
Update crates/meilisearch-types/src/settings.rs
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-01-07 16:30:14 +01:00
Tamo
dd128656cb
fix all the tests 2025-01-07 16:28:12 +01:00
Tamo
ac944f0960
review all the return type 2025-01-07 16:26:06 +01:00