11199 Commits

Author SHA1 Message Date
Kerollmops
5e6abcf50c
Prefer using WithoutTls for the auth env 2025-03-13 11:29:00 +01:00
Kerollmops
a4aaf932ba
Fix some test (invalid anyway) 2025-03-13 11:29:00 +01:00
Kerollmops
16c962eb30
Enable debug assertions of heed 2025-03-13 11:07:49 +01:00
Kerollmops
55ca2c4481
Avoid opening the Auth environment multiple times 2025-03-13 11:07:49 +01:00
Kerollmops
fedb444e66
Fix the upgrade arroy calls 2025-03-13 11:07:49 +01:00
Kerollmops
bef5954741
Use a WithoutTls env 2025-03-13 11:07:49 +01:00
Kerollmops
ff8cf38d6b
Move to the latest version of arroy 2025-03-13 11:07:48 +01:00
Kerollmops
f8ac575ec5
Move to the latest version of arroy 2025-03-13 11:07:48 +01:00
Kerollmops
566b4efb06
Dumpless upgrade from v1.13 to v1.14 2025-03-13 11:07:44 +01:00
Kerollmops
1d499ed9b2
Use the new arroy upgrade method to move from 0.4 to 0.5 2025-03-13 11:07:44 +01:00
Kerollmops
3bc62f0549
WIP: Still need to introduce a Env::copy_to_path method 2025-03-13 11:07:39 +01:00
Kerollmops
21bbbdec76
Specify WithoutTls everywhere 2025-03-13 11:07:38 +01:00
Kerollmops
78ebd8dba2
Fix the error variants 2025-03-13 11:07:38 +01:00
Kerollmops
34df44a002
Open Env without TLS 2025-03-13 11:07:38 +01:00
Kerollmops
48a27f669e
Bump heed and other dependencies 2025-03-13 11:07:37 +01:00
meili-bors[bot]
e2d0ce52ba
Merge #5384
5384: Get multiple documents by ids r=irevoire a=dureuill

# Pull Request

## Related issue
Fixes #5345 

## What does this PR do?
- Implements [public usage](https://www.notion.so/meilisearch/Get-documents-by-ID-1994b06b651f805ba273e1c6b75ce4d8)
- Slightly refactor error messages for the `/similar` route

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-03-12 17:26:49 +00:00
meili-bors[bot]
995f8962bd
Merge #5398
5398: Bump ring from 0.17.8 to 0.17.13 r=Kerollmops a=dependabot[bot]

Bumps [ring](https://github.com/briansmith/ring) from 0.17.8 to 0.17.13.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/briansmith/ring/blob/main/RELEASES.md">ring's changelog</a>.</em></p>
<blockquote>
<h1>Version 0.17.13 (2025-03-06)</h1>
<p>Increased MSRV to 1.66.0 to avoid bugs in earlier versions so that we can
safely use <code>core::arch::x86_64::__cpuid</code> and <code>core::arch::x86::__cpuid</code> from
Rust in future releases.</p>
<p>AVX2-based VAES-CLMUL implementation. This will be a notable performance
improvement for most newish x86-64 systems. This will likely raise the minimum
binutils version supported for very old Linux distros.</p>
<h1>Version 0.17.12 (2025-03-05)</h1>
<p>Bug fix: <a href="https://redirect.github.com/briansmith/ring/pull/2447">briansmith/ring#2447</a> for denial of service (DoS).</p>
<ul>
<li>
<p>Fixes a panic in <code>ring::aead::quic::HeaderProtectionKey::new_mask()</code> when
integer overflow checking is enabled. In the QUIC protocol, an attacker can
induce this panic by sending a specially-crafted packet. Even unintentionally
it is likely to occur in 1 out of every 2**32 packets sent and/or received.</p>
</li>
<li>
<p>Fixes a panic on 64-bit targets in <code>ring::aead::{AES_128_GCM, AES_256_GCM}</code>
when overflow checking is enabled, when encrypting/decrypting approximately
68,719,476,700 bytes (about 64 gigabytes) of data in a single chunk. Protocols
like TLS and SSH are not affected by this because those protocols break large
amounts of data into small chunks. Similarly, most applications will not
attempt to encrypt/decrypt 64GB of data in one chunk.</p>
</li>
</ul>
<p>Overflow checking is not enabled in release mode by default, but
<code>RUSTFLAGS=&quot;-C overflow-checks&quot;</code> or <code>overflow-checks = true</code> in the Cargo.toml
profile can override this. Overflow checking is usually enabled by default in
debug mode.</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a href="https://github.com/briansmith/ring/commits">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=ring&package-manager=cargo&previous-version=0.17.8&new-version=0.17.13)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/meilisearch/meilisearch/network/alerts).

</details>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-12 13:20:30 +00:00
meili-bors[bot]
1cd00f37c0
Merge #5413
5413: Make sure to delete useless prefixes r=ManyTheFish a=Kerollmops

We discovered a bug where the new indexer was still writing empty roaring bitmaps instead of deleting the prefix entry from the prefix database.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2025-03-12 10:54:04 +00:00
Louis Dureuil
60ff1b19a8
Searching for a document that does not exist no longer raises an error 2025-03-12 11:50:39 +01:00
Louis Dureuil
7df5e3f059
Fix error message
Co-authored-by: Tamo <tamo@meilisearch.com>
2025-03-12 11:48:40 +01:00
Kerollmops
0197dc87e0
Make sure to delete useless prefixes 2025-03-12 11:24:13 +01:00
meili-bors[bot]
a2a86ef4e2
Merge #5254
5254: Granular Filterable attribute settings r=ManyTheFish a=ManyTheFish

# Related
**Issue:** https://github.com/meilisearch/meilisearch/issues/5163
**PRD:** https://meilisearch.notion.site/API-usage-Settings-to-opt-out-indexing-features-filterableAttributes-1764b06b651f80aba8bdf359b2df3ca8

# Summary
Change the `filterableAttributes` settings to let the user choose which facet feature he wants to activate or not.
Deactivating a feature will avoid some database computation in the indexing process and save time and disk size.

# Example

`PATCH /indexes/:index_uid/settings`

```json
{
  "filterableAttributes": [
    {
      "patterns": [
        "cattos",
        "doggos.age"
      ],
      "features": {
        "facetSearch": false,
        "filter": {
          "equality": true,
          "comparison": false
        }
      }
    }
  ]
}
```

# Impact on the codebase
- Settings API:
  - `/settings`
  - `/settings/filterable-attributes`
  - OpenAPI 
  - may impact the LocalizedAttributesRules due to the AttributePatterns factorization
- Database:
  - Filterable attributes format changed
  - Faceted field_ids are no more stored in the database
  - FieldIdsMap has no more unexisting fields
- Search:
  - Search using filters
  - Facet search
  - `Attributes` ranking rule
  - Distinct attribute
  - Facet distribution
- Settings reindexing:
  - searchable
  - facet
  - vector
  - geo
- Document indexing:
  - searchable
  - facet
  - vector
  - geo
- Dump import

# Note for the reviewers
The changes are huge and have been split in different commits with a dedicated explanation, I suggest reviewing the commit 1by1

Co-authored-by: ManyTheFish <many@meilisearch.com>
2025-03-12 09:00:43 +00:00
ManyTheFish
d500c7f625 Add default deserialize value 2025-03-11 17:55:49 +01:00
ManyTheFish
ea7e299663 Update has_changed_for_fields documentation 2025-03-11 16:48:55 +01:00
ManyTheFish
a370b467fe Merge MetadataBuilder::_new into MetadataBuilder::new 2025-03-11 15:31:57 +01:00
ManyTheFish
8790880589 Fix clippy 2025-03-11 15:22:39 +01:00
ManyTheFish
7072fe9780 Fix typos in comments and messages 2025-03-11 15:22:00 +01:00
meili-bors[bot]
d0dda78f3d
Merge #5401
5401: Make composite embedders an experimental feature r=irevoire a=dureuill

# Pull Request

## Related issue
Fixes #5343 

## What does this PR do?
- Introduce new `compositeEmbedders` experimental feature
- Guard `source = "composite"` and `searchEmbedder`, `indexingEmbedder` behind enabling the feature.
- Update tests accordingly

## Dumpless upgrade

- Adding an experimental feature is never a breaking change

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-03-11 14:20:36 +00:00
Louis Dureuil
fa8afc5cfd
Style change after review
Co-authored-by: Tamo <tamo@meilisearch.com>
2025-03-11 13:25:35 +01:00
ManyTheFish
6d52c6e711 Merge branch 'main' into granular-filterable-attributes 2025-03-11 10:05:58 +01:00
ManyTheFish
dfb8411647 Revert "Remove filter pre-check"
This reverts commit b12ffd13569e1c90f7ae1b3a45211eec4594b0e2.
2025-03-11 09:48:30 +01:00
ManyTheFish
6269f757ff Revert document creation in tests 2025-03-10 18:35:10 +01:00
ManyTheFish
40c5f911fd Revert metadata creation when computing the facet-distribution 2025-03-10 17:05:41 +01:00
ManyTheFish
abef655849 Revert metadata creation when computing facet search and distinct 2025-03-10 15:45:59 +01:00
ManyTheFish
b12ffd1356 Remove filter pre-check 2025-03-10 14:29:45 +01:00
ManyTheFish
c9a4c6ed96 REvert metadata creation when computing filters at search time 2025-03-10 14:29:44 +01:00
Louis Dureuil
aa32b719c7
Add tests about experimentalness of the feature and fix existing 2025-03-10 14:23:22 +01:00
Louis Dureuil
41d2b1e52b
Analytics 2025-03-10 14:23:07 +01:00
Louis Dureuil
54ee81bb09
Make composite embedders experimental 2025-03-10 14:22:47 +01:00
ManyTheFish
689e69d6d2 Take into account PR messages 2025-03-10 13:46:33 +01:00
Louis Dureuil
9d9e0d4c54
Add analytics 2025-03-10 11:33:15 +01:00
Louis Dureuil
19c9caed39
Fix tests 2025-03-10 11:11:48 +01:00
Louis Dureuil
21c3b3957e
tests: Change get_document_by_filter to fetch_documents 2025-03-10 11:11:48 +01:00
Louis Dureuil
f292fc9ac0
Add ids parameter to GET documents and POST documents/fetch 2025-03-10 11:11:48 +01:00
Louis Dureuil
1d3c4642a6
Don't use Deserr for ExternalDocumentId, instead convert to error afterward 2025-03-10 11:11:48 +01:00
meili-bors[bot]
9a282be0a2
Merge #5393
5393: Bring back changes from v1.13.3 into main r=irevoire a=Kerollmops



Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: Kerollmops <Kerollmops@users.noreply.github.com>
Co-authored-by: Strift <lau.cazanove@gmail.com>
2025-03-10 07:57:02 +00:00
dependabot[bot]
bea28968a0
Bump ring from 0.17.8 to 0.17.13
Bumps [ring](https://github.com/briansmith/ring) from 0.17.8 to 0.17.13.
- [Changelog](https://github.com/briansmith/ring/blob/main/RELEASES.md)
- [Commits](https://github.com/briansmith/ring/commits)

---
updated-dependencies:
- dependency-name: ring
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-07 17:04:57 +00:00
ManyTheFish
ed1dcbe0f7 Fix behavior change in the Attributes criterion 2025-03-06 14:18:25 +01:00
ManyTheFish
5ceddbda84 Add the max_weight of the weight map if it's lacking 2025-03-06 13:58:28 +01:00
ManyTheFish
ca41ce3bbd Old indexer document addition now check if facet search is globally activated 2025-03-06 11:43:42 +01:00