11441 Commits

Author SHA1 Message Date
CodeMan62
9162e8ba04 Enhance error messages for filterable attributes and improve error handling 2025-03-17 22:04:18 +05:30
shu-kitamura
2118cc092e rm db.snapshot 2025-03-17 23:04:13 +09:00
shu-kitamura
c7564d500f Split unit test in tasks.rs 2025-03-17 22:55:23 +09:00
ManyTheFish
bf144a94d8 No more use FST to find a word without any typo prototype-isolate-word-fst-usage-00 2025-03-17 14:20:10 +01:00
ManyTheFish
b0b1888ef9 Add test 2025-03-17 14:20:10 +01:00
meili-bors[bot]
6ec1d2b712
Merge #5423
5423: Bump ring to v0.17.14 to compile on old aarch64 r=irevoire a=Kerollmops

This PR will fix [this CI issue](https://github.com/meilisearch/meilisearch/actions/runs/13896085925/job/38876941154) where ring v0.17.13 breaks the compilation on old aarch64 machines by bumping its version to v0.17.14.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2025-03-17 12:53:02 +00:00
meili-bors[bot]
cbdf80893d
Merge #5422
5422: Add more progress levels to measure merging r=Kerollmops a=Kerollmops

I found out that Meilisearch was not correctly reporting the long indexing times in the progress and that a lot of time was spent on extracting words with all documents already extracted. The reason was that there was no step to report merging the cache and sending the entries to write to the writer thread. This PR adds these entries to the progress.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2025-03-17 12:02:46 +00:00
Kerollmops
e2156ddfc7
Simplify the IndexingStep progress enum 2025-03-17 11:40:50 +01:00
Kerollmops
49dd50dab2
Bump ring to v0.17.14 to compile on old aarch64 2025-03-17 11:29:17 +01:00
meili-bors[bot]
13a88d6131
Merge #5407
5407: Geo update bug r=irevoire a=ManyTheFish

# Pull Request

## Related issue
Fixes #5380
Fixes #5399



Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: ManyTheFish <many@meilisearch.com>
2025-03-17 10:24:33 +00:00
meili-bors[bot]
d9875b782d
Merge #5421
5421: Accept total batch size in human size r=irevoire a=Kerollmops

This PR fixes the new `experimental-limit-batched-tasks-total-size` to accept human-defined sizes in bytes.

Co-authored-by: Kerollmops <clement@meilisearch.com>
2025-03-17 09:41:22 +00:00
Kerollmops
cb16baab18
Add more progress levels to measure merging 2025-03-17 10:13:29 +01:00
meili-bors[bot]
2500e3c067
Merge #5414
5414: Update version for the next release (v1.14.0) in Cargo.toml r=Kerollmops a=meili-bot

⚠️ This PR is automatically generated. Check the new version is the expected one and Cargo.lock has been updated before merging. Fixes https://github.com/meilisearch/meilisearch/issues/5268.

Co-authored-by: Kerollmops <Kerollmops@users.noreply.github.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
v1.14.0-rc.0
2025-03-14 13:35:54 +00:00
Kerollmops
d3e4b2dfe7
Accept total batch size in human size 2025-03-14 13:07:51 +01:00
meili-bors[bot]
2a46624e19
Merge #5420
5420: Add support for the progress API of arroy r=Kerollmops a=irevoire

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/5419

## What does this PR do?
- Convert the arroy progress to the meilisearch progress
- Use the new arroy closure to support the progress of arroy


Co-authored-by: Tamo <tamo@meilisearch.com>
2025-03-13 18:03:08 +00:00
Tamo
009c36a4d0 Add support for the progress API of arroy 2025-03-13 19:00:43 +01:00
Kerollmops
2a47e25e6d
Update the upgrade path snap 2025-03-13 18:35:06 +01:00
meili-bors[bot]
82912e191b
Merge #5418
5418: Cache embeddings in search r=Kerollmops a=dureuill

# Pull Request

## Related issue
TBD

## What does this PR do?
- Adds a cache for embeddings produced in search
- The cache is disabled by default, and can be enabled following the instructions [here](https://github.com/orgs/meilisearch/discussions/818).
- Had to accommodate the `timeout` test for openai that uses a mock that simulates a timeout on subsequent responses: since the test was reusing the same query, the cache would kick-in and no request would be made to the mock, meaning no timeout any longer and so a failing test 😅 
- `Embedder::embed_search` now accepts a reference instead of an owned `String`.

## Manual testing

- I created 4 indexes on a fresh DB with the same settings (one embedder from openai)
- I sent 1/4 of movies.json to each index
- I sent a federated search request against all 4 indexes, with the same query for each index, using the embedder of each index.

Results:

- The first call took 400ms to 1s. Before this change, it took in the 3s range.
- Any repeated call with the same query took in the range of 25ms.
- Looking at the details at trace log level, I can see that the first index that needs the embedding is taking most of the 400ms in `embed_one`. The other indexes report that the query text is found in the cache and they each take a few µs.

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-03-13 16:37:15 +00:00
Louis Dureuil
e2d372823a
Disable the cache by default and make it experimental 2025-03-13 17:22:51 +01:00
Louis Dureuil
1876132172
Mutex-based implementation 2025-03-13 17:22:50 +01:00
Louis Dureuil
d0b0b90d17
fixup tests, in particular foil the cache for the timeout test 2025-03-13 17:22:50 +01:00
Louis Dureuil
b08544e86d
Add embedding cache 2025-03-13 17:22:50 +01:00
Louis Dureuil
d9111fe8ce
Add lru crate to milli again 2025-03-13 17:22:50 +01:00
Kerollmops
41d8161017
Update the versions 2025-03-13 17:22:32 +01:00
Clément Renault
7df5715d39
Merge pull request #5406 from meilisearch/bump-heed
Bump heed to v0.22 and arroy to v0.6
2025-03-13 16:52:45 +01:00
Kerollmops
5fe02ab5e0
Move to heed 0.22 and arroy 0.6 2025-03-13 15:48:18 +01:00
Tamo
5ef7767429 Let arroy uses all the memory available instead of 50% of the 70% 2025-03-13 15:06:03 +01:00
Tamo
3fad48167b remove arroy dependency in the index-scheduler 2025-03-13 14:57:56 +01:00
Clément Renault
a92a48b9b9
Do not recompute stats on dumpless upgrade
Co-authored-by: Tamo <tamo@meilisearch.com>
2025-03-13 13:58:58 +01:00
Tamo
d53225bf64 uses a random seed instead of 42 2025-03-13 12:43:31 +01:00
Kerollmops
20896500c2
Bump arroy to the latest version 2025-03-13 12:37:10 +01:00
Kerollmops
1af520077c
Call the underlying Env::copy_to_path method 2025-03-13 11:49:25 +01:00
Kerollmops
7e07cb9de1
Make meilitool prefer WithoutTls Env 2025-03-13 11:47:19 +01:00
meili-bors[bot]
a12b06d99d
Merge #5369
5369: exhaustive facet search r=ManyTheFish a=ManyTheFish

Fixes #5403

This PR adds an `exhaustiveFacetCount` field to the `/facet-search` API allowing the end-user to have a better facet count when having a distinct attribute set in the index settings.

 # Usage

`POST /index/:index_uid/facet-search`
**Body:**
```json
{
  "facetQuery": "blob",
  "facetName": "genres",
  "q": "",
  "exhaustiveFacetCount": true
}
```

# Prototype Docker images

```sh
$ docker pull getmeili/meilisearch:prototype-exhaustive-facet-search-00
```

Co-authored-by: ManyTheFish <many@meilisearch.com>
2025-03-13 10:36:04 +00:00
Kerollmops
331dc3d241
Add a comment to explain why we keep debug assertions 2025-03-13 11:29:00 +01:00
Tamo
ef9d9f8481
set the memory in arroy 2025-03-13 11:29:00 +01:00
Kerollmops
d3d22d8ed4
Prefer waiting for the task before getting the indexes 2025-03-13 11:29:00 +01:00
Kerollmops
5e6abcf50c
Prefer using WithoutTls for the auth env 2025-03-13 11:29:00 +01:00
Kerollmops
a4aaf932ba
Fix some test (invalid anyway) 2025-03-13 11:29:00 +01:00
Kerollmops
16c962eb30
Enable debug assertions of heed 2025-03-13 11:07:49 +01:00
Kerollmops
55ca2c4481
Avoid opening the Auth environment multiple times 2025-03-13 11:07:49 +01:00
Kerollmops
fedb444e66
Fix the upgrade arroy calls 2025-03-13 11:07:49 +01:00
Kerollmops
bef5954741
Use a WithoutTls env 2025-03-13 11:07:49 +01:00
Kerollmops
ff8cf38d6b
Move to the latest version of arroy 2025-03-13 11:07:48 +01:00
Kerollmops
f8ac575ec5
Move to the latest version of arroy 2025-03-13 11:07:48 +01:00
Kerollmops
566b4efb06
Dumpless upgrade from v1.13 to v1.14 2025-03-13 11:07:44 +01:00
Kerollmops
1d499ed9b2
Use the new arroy upgrade method to move from 0.4 to 0.5 2025-03-13 11:07:44 +01:00
Kerollmops
3bc62f0549
WIP: Still need to introduce a Env::copy_to_path method 2025-03-13 11:07:39 +01:00
Kerollmops
21bbbdec76
Specify WithoutTls everywhere 2025-03-13 11:07:38 +01:00
Kerollmops
78ebd8dba2
Fix the error variants 2025-03-13 11:07:38 +01:00