74 Commits

Author SHA1 Message Date
Kerollmops
5e6abcf50c
Prefer using WithoutTls for the auth env 2025-03-13 11:29:00 +01:00
Kerollmops
55ca2c4481
Avoid opening the Auth environment multiple times 2025-03-13 11:07:49 +01:00
Kerollmops
3bc62f0549
WIP: Still need to introduce a Env::copy_to_path method 2025-03-13 11:07:39 +01:00
ManyTheFish
7072fe9780 Fix typos in comments and messages 2025-03-11 15:22:00 +01:00
ManyTheFish
6d52c6e711 Merge branch 'main' into granular-filterable-attributes 2025-03-11 10:05:58 +01:00
ManyTheFish
c9a4c6ed96 REvert metadata creation when computing filters at search time 2025-03-10 14:29:44 +01:00
meili-bors[bot]
9a282be0a2
Merge #5393
5393: Bring back changes from v1.13.3 into main r=irevoire a=Kerollmops



Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: Kerollmops <Kerollmops@users.noreply.github.com>
Co-authored-by: Strift <lau.cazanove@gmail.com>
2025-03-10 07:57:02 +00:00
meili-bors[bot]
3fd86e8d76
Merge #5371
5371: Composite embedders r=irevoire a=dureuill

# Pull Request

## Related issue
Fixes #5343 

## What does this PR do?
- Implement [public usage](https://www.notion.so/meilisearch/Composite-embedder-usage-14a4b06b651f81859dc3df21e8cd02a0)
- Refactor the way we check if a parameter is mandatory/allowed/disallowed for a given source
- Take the "nesting context" into account for computer if a parameter is mandatory/allowed/disallowed
- Add tests checking all parameters with all sources, and made sure the results didn't change compared with v1.13

## Dumpless Upgrade

- This adds a new value for an existing parameter => compatible without change
- This adds new optional parameters => compatible without change

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-03-05 17:18:11 +00:00
Kerollmops
ba30747de3
Bump v1.13.2 to v1.13.3 in the TOMLs and snaps 2025-03-05 15:24:53 +01:00
Kerollmops
c8c0951c43
Update the snapshots 2025-03-05 15:24:21 +01:00
meili-bors[bot]
683a2ac685
Merge #5379
5379: Bring back the changes from v1.13.2 into main r=dureuill a=Kerollmops



Co-authored-by: Kerollmops <Kerollmops@users.noreply.github.com>
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-03-04 13:24:25 +00:00
ManyTheFish
1994494155 Update snapshot using the new filterableAttributes type 2025-03-03 10:33:39 +01:00
ManyTheFish
9a75dc6ab3 Update tests using filterable attributes rules
**Changes:**
Replace the BTreeSet<String> by Vec<FilterableAttributesRule> without changing the test results

**Impact:**
- None
2025-03-03 10:33:34 +01:00
meili-bors[bot]
c63c25a9a2
Merge #5355
5355: Support fetching the pooling method from the model configuration r=Kerollmops a=dureuill

# Pull Request

## Related issue
Fixes #5354 

## What does this PR do?
- Fetches the pooling configuration from the model repository
- Use a pooling method that depends on the pooling configuration of that model.
- Allow overriding the pooling method with a new huggingFace embedder parameter `pooling`
  - for backward-compatibility with Meilisearch v1.13
  - for compatibility with embedders that exhibit the same behavior as Meilisearch v1.13
- Handle the default value of that new parameter
   - for compatibility, when importing a db/a dump, it should be set to `forceMean`
   - when (re)set from the settings for an embedder, it should be set to `useModel`


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-02-27 14:55:13 +00:00
Louis Dureuil
754f254a00
Update snapshots following version bump 2025-02-27 15:17:23 +01:00
ManyTheFish
5d421abdc4
Update Snapshots 2025-02-26 17:01:37 +01:00
ManyTheFish
9a6c1730aa
Add document database stats 2025-02-26 17:01:25 +01:00
Kerollmops
025b9b79bb
Update the snapshots 2025-02-26 17:01:21 +01:00
Kerollmops
dfce20be21
Rename callTrace into progressTrace 2025-02-25 10:09:03 +01:00
Louis Dureuil
e374b095a2
Fix tests 2025-02-24 14:11:26 +01:00
Louis Dureuil
4a2643daa2
Rename embed_one to embed_search and embed_chunks* to embed_index* 2025-02-24 13:58:26 +01:00
Kerollmops
243a5fa6a8
Log the call trace and congestion 2025-02-20 14:17:34 +01:00
Kerollmops
05cc8c650c
Expose the write channel congestion in the batches 2025-02-19 15:47:54 +01:00
Kerollmops
3ff1de0a21
Expose the call trace in the batch stats 2025-02-19 11:24:11 +01:00
Kerollmops
11a11fc870
Accumulate step durations from the progress system 2025-02-18 18:33:19 +01:00
Louis Dureuil
cd0dfa3f1b
Fix test cases 2025-02-18 17:21:52 +01:00
meili-bors[bot]
0f1aeb8eaa
Merge #5351
5351: Bring back v1.13.0 changes into main r=irevoire a=Kerollmops

This PR brings back the changes made in v1.13 into the main branch.

Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
Co-authored-by: Clémentine <clementine@meilisearch.com>
Co-authored-by: meili-bors[bot] <89034592+meili-bors[bot]@users.noreply.github.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
2025-02-18 08:05:02 +00:00
ManyTheFish
49e9655c24 Update snapshots 2025-02-12 14:05:32 +01:00
Tamo
84e2a1f836 rename the atomic to something more meaningful 2025-02-11 11:14:49 +01:00
Tamo
fa00b42c93 fix the missing batch in the dumps in meilisearch and meilitools 2025-02-11 11:14:49 +01:00
Tamo
45f843ccb9 fmt 2025-02-10 10:46:42 +01:00
Louis Dureuil
e34afca6d7
Support network in dumps 2025-02-05 15:03:16 +01:00
Kerollmops
a6f9e0ddf0
Fix auto batching related tests 2025-02-03 10:34:07 +01:00
Kerollmops
8e6893ddbe
Make sure we correctly mix different document operations 2025-02-03 10:34:06 +01:00
Kerollmops
d018346f18
Make the auto-batcher batche replacement with updates 2025-02-03 10:34:05 +01:00
Tamo
bdd3005d10
Log the progress when a batch fails 2025-01-29 16:36:23 +01:00
Tamo
8439aeb7cf
improve error message in case of unexpected panic while processing tasks 2025-01-29 11:51:06 +01:00
Tamo
ef47a0d820
apply review comment 2025-01-28 16:53:50 +01:00
Tamo
58f90b70c7
store the enqueued at to eases the batch deletion 2025-01-28 16:53:50 +01:00
Tamo
508db9020d
update the snapshots 2025-01-28 16:53:50 +01:00
Kerollmops
6ff37c6fc4
Fix the insta snapshots 2025-01-28 16:53:50 +01:00
Kerollmops
f21ae1f5d1
Remove the batch id from the date time databases 2025-01-28 16:53:50 +01:00
Tamo
787472453d
write the version of the index while upgrading it 2025-01-23 16:51:24 +01:00
Tamo
8f65f35de9
rewrite part of the index-scheduler upgrade test 2025-01-23 16:51:23 +01:00
Tamo
c27c923439
introduce a trait to upgrade the indexes 2025-01-23 16:51:23 +01:00
Tamo
fd5649091d
add the upgradeTo field in the details 2025-01-23 16:51:23 +01:00
Tamo
7740997ea8
reintroduce the unrecoverable error and use it where its supposed to be used 2025-01-23 16:51:22 +01:00
Tamo
27bf2f1298
remove the empty progress made for the upgrade database 2025-01-23 16:51:22 +01:00
Tamo
d4d82fbd0c
commit the index wtxn before the index-scheduler wtxn 2025-01-23 16:51:21 +01:00
Tamo
b132d70413
fix the details in all cases 2025-01-23 16:51:21 +01:00