369 Commits

Author SHA1 Message Date
Kerollmops
76968179de
Fix the upgrade arroy calls 2025-03-11 15:33:43 +01:00
Kerollmops
7d2f26a6db
Move to the latest version of arroy 2025-03-11 15:33:43 +01:00
Kerollmops
39c58d8085
Dumpless upgrade from v1.13 to v1.14 2025-03-11 15:08:07 +01:00
Kerollmops
c1fa3e81a0
Use the new arroy upgrade method to move from 0.4 to 0.5 2025-03-11 14:52:46 +01:00
Kerollmops
c63dae8267
WIP: Still need to introduce a Env::copy_to_path method 2025-03-10 16:33:35 +01:00
Kerollmops
b9da33fcbd
Specify WithoutTls everywhere 2025-03-10 15:41:53 +01:00
Kerollmops
26f0f1d638
Fix the error variants 2025-03-10 15:41:32 +01:00
Kerollmops
0a4dbea30d
Open Env without TLS 2025-03-10 15:41:11 +01:00
Kerollmops
1740940809
Bump heed and other dependencies 2025-03-10 15:40:23 +01:00
meili-bors[bot]
3fd86e8d76
Merge #5371
5371: Composite embedders r=irevoire a=dureuill

# Pull Request

## Related issue
Fixes #5343 

## What does this PR do?
- Implement [public usage](https://www.notion.so/meilisearch/Composite-embedder-usage-14a4b06b651f81859dc3df21e8cd02a0)
- Refactor the way we check if a parameter is mandatory/allowed/disallowed for a given source
- Take the "nesting context" into account for computer if a parameter is mandatory/allowed/disallowed
- Add tests checking all parameters with all sources, and made sure the results didn't change compared with v1.13

## Dumpless Upgrade

- This adds a new value for an existing parameter => compatible without change
- This adds new optional parameters => compatible without change

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-03-05 17:18:11 +00:00
Louis Dureuil
4fab72cbea
Rename SettingsDiff::diff to SettingsDiff::apply_and_diff 2025-03-05 18:16:57 +01:00
Louis Dureuil
afb4b9677f
Remove Embedder:embed 2025-03-05 18:16:57 +01:00
Louis Dureuil
73d2dbd60f
Error handling 2025-03-05 18:16:57 +01:00
meili-bors[bot]
683a2ac685
Merge #5379
5379: Bring back the changes from v1.13.2 into main r=dureuill a=Kerollmops



Co-authored-by: Kerollmops <Kerollmops@users.noreply.github.com>
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-03-04 13:24:25 +00:00
meili-bors[bot]
c63c25a9a2
Merge #5355
5355: Support fetching the pooling method from the model configuration r=Kerollmops a=dureuill

# Pull Request

## Related issue
Fixes #5354 

## What does this PR do?
- Fetches the pooling configuration from the model repository
- Use a pooling method that depends on the pooling configuration of that model.
- Allow overriding the pooling method with a new huggingFace embedder parameter `pooling`
  - for backward-compatibility with Meilisearch v1.13
  - for compatibility with embedders that exhibit the same behavior as Meilisearch v1.13
- Handle the default value of that new parameter
   - for compatibility, when importing a db/a dump, it should be set to `forceMean`
   - when (re)set from the settings for an embedder, it should be set to `useModel`


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-02-27 14:55:13 +00:00
Louis Dureuil
5e7f226ac9
Support dumpless upgrade for all v1.13 patches 2025-02-27 15:17:23 +01:00
ManyTheFish
d4063c9dcd
Fix fmt 2025-02-26 17:02:45 +01:00
Many the fish
abebc574f6
Update crates/milli/src/index.rs
Co-authored-by: Tamo <tamo@meilisearch.com>
2025-02-26 17:02:45 +01:00
Many the fish
f32ab67819
Update crates/milli/src/index.rs
Co-authored-by: Tamo <tamo@meilisearch.com>
2025-02-26 17:02:44 +01:00
ManyTheFish
d25953f322
fix clippy 2025-02-26 17:02:43 +01:00
ManyTheFish
405bbd04c1
Dumpless upgrade 2025-02-26 17:01:38 +01:00
ManyTheFish
9f3663e768
Implement Incremental document database stats computing 2025-02-26 17:01:35 +01:00
ManyTheFish
d9642ec916
Use checked_div in average computation 2025-02-26 17:01:34 +01:00
ManyTheFish
818e8b0237
Fix zero division 2025-02-26 17:01:31 +01:00
ManyTheFish
4f77a7fba5
fix clippy 2025-02-26 17:01:29 +01:00
ManyTheFish
9a6c1730aa
Add document database stats 2025-02-26 17:01:25 +01:00
ManyTheFish
15788773af
Check the exact_word database when computing zero typo query 2025-02-26 17:01:22 +01:00
Louis Dureuil
24fe6cd205
Fix multiple embeddings in hf 2025-02-24 16:24:04 +01:00
Louis Dureuil
e374b095a2
Fix tests 2025-02-24 14:11:26 +01:00
Louis Dureuil
9f3e4801b1
Refactor settings validation and introduce SubEmbedderSettings 2025-02-24 13:58:26 +01:00
Louis Dureuil
b85180fedb
Error types 2025-02-24 13:58:26 +01:00
Louis Dureuil
294cf39cad
Integrate composite embedder 2025-02-24 13:58:26 +01:00
Louis Dureuil
4a2643daa2
Rename embed_one to embed_search and embed_chunks* to embed_index* 2025-02-24 13:58:26 +01:00
Louis Dureuil
8d2d9066ba
Add composite embedder 2025-02-24 13:58:26 +01:00
Louis Dureuil
526476e168
Move settings test to its own file 2025-02-24 13:58:26 +01:00
Kerollmops
76fd5d92d7
Clarify the tail writing to database 2025-02-20 17:35:23 +01:00
Kerollmops
245a55722a
Remove commented code 2025-02-20 16:48:18 +01:00
Kerollmops
05cc8c650c
Expose the write channel congestion in the batches 2025-02-19 15:47:54 +01:00
Louis Dureuil
14e1459bf5
Document settings 2025-02-19 15:06:22 +01:00
Kerollmops
e9add14189
Reorder steps 2025-02-18 19:26:41 +01:00
Kerollmops
4a058a080e
Simplify the name generation 2025-02-18 18:48:44 +01:00
Kerollmops
11a11fc870
Accumulate step durations from the progress system 2025-02-18 18:33:19 +01:00
Louis Dureuil
7b4ce468a6
Allow overriding pooling method 2025-02-18 17:12:23 +01:00
Louis Dureuil
11759c4be4
Support pooling 2025-02-18 16:10:51 +01:00
meili-bors[bot]
0f1aeb8eaa
Merge #5351
5351: Bring back v1.13.0 changes into main r=irevoire a=Kerollmops

This PR brings back the changes made in v1.13 into the main branch.

Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
Co-authored-by: Clémentine <clementine@meilisearch.com>
Co-authored-by: meili-bors[bot] <89034592+meili-bors[bot]@users.noreply.github.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
2025-02-18 08:05:02 +00:00
meili-bors[bot]
885710a07b
Merge #5341
5341: Embeddings stats r=ManyTheFish a=ManyTheFish

# Pull Request

## Related issue
Fixes #5321

## What does this PR do?
- Add embedding stats
- force dumpless upgrade to recompute stats
- add tests


Co-authored-by: ManyTheFish <many@meilisearch.com>
2025-02-12 15:46:37 +00:00
ManyTheFish
c55fdad2c3 Fix dumpless upgrade target version 2025-02-12 16:35:05 +01:00
ManyTheFish
8419ed52a1 fix clippy 2025-02-12 14:38:51 +01:00
Louis Dureuil
8e0d8d31f9
Add back timeout from v1.11.3 2025-02-12 11:53:00 +01:00
ManyTheFish
bd27fe7d02 force dumpless upgrade to recompute stats 2025-02-12 11:45:02 +01:00