5223: Limit batched tasks total size r=curquiza a=Kerollmops
Introduce a new engine parameter (env and config, too) to limit the maximum payload size processed by the engine in batches. You can [review the Discussion and usage on GitHub](https://github.com/orgs/meilisearch/discussions/801).
Co-authored-by: Clément Renault <clement@meilisearch.com>
4867: Autogenerate the openAPI spec r=irevoire a=irevoire
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/5073
## What does this PR do?
- Introduce utoipa and the auto-generation of the openAPI file
- Introduce the scalar swagger when the `swagger` feature flag is enabled.
Generating the openAPI file takes between 15 and 20ms at startup time on my computer. That could be an issue if we plan to stabilize the feature.
Co-authored-by: Tamo <tamo@meilisearch.com>
5168: Refactor indexer r=ManyTheFish a=dureuill
# Pull Request
Split the indexer mod into multiple submodules.
This restores the ability of rustfmt to format the file 🎉
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
5166: fix list indexes r=dureuill a=irevoire
# Pull Request
### Smol benchmark on a meilisearch with 1009 indexes:
**Before** this PR on my computer, it was taking 5.5s to call the `GET /indexes` route on a cold computer where all the indexes were closed.
**After** this PR it takes 0.009s to call the route on the first 20 indexes, and 0.176 for the last 20 indexes (retrieving the first or last indexes on main has no impact on performances).
If my computations are right, that's between 61111.1% and 3125% faster on this test 😂
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4694
## What does this PR do?
- Add the primary key to the cache we already have in the index-mapper
- Provide a new route to retrieve the paginated indexes straight from the cache without opening them
- Fix a bug where the cache was not computed when loading a dump and was forcing us to open the indexes to compute their stats on the fly
## Is it breaking?
Since the field I added is an `Option` I think we should consider it as non-breaking and let it update itself automatically on the next operation of this index.
I also tested to run my patch over a DB generated on release-v1.12.0 and it works. The importing a dump also works.
Co-authored-by: Tamo <tamo@meilisearch.com>
5178: Add Prometheus metrics to measure task queue latency r=irevoire a=takaebato
# Pull Request
## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/5046
## What does this PR do?
- Added Prometheus metrics to measure task queue latency
(Confirmed locally that latency is measured during parallel task execution in the benchmark.)
## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?
Thank you so much for contributing to Meilisearch!
Co-authored-by: Takahiro Ebato <takahiro.ebato@gmail.com>
