431 Commits

Author SHA1 Message Date
Louis Dureuil
1cc6cd78e0
Fix uselessly deep stack trace 2025-01-09 10:36:37 +01:00
Louis Dureuil
c204afdc79
Update snapshot 2025-01-09 10:36:37 +01:00
Louis Dureuil
c14967eeac
Use new incremental facet indexing and enable sanity checks in debug 2025-01-09 10:36:35 +01:00
Louis Dureuil
f38db86120
Add new incremental facet indexing 2025-01-09 10:24:36 +01:00
Louis Dureuil
50b155fa2d
add valid_facet_value utility function 2025-01-09 10:24:36 +01:00
Louis Dureuil
a533c8e041
Add sanity checks for facet values 2025-01-09 10:24:36 +01:00
Tamo
908adee6fc
Fix the addition of empty payload 2025-01-09 10:24:36 +01:00
Tamo
7b3353252f
update the test to ensure it works when specifying the primary key or not: it doesn't work 2025-01-09 10:24:35 +01:00
Tamo
647a10bf18
stop skipping empty tasks when adding documents 2025-01-09 10:24:34 +01:00
Tamo
08c332980b
add a test reproducing the bug 2025-01-09 10:12:12 +01:00
meili-bors[bot]
0e6b6bd130
Merge #4867
4867: Autogenerate the openAPI spec r=irevoire a=irevoire

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/5073

## What does this PR do?
- Introduce utoipa and the auto-generation of the openAPI file
- Introduce the scalar swagger when the `swagger` feature flag is enabled.

Generating the openAPI file takes between 15 and 20ms at startup time on my computer. That could be an issue if we plan to stabilize the feature.

Co-authored-by: Tamo <tamo@meilisearch.com>
2025-01-08 09:45:50 +00:00
meili-bors[bot]
b1b0b0b67c
Merge #5168
5168: Refactor indexer r=ManyTheFish a=dureuill

# Pull Request

Split the indexer mod into multiple submodules. 

This restores the ability of rustfmt to format the file 🎉

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-01-08 09:12:49 +00:00
Tamo
99f5e09a79
fix the tests 2025-01-07 16:42:53 +01:00
Tamo
a8ef6f08e0
Update crates/meilisearch-types/src/settings.rs
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-01-07 16:31:39 +01:00
Tamo
ae5a04e85c
apply review comments 2025-01-07 16:30:14 +01:00
Tamo
8ebfc9fa92
Update crates/meilisearch-types/src/settings.rs
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-01-07 16:30:14 +01:00
Tamo
21026f0ca8
move the swagger behind a feature flag 2025-01-07 16:30:14 +01:00
Tamo
e579554c84
fmt 2025-01-07 16:30:12 +01:00
Tamo
ff49250c1a
remove useless doc 2025-01-07 16:29:09 +01:00
Tamo
8b95c6ae56
improve the description of all the settings route 2025-01-07 16:29:09 +01:00
Tamo
28162759a4
fix imports after rebase 2025-01-07 16:29:08 +01:00
Tamo
dd128656cb
fix all the tests 2025-01-07 16:28:12 +01:00
Tamo
4456df5a46
fix some tests 2025-01-07 16:28:11 +01:00
Tamo
0b104b3efa
fix the list indexes 2025-01-07 16:26:06 +01:00
Tamo
ac944f0960
review all the return type 2025-01-07 16:26:06 +01:00
Tamo
5f55e88484
review all the parameters and tags 2025-01-07 16:26:06 +01:00
Tamo
aab6ffec30
fix and review all the documents route 2025-01-07 16:26:06 +01:00
Tamo
1dd33af8a3
add the batches 2025-01-07 16:26:06 +01:00
Tamo
8a2a1e4d27
add the experimental features route 2025-01-07 16:26:06 +01:00
Tamo
e2686c0fce
add the swap indexes 2025-01-07 16:26:06 +01:00
Tamo
9473a2a6ca
add the multi-search 2025-01-07 16:26:06 +01:00
Tamo
11ce3b9636
fix the settings 2025-01-07 16:26:06 +01:00
Tamo
0bf4157a75
try my best to make the sub-settings routes works, it doesn't 2025-01-07 16:26:06 +01:00
Tamo
4eaa626bca
add the similar route 2025-01-07 16:26:06 +01:00
Tamo
668b26b641
add the facet search 2025-01-07 16:26:06 +01:00
Tamo
04e4586fb3
add the searches route and fix a few broken things 2025-01-07 16:26:06 +01:00
Tamo
78f6f22a80
implement all the /indexes/documents route 2025-01-07 16:26:06 +01:00
Tamo
13afdaf393
finish rebase and update utoipa to the latest version 2025-01-07 16:26:06 +01:00
Tamo
742d0ee531
Implements the get and delete tasks route 2025-01-07 16:26:04 +01:00
Louis Dureuil
4275833bab
Rename compute.rs to post_process.rs 2025-01-07 15:31:20 +01:00
Louis Dureuil
de7f8c4406
refactor indexer mod 2025-01-07 15:29:02 +01:00
Tamo
43bb02e7b4
split the autobatcher in two 2025-01-07 15:02:03 +01:00
Tamo
cb82b0798a
Split the index-scheduler in ~500 loc modules 2025-01-06 14:08:26 +01:00
meili-bors[bot]
baeefa4817
Merge #5166
5166: fix list indexes r=dureuill a=irevoire

# Pull Request

### Smol benchmark on a meilisearch with 1009 indexes:

**Before** this PR on my computer, it was taking 5.5s to call the `GET /indexes` route on a cold computer where all the indexes were closed.
**After** this PR it takes 0.009s to call the route on the first 20 indexes, and 0.176 for the last 20 indexes (retrieving the first or last indexes on main has no impact on performances).

If my computations are right, that's between 61111.1% and 3125% faster on this test 😂 

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4694

## What does this PR do?
- Add the primary key to the cache we already have in the index-mapper
- Provide a new route to retrieve the paginated indexes straight from the cache without opening them
- Fix a bug where the cache was not computed when loading a dump and was forcing us to open the indexes to compute their stats on the fly

## Is it breaking?

Since the field I added is an `Option` I think we should consider it as non-breaking and let it update itself automatically on the next operation of this index.
I also tested to run my patch over a DB generated on release-v1.12.0 and it works. The importing a dump also works.

Co-authored-by: Tamo <tamo@meilisearch.com>
2024-12-31 10:55:22 +00:00
Tamo
e8ba7833ec
Update crates/meilisearch/src/routes/indexes/mod.rs
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-12-31 10:43:22 +01:00
Tamo
db676aee73
Update crates/meilisearch/src/routes/indexes/mod.rs
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-12-31 10:43:12 +01:00
meili-bors[bot]
1a0d8810e5
Merge #5178
5178: Add Prometheus metrics to measure task queue latency r=irevoire a=takaebato

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/5046

## What does this PR do?

- Added Prometheus metrics to measure task queue latency

(Confirmed locally that latency is measured during parallel task execution in the benchmark.)

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Takahiro Ebato <takahiro.ebato@gmail.com>
2024-12-30 14:47:15 +00:00
Gnosnay
525e67ba93 Fix the format and linter error 2024-12-28 20:35:55 +08:00
Gnosnay
44eb153619 Replace hardcoded string with constants 2024-12-28 20:35:55 +08:00
Tamo
4eae92f411
fix list indexes 2024-12-26 18:48:25 +01:00