49 Commits

Author SHA1 Message Date
Tamo
4456df5a46
fix some tests 2025-01-07 16:28:11 +01:00
Tamo
0b104b3efa
fix the list indexes 2025-01-07 16:26:06 +01:00
Tamo
ac944f0960
review all the return type 2025-01-07 16:26:06 +01:00
Tamo
5f55e88484
review all the parameters and tags 2025-01-07 16:26:06 +01:00
Tamo
aab6ffec30
fix and review all the documents route 2025-01-07 16:26:06 +01:00
Tamo
1dd33af8a3
add the batches 2025-01-07 16:26:06 +01:00
Tamo
8a2a1e4d27
add the experimental features route 2025-01-07 16:26:06 +01:00
Tamo
e2686c0fce
add the swap indexes 2025-01-07 16:26:06 +01:00
Tamo
9473a2a6ca
add the multi-search 2025-01-07 16:26:06 +01:00
Tamo
11ce3b9636
fix the settings 2025-01-07 16:26:06 +01:00
Tamo
0bf4157a75
try my best to make the sub-settings routes works, it doesn't 2025-01-07 16:26:06 +01:00
Tamo
4eaa626bca
add the similar route 2025-01-07 16:26:06 +01:00
Tamo
668b26b641
add the facet search 2025-01-07 16:26:06 +01:00
Tamo
04e4586fb3
add the searches route and fix a few broken things 2025-01-07 16:26:06 +01:00
Tamo
78f6f22a80
implement all the /indexes/documents route 2025-01-07 16:26:06 +01:00
Tamo
13afdaf393
finish rebase and update utoipa to the latest version 2025-01-07 16:26:06 +01:00
Tamo
742d0ee531
Implements the get and delete tasks route 2025-01-07 16:26:04 +01:00
Tamo
cb82b0798a
Split the index-scheduler in ~500 loc modules 2025-01-06 14:08:26 +01:00
meili-bors[bot]
baeefa4817
Merge #5166
5166: fix list indexes r=dureuill a=irevoire

# Pull Request

### Smol benchmark on a meilisearch with 1009 indexes:

**Before** this PR on my computer, it was taking 5.5s to call the `GET /indexes` route on a cold computer where all the indexes were closed.
**After** this PR it takes 0.009s to call the route on the first 20 indexes, and 0.176 for the last 20 indexes (retrieving the first or last indexes on main has no impact on performances).

If my computations are right, that's between 61111.1% and 3125% faster on this test 😂 

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4694

## What does this PR do?
- Add the primary key to the cache we already have in the index-mapper
- Provide a new route to retrieve the paginated indexes straight from the cache without opening them
- Fix a bug where the cache was not computed when loading a dump and was forcing us to open the indexes to compute their stats on the fly

## Is it breaking?

Since the field I added is an `Option` I think we should consider it as non-breaking and let it update itself automatically on the next operation of this index.
I also tested to run my patch over a DB generated on release-v1.12.0 and it works. The importing a dump also works.

Co-authored-by: Tamo <tamo@meilisearch.com>
2024-12-31 10:55:22 +00:00
Tamo
e8ba7833ec
Update crates/meilisearch/src/routes/indexes/mod.rs
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-12-31 10:43:22 +01:00
Tamo
db676aee73
Update crates/meilisearch/src/routes/indexes/mod.rs
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-12-31 10:43:12 +01:00
meili-bors[bot]
1a0d8810e5
Merge #5178
5178: Add Prometheus metrics to measure task queue latency r=irevoire a=takaebato

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/5046

## What does this PR do?

- Added Prometheus metrics to measure task queue latency

(Confirmed locally that latency is measured during parallel task execution in the benchmark.)

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Takahiro Ebato <takahiro.ebato@gmail.com>
2024-12-30 14:47:15 +00:00
Tamo
4eae92f411
fix list indexes 2024-12-26 18:48:25 +01:00
meili-bors[bot]
d3491851bc
Merge #5187
5187: Bring back v1.12.0 of pre-release changes into `main` r=irevoire a=curquiza



Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: meili-bors[bot] <89034592+meili-bors[bot]@users.noreply.github.com>
Co-authored-by: Many the fish <many@meilisearch.com>
2024-12-23 10:59:33 +00:00
Takahiro Ebato
47827ca5c1 Add Prometheus metrics to measure task queue latency 2024-12-21 18:29:30 +09:00
Tamo
b004db37c7
fmt 2024-12-16 15:59:26 +01:00
Tamo
0c04cd1d9f
make clippy happy 2024-12-16 15:52:47 +01:00
Tamo
8cd3a1aa57
fmt 2024-12-11 18:18:40 +01:00
Kerollmops
5622b9607d
Wrap the read NDJSON pass into a tokio blocking 2024-12-11 12:18:36 +01:00
Kerollmops
69c931334f
Fix the error messages categorization with invalid NDJson 2024-12-11 12:02:48 +01:00
Kerollmops
d683f5980c
Do not duplicate NDJson when unecessary 2024-12-11 12:02:48 +01:00
Kushal Kumar
34254b42b6 refactor: use test configuration on import
Signed-off-by: Kushal Kumar <kushalkumargupta4@gmail.com>
2024-12-10 00:00:43 +05:30
Kushal Kumar
c0aa018c87 tests: split test in separate file
Signed-off-by: Kushal Kumar <kushalkumargupta4@gmail.com>
2024-12-08 00:32:32 +05:30
airycanon
b75f1f4c17 fix tests
# Conflicts:
#	crates/index-scheduler/src/batch.rs
#	crates/index-scheduler/src/snapshots/lib.rs/fail_in_process_batch_for_document_deletion/after_removing_the_documents.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_bad_primary_key/fifth_task_succeeds.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_bad_primary_key/fourth_task_fails.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_multiple_primary_key/second_task_fails.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_multiple_primary_key/third_task_fails.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_multiple_primary_key_batch_wrong_key/second_and_third_tasks_fails.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_set_and_null_primary_key_inference_works/all_other_tasks_succeeds.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_set_and_null_primary_key_inference_works/second_task_fails.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_set_and_null_primary_key_inference_works/third_task_succeeds.snap

# Conflicts:
#	crates/index-scheduler/src/batch.rs
#	crates/meilisearch/src/search/mod.rs
#	crates/meilisearch/tests/vector/mod.rs

# Conflicts:
#	crates/index-scheduler/src/batch.rs
2024-12-06 02:03:02 +08:00
airycanon
95ed079761 attach index name in errors
# Conflicts:
#	crates/index-scheduler/src/batch.rs

# Conflicts:
#	crates/index-scheduler/src/batch.rs
#	crates/meilisearch/src/search/mod.rs
2024-12-06 01:12:13 +08:00
ManyTheFish
90b428a8c3 Apply change requests 2024-11-28 15:16:13 +01:00
ManyTheFish
9f36ffcbdb Polish make_setting_routes! 2024-11-28 11:44:09 +01:00
ManyTheFish
68c4717e21 Change the settings tests and macros to avoid oversights 2024-11-28 11:34:35 +01:00
ManyTheFish
d66dc363ed Test and implement settings opt-out 2024-11-25 18:23:22 +01:00
Louis Dureuil
ba7500998e
Fix getting a single batch through the GET route 2024-11-21 17:59:31 +01:00
Tamo
b906e3ed70
improve the way we access the mutex 2024-11-20 10:51:06 +01:00
Tamo
d489f5635f
add the mapping between the task and batches 2024-11-20 10:49:23 +01:00
Tamo
a1251c3c83
Implements the get all batches route with filters working 2024-11-20 10:42:55 +01:00
Tamo
6062914654
add the batch_id to the tasks 2024-11-20 10:42:54 +01:00
Tamo
2eb1801e85 reverse the order of the task queue 2024-11-07 19:19:44 +01:00
Pedro Turik Firmino
8b95f5ccc6 Adds new metrics to prometheus: SearchQueue size, SearchQueue searches running, and Search Queue searches waiting. 2024-11-06 15:37:16 -03:00
Tamo
106cc7fe3a fmt 2024-11-04 17:51:40 +01:00
Tamo
cf6ad1ae5e Merge branch 'main' into tmp-release-v1.11.0 2024-11-04 16:14:44 +01:00
Clément Renault
9c1e54a2c8
Move crates under a sub folder to clean up the code 2024-10-21 08:18:43 +02:00