84 Commits

Author SHA1 Message Date
Tamo
381b2f1504
Add the new tasks with most of the job done 2025-01-14 14:56:13 +01:00
meili-bors[bot]
e568dbbabb
Merge #5182
5182: Remove hard coded task ids to prevent flaky tests r=irevoire a=mhmoudr

# Pull Request

## Related issue
Fixes partial #4840

## What does this PR do?
- Mainly scan the test code for any hard coded task Id and replace it by the returned task Id once the action or task is performed on an index.
- PS: _PR is not split by files as it has one theme applied to all tests which make it easy to review_ 


## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Mahmoud Rawas <mhmoudr@gmail.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
2025-01-13 15:18:55 +00:00
Tamo
8ff15b3dfb
fix the tests 2025-01-13 16:17:50 +01:00
meili-bors[bot]
cc4aca78c4
Merge #5220
5220: Merge back changes of v1.12.2 in main r=dureuill a=dureuill



Co-authored-by: curquiza <curquiza@users.noreply.github.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: dureuill <dureuill@users.noreply.github.com>
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2025-01-13 10:54:36 +00:00
Tamo
7b3353252f
update the test to ensure it works when specifying the primary key or not: it doesn't work 2025-01-09 10:24:35 +01:00
Tamo
647a10bf18
stop skipping empty tasks when adding documents 2025-01-09 10:24:34 +01:00
Tamo
08c332980b
add a test reproducing the bug 2025-01-09 10:12:12 +01:00
Clément Renault
71e5605daa
Make clippy happy 2025-01-08 18:24:39 +01:00
Mahmoud Rawas
27155f845c adding back a missing task wait. 2025-01-08 15:36:10 +11:00
Mahmoud Rawas
c6f14279d7 remove unused imports. 2025-01-08 15:11:34 +11:00
Tamo
dd128656cb
fix all the tests 2025-01-07 16:28:12 +01:00
Mahmoud Rawas
0625d08e4e adding a function toextract batch_uid from json and modifying the get_batch interface for easier call - did not work, so falling back to hard coded batch id for now. 2025-01-07 12:07:33 +11:00
Mahmoud Rawas
9269086fda fixing a rebase issue 2025-01-07 11:48:09 +11:00
Mahmoud Rawas
98e3ecb86b Format fixes after running: cargo +nightly fmt 2025-01-07 11:16:37 +11:00
Mahmoud Rawas
d7cb319217 #4840 - Partial fix - Confirm task success after waiting for it - continued, few missing cases - batch 2 2024-12-24 23:07:43 +11:00
Mahmoud Rawas
15062e7dba #4840 - Partial fix - Confirm task success after waiting for it - continued, few missing cases. 2024-12-24 23:06:07 +11:00
Mahmoud Rawas
bf19f86e38 #4840 - Partial fix - Confirm task success after waiting for it. 2024-12-24 23:06:07 +11:00
Mahmoud Rawas
91c7ef8723 #4840 - Partial fix - Remove hard coded task ids to prevent flaky tests.
# Conflicts:
#	crates/meilisearch/tests/documents/add_documents.rs
#	crates/meilisearch/tests/search/facet_search.rs
#	crates/meilisearch/tests/settings/get_settings.rs
#	crates/meilisearch/tests/snapshot/mod.rs
2024-12-24 23:05:59 +11:00
meili-bors[bot]
d3491851bc
Merge #5187
5187: Bring back v1.12.0 of pre-release changes into `main` r=irevoire a=curquiza



Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: meili-bors[bot] <89034592+meili-bors[bot]@users.noreply.github.com>
Co-authored-by: Many the fish <many@meilisearch.com>
2024-12-23 10:59:33 +00:00
Tamo
4bcdd7a9f9
fix the flaky batches test 2024-12-18 11:51:12 +01:00
ManyTheFish
18ce95dcbf Add test reproducing the bug 2024-12-12 14:56:45 +01:00
meili-bors[bot]
1fc90fbacb
Merge #5147
5147: Batch progress r=dureuill a=irevoire

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/5068

## What does this PR do?
- ...

## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [ ] Have you read the contributing guidelines?
- [ ] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Tamo <tamo@meilisearch.com>
2024-12-12 09:15:54 +00:00
Tamo
ab75f53efd
update all snapshots 2024-12-11 16:25:02 +01:00
Louis Dureuil
d075be798a
Fix tests 2024-12-10 13:39:07 +01:00
Louis Dureuil
7cf6707ed3
Extend test to add the ==512 bytes case 2024-12-10 11:05:42 +01:00
airycanon
b75f1f4c17 fix tests
# Conflicts:
#	crates/index-scheduler/src/batch.rs
#	crates/index-scheduler/src/snapshots/lib.rs/fail_in_process_batch_for_document_deletion/after_removing_the_documents.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_bad_primary_key/fifth_task_succeeds.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_bad_primary_key/fourth_task_fails.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_multiple_primary_key/second_task_fails.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_multiple_primary_key/third_task_fails.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_multiple_primary_key_batch_wrong_key/second_and_third_tasks_fails.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_set_and_null_primary_key_inference_works/all_other_tasks_succeeds.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_set_and_null_primary_key_inference_works/second_task_fails.snap
#	crates/index-scheduler/src/snapshots/lib.rs/test_document_addition_with_set_and_null_primary_key_inference_works/third_task_succeeds.snap

# Conflicts:
#	crates/index-scheduler/src/batch.rs
#	crates/meilisearch/src/search/mod.rs
#	crates/meilisearch/tests/vector/mod.rs

# Conflicts:
#	crates/index-scheduler/src/batch.rs
2024-12-06 02:03:02 +08:00
meili-bors[bot]
26be5e0733
Merge #5123
5123: Fix batch details r=dureuill a=irevoire

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/5079
Fixes https://github.com/meilisearch/meilisearch/issues/5112

## What does this PR do?
- Make the processing tasks actually processing in the stats of the batch instead of enqueued
- Stop counting one extra task for all non-prioritized batches in the stats
- Add a test

Co-authored-by: Tamo <tamo@meilisearch.com>
2024-12-05 15:21:55 +00:00
ManyTheFish
a0a3b55700 Change error code 2024-12-05 14:48:29 +01:00
Tamo
214b51de87
try to fix the snapshot on demand flaky test 2024-12-05 14:45:54 +01:00
ManyTheFish
fc1df5793c fix tests 2024-12-04 14:35:20 +01:00
ManyTheFish
953a82ca04 Add new error message 2024-12-04 11:15:29 +01:00
Tamo
a439fa3e1a
While spamming the batches route we could see a processing batch becoming missing and then finished, this commit ensures the batches goes from processing to finished directly 2024-12-02 12:02:16 +01:00
ManyTheFish
90b428a8c3 Apply change requests 2024-11-28 15:16:13 +01:00
ManyTheFish
5383f41bba Polish test_setting_routes! 2024-11-28 12:04:21 +01:00
ManyTheFish
68c4717e21 Change the settings tests and macros to avoid oversights 2024-11-28 11:34:35 +01:00
meili-bors[bot]
d0b2c0a523
Merge #5091
5091: Settings opt out r=Kerollmops a=ManyTheFish

# Pull Request

Related PRD: https://www.notion.so/meilisearch/API-usage-Settings-to-opt-out-indexing-features-fff4b06b651f8108ade3f858aeb16b14?pvs=4

## Related issue
Fixes #4979 

- [x] Add setting opt-out
- [x] Add analytics
- [x] Add tests


Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Many the fish <many@meilisearch.com>
2024-11-26 15:50:28 +00:00
ManyTheFish
2e896f30a5 Fix PR comments 2024-11-26 16:06:33 +01:00
meili-bors[bot]
e241f91285
Merge #5062
5062: Fix bugs for v1.12 r=Kerollmops a=ManyTheFish

# Pull Request

## Related issue
Fixes #4984
Fixes https://github.com/meilisearch/meilisearch/issues/4974
Fixes [SDK test](https://github.com/meilisearch/meilisearch/actions/runs/11886701996/job/33118278794)
## What does this PR do?
- add 3 tests
- fix bugs

Co-authored-by: ManyTheFish <many@meilisearch.com>
2024-11-26 08:10:50 +00:00
ManyTheFish
d66dc363ed Test and implement settings opt-out 2024-11-25 18:23:22 +01:00
meili-bors[bot]
19e6f675b3
Merge #4900
4900: Indexer edition 2024 r=Kerollmops a=dureuill

This PR is implementing the indexer edition 2024, largely inspired by [the ideas from this blog post](https://blog.kerollmops.com/meilisearch-is-too-slow).

Fixes https://github.com/meilisearch/meilisearch/issues/4985

## Features
- Stream-first approach to reading documents.
- Minimum disk write operations.
- RAM usage-first approach to avoid modifying common bitmaps on disk but in memory.
- Reduced LMDB fragmentation by writing entries only once...
- ...computing the final version of the entries in parallel...
- ...and storing them in write-optimized data structures before sending them to the BTree (LMDB).
- Indexing in multiple transactions to improve large dataset support (dumps).


Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-11-21 16:19:10 +00:00
ManyTheFish
36962b943b First batch of PR comment 2024-11-21 16:38:11 +01:00
Tamo
1e694ae432
improve the count of the number of tasks in a batch 2024-11-20 17:48:26 +01:00
Louis Dureuil
bda2b41d11
update snaps after merge 2024-11-20 17:08:30 +01:00
Louis Dureuil
6e6acfcf1b
Merge branch 'main' into indexer-edition-2024 2024-11-20 16:59:58 +01:00
ManyTheFish
4d616f8794 Parse every attributes and filter before tokenization 2024-11-20 15:15:25 +01:00
ManyTheFish
35bbe1c2a2 Add failing test on settings changes 2024-11-20 14:48:12 +01:00
Tamo
a7ac590e9e
implements the reverse query parameter for the batches 2024-11-20 13:29:52 +01:00
Tamo
7e379b3d14
remove useless prints 2024-11-20 12:27:12 +01:00
Tamo
56eacd221f
update the tests after the rebase 2024-11-20 10:54:38 +01:00
Tamo
b906e3ed70
improve the way we access the mutex 2024-11-20 10:51:06 +01:00