Commit Graph

298 Commits

Author SHA1 Message Date
Tamo
2d9a055fb9 stops snapshotting in advance_till when we don't need to 2024-07-30 13:57:12 +02:00
Tamo
988552e178
add tests on the rest embedder 2024-07-24 14:34:17 +02:00
meili-bors[bot]
677ed6bbf6
Merge #4787
4787: Add index exists function in index_scheduler which stops opening indexes to only check if they exist. r=Kerollmops a=Karribalu

# Pull Request

## Related issue
Fixes #4784

## What does this PR do?
- Added index_exists function in the index_scheduler.
- Resolved opening indexes to only check if they exist.
- Made changes to existing tests to test this function.

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: karribalu <karri.balu123456@gmail.com>
2024-07-11 13:05:20 +00:00
Clément Renault
bcd0c5f5a4
Support DocumentEdition in dumps 2024-07-10 16:29:17 +02:00
Clément Renault
f07256971a
Fix tests 2024-07-10 16:28:14 +02:00
karribalu
47e526f5ea Add index exists function in index_scheduler 2024-07-08 22:27:10 +01:00
ManyTheFish
015d90a962 merge main 2024-07-01 11:50:36 +02:00
Louis Dureuil
27496354e2
Grow by 1TB instead of 1MB 2024-06-25 09:01:11 +02:00
Tamo
1693332cab Update arroy and always build the tree that need to be built 2024-06-24 10:14:03 +02:00
Louis Dureuil
3f212a8202
Update tests 2024-06-12 18:13:34 +02:00
Louis Dureuil
34fabed214
Add test for vector writeback 2024-06-12 17:09:34 +02:00
Louis Dureuil
fca9fe39b3
Update test snapshots 2024-06-12 14:50:55 +02:00
Tamo
3493093c4f add a batch of tests 2024-06-11 16:03:54 +02:00
Tamo
0502b17501 log the state of the index-scheduler in all failed tests 2024-06-10 10:52:49 +02:00
Tamo
ea61e5cbec makes clippy happy x2 2024-06-06 11:39:29 +02:00
Tamo
31a793d226 fix the regeneration of the embeddings in the search 2024-06-06 11:39:29 +02:00
Tamo
d85ab23b82 rename all occurences of user_defined to user_provided for consistency 2024-06-06 11:39:29 +02:00
Tamo
b7349910d9 implements mor review comments 2024-06-06 11:39:29 +02:00
Tamo
49fa41ce65 apply first round of review comments 2024-06-06 11:39:29 +02:00
Tamo
b867829ef1 remove useless dbg 2024-06-06 11:39:29 +02:00
Tamo
5d50850e12 always push the user defined vectors in arroy 2024-06-06 11:39:29 +02:00
Tamo
9eb6f522ea wraps the index embedding config in a struct 2024-06-06 11:37:30 +02:00
Tamo
04f6523f3c expose a new parameter to retrieve the embedders at search time 2024-06-06 11:36:11 +02:00
Tamo
30d66abf8d fix the test 2024-06-06 11:36:11 +02:00
Tamo
84e498299b Remove the vectors from the documents database 2024-06-06 11:36:11 +02:00
Clément Renault
dc949ab46a
Remove puffin usage 2024-05-27 15:59:14 +02:00
Louis Dureuil
eccbcf5130
Increase index-scheduler test timeouts 2024-05-21 14:59:08 +02:00
Louis Dureuil
9969f7a638
Add test on index-scheduler 2024-05-20 14:44:10 +02:00
Tamo
897d25780e update milli to latest version 2024-05-16 18:31:32 +02:00
ManyTheFish
bad46f88d6 Fix embedder test 2024-04-16 14:39:30 +02:00
meili-bors[bot]
78668584cd
Merge #4533
4533: Hide api key in settings and task queue r=dureuill a=dureuill

# Pull Request

See [Usage page](https://meilisearch.notion.site/v1-8-AI-search-API-usage-135552d6e85a4a52bc7109be82aeca42#117f5ff7b19f4d95bb3ae0005f6c6633)

## Motivation

See [slack discussion (internal link)](https://meilisearch.slack.com/archives/C06GQP7FQ6P/p1709804022298749)


## Changes

- The value of the `apiKey` parameter is now hidden in the settings and the details of the task queue.

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-03-28 16:02:53 +00:00
Bruno Casali
8f2606d79d
fixes typos 2024-03-27 14:26:47 -03:00
Louis Dureuil
9a95ed619d
Add tests 2024-03-26 10:36:56 +01:00
Tamo
f2f1367ec3
add a timeout to the webhook 2024-03-20 13:59:43 +01:00
Tamo
b130917933 add the content type in the webhook + improve the test 2024-03-05 11:22:29 +01:00
meili-bors[bot]
b005eb3289
Merge #4435
4435: Make update file deletion atomic r=Kerollmops a=irevoire

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4432
Fixes https://github.com/meilisearch/meilisearch/issues/4438 by adding the logs the user asked

## What does this PR do?
- Adds a bunch of logs to help debug this kind of issue in the future
- Delete the update files AFTER committing the update in the `index-scheduler` (thus, if a restart happens, we are able to re-process the batch successfully)
- Multi-thread the deletion of all update files.


Co-authored-by: Tamo <tamo@meilisearch.com>
2024-02-26 17:54:40 +00:00
Tamo
0562818c2a fix and remove the file-store hack of /dev/null 2024-02-26 13:59:41 +01:00
Tamo
36c27a18a1 implement the dry run ha parameter 2024-02-26 13:58:04 +01:00
Tamo
1eb1c043b5 disable the auto deletion of tasks when the ha mode is enabled 2024-02-26 13:58:04 +01:00
Tamo
eb25b07390 let you specify your task id 2024-02-26 13:56:31 +01:00
Tamo
066a7a3cde takes only one read transaction per thread 2024-02-26 10:43:04 +01:00
Tamo
91cdd502f8 When processing tasks, make the update file deletion atomic 2024-02-22 14:56:22 +01:00
Tamo
7ff722b72e
get rids of the log dependencies everywhere 2024-02-08 15:04:05 +01:00
Louis Dureuil
02e6c8a440
Add tracing to index-scheduler 2024-02-08 15:03:31 +01:00
meili-bors[bot]
1ccde9bf0b
Merge #4316
4316: Autobatch the task deletions r=curquiza a=irevoire

# Pull Request

## Related issue
Fix part of https://github.com/meilisearch/meilisearch-support/issues/69
Fix #4315 

## What does this PR do?
- Autobatch the task deletions

Co-authored-by: Tamo <tamo@meilisearch.com>
2024-01-15 17:54:50 +00:00
Tamo
b4d7d80ad9
autobatch the task deletions 2024-01-11 14:58:07 +01:00
Louis Dureuil
97bb1ff9e2
Move currently_updating_index to IndexMapper 2024-01-09 15:37:27 +01:00
Clément Renault
fa2b96b9a5
Add an Authorization Header along with the webhook calls 2023-12-19 12:18:45 +01:00
Tamo
4fb25b8782
fix clippy 2023-12-19 10:35:51 +01:00
Tamo
c83a33017e
stream and chunk the data 2023-12-19 10:35:51 +01:00