378 Commits

Author SHA1 Message Date
Kerollmops
02f468afb7
Remove span that is called too many times 2024-12-19 11:33:58 +01:00
Kerollmops
0784e8dddc
Add more spans to debug compression 2024-12-19 10:45:59 +01:00
Kerollmops
4232e522ea
Generate the dictionary only when necessary 2024-12-18 17:11:26 +01:00
Kerollmops
bfaebb50b2
Clean up the tests 2024-12-18 15:06:29 +01:00
Clément Renault
6cdae51626
Remove last warning by storing rtxn and compressor on each thread 2024-12-17 17:37:05 +01:00
Clément Renault
d2e628e3b5
Remove TODO and rely on the PR checklist 2024-12-17 17:18:15 +01:00
Clément Renault
cdd6c34d67
Fix some issues after a rebase 2024-12-17 17:16:41 +01:00
Clément Renault
7d75988a53
Fetch the compression dictionary only once to decompress documents 2024-12-17 17:10:15 +01:00
Clément Renault
c1dd489adc
Fix the usage of compressed documents 2024-12-17 17:10:15 +01:00
Clément Renault
b7ae720a7e
Compress and send compressed documents to the writer 2024-12-17 17:10:15 +01:00
Clément Renault
a466cf4f2c
Compress the right documents when a new dictionary is computed 2024-12-17 17:10:15 +01:00
Clément Renault
742709450d
Move the compression extractor into a dedicated module 2024-12-17 17:10:15 +01:00
Kerollmops
b7a9dbfdf8
Allocate the decompressed documents in the extractor allocator 2024-12-17 17:08:55 +01:00
Clément Renault
409f17b1a0
Squash in a single commit and rebase 2024-12-17 17:06:19 +01:00
meili-bors[bot]
ba11121cfc
Merge #5159
5159: Fix the New Indexer Spilling r=irevoire a=Kerollmops

Fix two bugs in the merging of the spilled caches. Thanks to `@ManyTheFish` and `@irevoire` 👏

Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: ManyTheFish <many@meilisearch.com>
2024-12-12 17:16:53 +00:00
ManyTheFish
acdd5aa6ea
Use the thread source id instead of the destination id
when filtering on the cache to merge
2024-12-12 18:12:00 +01:00
Kerollmops
2f3cc8cdd2
Fix the merge_caches_sorted function 2024-12-12 16:15:37 +01:00
meili-bors[bot]
7a95fed23f
Merge #5158
5158: Indexer edition 2024 fix facet fst r=Kerollmops a=ManyTheFish

# Pull Request
Fix a regression in the new indexer; when several filterable attributes containing strings were set, all the field IDs were shifted, and the last one was overwriting the previous FST.

## What does this PR do?
- Add a test reproducing the bug
- fix the bug

Co-authored-by: ManyTheFish <many@meilisearch.com>
2024-12-12 14:14:44 +00:00
ManyTheFish
961de4d34e Fix facet fst 2024-12-12 15:12:28 +01:00
ManyTheFish
18ce95dcbf Add test reproducing the bug 2024-12-12 14:56:45 +01:00
meili-bors[bot]
c177210b1b
Merge #5152
5152: Make xtasks be able to use the specified binary r=dureuill a=Kerollmops

Makes it possible to specify the binary to run. It is useful to run PGO optimized binaries.

Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
2024-12-12 12:28:16 +00:00
meili-bors[bot]
1fc90fbacb
Merge #5147
5147: Batch progress r=dureuill a=irevoire

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/5068

## What does this PR do?
- ...

## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [ ] Have you read the contributing guidelines?
- [ ] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Tamo <tamo@meilisearch.com>
2024-12-12 09:15:54 +00:00
Clément Renault
6c72559457
Update the binary-path description
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-12-12 09:39:39 +01:00
Kerollmops
1fdfa3f208
Change the exit code to 130 when Ctrl-Ced 2024-12-12 09:26:14 +01:00
Tamo
0d0c18f519
rename the Step::name into Step::current_step 2024-12-11 18:41:03 +01:00
Tamo
d12364c1e0
fix the tests 2024-12-11 18:30:48 +01:00
Tamo
8cd3a1aa57
fmt 2024-12-11 18:18:40 +01:00
Tamo
08fd026ebd
fix warning 2024-12-11 18:18:13 +01:00
Tamo
75d5cea624
use a with_capacity while allocating the progress view 2024-12-11 18:17:33 +01:00
Tamo
ab9213fa94
ensure we never write the progress to the db 2024-12-11 18:16:20 +01:00
Tamo
45d5d4bf40
make the progressview public 2024-12-11 18:15:33 +01:00
Tamo
fa885e75b4
rename the send_progress in progress 2024-12-11 18:13:12 +01:00
Tamo
29fc77ee5b
remove usuless print 2024-12-11 18:11:19 +01:00
Tamo
ad4dc70720
rename the ComputingTheChanges to ComputingDocumentChanges in the edit document progress 2024-12-11 18:09:54 +01:00
Tamo
5d682b4700
rename the ComputingTheChanges to ComputingDocumentChanges 2024-12-11 18:08:45 +01:00
Tamo
f1beb60204
make the progress use payload instead of documents 2024-12-11 18:07:45 +01:00
Tamo
85577e70cd
reuse the enqueued 2024-12-11 18:05:34 +01:00
Tamo
c5536c37b5
rename the atomic::name to unit_name 2024-12-11 18:03:06 +01:00
Tamo
9245c89cfe
move the macros to milli 2024-12-11 18:00:46 +01:00
meili-bors[bot]
eaabc1af2f
Merge #5144
5144: Exactly 512 bytes docid fails r=Kerollmops a=dureuill

# Pull Request

## Related issue
Fixes #5050 

## What does this PR do?
- Return a user error rather than an internal one for docids of exactly 512 bytes
- Fix up error message to indicate that exactly 512 bytes long docids are not supported.
- Fix up error message to reflect that index uids are actually limited to 400 bytes in length

## Impact

- Impacts docs: 
    - update [this paragraph](https://www.meilisearch.com/docs/learn/resources/known_limitations#length-of-primary-key-values) to say 511 bytes instead of 512 

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-12-11 15:41:05 +00:00
Kerollmops
04a24a9239
Kill Meilisearch with a TERM signal 2024-12-11 16:27:07 +01:00
Tamo
1f54dfa883
update the macro to look more like an enum 2024-12-11 16:26:09 +01:00
Tamo
786b0fabea
implement the progress for almost all the tasks 2024-12-11 16:26:08 +01:00
Tamo
26733c705d
add progress for the task deletion and task cancelation 2024-12-11 16:25:02 +01:00
Tamo
ab75f53efd
update all snapshots 2024-12-11 16:25:02 +01:00
Tamo
867e6a8f1d
rename the send_progress field to progress since it s not sending anything 2024-12-11 16:25:01 +01:00
Tamo
6f4823fc97
make the number of document in the document tasks more incremental 2024-12-11 16:25:01 +01:00
Tamo
df9b68f8ed
inital implementation of the progress 2024-12-11 16:25:01 +01:00
meili-bors[bot]
5bc6391700
Merge #5153
5153: Return docid in case of errors while rendering the document template r=Kerollmops a=dureuill

Improves error message:

Before: 

```
ERROR index_scheduler: Batch failed Index `mieli`: user error: missing field in document: liquid: Unknown index
  with:
    variable=doc
    requested index=title
    available indexes=by, id, kids, parent, text, time, type
```

After:

```
ERROR index_scheduler: Batch failed Index `mieli`: user error: missing field in document `11345147`: liquid: Unknown index
  with:
    variable=doc
    requested index=title
    available indexes=by, id, kids, parent, text, time, type
```

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-12-11 15:01:40 +00:00
Kerollmops
eaa897d983
Avoid compiling when unecessary 2024-12-11 15:57:16 +01:00