9206 Commits

Author SHA1 Message Date
Tamo
f70a615ed9
update the github discussion links 2024-02-08 15:04:05 +01:00
Tamo
7ff722b72e
get rids of the log dependencies everywhere 2024-02-08 15:04:05 +01:00
Tamo
bcf7909bba
add a profile_memory parameter disabled by default 2024-02-08 15:04:05 +01:00
Tamo
ceb211c515
move the /logs route to the /logs/stream route 2024-02-08 15:04:05 +01:00
Clément Renault
f3c34d5b8c
Simplify MemoryStats fetching 2024-02-08 15:04:05 +01:00
Tamo
4de2db6786
add back the actix-web logs 2024-02-08 15:04:05 +01:00
Louis Dureuil
661baa716b
logs route profile mode: don't barf bytes if the buffer is not empty 2024-02-08 15:04:05 +01:00
Clément Renault
02dcaf07db
Replace the procfs by libproc 2024-02-08 15:04:05 +01:00
Louis Dureuil
d78ada07b5
spanstats: change field names 2024-02-08 15:04:05 +01:00
Louis Dureuil
bc097d90cb
tracing-trace: Spanstats deserializable + public fields 2024-02-08 15:04:05 +01:00
Clément Renault
b393823f36
Replace stats_alloc with procfs 2024-02-08 15:04:05 +01:00
Tamo
e773dfa9ba
get rids of log in milli and add logs for the bucket sort 2024-02-08 15:04:05 +01:00
Tamo
f158e96fe7
fix the auth 2024-02-08 15:04:05 +01:00
Tamo
e23ec4886d
fix the tests and add tests on the experimental features 2024-02-08 15:04:03 +01:00
Tamo
7793ba67a4
hide the route logs behind a feature flag 2024-02-08 15:03:33 +01:00
Tamo
80774148fd
handle and tests errors 2024-02-08 15:03:33 +01:00
Tamo
bf5cea8b10
add a test 2024-02-08 15:03:33 +01:00
Louis Dureuil
38e1c40f38
meilisearch: logs route disconnects in profile mode 2024-02-08 15:03:33 +01:00
Louis Dureuil
afc0585c1c
meilisearch: don't spawn a report everytime Meilisearch starts 2024-02-08 15:03:33 +01:00
Louis Dureuil
0e7a411d4d
tracing-trace: introduce TraceWriter, trace now only exposes the channel 2024-02-08 15:03:33 +01:00
Louis Dureuil
0f327f2821
tracing-trace: implement Error on Error 2024-02-08 15:03:33 +01:00
Tamo
77254765e8
get rids of env loggegr and fix the tests 2024-02-08 15:03:33 +01:00
Tamo
ce6e6ec2c5
stops profiling in a file by default 2024-02-08 15:03:32 +01:00
Louis Dureuil
91a8f74763
Add cancel log route 2024-02-08 15:03:32 +01:00
Tamo
abaa72e2bf
start handling reloads with profiling 2024-02-08 15:03:32 +01:00
Tamo
3c3a258a22
start exposing the profiling layer 2024-02-08 15:03:32 +01:00
Louis Dureuil
73e66d5a97
Add dummy log when calling tasks 2024-02-08 15:03:32 +01:00
Louis Dureuil
b8da117b9c
Simplify stream implementation 2024-02-08 15:03:32 +01:00
Louis Dureuil
5e52107474
better than before??? 2024-02-08 15:03:32 +01:00
Tamo
bcf1c4dae5
make it compile and runtime error 2024-02-08 15:03:32 +01:00
Tamo
50f84d43f5
init commit 2024-02-08 15:03:32 +01:00
Tamo
f76cc0806e
WIP: first draft at introducing a new log route 2024-02-08 15:03:32 +01:00
Tamo
2f1abd2c03
nelson is not used anymore 2024-02-08 15:03:32 +01:00
Louis Dureuil
dedc91e2cf
use json lines 2024-02-08 15:03:32 +01:00
Louis Dureuil
a61d8c59ff
Add span stats processor 2024-02-08 15:03:32 +01:00
Louis Dureuil
6e23040464
Use with tokio channel in Meilisearch 2024-02-08 15:03:32 +01:00
Louis Dureuil
8febbf64ce
Switch to tokio channel 2024-02-08 15:03:32 +01:00
Louis Dureuil
b141c82a04
Support Events in trace layer 2024-02-08 15:03:32 +01:00
Louis Dureuil
cc79cd0b04
Switch to a single view indicating current usage 2024-02-08 15:03:32 +01:00
Louis Dureuil
256538ccb9
Refactor memory handling and add markers 2024-02-08 15:03:31 +01:00
Clément Renault
ca8990394e
Remove the stats_alloc from the default features 2024-02-08 15:03:31 +01:00
Clément Renault
83fb2949c3
Give the allocator to the tracer when necessary 2024-02-08 15:03:31 +01:00
Louis Dureuil
6cf703387d
Format the bytes as human readable bytes
Uses the same `byte_unit` version as `meilisearch`
2024-02-08 15:03:31 +01:00
Clément Renault
771861599b
Logging the memory usage over time 2024-02-08 15:03:31 +01:00
Louis Dureuil
7e47cea0c4
Add tracing to Meilisearch 2024-02-08 15:03:31 +01:00
Louis Dureuil
5d7061682e
Add tracing to milli 2024-02-08 15:03:31 +01:00
Louis Dureuil
02e6c8a440
Add tracing to index-scheduler 2024-02-08 15:03:31 +01:00
Louis Dureuil
89401d097b
Add tracing-trace 2024-02-08 15:03:30 +01:00
meili-bors[bot]
72ebac1fbb
Merge #4388
4388: Cap the maximum memory of the grenad sorters r=curquiza a=Kerollmops

This PR clamps the memory usage of the grenad sorters to a reasonable maximum. Grenad sorters are opened on multiple threads at a time. This can result in higher memory usage than expected, even though it shouldn't consume more than the memory available.

Fixes #4152.

Co-authored-by: Clément Renault <clement@meilisearch.com>
2024-02-08 13:19:28 +00:00
meili-bors[bot]
a616a1d37b
Merge #4389
4389: Stabilize scoreDetails r=dureuill a=dureuill

# Pull Request

## Related issue
Fixes #4359

## What does this PR do?

### User standpoint

- Users no longer need to enable the `scoreDetails` experimental feature to use `showRankingScoreDetails` in search queries.
- ⚠️ **Breaking change**: sending an object containing the key `"scoreDetails"` to the `/experimental-features` route is now an error. However, importing a dump of a database where that feature was enabled completes successfully.

### Implementation standpoint
- remove `scoreDetails` from the experimental features
- remove check on the experimental feature `scoreDetails` before accepting `showRankingScoreDetails`
- remove `scoreDetails` from the accepted fields in the `/experimental-features` route
- fix tests accordingly

## Manual tests

1. exported a dump with the `scoreDetails` feature enabled on `main`
    - tried to import the dump after the changes in this PR
    - the dump imported successfully
2. tried to make a search with `showRankingScoreDetails: true`
    - the ranking score details are displayed
    - an automated test case also exists and passes
3. tried to enable the `scoreDetails` in `/experimental-features`
    - get error message 
      ```
       Unknown field `scoreDetails`: expected one of `vectorStore`, `metrics`, `exportPuffinReports`
      ```

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-02-08 10:40:00 +00:00