Commit Graph

10355 Commits

Author SHA1 Message Date
meili-bors[bot]
c7a3f80de6
Merge #4073
4073: Simplify Puffin report exports r=ManyTheFish a=Kerollmops

This PR changes how we export Puffin reports by directly writing them to disk when the `exportPuffinReports` [experimental feature is enabled](https://www.meilisearch.com/docs/learn/experimental/overview) on the `/experimental-features` route. It also adds more puffing logging to the deletion phase and grenad helpers. The puffin reports are identified by the date and time at which they are exported.

## Todo List
 - [x] Change the CLI flag to be an API experimental option.
 - [x] Create [a PRD for this experimental feature (private)](https://www.notion.so/meilisearch/Export-Puffin-Reports-091df151e71c4edfb7d72f4bf995b3ea).
 - [x] Create and complete [a product discussion](https://github.com/meilisearch/product/discussions/693) (copy/paste PROFILING markdown?).
 - [x] Update the _PROFILING.md_ markdown file instructions.
 - [x] Change the debug logs of the processing operation (visible in puffin viewer).

Co-authored-by: Clément Renault <clement@meilisearch.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
2023-10-16 15:48:15 +00:00
curquiza
029d4de043 Add webinar banner in README 2023-10-16 14:38:10 +02:00
meili-bors[bot]
549f1bcccf
Merge #4125
4125: Rename benchmark CI file to find it easily in the manifest list r=Kerollmops a=curquiza



Co-authored-by: curquiza <clementine@meilisearch.com>
2023-10-16 11:38:28 +00:00
bwbonanno
689ec7c7ad Make the experimental route /metrics activable via HTTP 2023-10-13 22:12:54 +00:00
Clément Renault
3655d4bdca
Move the puffin file export logic into the run function 2023-10-13 13:11:30 +02:00
Clément Renault
055ca3935b
Update index-scheduler/src/batch.rs
Co-authored-by: Tamo <tamo@meilisearch.com>
2023-10-13 13:11:30 +02:00
Kerollmops
1b8871a585
Make cargo insta happy 2023-10-13 13:11:30 +02:00
Kerollmops
bf8fac6676
Fix the tests 2023-10-13 13:11:30 +02:00
Kerollmops
f2a9e1ebbb
Improve the debugging experience in the puffin reports 2023-10-13 13:11:30 +02:00
Kerollmops
c45c6cf54c
Update the PROFILING.md file 2023-10-13 13:11:30 +02:00
Kerollmops
513e61e9a3
Remove the experimental CLI flag 2023-10-13 13:11:29 +02:00
Kerollmops
90a626bf80
Use the runtime feature to enable puffin report exporting 2023-10-13 13:11:29 +02:00
Kerollmops
0d4acf2daa
Fix the metrics product URL 2023-10-13 13:11:29 +02:00
Kerollmops
58db8d85ec
Add the exportPuffinReports option to the runtime features route 2023-10-13 13:11:29 +02:00
Clément Renault
62dfd09dc6
Add more puffin logs to the deletion functions 2023-10-13 13:11:09 +02:00
Clément Renault
656dadabea
Expose an experimental flag to write the puffin reports to disk 2023-10-13 13:11:09 +02:00
Clément Renault
c5f7893fbb
Remove the puffin http dependency 2023-10-13 13:11:08 +02:00
curquiza
8cf2ccf168 Rename benchmark CI file to find it easily in the manifest list 2023-10-12 18:41:26 +02:00
meili-bors[bot]
0913373a5e
Merge #4122
4122: Bring back changes from `release-v1.4.1` into `main` r=Kerollmops a=curquiza



Co-authored-by: curquiza <curquiza@users.noreply.github.com>
Co-authored-by: meili-bors[bot] <89034592+meili-bors[bot]@users.noreply.github.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Vivek Kumar <vivek.26@outlook.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
2023-10-12 15:57:47 +00:00
Clément Renault
1a7f1282af
Fix test to use new common Value type 2023-10-12 17:37:04 +02:00
Clément Renault
bc747aac3a
Cut the first 8 characters 2023-10-12 15:04:37 +02:00
Kerollmops
be92376ab3
Fix originating commit branch 2023-10-12 13:51:41 +02:00
Kerollmops
cf7e355735
Fix originating commit command 2023-10-12 13:12:53 +02:00
Clément Renault
5f09d89ad1
Fetch the whole git history when cloning 2023-10-12 12:25:26 +02:00
Clément Renault
6ecb26a3f8
Add more info on the commenting CI command 2023-10-12 11:54:56 +02:00
meili-bors[bot]
76c6f554d6
Merge #4101
4101: Bump webpki from 0.22.1 to 0.22.2 r=curquiza a=dependabot[bot]

Bumps [webpki](https://github.com/briansmith/webpki) from 0.22.1 to 0.22.2.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a href="https://github.com/briansmith/webpki/commits">compare view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=webpki&package-manager=cargo&previous-version=0.22.1&new-version=0.22.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/meilisearch/meilisearch/network/alerts).

</details>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-10-12 08:46:04 +00:00
meili-bors[bot]
f343ef5f2f
Merge #4108
4108: Fix bug where search with distinct attribute and no ranking, returns offset+limit hits r=curquiza a=vivek-26

# Pull Request

## Related issue
Fixes #4078 

## What does this PR do?
This PR - 
- Fixes bug where search with distinct attribute and no ranking, returns offset+limit hits.
- Adds unit and integration tests.

## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Vivek Kumar <vivek.26@outlook.com>
2023-10-12 07:51:29 +00:00
Kerollmops
96982a768a
Triggers for every type of issue_comment 2023-10-11 23:18:29 +02:00
meili-bors[bot]
fca78fbc46
Merge #4082
4082: Update sprint_issue.md r=curquiza a=curquiza

Following internal recent discussions

Co-authored-by: Clémentine U. - curqui <clementine@meilisearch.com>
2023-10-11 15:12:38 +00:00
meili-bors[bot]
67a678cfb6
Merge #4089
4089: Use a bufreader and bufwriter everytime there is a grenad<file> r=curquiza a=irevoire

# Pull Request
Wrap all the files we give to a grenad in a `BufReader` or `BufWriter`.

The dump import I tried in the issue went from 2h to 10 minutes on my machine.

I also ran a bunch of benchmarks on my machine, and we're faster by a few seconds everywhere but nothing huge.

-----

The one thing I’m afraid about is if we used to get the inner file in a grenad and then do a read right after without a seek at the beginning of the file or a reopen.
Since we now use a bufreader our read would return the bytes one buffer later and probably completely corrupt what we were supposed to read.

From what I see, it looks like it works, but I may have missed something, I don't know much about this part of the codebase.

This issue should not arise on the bufwriter, though, because if we're not able to write the content of the buffer I ensured that the `into_inner` of the bufwriter should return an internal error.

## Related issue
Fixes #4087


Co-authored-by: Tamo <tamo@meilisearch.com>
2023-10-11 14:27:00 +00:00
Vivek Kumar
d1331d8abf
add integration test for distinct search with no ranking 2023-10-11 19:12:56 +05:30
Vivek Kumar
19ba129165
add unit test for distinct search with no ranking 2023-10-11 19:02:27 +05:30
Vivek Kumar
d4da06ff47
fix bug where distinct search with no ranking returns offset+limit hits 2023-10-11 19:02:16 +05:30
Clément Renault
3e0471edae
Only trigger CI on created or edited comments 2023-10-11 15:15:15 +02:00
Clément Renault
432df03c4c
Use the correct base filename in the comment bench CI 2023-10-11 14:57:03 +02:00
Clément Renault
11958016dd
Force a small if to evoid triggering the CI every time 2023-10-11 14:27:51 +02:00
Kerollmops
63c250a04d
Do not use the GITHUB_REF variable 2023-10-11 13:05:54 +02:00
Clément Renault
06d8cd5b72
Make sure that we checkout on the right branch 2023-10-11 12:02:44 +02:00
Tamo
c0f2724c2d get rids of the new introduced error code in favor of an io::Error 2023-10-10 15:12:23 +02:00
Tamo
d772073dfa use a bufreader everytime there is a grenad<file> 2023-10-10 15:00:30 +02:00
meili-bors[bot]
8fe8ddea79
Merge #4112
4112: Update version for the next release (v1.4.1) in Cargo.toml r=curquiza a=meili-bot

⚠️ This PR is automatically generated. Check the new version is the expected one and Cargo.lock has been updated before merging.

Co-authored-by: curquiza <curquiza@users.noreply.github.com>
2023-10-10 09:05:10 +00:00
curquiza
8a95bf28e5 Update version for the next release (v1.4.1) in Cargo.toml 2023-10-10 09:01:45 +00:00
Clément Renault
c0fd3dffb8
Setup a Github Token env var 2023-10-09 18:04:49 +02:00
Clément Renault
c42fd5375f
Fix the git commands again 2023-10-09 17:36:19 +02:00
Clément Renault
b418c3a756
Use the PAT token instead 2023-10-09 16:52:04 +02:00
Clément Renault
1cde455758
Fix workflow CI 2023-10-09 16:30:46 +02:00
Clément Renault
ca19bae72f
Prefer using a action to manage commands 2023-10-09 14:56:41 +02:00
meili-bors[bot]
705878ff59
Merge #4102
4102: Introduce the first bot that shows benchmarks results r=curquiza a=Kerollmops

TBD

Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
2023-10-05 10:11:06 +00:00
Clément Renault
92c280d1c8
Update .github/workflows/trigger-benchmarks-on-message.yml
Co-authored-by: Clémentine U. - curqui <clementine@meilisearch.com>
2023-10-05 12:09:52 +02:00
Kerollmops
181e7a1e53
Introduce the first bot that triggers benchmarks 2023-10-05 12:05:38 +02:00