From 7c267a8a0ee0d39cc4ba5430dcc0bfefeb870e43 Mon Sep 17 00:00:00 2001 From: Tamo Date: Thu, 6 Mar 2025 15:50:29 +0100 Subject: [PATCH 1/4] update the issue template for the sprint issue --- .github/ISSUE_TEMPLATE/sprint_issue.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/.github/ISSUE_TEMPLATE/sprint_issue.md b/.github/ISSUE_TEMPLATE/sprint_issue.md index 84b8f1066..fa558487e 100644 --- a/.github/ISSUE_TEMPLATE/sprint_issue.md +++ b/.github/ISSUE_TEMPLATE/sprint_issue.md @@ -22,6 +22,18 @@ Related product discussion: +### Are you modifying a database? +- [ ] If not, add the `no db change` label to your PR, and you're good to go. +- [ ] If yes, add the `db change` label to your PR and you're good to go. + - [ ] /!\ Ensure all the read operations still work! + - If the change happened in milli, you may need to check the version of the database before doing any read operation + - If the change happened in the index-scheduler, make sure the new code can immediately read the old database + - If the change happened in the meilisearch-auth database, reach out to the team; we don't know yet how to handle these changes + - [ ] Write the code to go from the old database to the new one + - If the change happened in milli, the upgrade function should be written and called [here](https://github.com/meilisearch/meilisearch/blob/3fd86e8d76d7d468b0095d679adb09211ca3b6c0/crates/milli/src/update/upgrade/mod.rs#L24-L47) + - If the change happened in the index-scheduler, we've never done it yet, but the right place to do it should be [here](https://github.com/meilisearch/meilisearch/blob/3fd86e8d76d7d468b0095d679adb09211ca3b6c0/crates/index-scheduler/src/scheduler/process_upgrade/mod.rs#L13) + - [ ] Write an integration test [here](https://github.com/meilisearch/meilisearch/blob/main/crates/meilisearch/tests/upgrade/mod.rs) ensuring you can read the old database, upgrade to the new database, and read the new database as expected + ### Reminders when modifying the API - [ ] Update the openAPI file with utoipa: From 2ddc1d2258138ebead3427e25b0c156efdb9e8cd Mon Sep 17 00:00:00 2001 From: Tamo Date: Thu, 6 Mar 2025 16:30:48 +0100 Subject: [PATCH 2/4] update the CI to enforce the db change label on PR --- .github/workflows/db-change.yml | 28 ++++++++++++++++++++++++++++ bors.toml | 1 + 2 files changed, 29 insertions(+) create mode 100644 .github/workflows/db-change.yml diff --git a/.github/workflows/db-change.yml b/.github/workflows/db-change.yml new file mode 100644 index 000000000..dbd6bb82c --- /dev/null +++ b/.github/workflows/db-change.yml @@ -0,0 +1,28 @@ +name: Check db change labels + +on: + pull_request: + types: [opened, synchronize, reopened, labeled, unlabeled] + +env: + GH_TOKEN: ${{ secrets.MEILI_BOT_GH_PAT }} + +jobs: + check-labels: + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v2 + - name: Check db change labels + id: check_labels + run: | + URL=/repos/meilisearch/meilisearch/pulls/${{ github.event.pull_request.number }}/labels + echo ${{ github.event.pull_request.number }} + echo $URL + LABELS=$(gh api -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2022-11-28" /repos/meilisearch/meilisearch/issues/${{ github.event.pull_request.number }}/labels -q .[].name) + if [[ ! "$LABELS" =~ "db change" && ! "$LABELS" =~ "no db change" ]]; then + echo "::error::Pull request must contain either the 'db change' or 'no db change' label." + exit 1 + else + echo "The label is set" + fi diff --git a/bors.toml b/bors.toml index 3d04b834c..8671ab305 100644 --- a/bors.toml +++ b/bors.toml @@ -5,6 +5,7 @@ status = [ 'Run Clippy', 'Run Rustfmt', 'Run tests in debug', + 'Check db change labels', ] # 3 hours timeout timeout-sec = 10800 From 3928fb36b3be2779c5544b3784c2de982713aad1 Mon Sep 17 00:00:00 2001 From: Tamo Date: Wed, 19 Mar 2025 12:16:59 +0100 Subject: [PATCH 3/4] Introduce a second github action that post the right message when we declare there are db changes --- .github/workflows/db-change-comments.yml | 57 +++++++++++++++++++ .../{db-change.yml => db-change-missing.yml} | 0 2 files changed, 57 insertions(+) create mode 100644 .github/workflows/db-change-comments.yml rename .github/workflows/{db-change.yml => db-change-missing.yml} (100%) diff --git a/.github/workflows/db-change-comments.yml b/.github/workflows/db-change-comments.yml new file mode 100644 index 000000000..794142044 --- /dev/null +++ b/.github/workflows/db-change-comments.yml @@ -0,0 +1,57 @@ +name: Comment when db change labels are added + +on: + pull_request: + types: [labeled] + +env: + MESSAGE: | + ### Hello, I'm a bot 🤖 + + You are receiving this message because you declared that this PR make changes to the Meilisearch database. + Depending on the nature of the change, additional actions might be required on your part. The following sections detail the additional actions depending on the nature of the change, please copy the relevant section in the description of your PR, and make sure to perform the required actions. + + Thank you for contributing to Meilisearch :heart: + + ## This PR makes forward-compatible changes + + *Forward-compatible changes are changes to the database such that databases created in an older version of Meilisearch are still valid in the new version of Meilisearch. They usually represent additive changes, like adding a new optional attribute or setting.* + + - [ ] Detail the change to the DB format and why they are forward compatible + - [ ] Forward-compatibility: A database created before this PR and using the features touched by this PR was able to be opened by a Meilisearch produced by the code of this PR. + + + ## This PR makes breaking changes + + *Breaking changes are changes to the database such that databases created in an older version of Meilisearch need changes to remain valid in the new version of Meilisearch. This typically happens when the way to store the data changed (change of database, new required key, etc). This can also happen due to breaking changes in the API of an experimental feature. ⚠️ This kind of changes are more difficult to achieve safely, so proceed with caution and test dumpless upgrade right before merging the PR.* + + - [ ] Detail the changes to the DB format, + - [ ] which are compatible, and why + - [ ] which are not compatible, why, and how they will be fixed up in the upgrade + - [ ] /!\ Ensure all the read operations still work! + - If the change happened in milli, you may need to check the version of the database before doing any read operation + - If the change happened in the index-scheduler, make sure the new code can immediately read the old database + - If the change happened in the meilisearch-auth database, reach out to the team; we don't know yet how to handle these changes + - [ ] Write the code to go from the old database to the new one + - If the change happened in milli, the upgrade function should be written and called [here](https://github.com/meilisearch/meilisearch/blob/3fd86e8d76d7d468b0095d679adb09211ca3b6c0/crates/milli/src/update/upgrade/mod.rs#L24-L47) + - If the change happened in the index-scheduler, we've never done it yet, but the right place to do it should be [here](https://github.com/meilisearch/meilisearch/blob/3fd86e8d76d7d468b0095d679adb09211ca3b6c0/crates/index-scheduler/src/scheduler/process_upgrade/mod.rs#L13) + - [ ] Write an integration test [here](https://github.com/meilisearch/meilisearch/blob/main/crates/meilisearch/tests/upgrade/mod.rs) ensuring you can read the old database, upgrade to the new database, and read the new database as expected + + +jobs: + add-comment: + runs-on: ubuntu-latest + if: github.event.label.name == 'db change' + steps: + - name: Add comment + uses: actions/github-script@v6 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + const message = process.env.MESSAGE; + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: message + }) diff --git a/.github/workflows/db-change.yml b/.github/workflows/db-change-missing.yml similarity index 100% rename from .github/workflows/db-change.yml rename to .github/workflows/db-change-missing.yml From 5b51e8a08361092c3d3525e96dc789a136dabbbf Mon Sep 17 00:00:00 2001 From: Tamo Date: Wed, 19 Mar 2025 12:37:14 +0100 Subject: [PATCH 4/4] simplify the sprint issue to only tell you to add a label on your PR --- .github/ISSUE_TEMPLATE/sprint_issue.md | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/sprint_issue.md b/.github/ISSUE_TEMPLATE/sprint_issue.md index fa558487e..30b5e16ff 100644 --- a/.github/ISSUE_TEMPLATE/sprint_issue.md +++ b/.github/ISSUE_TEMPLATE/sprint_issue.md @@ -23,16 +23,8 @@ Related product discussion: ### Are you modifying a database? -- [ ] If not, add the `no db change` label to your PR, and you're good to go. -- [ ] If yes, add the `db change` label to your PR and you're good to go. - - [ ] /!\ Ensure all the read operations still work! - - If the change happened in milli, you may need to check the version of the database before doing any read operation - - If the change happened in the index-scheduler, make sure the new code can immediately read the old database - - If the change happened in the meilisearch-auth database, reach out to the team; we don't know yet how to handle these changes - - [ ] Write the code to go from the old database to the new one - - If the change happened in milli, the upgrade function should be written and called [here](https://github.com/meilisearch/meilisearch/blob/3fd86e8d76d7d468b0095d679adb09211ca3b6c0/crates/milli/src/update/upgrade/mod.rs#L24-L47) - - If the change happened in the index-scheduler, we've never done it yet, but the right place to do it should be [here](https://github.com/meilisearch/meilisearch/blob/3fd86e8d76d7d468b0095d679adb09211ca3b6c0/crates/index-scheduler/src/scheduler/process_upgrade/mod.rs#L13) - - [ ] Write an integration test [here](https://github.com/meilisearch/meilisearch/blob/main/crates/meilisearch/tests/upgrade/mod.rs) ensuring you can read the old database, upgrade to the new database, and read the new database as expected +- [ ] If not, add the `no db change` label to your PR, and you're good to merge. +- [ ] If yes, add the `db change` label to your PR. You'll receive a message explaining you what to do. ### Reminders when modifying the API