Commit Graph

3924 Commits

Author SHA1 Message Date
mlemesle
135499f398 Extract new env vars to const 2022-09-07 17:47:15 +02:00
mlemesle
ef3fa92536 Refactor default values for clap and serde 2022-09-07 16:58:03 +02:00
mlemesle
6520d3c474 Refactor build method and flag 2022-09-07 16:09:00 +02:00
mlemesle
403226a029 Add support for config file 2022-09-07 16:09:00 +02:00
bors[bot]
3a62e97c9f
Merge #2744
2744: Minor fixes in the just added update-version CI r=ManyTheFish a=curquiza

These fixes does not prevent us to use the current CI

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-06 13:14:06 +00:00
Clémentine Urquizar
55aa83d75a
Minor fixes in the just added update-version CI 2022-09-05 16:41:01 +02:00
bors[bot]
0c962f6a76
Merge #2740
2740: Update checkout v2 to v3 in CI manifests and use a unique GitHub PAT r=Kerollmops a=curquiza

Upgrade the missing checkout v2 into v3
Probably a bad merge conflicts that make them removed when merging `stable` into `main` after v0.28.0 release.

Also, use `MEILI_BOT_GH_PAT` instead of `PUBLISH_TOKEN` and the default github token, which allow us to remove useless GitHub secrets (once this PR is merged and v0.29.0 is release because `PUBLISH_TOKEN` is still used on `release-v0.29.0`)

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-05 12:14:40 +00:00
bors[bot]
2e0221cfc6
Merge #2739
2739: Add CI manifest to automate some steps when closing/creating a Milestone r=Kerollmops a=curquiza

For each Milestone created (not opened!), and if the release is NOT a patch release (only the patch changed)
- the roadmap issue is created, see the [template](https://github.com/meilisearch/core-team/blob/main/issue-templates/roadmap-issue.md)
- the changelog issue is created, see the [template](https://github.com/meilisearch/core-team/blob/main/issue-templates/changelog-issue.md)

For each Milestone closed
- the `vX.Y.Z` label is created
- this label is applied to all issues/PRs in the Milestone

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-05 11:42:10 +00:00
bors[bot]
2d34b239ab
Merge #2738
2738: Add missing env vars for dumps and snapshots features r=irevoire a=gmourier

# Pull Request

## What does this PR do?
Fixes #2721


## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
2022-09-05 11:12:09 +00:00
bors[bot]
55853815a5
Merge #2741
2741: Add CI to update the Meilisearch version in Cargo.toml files r=ManyTheFish a=curquiza

Add a CI we can trigger manually to create a PR updating the Meilisearch version
The next step is to create a Slack bot that will trigger this CI
In the meantime, we can trigger this CI manually in the [Actions tab](https://github.com/meilisearch/milli/actions)

The `MEILI_BOT_GH_PAT` secrets has been added to the organization level, and is accessible for the following repositories (so far): Meilisearch, Milli and Charabia

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-05 08:32:43 +00:00
Clémentine Urquizar
b897ce8dfa
Add CI to update th Meilisearch version in Cargo.toml files 2022-09-04 11:57:23 +02:00
Clémentine Urquizar
5f5b787483
Use meili-bot PAT everywhere 2022-09-04 11:32:22 +02:00
Clémentine Urquizar
d0aa8042e2
Fix job names 2022-09-03 17:53:37 +02:00
Clémentine Urquizar
9cb1e4af5c
Rename workflow file 2022-09-03 17:50:24 +02:00
Clémentine Urquizar
cc09aa8868
Refacto env var 2022-09-02 17:07:23 +02:00
Clémentine Urquizar
2eca723a91
Update checkout v2 to v3 in CI manifests 2022-09-02 16:45:32 +02:00
Clémentine Urquizar
ae14567f97
Add CI manifest to automate some step of the release management when creating/closing a Milestone 2022-09-02 16:28:58 +02:00
Guillaume Mourier
d0f1054f5c chore: cargo fmt 2022-09-01 22:37:07 +02:00
Guillaume Mourier
3878c289df feat: add missing env var for dumps and snapshots feature 2022-09-01 22:34:20 +02:00
bors[bot]
f45ddff312
Merge #2733
2733: Move if conditions to the steps and not to the whole job r=Kerollmops a=curquiza

Follows https://github.com/meilisearch/meilisearch/pull/2726
Fixes the CI. Moves the `if:` condition to the steps and not the whole `check-version` jobs, otherwise, the following jobs checking the compilation cannot run.
See: https://github.com/meilisearch/meilisearch/actions/runs/2968543407

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-01 11:49:40 +00:00
Clémentine Urquizar
8f3b590436
Move if conditions to the steps and not to the whole job 2022-09-01 13:34:34 +02:00
bors[bot]
4fbc8c705f
Merge #2726
2726: Add dry run for publishing binaries: check the compilation works r=Kerollmops a=curquiza

To avoid realizing the compilation of one type of binary does not work during the release, I create a dry run for binary compilation every day at 2am 😇 

See the problem we had recently because missing this CI: https://github.com/meilisearch/meilisearch/issues/2718

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-08-31 15:35:05 +00:00
Clémentine Urquizar
80b1f3e830
Add dry run for publishing binaries: check the compilation works 2022-08-31 17:28:42 +02:00
bors[bot]
3efd06168f
Merge #2719
2719: Update ubuntu-18.04 to 20.04 r=Kerollmops a=curquiza

ubuntu-18.04 is deprecated

<img width="1566" alt="Capture d’écran 2022-08-30 à 12 04 25" src="https://user-images.githubusercontent.com/20380692/187409661-5ac9c545-a737-47f5-ad2f-0369114be11a.png">

https://github.com/meilisearch/meilisearch/actions/runs/2949985452

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-08-30 13:42:09 +00:00
Clémentine Urquizar
07bb32b622
Update ubuntu-18.04 to 20.04 2022-08-29 18:26:01 +02:00
bors[bot]
0826aa35e2
Merge #2713
2713: Move prometheus behind a feature flag r=Kerollmops a=irevoire

We decided we wanted to continue working on this feature before making it public.

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-08-29 12:38:42 +00:00
Tamo
6aa3ad6b6c
move prometheus behind a feature flag 2022-08-29 14:36:59 +02:00
bors[bot]
47a1aa69f0
Merge #2702
2702: Add link to the main image r=curquiza a=brunoocasali

I have wrapped the image with a `<a>` link, and it seems to be working fine, WDYT?

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
2022-08-24 16:54:08 +00:00
Clémentine Urquizar - curqui
b3d89da74d
Update README.md 2022-08-24 18:52:34 +02:00
Bruno Casali
0798170a64
Add link to the main image 2022-08-24 13:47:09 -03:00
bors[bot]
446dfccc8c
Merge #2504 #2697
2504: New README 🌟 r=curquiza a=curquiza

⚠️ Please do not only look at the Markdown but also how the GitHub renders the README 😇 

👉 👉 [Rendered](https://github.com/meilisearch/meilisearch/blob/new-readme/README.md) 👈 👈

2697: Accept an environment variable to enable the metrics route r=ManyTheFish a=Kerollmops

With the PR Meilisearch is able to accept the `MEILI_ENABLE_METRICS_ROUTE` environment variable to enable the newly introduces metrics route.

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-08-24 15:41:44 +00:00
Clémentine Urquizar
43175cfb78
New README version
Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update according to guigui reuqest

Add demo link

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: CaroFG <48251481+CaroFG@users.noreply.github.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Put sentence in bold
2022-08-24 17:16:29 +02:00
bors[bot]
a1bb49c351
Merge #2696
2696: Add the new `metrics.get` and `metrics.all` actions rights r=Kerollmops a=Kerollmops

Follow the specification and add the new `metrics.get` and `metrics.all` actions, making the `/metrics` route only accessible with those rights.

Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-08-24 15:08:53 +00:00
Clément Renault
bebd76064a
Add test for the rights of /metrics route 2022-08-24 17:03:43 +02:00
Clément Renault
f0b2ac6efb
metrics.all must define metrics.get 2022-08-24 17:03:30 +02:00
Clément Renault
08d86e33ca
Accept an env variable to enable the metrics route 2022-08-24 16:39:56 +02:00
Clément Renault
2c2efc7ab6
Remove the hand written numbers of the actions rights 2022-08-24 16:33:12 +02:00
Clément Renault
381df43be4
Change the metrics route API access rights 2022-08-24 16:28:33 +02:00
bors[bot]
f87ebfe477
Merge #2692
2692: Slight changes for prometheus metrics r=Kerollmops a=gmourier

# Pull Request

## What does this PR do?

- Replace "MeiliSearch" with "Meilisearch"
- Brings some consistency between rust identifier and exposed metrics names
- Add suffix describing unit, in plural form. e.g `MEILISEARCH_DB_SIZE_BYTES` (https://prometheus.io/docs/practices/naming/#metric-names)
- Update dashboard.json

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
2022-08-24 10:12:24 +00:00
bors[bot]
c445334070
Merge #2636
2636: Upgrade milli to v0.33.0 r=Kerollmops a=ManyTheFish

# Summary
- Update milli to v0.33.0
- Classify the new InvalidLmdbOpenOptions error as an Internal error
- Update filter error check in tests
- Introduce Terms Matching Policies

fixes #2479
fixes #2484
fixes #2486
fixes #2516
fixes #2578
fixes #2580
fixes #2583
fixes #2600
fixes #2640
fixes #2672
fixes #2679
fixes #2686

# Terms Matching Policies
This PR allows end users to customize matching term policies

## Todo

- [x] Update the API to return the number of pages and allow users to directly choose a page instead of computing an offset
- [x] Change generation of the query tree depending on the chosen settings https://github.com/meilisearch/milli/pull/598

## Small Documentation

### Default search query

**request**:
```sh
curl \
  -X POST 'http://localhost:7700/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "doctor of tokio" }'
```

**result**:
```json
{
  "hits":[...],
  "estimatedTotalHits":32,
  "query":"doctor of tokio",
  "limit":20,
  "offset":0,
  "processingTimeMs":7
}
```

The default behavior doesn't change with the current Meilisearch behavior:
If we don't have enough documents to fit the requested limit, we remove the query words from the last to the first typed word.

## Search query with `optionalWords` parameter

**request**:
```sh
curl \
  -X POST 'http://localhost:7700/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "doctor of tokio", "matchingStrategy": "all"}'
```

**result**:
```json
{
  "hits":[...],
  "estimatedTotalHits":1,
  "query":"doctor of tokio",
  "limit":20,
  "offset":0,
  "processingTimeMs":7
}
```

### allowed `matchingStrategy` values

#### `last`
The default behavior, If we don't have enough documents to fit the requested limit, we remove the query words from the last to the first typed word.

#### `all`
No word will be removed, If we don't have enough documents to fit the requested limit, we return the number of documents we found.

### In charge of the feature

Core: `@ManyTheFish` & `@curquiza`  
Docs: TBD
Integration: `@bidoubiwa` 


Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Many the fish <many@meilisearch.com>
2022-08-23 16:21:00 +00:00
Many the fish
651a22b1ed
Enhance enum documentation
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-08-23 18:11:20 +02:00
Guillaume Mourier
ff59ae56f4 cargo fmt 2022-08-23 17:17:02 +02:00
Guillaume Mourier
b2577aac52 Add suffix describing the unit when needed; Replace MeiliSearch by Meilisearch; Precised some metrics name 2022-08-23 17:09:27 +02:00
ManyTheFish
c9bb111ef3 Implement all and last matching strategy 2022-08-23 17:07:43 +02:00
ManyTheFish
e2af8dccb8 Fix filter tests 2022-08-23 16:39:39 +02:00
ManyTheFish
5e206ee84b Classify InvalidLmdbOpenOptions as an Internal error 2022-08-23 16:39:39 +02:00
ManyTheFish
aff4b64265 Update dependencies 2022-08-23 16:39:39 +02:00
bors[bot]
0a2ef0037f
Merge #2689 #2690
2689: Use mimalloc as the global allocator r=Kerollmops a=loiclec

milli has switched its global allocator to mimalloc already, and we have seen some performance gains as a result. Furthermore, we can use mimalloc as the global allocator on all platforms whereas jemalloc was only activated on Linux. 

This PR brings mimalloc to Meilisearch as well. 

2690: Add LTO and codegen-units=1 to release compile options r=Kerollmops a=loiclec

This PR brings Meilisearch's release compile options in line with milli (see https://github.com/meilisearch/milli/pull/606 ). 

Adding LTO and codegen=units=1 will make compile times longer, but they also speed up the final binary significantly.

Co-authored-by: Loïc Lecrenier <loic@meilisearch.com>
2022-08-23 12:05:02 +00:00
bors[bot]
40ae26478a
Merge #2691
2691: Update version for next release (v0.29.0) in Cargo.toml files r=Kerollmops a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-08-23 11:41:22 +00:00
Clémentine Urquizar
6fe3f285ce
Update version for next release (v0.29.0) 2022-08-23 13:39:56 +02:00