Commit Graph

3933 Commits

Author SHA1 Message Date
arriven
88e69f4302 Increase max concurrent readers on indexes 2022-10-02 18:25:32 +03:00
Clémentine Urquizar - curqui
2827ff7957
Update README.md with Hacktoberfest section (#2794)
* Update README.md

* Update README.md

* Update README.md

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
2022-09-22 17:45:33 +02:00
Luna-meili
d166a97d67
Update CONTRIBUTING.md for Hacktoberfest (#2793)
* Update CONTRIBUTING.md

* Update CONTRIBUTING.md

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>

* Update CONTRIBUTING.md

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
2022-09-22 17:27:42 +02:00
bors[bot]
f4b81fa0a1
Merge #2772 #2773
2772: Improve issue template display to avoid support in Meilisearch issues r=curquiza a=curquiza

Move the support line higher to make it more visible
Like this: https://github.com/meilisearch/meilisearch/discussions/2780

2773: Allow building without specialized tokenizations r=curquiza a=jirutka

Fixes #2774

(Some of) these specialized tokenizations include huge dictionaries that currently account for 90% (!) of the meilisearch binary size.

This commit adds `chinese`, `hebrew`, `japanese`, and `thai` feature flags that are propagated via `milli` down to the `charabia` crate. To keep it backwards compatible, they are enabled by default.

Related to meilisearch/milli#632

Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
Co-authored-by: Jakub Jirutka <jakub@jirutka.cz>
2022-09-21 10:56:12 +00:00
bors[bot]
0fdbf128e9
Merge #2790
2790: Improve Docker CI for cloud team r=curquiza a=curquiza

Work with `@eskombro` 
Update CI to send a signal to Cloud team CI when Docker image is pushed

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-21 09:55:50 +00:00
Clémentine Urquizar
d3b984d862
Update CI to send a signal to Cloud team when Docker image is pushed
Co-authored-by: Samuel Jimenez <sjimenezre@gmail.com>
2022-09-21 11:39:58 +02:00
Jakub Jirutka
935f18efcf Allow building without specialized tokenizations
(Some of) these specialized tokenizations include huge dictionaries
that currently account for 90% (!) of the meilisearch binary size.

This commit adds chinese, hebrew, japanese, and thai feature flags
that are propagated via milli down to the charabia crate. To keep it
backward compatible, they are enabled by default.

Related to meilisearch/milli#632
2022-09-14 21:16:34 +02:00
Jakub Jirutka
5b57114771 Bump milli from 0.33.0 to 0.33.4 2022-09-14 20:52:11 +02:00
Clémentine Urquizar - curqui
c2ab7a7939
Update config.yml 2022-09-14 14:40:36 +02:00
bors[bot]
528a4721c1
Merge #2727
2727: Don't panic when the error length is slightly over 100 r=Kerollmops a=onyxcherry

# Pull Request

## What does this PR do?
Fixes PR #2207 as [the last commit](7ece7a9d9e) has changed number of the characters at the end to leave in place from `50` to `85` **but the lower limit of a string length wasn't changed**.
Therefore, any data (e.g. example string from issue #2680) was causing `meilisearch` to **panic**.

So I simply raised the minimum value from `100` to `135` (`50 + 85`) to ensure that `replace_range()` won't panic due to an inverted range.
At the same time I am in favor of the `85` value which was changed in the `@CNLHC's` last commit.


## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing ~issue~ pull request?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Tomasz Wiśniewski <tomasz@wisniewski.app>
2022-09-07 16:16:43 +00:00
Tomasz Wiśniewski
d1a30df23d Remove unneeded prints, format 2022-09-07 18:05:55 +02:00
bors[bot]
3a62e97c9f
Merge #2744
2744: Minor fixes in the just added update-version CI r=ManyTheFish a=curquiza

These fixes does not prevent us to use the current CI

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-06 13:14:06 +00:00
Clémentine Urquizar
55aa83d75a
Minor fixes in the just added update-version CI 2022-09-05 16:41:01 +02:00
bors[bot]
0c962f6a76
Merge #2740
2740: Update checkout v2 to v3 in CI manifests and use a unique GitHub PAT r=Kerollmops a=curquiza

Upgrade the missing checkout v2 into v3
Probably a bad merge conflicts that make them removed when merging `stable` into `main` after v0.28.0 release.

Also, use `MEILI_BOT_GH_PAT` instead of `PUBLISH_TOKEN` and the default github token, which allow us to remove useless GitHub secrets (once this PR is merged and v0.29.0 is release because `PUBLISH_TOKEN` is still used on `release-v0.29.0`)

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-05 12:14:40 +00:00
bors[bot]
2e0221cfc6
Merge #2739
2739: Add CI manifest to automate some steps when closing/creating a Milestone r=Kerollmops a=curquiza

For each Milestone created (not opened!), and if the release is NOT a patch release (only the patch changed)
- the roadmap issue is created, see the [template](https://github.com/meilisearch/core-team/blob/main/issue-templates/roadmap-issue.md)
- the changelog issue is created, see the [template](https://github.com/meilisearch/core-team/blob/main/issue-templates/changelog-issue.md)

For each Milestone closed
- the `vX.Y.Z` label is created
- this label is applied to all issues/PRs in the Milestone

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-05 11:42:10 +00:00
bors[bot]
2d34b239ab
Merge #2738
2738: Add missing env vars for dumps and snapshots features r=irevoire a=gmourier

# Pull Request

## What does this PR do?
Fixes #2721


## PR checklist
Please check if your PR fulfills the following requirements:
- [x] Does this PR fix an existing issue?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
2022-09-05 11:12:09 +00:00
bors[bot]
55853815a5
Merge #2741
2741: Add CI to update the Meilisearch version in Cargo.toml files r=ManyTheFish a=curquiza

Add a CI we can trigger manually to create a PR updating the Meilisearch version
The next step is to create a Slack bot that will trigger this CI
In the meantime, we can trigger this CI manually in the [Actions tab](https://github.com/meilisearch/milli/actions)

The `MEILI_BOT_GH_PAT` secrets has been added to the organization level, and is accessible for the following repositories (so far): Meilisearch, Milli and Charabia

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-05 08:32:43 +00:00
Clémentine Urquizar
b897ce8dfa
Add CI to update th Meilisearch version in Cargo.toml files 2022-09-04 11:57:23 +02:00
Clémentine Urquizar
5f5b787483
Use meili-bot PAT everywhere 2022-09-04 11:32:22 +02:00
Clémentine Urquizar
d0aa8042e2
Fix job names 2022-09-03 17:53:37 +02:00
Clémentine Urquizar
9cb1e4af5c
Rename workflow file 2022-09-03 17:50:24 +02:00
Clémentine Urquizar
cc09aa8868
Refacto env var 2022-09-02 17:07:23 +02:00
Clémentine Urquizar
2eca723a91
Update checkout v2 to v3 in CI manifests 2022-09-02 16:45:32 +02:00
Clémentine Urquizar
ae14567f97
Add CI manifest to automate some step of the release management when creating/closing a Milestone 2022-09-02 16:28:58 +02:00
Guillaume Mourier
d0f1054f5c chore: cargo fmt 2022-09-01 22:37:07 +02:00
Guillaume Mourier
3878c289df feat: add missing env var for dumps and snapshots feature 2022-09-01 22:34:20 +02:00
Tomasz Wiśniewski
d1b3642923 Extract input to trim lengths to variables 2022-09-01 20:50:11 +02:00
bors[bot]
f45ddff312
Merge #2733
2733: Move if conditions to the steps and not to the whole job r=Kerollmops a=curquiza

Follows https://github.com/meilisearch/meilisearch/pull/2726
Fixes the CI. Moves the `if:` condition to the steps and not the whole `check-version` jobs, otherwise, the following jobs checking the compilation cannot run.
See: https://github.com/meilisearch/meilisearch/actions/runs/2968543407

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-09-01 11:49:40 +00:00
Clémentine Urquizar
8f3b590436
Move if conditions to the steps and not to the whole job 2022-09-01 13:34:34 +02:00
Tomasz Wiśniewski
60792eebcf Fix #2207 - do not panic when the error message length is between 100 and 135 2022-08-31 19:29:53 +02:00
bors[bot]
4fbc8c705f
Merge #2726
2726: Add dry run for publishing binaries: check the compilation works r=Kerollmops a=curquiza

To avoid realizing the compilation of one type of binary does not work during the release, I create a dry run for binary compilation every day at 2am 😇 

See the problem we had recently because missing this CI: https://github.com/meilisearch/meilisearch/issues/2718

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-08-31 15:35:05 +00:00
Clémentine Urquizar
80b1f3e830
Add dry run for publishing binaries: check the compilation works 2022-08-31 17:28:42 +02:00
bors[bot]
3efd06168f
Merge #2719
2719: Update ubuntu-18.04 to 20.04 r=Kerollmops a=curquiza

ubuntu-18.04 is deprecated

<img width="1566" alt="Capture d’écran 2022-08-30 à 12 04 25" src="https://user-images.githubusercontent.com/20380692/187409661-5ac9c545-a737-47f5-ad2f-0369114be11a.png">

https://github.com/meilisearch/meilisearch/actions/runs/2949985452

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2022-08-30 13:42:09 +00:00
Clémentine Urquizar
07bb32b622
Update ubuntu-18.04 to 20.04 2022-08-29 18:26:01 +02:00
bors[bot]
0826aa35e2
Merge #2713
2713: Move prometheus behind a feature flag r=Kerollmops a=irevoire

We decided we wanted to continue working on this feature before making it public.

Co-authored-by: Tamo <tamo@meilisearch.com>
2022-08-29 12:38:42 +00:00
Tamo
6aa3ad6b6c
move prometheus behind a feature flag 2022-08-29 14:36:59 +02:00
bors[bot]
47a1aa69f0
Merge #2702
2702: Add link to the main image r=curquiza a=brunoocasali

I have wrapped the image with a `<a>` link, and it seems to be working fine, WDYT?

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
Co-authored-by: Clémentine Urquizar - curqui <clementine@meilisearch.com>
2022-08-24 16:54:08 +00:00
Clémentine Urquizar - curqui
b3d89da74d
Update README.md 2022-08-24 18:52:34 +02:00
Bruno Casali
0798170a64
Add link to the main image 2022-08-24 13:47:09 -03:00
bors[bot]
446dfccc8c
Merge #2504 #2697
2504: New README 🌟 r=curquiza a=curquiza

⚠️ Please do not only look at the Markdown but also how the GitHub renders the README 😇 

👉 👉 [Rendered](https://github.com/meilisearch/meilisearch/blob/new-readme/README.md) 👈 👈

2697: Accept an environment variable to enable the metrics route r=ManyTheFish a=Kerollmops

With the PR Meilisearch is able to accept the `MEILI_ENABLE_METRICS_ROUTE` environment variable to enable the newly introduces metrics route.

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-08-24 15:41:44 +00:00
Clémentine Urquizar
43175cfb78
New README version
Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update according to guigui reuqest

Add demo link

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: CaroFG <48251481+CaroFG@users.noreply.github.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Update README.md

Co-authored-by: gui machiavelli <hey@guimachiavelli.com>

Put sentence in bold
2022-08-24 17:16:29 +02:00
bors[bot]
a1bb49c351
Merge #2696
2696: Add the new `metrics.get` and `metrics.all` actions rights r=Kerollmops a=Kerollmops

Follow the specification and add the new `metrics.get` and `metrics.all` actions, making the `/metrics` route only accessible with those rights.

Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-08-24 15:08:53 +00:00
Clément Renault
bebd76064a
Add test for the rights of /metrics route 2022-08-24 17:03:43 +02:00
Clément Renault
f0b2ac6efb
metrics.all must define metrics.get 2022-08-24 17:03:30 +02:00
Clément Renault
08d86e33ca
Accept an env variable to enable the metrics route 2022-08-24 16:39:56 +02:00
Clément Renault
2c2efc7ab6
Remove the hand written numbers of the actions rights 2022-08-24 16:33:12 +02:00
Clément Renault
381df43be4
Change the metrics route API access rights 2022-08-24 16:28:33 +02:00
bors[bot]
f87ebfe477
Merge #2692
2692: Slight changes for prometheus metrics r=Kerollmops a=gmourier

# Pull Request

## What does this PR do?

- Replace "MeiliSearch" with "Meilisearch"
- Brings some consistency between rust identifier and exposed metrics names
- Add suffix describing unit, in plural form. e.g `MEILISEARCH_DB_SIZE_BYTES` (https://prometheus.io/docs/practices/naming/#metric-names)
- Update dashboard.json

Co-authored-by: Guillaume Mourier <guillaume@meilisearch.com>
2022-08-24 10:12:24 +00:00
bors[bot]
c445334070
Merge #2636
2636: Upgrade milli to v0.33.0 r=Kerollmops a=ManyTheFish

# Summary
- Update milli to v0.33.0
- Classify the new InvalidLmdbOpenOptions error as an Internal error
- Update filter error check in tests
- Introduce Terms Matching Policies

fixes #2479
fixes #2484
fixes #2486
fixes #2516
fixes #2578
fixes #2580
fixes #2583
fixes #2600
fixes #2640
fixes #2672
fixes #2679
fixes #2686

# Terms Matching Policies
This PR allows end users to customize matching term policies

## Todo

- [x] Update the API to return the number of pages and allow users to directly choose a page instead of computing an offset
- [x] Change generation of the query tree depending on the chosen settings https://github.com/meilisearch/milli/pull/598

## Small Documentation

### Default search query

**request**:
```sh
curl \
  -X POST 'http://localhost:7700/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "doctor of tokio" }'
```

**result**:
```json
{
  "hits":[...],
  "estimatedTotalHits":32,
  "query":"doctor of tokio",
  "limit":20,
  "offset":0,
  "processingTimeMs":7
}
```

The default behavior doesn't change with the current Meilisearch behavior:
If we don't have enough documents to fit the requested limit, we remove the query words from the last to the first typed word.

## Search query with `optionalWords` parameter

**request**:
```sh
curl \
  -X POST 'http://localhost:7700/indexes/movies/search' \
  -H 'Content-Type: application/json' \
  --data-binary '{ "q": "doctor of tokio", "matchingStrategy": "all"}'
```

**result**:
```json
{
  "hits":[...],
  "estimatedTotalHits":1,
  "query":"doctor of tokio",
  "limit":20,
  "offset":0,
  "processingTimeMs":7
}
```

### allowed `matchingStrategy` values

#### `last`
The default behavior, If we don't have enough documents to fit the requested limit, we remove the query words from the last to the first typed word.

#### `all`
No word will be removed, If we don't have enough documents to fit the requested limit, we return the number of documents we found.

### In charge of the feature

Core: `@ManyTheFish` & `@curquiza`  
Docs: TBD
Integration: `@bidoubiwa` 


Co-authored-by: ManyTheFish <many@meilisearch.com>
Co-authored-by: Many the fish <many@meilisearch.com>
2022-08-23 16:21:00 +00:00
Many the fish
651a22b1ed
Enhance enum documentation
Co-authored-by: Clément Renault <clement@meilisearch.com>
2022-08-23 18:11:20 +02:00