Commit Graph

9338 Commits

Author SHA1 Message Date
meili-bors[bot] 781e2d7750
Merge #4532
4532: Add `url` and `api_key` to ollama r=ManyTheFish a=dureuill

See [Usage page](https://meilisearch.notion.site/v1-8-AI-search-API-usage-135552d6e85a4a52bc7109be82aeca42#5c77ef49e78e43388c1d3d5429151357)

### Motivation

- Before this PR, the url for ollama is only read from the environment. This is a needless restriction that will be troublesome in settings where passing an environment variable is complex or impossible (e.g., the Cloud)
- Before this PR, ollama did not support an api_key. While ollama does not natively support API keys, [a common practice](https://github.com/ollama/ollama/issues/849) is to put a publicly accessible ollama server behind a proxy to support authentication.

### Skip changelog

ollama embedder was added to v1.8

Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-03-28 12:35:19 +00:00
Louis Dureuil 796213af9a
Merge branch 'main' into tmp-release-v1.7.4 2024-03-28 10:51:49 +01:00
Clément Renault 69f8b2730d
Fix the tests 2024-03-28 10:47:04 +01:00
meili-bors[bot] 7385067c42
Merge #4542
4542: fixes typos r=irevoire a=brunoocasali

Just fix a typo 😬 

Co-authored-by: Bruno Casali <brunoocasali@gmail.com>
2024-03-27 18:21:48 +00:00
meili-bors[bot] d1021c0f0d
Merge #4520
4520: Add automation to create openAPI issue r=dureuill a=curquiza

Create automatically an issue to remind us to update open-api file when opening a milestone

Co-authored-by: curquiza <clementine@meilisearch.com>
2024-03-27 17:33:22 +00:00
Bruno Casali 8f2606d79d
fixes typos 2024-03-27 14:26:47 -03:00
meili-bors[bot] 0259ad6082
Merge #4541
4541: Update version for the next release (v1.7.4) in Cargo.toml r=Kerollmops a=meili-bot

⚠️ This PR is automatically generated. Check the new version is the expected one and Cargo.lock has been updated before merging.

Co-authored-by: dureuill <dureuill@users.noreply.github.com>
2024-03-27 16:49:40 +00:00
Tamo 06a11b5b21
Improve error message 2024-03-27 17:34:49 +01:00
dureuill b50f518764 Update version for the next release (v1.7.4) in Cargo.toml 2024-03-27 16:12:54 +00:00
meili-bors[bot] 94b7afcc55
Merge #4539
4539: Don't optimize reindexing when fields contain dots r=Kerollmops a=dureuill

# Pull Request

## Related issue
Fixes https://github.com/meilisearch/meilisearch/issues/4525

## What does this PR do?
- Don't try to optimize the amount of reindexing operation when nested fields are used anywhere in:
    - the field distribution (e.g. a key actually contains a `.`)
    - the old faceted fields
    - the new faceted fields

This is because the facet distribution is not reporting on existing nested fields.



Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-03-27 16:07:49 +00:00
Louis Dureuil ee8cbea810
Don't optimize reindexing when fields contain dots 2024-03-27 17:04:45 +01:00
Tamo b7c582e4f3 connect the search queue with the health route 2024-03-27 15:49:43 +01:00
Tamo 03c886ac1b adds a bit of documentation 2024-03-27 15:38:36 +01:00
Louis Dureuil cde7ce4f44
Add test 2024-03-27 14:02:09 +01:00
Louis Dureuil 92224f109a
Fix tests 2024-03-27 12:19:10 +01:00
meili-bors[bot] 0d27d50740
Merge #4516
4516: Update sprint_issue.md r=Kerollmops a=curquiza

Following decision made about specification

Also
- removed useless parts of the template
- add automatic labels -> better to forget to remove them rather than forgetting to add them (some mistakes happened in the past)

Co-authored-by: Clémentine U. - curqui <clementine@meilisearch.com>
2024-03-27 11:04:06 +00:00
Louis Dureuil 572fb3a51d
Finer granularity for embedder needs reindex 2024-03-27 12:01:34 +01:00
Louis Dureuil 4ff0255783
remove unused function 2024-03-27 11:51:14 +01:00
Louis Dureuil a25456120d
Expose distribution in settings 2024-03-27 11:51:04 +01:00
Louis Dureuil 168ded3b9d
Deserr for distribution 2024-03-27 11:50:33 +01:00
Louis Dureuil afd1da5642
Add distribution to all embedders 2024-03-27 11:50:22 +01:00
Tamo 087a96d22e fix flaky test 2024-03-27 11:05:37 +01:00
meili-bors[bot] 34dfea72cc
Merge #4509
4509: Rest embedder r=ManyTheFish a=dureuill

Fixes #4531 

See [Usage page](https://meilisearch.notion.site/v1-8-AI-search-API-usage-135552d6e85a4a52bc7109be82aeca42?pvs=25#e6f58c3b742c4effb4ddc625ce12ee16)

### Implementation changes

- Remove tokio, futures, reqwests
- Add a new `milli::vector::rest::Embedder` embedder
- Update OpenAI and Ollama embedders to use the REST embedder internally
- Make Embedder::embed a sync method
- Add the new embedder source as described in the usage


Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-03-27 09:27:46 +00:00
Tamo 3a1f458139 fix a flaky test 2024-03-26 21:06:55 +01:00
Tamo 55df9daaa0 adds a comment about the safety of an operation 2024-03-26 19:34:55 +01:00
Tamo 2e36f069c2 fmt imports 2024-03-26 19:23:55 +01:00
Tamo 8f5d9f501a update the discussion link 2024-03-26 19:18:32 +01:00
Tamo 8127c9a115 handle the case of a queue of zero elements 2024-03-26 19:04:39 +01:00
Tamo e7704f1fc1 add a test to ensure we effectively returns a retry-after when the search queue is full 2024-03-26 18:08:59 +01:00
Clément Renault 34262c7a0d
Add analytics for the negative operator 2024-03-26 18:01:27 +01:00
Tamo e2a1bbae37 simplify and improve the http error 2024-03-26 17:53:37 +01:00
Clément Renault 1da9e0f246
Better support space around the negative operator (-) 2024-03-26 17:47:13 +01:00
Clément Renault e4a3e603b3
Expose a first working version of the negative keyword 2024-03-26 17:47:13 +01:00
Tamo e433fd53e6 rename the method to get a permit and use it in all search requests 2024-03-26 17:28:03 +01:00
Tamo 3f23fbb46d create the experimental CLI argument 2024-03-26 16:43:40 +01:00
Tamo c41e1274dc push and test the search queue datastructure 2024-03-26 15:56:43 +01:00
Louis Dureuil 9a95ed619d
Add tests 2024-03-26 10:36:56 +01:00
Louis Dureuil f82d056072
Hide secrets in settings and task queue 2024-03-26 10:36:24 +01:00
meili-bors[bot] 5ea017b922
Merge #4530
4530: fix: set the histogram bucket boundaries to follow the otel spec r=curquiza a=rohankmr414

# Pull Request

## What does this PR do?
- Fixes the http request duration histogram bucket boundaries to follow the opentelemetry spec, currently the bucket boundaries are too granular and only track latencies below 1s.

## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Rohan Kumar <rohankmr414@gmail.com>
2024-03-25 12:23:31 +00:00
Louis Dureuil 817ccc089a
also allow `api_key` 2024-03-25 11:50:00 +01:00
meili-bors[bot] 2ddd872ce6
Merge #4373
4373: feat: add status code label to prometheus http request counter r=irevoire a=rohankmr414

# Pull Request

## What does this PR do?
- This PR adds the `status` label (the value is http status code) to the `meilisearch_http_requests_total` metric.

## PR checklist
Please check if your PR fulfills the following requirements:
- [ ] Does this PR fix an existing issue, or have you listed the changes applied in the PR description (and why they are needed)?
- [x] Have you read the contributing guidelines?
- [x] Have you made sure that the title is accurate and descriptive of the changes?

Thank you so much for contributing to Meilisearch!


Co-authored-by: Rohan Kumar <rohankmr414@gmail.com>
2024-03-25 10:40:50 +00:00
Louis Dureuil 4136630ea5
Use constants instead of raw strings in set_*set() 2024-03-25 11:39:33 +01:00
Louis Dureuil 58972f35cb
Allow `url` parameter for ollama embedder 2024-03-25 11:32:55 +01:00
Louis Dureuil dfa5e41ea6
Check validity of the URL setting 2024-03-25 11:23:16 +01:00
Louis Dureuil a1db342f01
Expose REST embedder to the API 2024-03-25 11:23:15 +01:00
Louis Dureuil f87747f4d3
Remove unwraps 2024-03-25 11:23:04 +01:00
Louis Dureuil b6b4b6bab7
Remove the tokio and the reqwests 2024-03-25 11:23:03 +01:00
Louis Dureuil f649f58013
embed no longer async 2024-03-25 11:23:03 +01:00
Louis Dureuil ac52c857e8
Update ollama and openai impls to use the rest embedder internally 2024-03-25 11:23:03 +01:00
Louis Dureuil 8708cbef25
Add RestEmbedder 2024-03-25 11:23:03 +01:00