Louis Dureuil
7b55462610
BREAKING CHANGE: errors if queries.facets in federated search
2024-09-17 10:39:48 +02:00
Louis Dureuil
f6114a1ff2
Introduce ComputedFacets and compute_facet_distribution_stats
2024-09-17 10:39:48 +02:00
Louis Dureuil
7c084b1286
SearchQueriesWithIndex changes
2024-09-17 10:39:47 +02:00
Louis Dureuil
57f9517a98
Required changes to IndexUid
2024-09-17 10:39:47 +02:00
Louis Dureuil
72cc573e0a
Add new error types
2024-09-17 10:39:47 +02:00
Louis Dureuil
a48b1d5a79
Update existing tests following error message changes
2024-09-17 10:39:47 +02:00
Louis Dureuil
a94a87ee54
Slightly changes existing error messages
2024-09-17 10:39:47 +02:00
Clément Renault
f4ab1f168e
Prefer using Rc<str> than String when cloning a lot
2024-09-16 15:41:29 +02:00
meili-bors[bot]
4b55ba68bc
Merge #4911
...
4911: Bump quinn-proto from 0.11.3 to 0.11.8 r=Kerollmops a=dependabot[bot]
Bumps [quinn-proto](https://github.com/quinn-rs/quinn ) from 0.11.3 to 0.11.8.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a href="https://github.com/quinn-rs/quinn/releases ">quinn-proto's releases</a>.</em></p>
<blockquote>
<h2>quinn-proto 0.11.5</h2>
<h2>What's Changed</h2>
<ul>
<li>No workspace lints by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1955 ">quinn-rs/quinn#1955</a></li>
</ul>
<h2>quinn-proto 0.11.4</h2>
<h2>What's Changed</h2>
<ul>
<li>Fix panic in example due to unset default crypto provider by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1882 ">quinn-rs/quinn#1882</a></li>
<li>Fix zero-length connection IDs by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1883 ">quinn-rs/quinn#1883</a></li>
<li>Add support for NetBSD, fix OpenBSD by <a href="https://github.com/flub "><code>`@flub</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1884 ">quinn-rs/quinn#1884</a></li>
<li>docs(udp): replace AsRawFd and AsRawSocket with AsFd and AsSocket by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1890 ">quinn-rs/quinn#1890</a></li>
<li>Resolve stopped/received_reset futures on lost connections by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1886 ">quinn-rs/quinn#1886</a></li>
<li>Bump version numbers (quinn 0.11.2, -proto 0.11.3) by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1891 ">quinn-rs/quinn#1891</a></li>
<li>udp: bump version to 0.5.2 by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1892 ">quinn-rs/quinn#1892</a></li>
<li>docs(quinn): Clarify effects of setting AckFrequencyConfig by <a href="https://github.com/gretchenfrage "><code>`@gretchenfrage</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1894 ">quinn-rs/quinn#1894</a></li>
<li>Apply clippy suggestions from Rust 1.79 by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1895 ">quinn-rs/quinn#1895</a></li>
<li>Only send MAX_STREAMS when >1/8 of flow control window is consumed by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1898 ">quinn-rs/quinn#1898</a></li>
<li>fix: remove unused dependency tracing-attributes by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1903 ">quinn-rs/quinn#1903</a></li>
<li>proto: make initial destination cid configurable by <a href="https://github.com/thynson "><code>`@thynson</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1897 ">quinn-rs/quinn#1897</a></li>
<li>Allow configuring rng seed through <code>EndpointConfig</code> by <a href="https://github.com/aochagavia "><code>`@aochagavia</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1901 ">quinn-rs/quinn#1901</a></li>
<li>quinn: introduce waking helpers by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1908 ">quinn-rs/quinn#1908</a></li>
<li>Wake blocked streams on 0-RTT rejection by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1905 ">quinn-rs/quinn#1905</a></li>
<li>Upgrade to rustc-hash 2 by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1909 ">quinn-rs/quinn#1909</a></li>
<li>Fix unnecessary Incoming warning on Endpoint drop by <a href="https://github.com/gretchenfrage "><code>`@gretchenfrage</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1907 ">quinn-rs/quinn#1907</a></li>
<li>Revise and add additional 0-rtt doc comments by <a href="https://github.com/gretchenfrage "><code>`@gretchenfrage</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1826 ">quinn-rs/quinn#1826</a></li>
<li>docs: remove reference to sendmmsg by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1914 ">quinn-rs/quinn#1914</a></li>
<li>Fix debug assert with reordered ACKs by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1893 ">quinn-rs/quinn#1893</a></li>
<li>quinn: Make <code>Endpoint::client</code> dual-stack V6 by default by <a href="https://github.com/gretchenfrage "><code>`@gretchenfrage</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1913 ">quinn-rs/quinn#1913</a></li>
<li>bench(udp): measure non-GSO & GSO on localhost by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1915 ">quinn-rs/quinn#1915</a></li>
<li>proto: avoid overflow in handshake done statistic by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1918 ">quinn-rs/quinn#1918</a></li>
<li>Use workspace dependencies for all external dependencies by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1919 ">quinn-rs/quinn#1919</a></li>
<li>Fix lack of reexport of ConnectionStats and ResetError by <a href="https://github.com/TirushOne "><code>`@TirushOne</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1920 ">quinn-rs/quinn#1920</a></li>
<li>[non-breaking] deps(udp): make tracing optional and add optional log by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1923 ">quinn-rs/quinn#1923</a></li>
<li>fix(udp): feature flag tracing in windows.rs by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1932 ">quinn-rs/quinn#1932</a></li>
<li>Bump MSRV to 1.70 following tokio 1.39 by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1939 ">quinn-rs/quinn#1939</a></li>
<li>Raise default idle timeout to 30 seconds by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1938 ">quinn-rs/quinn#1938</a></li>
<li>Discard pre-handshake packets after the handshake by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1937 ">quinn-rs/quinn#1937</a></li>
<li>Apply suggestions from Clippy 1.80 by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1941 ">quinn-rs/quinn#1941</a></li>
<li>chore(quinn): feature flag socket2 imports by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1933 ">quinn-rs/quinn#1933</a></li>
<li>refactor: move rust-version to workspace Cargo.toml by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1940 ">quinn-rs/quinn#1940</a></li>
<li>chore: move common package data to workspace Cargo.toml by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1943 ">quinn-rs/quinn#1943</a></li>
<li>Endpoint stats interface by <a href="https://github.com/ryleung-solana "><code>`@ryleung-solana</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1900 ">quinn-rs/quinn#1900</a></li>
<li>Expose the Handshake Confirmed state by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1944 ">quinn-rs/quinn#1944</a></li>
<li>Exclude metrics with freestanding getters from EndpointStats by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1945 ">quinn-rs/quinn#1945</a></li>
<li>Fix incorrect initial DCID indexing on retried connections by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1946 ">quinn-rs/quinn#1946</a></li>
<li>Add expect message to unwrap in PacketBuilder by <a href="https://github.com/casey "><code>`@casey</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1951 ">quinn-rs/quinn#1951</a></li>
<li>Revert "proto: yield transport error for Initial packets with no CRYPTO" by <a href="https://github.com/Ralith "><code>`@Ralith</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1952 ">quinn-rs/quinn#1952</a></li>
<li>refactor(udp): introduce log facade by <a href="https://github.com/mxinden "><code>`@mxinden</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1935 ">quinn-rs/quinn#1935</a></li>
<li>Update cargo-deny-action to v2 by <a href="https://github.com/djc "><code>`@djc</code></a>` in <a href="https://redirect.github.com/quinn-rs/quinn/pull/1953 ">quinn-rs/quinn#1953</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="7c09b02073https://redirect.github.com/quinn-rs/quinn/issues/1981 ">#1981</a>)</li>
<li><a href="59bccd2e7ea8ec510fd1c26e8cd2f7e01609ccd8c292a3c6a6bb02a12a845e5cc93645cef42cccefedf16a6f10https://github.com/quinn-rs/quinn/compare/quinn-proto-0.11.3...quinn-proto-0.11.8 ">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores )
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting ``@dependabot` rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- ``@dependabot` rebase` will rebase this PR
- ``@dependabot` recreate` will recreate this PR, overwriting any edits that have been made to it
- ``@dependabot` merge` will merge this PR after your CI passes on it
- ``@dependabot` squash and merge` will squash and merge this PR after your CI passes on it
- ``@dependabot` cancel merge` will cancel a previously requested merge and block automerging
- ``@dependabot` reopen` will reopen this PR if it is closed
- ``@dependabot` close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- ``@dependabot` show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency
- ``@dependabot` ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- ``@dependabot` ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/meilisearch/meilisearch/network/alerts ).
</details>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-16 13:32:32 +00:00
ManyTheFish
1a0e962299
Replace hashmap by vectors in wpp
2024-09-16 15:01:20 +02:00
ManyTheFish
f13e076b8a
Use hashmap instead of Btree in wpp extractor
2024-09-16 14:40:40 +02:00
ManyTheFish
7ba49b849e
Extract and write facet databases
2024-09-16 09:35:16 +02:00
Clément Renault
f7652186e1
WIP geo fields
2024-09-12 18:01:02 +02:00
Louis Dureuil
23e14138bb
facet distribution: implement Display for OrderBy
2024-09-12 17:43:50 +02:00
Louis Dureuil
e44325683a
Facet distribution: fix issue where truncated facet distribution would have a wrong order
2024-09-12 17:43:49 +02:00
Clément Renault
b2f4e67c9a
Do not store useless updates
2024-09-12 15:38:31 +02:00
Clément Renault
ff5d3b59f5
Move the document id extraction to the primary key code
2024-09-12 12:01:42 +02:00
ManyTheFish
aa69308e45
Use a bufWriter to build word FSTs
2024-09-12 11:48:00 +02:00
ManyTheFish
eb9a20ff0b
Fix fid_word_docids extraction
2024-09-12 11:08:18 +02:00
Clément Renault
0d868f36d7
Make sure we always use a BufWriter to write the update files
2024-09-11 18:38:04 +02:00
Clément Renault
e7d9db078f
Use the right key name when convertir from CSV to NDJSON
2024-09-11 18:27:00 +02:00
Clément Renault
3e9198ebaa
Support guessing primary key again
2024-09-11 17:25:40 +02:00
Clément Renault
2a0ad0982f
Fix the document counter
2024-09-11 15:59:36 +02:00
ManyTheFish
2b317c681b
Build mergers in parallel
2024-09-11 11:49:26 +02:00
ManyTheFish
39b5990f64
Mutualize tokenization
2024-09-11 10:22:38 +02:00
Clément Renault
3848adf5a2
Improve error management and simplify JSON read
2024-09-11 10:10:51 +02:00
Clément Renault
b4de06259e
Better CSV support
2024-09-11 10:02:00 +02:00
meili-bors[bot]
02c2b660f8
Merge #4920
...
4920: Change OpenAI default model r=dureuill a=dureuill
# Pull Request
## Related issue
Fixes #4856
See also [public usage](https://meilisearch.notion.site/v1-11-AI-search-changes-0e37727193884a70999f254fa953ce6e#b4685a48c4784262a149ec307ec58671 )
## What does this PR do?
- make the `text-embedding-3-small` the default model for OpenAI instead of `text-embedding-ada-002`. Existing embedders are not impacted
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-09-11 07:08:39 +00:00
Clément Renault
8287c2644f
Support CSV again
2024-09-10 21:10:28 +01:00
Clément Renault
c1c44a0b81
Impl serialize on TopLevelMap
2024-09-10 19:32:03 +01:00
Clément Renault
04596f3616
Move the TopLevelMap into a dedicated module
2024-09-10 18:01:17 +01:00
Clément Renault
24cb5839ad
Move the document changes sorting logic to a new trait
2024-09-10 17:37:52 +01:00
Clément Renault
8d97b7b28c
Support JSON payloads again (not perfectly though)
2024-09-10 17:09:49 +01:00
ManyTheFish
f69688e8f7
Fix several warnings in extractors and remove unreachable macros
2024-09-09 14:52:50 +02:00
Louis Dureuil
f18e9cb7b3
Change openai default model
2024-09-09 13:09:35 +02:00
Clément Renault
8fd0afaaaa
Make sure we iterate over the payload documents in order
2024-09-06 08:09:08 +02:00
Clément Renault
72c6a21a30
Use raw JSON to read the payloads
2024-09-05 20:08:23 +02:00
Clément Renault
8412be4a7d
Cleanup CowStr and TopLevelMap struct
2024-09-05 18:32:55 +02:00
Louis Dureuil
10f09c531f
add some commented code to read from json with raw values
2024-09-05 18:22:16 +02:00
ManyTheFish
8fd99b111b
Add tracing timers logs
2024-09-05 18:00:22 +02:00
Clément Renault
f6b3d1f9a5
Increase some channel sizes
2024-09-05 15:12:07 +02:00
meili-bors[bot]
db0cf3b2ed
Merge #4912
...
4912: Allow Meilitool to dumplessly, offline upgrade v1.9 -> v1.10 in some conditions r=Kerollmops a=dureuill
- bail early if the DB contains at least 1 REST embedder, providing the list of detected REST embedders, and without modifying the DB
- Might depend on the feature set that meilitool was compiled with and the featureset that the Meilisearch that created the DB was compiled with 💀 . In case of runtime error, try again with a different feature set (passing or not passing `-p meilitool` when building after a `cargo clean`)
Co-authored-by: Louis Dureuil <louis@meilisearch.com>
2024-09-05 09:11:23 +00:00
Clément Renault
73ce67862d
Use the word pair proximity and fid word count docids extractors
...
Co-authored-by: ManyTheFish <many@meilisearch.com>
2024-09-05 10:56:22 +02:00
Louis Dureuil
f6abf01d2c
Check REST embedders before touching the DB
2024-09-05 10:49:59 +02:00
Clément Renault
0fc02f7351
Move the facet extraction to dedicated modules
2024-09-05 10:32:27 +02:00
ManyTheFish
34f11e3380
Implement word count and word pair proximity extractors
2024-09-05 10:30:39 +02:00
Louis Dureuil
28da759f11
meilitool: Support dumpless upgrade from v1.9 to v1.10 when there are no REST embedders
2024-09-05 10:08:38 +02:00
Louis Dureuil
ea96d19525
Change versioning in meili
2024-09-05 10:08:06 +02:00
Louis Dureuil
d352b1ee83
Add serde to meilitool
2024-09-05 10:07:33 +02:00
Clément Renault
27308eaab1
Import the facet extractors
2024-09-04 17:58:15 +02:00
