Commit Graph

3068 Commits

Author SHA1 Message Date
bors[bot]
0a9d6e8210
Merge #1847
1847: Optimize document transform r=MarinPostma a=MarinPostma

integrate the optimization from https://github.com/meilisearch/milli/pull/402.

optimize payload read, by reading it to RAM first instead of streaming it. This means that the payload must fit into RAM, which should not be a problem.

Add BufWriter to the obkv writer to improve write speed.

I have measured a gain of 40-45% in speed after these optimizations.


Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-10-26 15:28:58 +00:00
bors[bot]
0ed800b612
Merge #1830
1830: Add MEILI_SERVER_PROVIDER to Dockerfile r=irevoire a=curquiza

Add docker information in `MEILI_SERVER_PROVIDER` env variable

It does not impact the telemetry spec since it's an already existing variable used on our side.

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-26 14:01:06 +00:00
marin postma
4ac005b094
optimize document transform
fix error types

bump milli
2021-10-26 13:51:15 +02:00
bors[bot]
e87146b0d9
Merge #1811
1811: Reducing ArmV8 binary build time with action-rs (cross build with Rust) r=curquiza a=patrickdung

This pull request is based on [discussion #1790](https://github.com/meilisearch/MeiliSearch/discussions/1790)

Note:
1) The binaries of this PR is additional to existing binary built
Existing binary would be produced (by existing GitHub workflow/action)

meilisearch-linux-amd64
meilisearch-linux-armv8
meilisearch-macos-amd64
meilisearch-windows-amd64.exe
meilisearch.deb

2) This PR produce these binaries. The name 'meilisearch-linux-aarch64' is used to avoid naming conflict with 'meilisearch-linux-armv8'.

meilisearch-linux-aarch64
meilisearch-linux-aarch64-musl
meilisearch-linux-aarch64-stripped
meilisearch-linux-amd64-musl

3) If it's fine (in next release), we should submit another PR to stop generating meilisearch-linux-armv8 (which could take two to three hours to build it)

Co-authored-by: Patrick Dung <38665827+patrickdung@users.noreply.github.com>
2021-10-24 12:24:39 +00:00
Patrick Dung
5caa79df67
Update .github/workflows/publish-crossbuild.yml
Update to use the correct syntax

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-24 11:31:04 +00:00
Patrick Dung
d519e1036f
Update .github/workflows/publish-crossbuild.yml
better naming

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-21 17:02:28 +00:00
Patrick Dung
19eebc0b0a
Update .github/workflows/publish-crossbuild.yml
better naming

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-21 17:02:16 +00:00
Patrick Dung
5585020753
Update .github/workflows/publish-crossbuild.yml
better spacing

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-21 17:02:00 +00:00
Patrick Dung
ef7e7a8f11 Only generate aarch64 binary with action-rs 2021-10-21 00:40:46 +08:00
Clémentine Urquizar
eb91f27b65
Add MEILI_SERVER_PROVIDER to dockerfile 2021-10-20 17:53:43 +02:00
bors[bot]
24eef577c5
Merge #1822
1822: Tiny improvements in download-latest.sh r=irevoire a=curquiza

- Add check on `$latest` to check if it's empty. We have some issue on the swift SDK currently where the version number seems not to be retrieved, but we don't why https://github.com/meilisearch/meilisearch-swift/pull/216
- Replace some `"` by `'`
- Rename `$BINARY_NAME` by `$binary_name` to make them consistent with the other variables that are filled all along the script

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-19 09:32:27 +00:00
Clémentine Urquizar
e7e4ccf74f
Merge pull request #1817 from nfsec/patch-1
Improve RUNs in Dockerfile
2021-10-19 00:00:22 +02:00
Clémentine Urquizar
017ecf76e3
Replace double quotes by single ones 2021-10-18 23:39:07 +02:00
bors[bot]
1c9ceadd8d
Merge #1824
1824: Fix indexation perfomances on mounted disk r=ManyTheFish a=ManyTheFish

We were creating all of our tempfiles in data.ms directory, but when the database directory is stored in a mounted disk, tempfiles I/O throughput decreases, impacting the indexation time.

Now, only the persisting tempfiles will be created in the database directory. Other tempfiles will stay in the default tmpdir.

Co-authored-by: many <maxime@meilisearch.com>
2021-10-18 12:42:47 +00:00
many
36ab7b3ebd
Fix small typo 2021-10-18 14:17:32 +02:00
many
b4038597ba
Keep persisting tmp files in database directory and put non-persisting tmp files in default tmp dir 2021-10-18 14:16:35 +02:00
bors[bot]
79817bd465
Merge #1813
1813: Apply highlight tags on numbers in the formatted search result output r=irevoire a=Jhnbrn90

This is my first ever Rust related PR. 

As described in #1758, I've attempted to highlighting numbers correctly under the `_formatted` key.

Additionally, I added a test which should assert appropriate highlighting. 

I'm open to suggestions and improvements. 


Co-authored-by: John Braun <john@brn.email>
2021-10-18 09:05:01 +00:00
Clémentine Urquizar
93ad8f04b5
Add check if $latest is empty 2021-10-16 17:36:36 +02:00
Clémentine Urquizar
e4cb7ed30f
Tiny improvement in download-latest.sh 2021-10-16 17:23:50 +02:00
bors[bot]
b9e060423f
Merge #1760
1760: Add option to use environment variable to increase rate limit r=curquiza a=nav1s

This closes #1655.

Added GITHUB_PAT environment variable and a comment to explain how to create it (I found the ```public_repo``` scope to be the best fit out of the available [scopes](https://docs.github.com/en/developers/apps/building-oauth-apps/scopes-for-oauth-apps#available-scopes)).

Co-authored-by: Aviv <avivnt@gmail.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-16 12:56:02 +00:00
Clémentine Urquizar
ead1ec3396
Update download-latest.sh 2021-10-16 14:55:10 +02:00
Clémentine Urquizar
306a8cd059
Update download-latest.sh 2021-10-16 14:55:06 +02:00
Patryk Krawaczyński
4c50deb4b7
2 RUNs less. 2021-10-16 11:37:01 +02:00
John Braun
be75426e64 Apply formatting according code style guidelines 2021-10-15 21:32:29 +02:00
Patryk Krawaczyński
23458de588
One RUN less
Align apk add commands between images.
2021-10-15 21:18:31 +02:00
bors[bot]
9fd849d48b
Merge #1808
1808: Add Milestone Check status to bors.toml r=curquiza a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-14 13:09:32 +00:00
bors[bot]
2b28bc9510
Merge #1693
1693: Remove dataset r=Kerollmops a=curquiza

Fixes https://github.com/meilisearch/MeiliSearch/issues/1230

⚠️ Should be merged once https://github.com/meilisearch/documentation/pull/1109 is merged! ⚠️

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-14 12:56:23 +00:00
bors[bot]
d107b3f46c
Merge #1759
1759: Feature docker as non root r=curquiza a=igaul

This closes #1757 . 
Adding a non root user with default name meiliuser.

Co-authored-by: gaul@pdx.edu <gaul@pdx.edu>
Co-authored-by: igaul <40813772+igaul@users.noreply.github.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-14 12:45:49 +00:00
Clémentine Urquizar
44149bec60
Merge branch 'main' into feature-docker-as-non-root 2021-10-14 14:45:28 +02:00
Clémentine Urquizar
f80b4fdedd
Use pr_status isntead of status 2021-10-14 14:21:42 +02:00
bors[bot]
fd4a90549b
Merge #1803
1803: Import hotfix from `stable` into `main` (v0.23.1) r=curquiza a=curquiza



Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
2021-10-14 11:43:45 +00:00
Clémentine Urquizar
b602a0836a
Merge branch 'main' into stable 2021-10-14 13:43:21 +02:00
Clémentine Urquizar
7349fca607
Add Milestone Check status to bors.toml 2021-10-13 19:10:20 +02:00
bors[bot]
4bacc8e47d
Merge #1806
1806: Fix csv content-type error message r=curquiza a=sanders41

Fixes #1805

I was not sure if the `application/csv` [here](23f11e355d/meilisearch-http/tests/content_type.rs (L29)) should also be changed? I'm thinking yes, but `applicaiton/csv` is a bad type.

Co-authored-by: Paul Sanders <psanders1@gmail.com>
2021-10-13 10:11:47 +00:00
igaul
7141f89c5f
Split entrypoint and cmd 2021-10-12 11:44:59 -07:00
igaul
893654fb15
change default user name
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-12 11:42:08 -07:00
Paul Sanders
c9e1d054c7 Fix csv content-type error 2021-10-12 13:38:48 -04:00
bors[bot]
2e2eeb0a42
Merge #1801
1801: Update milli version to v0.17.3 to fix inference issue r=curquiza a=curquiza

Fixes #1798

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-12 14:47:06 +00:00
Clémentine Urquizar
0f342ac46e
Update MeiliSearch version 2021-10-12 16:43:31 +02:00
Clémentine Urquizar
29ac324e90
Update milli version to v0.17.3 2021-10-12 16:12:16 +02:00
bors[bot]
23f11e355d
Merge #1799
1799: Update README.md with Telemetry page r=curquiza a=maryamsulemani97

Updated readme to link the Telemetry page in the documentation 

Co-authored-by: maryamsulemani97 <90181761+maryamsulemani97@users.noreply.github.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-12 11:50:14 +00:00
Clémentine Urquizar
f09016b2bc
Update README.md 2021-10-12 13:49:31 +02:00
Clémentine Urquizar
1fa3aeceeb
Update README.md 2021-10-12 13:47:38 +02:00
maryamsulemani97
443afdc412
Update README.md 2021-10-12 14:37:19 +04:00
bors[bot]
776befc1f0
Merge #1797
1797: Import stable into main (v0.22.0) r=MarinPostma a=curquiza



Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: many <maxime@meilisearch.com>
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-11 16:43:01 +00:00
Clémentine Urquizar
3edbc74430
Merge branch 'main' into stable 2021-10-11 18:30:10 +02:00
bors[bot]
3172c96042
Merge #1795
1795: Update milli version r=irevoire a=curquiza

Closes #1788 

Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-10-11 14:27:46 +00:00
Clémentine Urquizar
60473637fe
Update milli version 2021-10-11 16:21:19 +02:00
bors[bot]
b969f34317
Merge #1793
1793: Remove memmap dependency r=curquiza a=palfrey

Fixes #1792. I was going to replace with [memmap2](https://github.com/RazrFalcon/memmap2-rs) which should be a drop-in replacement, but I couldn't actually find anything that actually directly used it. It ends up being a dependency in [milli](https://github.com/meilisearch/milli) so I'm going to go there next and fix that.

Co-authored-by: Tom Parker-Shemilt <palfrey@tevp.net>
2021-10-11 08:29:40 +00:00
Tom Parker-Shemilt
6c46fbbc57 Remove memmap dependency 2021-10-10 22:33:40 +01:00