When loading a dump with docker, we had two problems.
After creating a tempdirectory, uncompressing and re-indexing the dump:
1. We try to `move` the new “data.ms” onto the currently present
one. The problem is that if the `data.ms` is a mount point because
that's what peoples do with docker usually. We can't override
a mount point, and thus we were throwing an error.
2. The tempdir is created in `/tmp`, which is usually quite small AND may not
be on the same partition as the `data.ms`. This means when we tried to move
the dump over the `data.ms`, it was also failing because we can't move data
between two partitions.
==============
1 was fixed by deleting the *content* of the `data.ms` and moving the *content*
of the tempdir *inside* the `data.ms`. If someone tries to create volumes inside
the `data.ms` that's his problem, not ours.
2 was fixed by creating the tempdir *inside* of the `data.ms`. If a user mounted
its `data.ms` on a large partition, there is no reason he could not load a big
dump because his `/tmp` was too small. This solves the issue; now the dump is
extracted and indexed on the same partition the `data.ms` will lay.
fix#1833
2059: change indexed doc count on error r=irevoire a=MarinPostma
change `indexed_documents` and `deleted_documents` to return 0 instead of null when empty when the task has failed.
close#2053
Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2056: Allow any header for CORS r=curquiza a=curquiza
Bug fix: trigger a CORS error when trying to send the `User-Agent` header via the browser
`@bidoubiwa` thanks for the bug report!
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
429: Benchmark CIs: not use a default label to call the GH runner r=irevoire a=curquiza
Since we now have multiple self-hosted github runners, we need to differentiate them calling them in the CI. The `self-hosted` label is the default one, so we need to use the unique and appropriate one for the benchmark machine
<img width="925" alt="Capture d’écran 2022-01-04 à 15 42 18" src="https://user-images.githubusercontent.com/20380692/148079840-49cd7878-5912-46ff-8ab8-bf646777f782.png">
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
428: Reintroduce the gitignore for the fuzzer r=Kerollmops a=irevoire
Reintroduce the gitignore in the fuzz directory
Co-authored-by: Tamo <tamo@meilisearch.com>
425: Push the result of the benchmarks to influxdb r=irevoire a=irevoire
Now execute a benchmark for every PR merged into main and then upload the results to influxdb.
Co-authored-by: Tamo <tamo@meilisearch.com>
2011: bug(lib): drop env on last use r=curquiza a=MarinPostma
fixes the `too many open files` error when running tests by closing the
environment on last drop
To check that we are actually the last owner of the `env` we plan to drop, I have wrapped all envs in `Arc`, and check that we have the last reference to it.
Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2036: chore(ci): Enable rust_backtrace in the ci r=curquiza a=irevoire
This should help us to understand unreproducible panics that happens in the CI all the time
Co-authored-by: Tamo <tamo@meilisearch.com>
2035: Use self hosted GitHub runner r=curquiza a=curquiza
Checked with `@tpayet,` we have created a self hosted github runner to save time when pushing the docker images.
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2037: test: Ignore the auths tests on windows r=irevoire a=irevoire
Since the auths tests fail sporadically on the windows CI but we can't reproduce these failures with a real windows machine we are going to ignore these ones.
But we still ensure they compile.
Co-authored-by: Tamo <tamo@meilisearch.com>
Since the auths tests fail sporadically on the windows CI but we can't
reproduce these failures with a real windows machine we are going to
ignore theses one.
But we still ensure they compile.
2032: Revert docker as non root PR r=curquiza a=ManyTheFish
Revert #1759
hotfix for #1969
Co-authored-by: Maxime Legendre <maximelegendre@mbp-de-maxime.home>
2033: Bug(FS): Consider empty pre-created directory as unexisting DB r=curquiza a=ManyTheFish
When the database directory was pre-created we were considering that DB is invalid, we are now accepting to create a database in it.
Co-authored-by: Maxime Legendre <maximelegendre@mbp-de-maxime.home>
2026: Bug(auth): Parse YMD date r=curquiza a=ManyTheFish
Use NaiveDate to parse YMD date instead of NaiveDatetime
fix#2017
Co-authored-by: Maxime Legendre <maximelegendre@mbp-de-maxime.home>
2025: Fix security index creation r=ManyTheFish a=ManyTheFish
Forbid index creation on alternates routes when the action `index.create` is not given
fix#2024
Co-authored-by: Maxime Legendre <maximelegendre@MacBook-Pro-de-Maxime.local>
2008: bug(lib): fix get dumps bad error code r=curquiza a=MarinPostma
fix bad error code being returned whet getting a dump status, and add a test
close#1994
Co-authored-by: Marin Postma <postma.marin@protonmail.com>
2006: chore(http): rename task types r=curquiza a=MarinPostma
Rename
- documentsAddition into documentAddition
- documentsPartial into documentPartial
- documentsDeletion into documentDeletion
close#1999
2007: bug(lib): ignore primary if already set on document addition r=curquiza a=MarinPostma
Ignore the primary key if it is already set on documents updates. Add a test for verify behaviour.
close#2002
Co-authored-by: Marin Postma <postma.marin@protonmail.com>
1989: Extend API keys r=curquiza a=ManyTheFish
# Pull Request
## What does this PR do?
- Add API keys in snapshots
- Add API keys in dumps
- fix QA #1979fix#1979fix#1995fix#2001fix#2003
related to #1890
Co-authored-by: many <maxime@meilisearch.com>
- Add API keys in snapshots
- Add API keys in dumps
- Rename action indexes.add to indexes.create
- fix QA #1979fix#1979fix#1995fix#2001fix#2003
related to #1890
1982: Set fail-fast to false in publish-binaries CI r=curquiza a=curquiza
This avoids the other jobs to fail if one of the jobs fails.
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
