1172: Fix atomic snapshot creation r=MarinPostma a=raszi
Compress gzip files to a temporary file first and then do an atomic rename.
In our setup we have an indexer which does snapshoting for the instances serving the requests. Since currently the snapshoting mechanism is replacing the file in place therefore the indexer could not share the snapshot with a live instance.
With this small patch we first create a new temporary file in the same directory as the snapshot dir and then we do an atomic rename therefore the snapshot path would always contain a valid snapshot.
After applying this change it would be enough to simply restart the serving instances to pick up the new snapshot from a shared storage without worrying them to die because of an incomplete snapshot.
Co-authored-by: KARASZI István <ikaraszi@gmail.com>
1176: fix race condition in document addition r=Kerollmops a=MarinPostma
As described in #1160, there was a race condition when updating settings and adding documents simultaneously. This was due to the schema being updated and document addition being processed in two different transactions. This PR moves the schema update logic for the primary key in the same transaction as the document addition, while maintaining the input checks for the validity of the primary key in the http route, in order not to break the error reporting for the document addition route.
close#1160.
Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: marin <postma.marin@protonmail.com>
1. NEEDS to ensure that service is completely up if it returns 204
2. DOES NOT block service process (write transaction)
3. NEEDS to use the less network bandwidth as possible when it's triggered
4. NEEDS to use the less service resources as possible when it's triggered
5. DOES NOT NEED any authentication
6. MAY be named /health
914: lazily create an index on documents push r=LegendreM a=qdequele
Create an index if it's possible when a user trying to send data to a non-existing index. https://github.com/meilisearch/MeiliSearch/issues/918
Co-authored-by: qdequele <quentin@meilisearch.com>
Co-authored-by: qdequele <quentin@dequelen.me>
946: Sort displayedAttributes field r=MarinPostma a=gorogoroumaru
Fix#943
displayedAttributes use the HashSet struct which is an unsorted structure, so I changed the implementation from HashSet into BTreeSet.
Co-authored-by: gorogoroumaru <zokutyou2@gmail.com>
* trigger backup importation via http route
* follow backup advancement with status route
* import backup via a command line
* let user choose batch size of documents to import (command lines)
closes#884closes#840
