Commit Graph

  • 31d460044b removing .await Santhosh Reddy Vootukuri (SUNNY) (from Dev Box) 2024-09-08 09:52:22 -07:00
  • 8fd0afaaaa
    Make sure we iterate over the payload documents in order Clément Renault 2024-09-05 22:31:17 +02:00
  • 72c6a21a30
    Use raw JSON to read the payloads Clément Renault 2024-09-05 20:08:23 +02:00
  • 8412be4a7d
    Cleanup CowStr and TopLevelMap struct Clément Renault 2024-09-05 18:32:55 +02:00
  • 10f09c531f
    add some commented code to read from json with raw values Louis Dureuil 2024-09-05 18:22:16 +02:00
  • 8fd99b111b Add tracing timers logs ManyTheFish 2024-09-05 17:36:19 +02:00
  • f6b3d1f9a5
    Increase some channel sizes Clément Renault 2024-09-05 15:12:07 +02:00
  • db0cf3b2ed
    Merge #4912 prototype-upgrade-offline-v1_9-v1_10-0 meili-bors[bot] 2024-09-05 09:11:23 +00:00
  • 73ce67862d
    Use the word pair proximity and fid word count docids extractors Clément Renault 2024-09-05 10:56:22 +02:00
  • f6abf01d2c
    Check REST embedders before touching the DB Louis Dureuil 2024-09-05 10:49:59 +02:00
  • 0fc02f7351
    Move the facet extraction to dedicated modules Clément Renault 2024-09-05 10:32:22 +02:00
  • 34f11e3380 Implement word count and word pair proximity extractors ManyTheFish 2024-09-05 10:30:39 +02:00
  • 28da759f11
    meilitool: Support dumpless upgrade from v1.9 to v1.10 when there are no REST embedders Louis Dureuil 2024-09-05 10:08:38 +02:00
  • ea96d19525
    Change versioning in meili Louis Dureuil 2024-09-05 10:08:06 +02:00
  • d352b1ee83
    Add serde to meilitool Louis Dureuil 2024-09-05 10:07:33 +02:00
  • 27308eaab1
    Import the facet extractors Clément Renault 2024-09-04 17:58:15 +02:00
  • b33ec9ba3f
    Introduce the FieldIdFacetIsNullDocidsExtractor Clément Renault 2024-09-04 17:50:08 +02:00
  • 9c0a1cd9fd
    Introduce the FieldIdFacetExistsDocidsExtractor Clément Renault 2024-09-04 17:48:49 +02:00
  • 0b061f1e70
    Introduce the FieldIdFacetIsEmptyDocidsExtractor Clément Renault 2024-09-04 17:40:24 +02:00
  • 19d937ab21
    Introduce the facet extractors Clément Renault 2024-09-04 17:03:09 +02:00
  • 1d59c19cd2
    Send the WordsFst by using an Mmap Clément Renault 2024-09-04 14:30:09 +02:00
  • 98e48371c3
    Factorize some stuff Clément Renault 2024-09-04 12:17:13 +02:00
  • 6d74fb0229
    Introduce the WordFidWordDocids database Clément Renault 2024-09-04 11:39:53 +02:00
  • 1eb75a1040 remove milli/src/update/new/extract/tokenize_document.rs ManyTheFish 2024-09-04 11:40:26 +02:00
  • 3b82d8b5b9
    Fix the cache to serialize entries correctly Clément Renault 2024-09-04 10:55:06 +02:00
  • 781a186f75 remove milli/src/update/new/extract/extract_word_docids.rs ManyTheFish 2024-09-04 10:28:31 +02:00
  • 6a399556b5 Implement more searchable extractor ManyTheFish 2024-09-04 10:20:18 +02:00
  • 27b4cab857
    Extract and write the documents and words fst in the database Clément Renault 2024-09-04 09:59:19 +02:00
  • 3f3cebf5f9
    Bump quinn-proto from 0.11.3 to 0.11.8 dependabot[bot] 2024-09-03 20:50:30 +00:00
  • b278815617
    Merge #4908 meili-bors[bot] 2024-09-03 14:28:12 +00:00
  • 52d32b4ee9
    Move the channel sender in the closure to stop the merger thread Clément Renault 2024-09-03 16:08:33 +02:00
  • da61408e52 Remove unimplemented from document changes ManyTheFish 2024-09-03 15:14:16 +02:00
  • fe69385bd7 Fix tokenizer test ManyTheFish 2024-09-03 14:24:37 +02:00
  • 40e13ceef3
    Merge #4892 meili-bors[bot] 2024-09-03 11:50:07 +00:00
  • 18a2c13e4e
    add analytics Louis Dureuil 2024-09-02 14:04:24 +02:00
  • ed19b7c3c3
    Only reindex if the size increased Louis Dureuil 2024-09-02 13:52:18 +02:00
  • 66bda2ce8a
    fix tests Louis Dureuil 2024-08-28 09:10:09 +02:00
  • 1ac008926b
    Add maxBytes parameter Louis Dureuil 2024-08-27 17:52:09 +02:00
  • c49d892c82
    Changes to prompt Louis Dureuil 2024-09-03 12:07:10 +02:00
  • de962a26f3
    New error type when maxBytes is null Louis Dureuil 2024-08-27 17:47:02 +02:00
  • c1557734dc
    Use the GlobalFieldsIdsMap everywhere and write it to disk Clément Renault 2024-09-03 12:01:01 +02:00
  • 005204e9e5 make the code of init_web_app in common between most tests Tamo 2024-09-03 11:40:05 +02:00
  • 1040e5e2b4 spawn on search queue per test Tamo 2024-09-03 11:20:25 +02:00
  • c50d3edc4a Integrate first searchable exctrator ManyTheFish 2024-09-03 11:02:39 +02:00
  • 7bfcd76628
    feat(prometheus): add build_info metric Nicolas Lamirault 2024-09-03 10:12:08 +02:00
  • 80408c92dc
    Merge #4906 meili-bors[bot] 2024-09-03 07:14:58 +00:00
  • 5369bf4a62
    Change some lifetimes Clément Renault 2024-09-02 19:51:22 +02:00
  • bcb1aa3d22
    Find a temporary solution to par into iter on an HashMap Clément Renault 2024-09-02 19:39:48 +02:00
  • fa1a0beb0c fix conflicts after rebase Tamo 2024-09-02 18:15:42 +02:00
  • 5aefe7cd17 add the snapshots Tamo 2024-09-02 10:53:33 +02:00
  • e6dd66e4a0 Do not fail the whole batch when a single document deletion by filter fails Tamo 2024-09-02 10:34:28 +02:00
  • 6e3839d8b6 autobatch document deletion by filter Tamo 2024-08-29 15:56:24 +02:00
  • cd271b8762 stop trying to process searches after one minute Tamo 2024-08-28 19:01:54 +02:00
  • 3ce8500d4c ensure we never early exit when we have a permit and remove the warning when we implicitely drop a permit Tamo 2024-08-28 15:17:10 +02:00
  • 588000d398 add a warning to help us find when we forget to drop explicitely drop a permit Tamo 2024-08-28 14:37:55 +02:00
  • 92b151607c explicitely drop the search permit Tamo 2024-08-28 14:29:25 +02:00
  • 42e7499260 Update version for the next release (v1.10.1) in Cargo.toml irevoire 2024-08-27 16:24:41 +00:00
  • 41aa1e1424 Only spawn one search queue in actix-web Tamo 2024-08-27 17:19:25 +02:00
  • 9b7858fb90
    Expose the new indexer Clément Renault 2024-09-02 15:21:59 +02:00
  • ab01679a8f
    Remove the useless option from the document changes Clément Renault 2024-09-02 15:21:00 +02:00
  • 521775f788
    I push for Many Clément Renault 2024-09-02 15:10:21 +02:00
  • 72e7b7846e
    Renaming the indexers Clément Renault 2024-09-02 14:42:27 +02:00
  • 6526ce1208
    Fix the merging of documents Clément Renault 2024-09-02 14:41:20 +02:00
  • 24ace5c381
    Add reindexing test Louis Dureuil 2024-09-02 13:37:01 +02:00
  • 21296190a3
    Reindex embedders Louis Dureuil 2024-09-02 12:58:09 +02:00
  • 03fda78901
    update other tests Louis Dureuil 2024-09-02 11:31:31 +02:00
  • 30a143f149
    Test new facilities Louis Dureuil 2024-09-02 11:31:23 +02:00
  • 4464d319af
    Change default template to use the new facility Louis Dureuil 2024-09-02 11:30:59 +02:00
  • 580ea2f450
    Pass the fields <-> ids map with metadata to render Louis Dureuil 2024-09-02 11:30:10 +02:00
  • 915cf4bae5
    Add field.is_searchable property to fields Louis Dureuil 2024-09-02 11:28:53 +02:00
  • 3f517dfae6
    Merge #4905 v1.10.1 release-v1.10.1 meili-bors[bot] 2024-09-02 09:00:11 +00:00
  • 11a8e537ed add the snapshots Tamo 2024-09-02 10:53:33 +02:00
  • e639ec79d1
    Move the indexers into their own modules Clément Renault 2024-09-02 10:42:19 +02:00
  • 959aeb0df3 Do not fail the whole batch when a single document deletion by filter fails Tamo 2024-09-02 10:34:28 +02:00
  • bb885a5810
    Fix the merge for roaring bitmap Clément Renault 2024-09-01 23:20:19 +02:00
  • b625d31c7d
    Introduce the PartialDumpIndexer indexer that generates document ids in parallel Clément Renault 2024-08-30 15:07:21 +02:00
  • 6487a67f2b
    Introduce the ConcurrentAvailableIds struct and rename the other to AvailableIds Clément Renault 2024-08-30 15:06:50 +02:00
  • 271ce91b3b
    Add the rayon Threadpool to the index function parameter Clément Renault 2024-08-30 14:34:24 +02:00
  • 54f2eb4507
    Remove duplication of grenad merger Clément Renault 2024-08-30 14:33:58 +02:00
  • 794ebcd582
    Replace grenad with the new grenad various-improvement branch Clément Renault 2024-08-30 11:49:47 +02:00
  • b7c77c7a39
    Use the latest version of the obkv crate Clément Renault 2024-08-30 10:03:54 +02:00
  • 0c57cf7565
    Replace obkv with the temporary new version of it Clément Renault 2024-08-29 19:20:10 +02:00
  • 27df9e6c73
    Introduce the indexer::index function that runs the indexation Clément Renault 2024-08-29 18:27:02 +02:00
  • 45c060831e
    Introduce typed channels and the merger loop Clément Renault 2024-08-29 17:51:42 +02:00
  • 874c1ac538
    First channels types Clément Renault 2024-08-29 15:07:59 +02:00
  • e6ffa4d454
    Implement the document merge function for the replace method Clément Renault 2024-08-29 14:08:31 +02:00
  • 637a9c8bdd
    Implement the document merge function for the update method Clément Renault 2024-08-29 12:06:44 +02:00
  • c683fa98e6
    WIP Louis Dureuil 2024-08-28 18:45:16 +02:00
  • 4daa27a8e8
    Merge #4901 meili-bors[bot] 2024-08-29 15:46:16 +00:00
  • 34ebcd378e autobatch document deletion by filter Tamo 2024-08-29 15:56:24 +02:00
  • 5fed4d035a
    Merge #4899 meili-bors[bot] 2024-08-29 07:10:42 +00:00
  • 11bee34bf0 stop trying to process searches after one minute Tamo 2024-08-28 19:01:54 +02:00
  • 541a23b17a
    Merge #4898 meili-bors[bot] 2024-08-28 13:33:38 +00:00
  • 69ab09c149 ensure we never early exit when we have a permit and remove the warning when we implicitely drop a permit Tamo 2024-08-28 15:17:10 +02:00
  • 25534cc794
    Merge #4895 meili-bors[bot] 2024-08-28 12:52:29 +00:00
  • 17d0b10765 Update version for the next release (v1.10.1) in Cargo.toml irevoire 2024-08-27 16:24:41 +00:00
  • 241746e7f4 add a warning to help us find when we forget to drop explicitely drop a permit Tamo 2024-08-28 14:37:55 +02:00
  • 1b90e6ce5f explicitely drop the search permit Tamo 2024-08-28 14:29:25 +02:00
  • c10d06febc
    Merge #4896 meili-bors[bot] 2024-08-28 07:46:05 +00:00
  • 9eee467226
    Merge #4893 meili-bors[bot] 2024-08-27 20:26:59 +00:00