MeiliSearch/meilisearch/tests
meili-bors[bot] d4f10800f2
Merge #3834
3834: Define searchable fields at runtime r=Kerollmops a=ManyTheFish

## Summary
This feature allows the end-user to search in one or multiple attributes using the search parameter `attributesToSearchOn`:

```json
{
  "q": "Captain Marvel",
  "attributesToSearchOn": ["title"]
}
```

This feature act like a filter, forcing Meilisearch to only return the documents containing the requested words in the attributes-to-search-on. Note that, with the matching strategy `last`, Meilisearch will only ensure that the first word is in the attributes-to-search-on, but, the retrieved documents will be ordered taking into account the word contained in the attributes-to-search-on. 

## Trying the prototype

A dedicated docker image has been released for this feature:

#### last prototype version:

```bash
docker pull getmeili/meilisearch:prototype-define-searchable-fields-at-search-time-1
```

#### others prototype versions:

```bash
docker pull getmeili/meilisearch:prototype-define-searchable-fields-at-search-time-0
```

## Technical Detail

The attributes-to-search-on list is given to the search context, then, the search context uses the `fid_word_docids`database using only the allowed field ids instead of the global `word_docids` database. This is the same for the prefix databases.
The database cache is updated with the merged values, meaning that the union of the field-id-database values is only made if the requested key is missing from the cache.

### Relevancy limits

Almost all ranking rules behave as expected when ordering the documents.
Only `proximity` could miss-order documents if all the searched words are in the restricted attribute but a better proximity is found in an ignored attribute in a document that should be ranked lower. I put below a failing test showing it:
```rust
#[actix_rt::test]
async fn proximity_ranking_rule_order() {
    let server = Server::new().await;
    let index = index_with_documents(
        &server,
        &json!([
        {
            "title": "Captain super mega cool. A Marvel story",
            // Perfect distance between words in an ignored attribute
            "desc": "Captain Marvel",
            "id": "1",
        },
        {
            "title": "Captain America from Marvel",
            "desc": "a Shazam ersatz",
            "id": "2",
        }]),
    )
    .await;

    // Document 2 should appear before document 1.
    index
        .search(json!({"q": "Captain Marvel", "attributesToSearchOn": ["title"], "attributesToRetrieve": ["id"]}), |response, code| {
            assert_eq!(code, 200, "{}", response);
            assert_eq!(
                response["hits"],
                json!([
                    {"id": "2"},
                    {"id": "1"},
                ])
            );
        })
        .await;
}
```

Fixing this would force us to create a `fid_word_pair_proximity_docids` and a `fid_word_prefix_pair_proximity_docids` databases which may multiply the keys of `word_pair_proximity_docids` and `word_prefix_pair_proximity_docids` by the number of attributes in the searchable_attributes list. If we think we should fix this test, I'll suggest doing it in another PR.

## Related

Fixes #3772

Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: ManyTheFish <many@meilisearch.com>
2023-06-28 08:19:23 +00:00
..
assets serde ndjson fix 2022-12-21 11:27:15 +08:00
auth fix tests 2023-06-26 16:29:43 +02:00
common Use insta snapshot 2023-06-26 14:56:19 +02:00
dashboard Renames meilisearch-http to meilisearch 2022-12-08 08:22:53 -07:00
documents get rid of the invalid document delete filter in favor of the invalid document filter 2023-05-24 11:53:16 +02:00
dumps wait until all tasks are processed before running our dump integration tests 2023-04-05 18:35:43 +02:00
index add tests on the index resource 2023-01-24 13:20:20 +01:00
search Use insta snapshot 2023-06-26 14:56:19 +02:00
settings Fix tests 2023-01-19 15:48:20 +01:00
snapshot put back the sleep as it was and fix the from 2023-03-06 18:09:09 +01:00
stats Renames meilisearch-http to meilisearch 2022-12-08 08:22:53 -07:00
swap_indexes Fix tests 2023-01-19 15:48:20 +01:00
tasks fix the type of the document deletion by filter tasks 2023-05-30 15:18:52 +02:00
content_type.rs Fix non insta tests 2023-01-19 16:10:05 +01:00
integration.rs add functionnal + error tests on the swap_indexes route 2023-01-18 09:36:04 +01:00