Kerollmops
46e6d23dd2
Remove the zstd dependency comming from actix-web/http
2022-03-17 11:00:25 +01:00
Kerollmops
86c1e83ea1
Remove three unused dependencies
2022-03-17 11:00:24 +01:00
Sai Kumar
e271395971
chore(all): bump milli
...
* updates to Use the milli's heed dependency #2210
* Update index.rs
* Update store.rs
* Update mod.rs
* cargo fmt
2022-03-16 16:34:44 +01:00
ad hoc
b57c59baa4
sequential extractor
2022-03-04 20:43:12 +01:00
Rob Ede
15150db957
clippy
2022-02-28 19:03:38 +01:00
Rob Ede
3b2e467ca6
update actix-web dependency to 4.0
2022-02-28 19:03:37 +01:00
bors[bot]
c3e3c900f2
Merge #2173
...
2173: chore(all): replace chrono with time r=irevoire a=irevoire
Chrono has been unmaintained for a few month now and there is a CVE on it.
Also I updated all the error messages related to the API key as you can see here: https://github.com/meilisearch/specifications/pull/114
fix #2172
Co-authored-by: Irevoire <tamo@meilisearch.com>
2022-02-17 14:12:23 +00:00
Irevoire
05c8d81e65
chore: get rid of chrono in favor of time
...
Chrono has been unmaintened for a few month now and there is a CVE on it.
make clippy happy
bump milli
2022-02-16 18:14:29 +01:00
Clémentine Urquizar
e2a9414c7a
Update version (v0.26.0)
2022-02-14 16:11:07 +01:00
Clémentine Urquizar
1a87b2f37d
Bump milli to v0.22.1
2022-02-08 11:21:44 +01:00
mpostma
c9a236b0af
feat(lib): auto-batching
2022-02-01 18:06:20 +01:00
bors[bot]
622c15e825
Merge #2096
...
2096: feat(auth): Tenant token r=Kerollmops a=ManyTheFish
Make meilisearch support JWT authentication signed with meilisearch API keys
using HS256, HS384 or HS512 algorithms.
Related spec: [specifications#89](https://github.com/meilisearch/specifications/pull/89 ) [rendered](https://github.com/meilisearch/specifications/blob/scoped-api-keys/text/0089-tenant-tokens.md )
Fix #1991
Co-authored-by: ManyTheFish <many@meilisearch.com>
2022-01-27 10:38:41 +00:00
ManyTheFish
7ca647f0d0
feat(auth): Implement Tenant token
...
Make meilisearch support JWT authentication signed with meilisearch API keys
using HS256, HS384 or HS512 algorithms.
Related spec: https://github.com/meilisearch/specifications/pull/89
Fix #1991
2022-01-27 08:25:39 +01:00
Clémentine Urquizar - curqui
aa50fcb1f0
Merge branch 'main' into stable
2022-01-26 20:17:41 +01:00
Tamo
7828da15c3
feat(analytics): send the whole set of cli options instead of only the snapshot
2022-01-26 13:52:41 +01:00
Rob Ede
9f5fee404b
chore(all): update actix-web dependency to 4.0.0-beta.21
2022-01-21 20:44:17 +00:00
Tamo
436f61a7f4
chore: bump meilisearch
2022-01-18 12:27:15 +01:00
Tamo
3fab5869fa
chore: bump milli
2022-01-18 11:50:17 +01:00
mpostma
f6d53e03f1
chore(http): migrate from structopt to clap3
2022-01-12 14:07:19 +01:00
many
ee7970f603
feat(auth): Extend API keys
...
- Add API keys in snapshots
- Add API keys in dumps
- Rename action indexes.add to indexes.create
- fix QA #1979
fix #1979
fix #1995
fix #2001
fix #2003
related to #1890
2021-12-14 17:33:39 +01:00
Clémentine Urquizar
ae73386723
Update version for the next release (v0.25.0)
2021-12-07 14:00:43 +01:00
many
ae2b0e7aa7
Use milli reexported tokenizer instead of importing meilisearch-tokenizer dependency
2021-12-06 17:18:28 +01:00
many
ffefd0caf2
feat(auth): API keys
...
implements:
https://github.com/meilisearch/specifications/blob/develop/text/0085-api-keys.md
- Add tests on API keys management route (meilisearch-http/tests/auth/api_keys.rs)
- Add tests checking authorizations on each meilisearch routes (meilisearch-http/tests/auth/authorization.rs)
- Implement API keys management routes (meilisearch-http/src/routes/api_key.rs)
- Create module to manage API keys and authorizations (meilisearch-auth)
- Reimplement GuardedData to extend authorizations (meilisearch-http/src/extractors/authentication/mod.rs)
- Change X-MEILI-API-KEY by Authorization Bearer (meilisearch-http/src/extractors/authentication/mod.rs)
- Change meilisearch routes to fit to the new authorization feature (meilisearch-http/src/routes/)
- close #1867
2021-12-06 09:52:41 +01:00
Marin Postma
a30e02c18c
feat(all): Task store
...
implements:
https://github.com/meilisearch/specifications/blob/develop/text/0060-refashion-updates-apis.md
linked PR:
- #1889
- #1891
- #1892
- #1902
- #1906
- #1911
- #1914
- #1915
- #1916
- #1918
- #1924
- #1925
- #1926
- #1930
- #1936
- #1937
- #1942
- #1944
- #1945
- #1946
- #1947
- #1950
- #1951
- #1957
- #1959
- #1960
- #1961
- #1962
- #1964
- https://github.com/meilisearch/milli/pull/414
- https://github.com/meilisearch/milli/pull/409
- https://github.com/meilisearch/milli/pull/406
- https://github.com/meilisearch/milli/pull/418
- close #1687
- close #1786
- close #1940
- close #1948
- close #1949
- close #1932
- close #1956
2021-12-02 20:14:42 +01:00
many
cc6306c0e1
Update milli version
2021-11-04 14:57:45 +01:00
many
b664a46e91
Update milli version
2021-11-03 16:11:20 +01:00
Tamo
c4737749ab
bump segment to be able to display a user
2021-10-29 17:25:53 +02:00
Tamo
de35a9a605
use an official release of segment
2021-10-29 17:25:53 +02:00
Tamo
9be90011c6
save the user-id in the config dir of the OS
2021-10-29 17:25:51 +02:00
Tamo
664d09e86a
makes the analytics works with the option and the feature
2021-10-29 17:25:47 +02:00
Tamo
e226b1a87f
rewrite the main analytics module and the information sent in the tick
2021-10-29 17:25:42 +02:00
bors[bot]
cf67964133
Merge #1848
...
1848: Error format and Definition r=MarinPostma a=ManyTheFish
Co-authored-by: many <maxime@meilisearch.com>
2021-10-28 14:15:35 +00:00
many
3a29cbf0ae
Use milli v0.20.0
2021-10-28 15:59:06 +02:00
many
cbaca2b579
Fix PR comments
2021-10-28 15:42:42 +02:00
Clémentine Urquizar
a76d9b15c9
Update version for the next release (v0.24.0)
2021-10-28 12:24:49 +02:00
many
ff0908d3fa
Ignore errors tests that show unrelated bugs
2021-10-28 11:41:59 +02:00
many
7464720426
Fix some errors
2021-10-28 10:47:59 +02:00
marin postma
4ac005b094
optimize document transform
...
fix error types
bump milli
2021-10-26 13:51:15 +02:00
Clémentine Urquizar
0f342ac46e
Update MeiliSearch version
2021-10-12 16:43:31 +02:00
Clémentine Urquizar
29ac324e90
Update milli version to v0.17.3
2021-10-12 16:12:16 +02:00
Clémentine Urquizar
3edbc74430
Merge branch 'main' into stable
2021-10-11 18:30:10 +02:00
Clémentine Urquizar
60473637fe
Update milli version
2021-10-11 16:21:19 +02:00
Tom Parker-Shemilt
6c46fbbc57
Remove memmap dependency
2021-10-10 22:33:40 +01:00
bors[bot]
ddbcf449da
Merge #1763
...
1763: Index tests r=MarinPostma a=MarinPostma
This pr aims to test more thorougly the usage on index in the meilisearch database, by writing unit tests.
work included:
- [x] Create index mock and stub methods
- [x] Test snapshot creation
- [x] Test Dumps
- [x] Test search
Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-10-06 14:39:53 +00:00
mpostma
85ae34cf9f
test snapshots
2021-10-06 14:10:23 +02:00
Tamo
fca686e7f8
bump meilisearch
2021-10-04 13:52:37 +02:00
Clémentine Urquizar
b17dae9ac0
Update version for the next release (v0.23.0)
2021-09-29 18:40:35 +02:00
bors[bot]
5fad37aebd
Merge #1711
...
1711: MeiliSearch refactor introducing OBKV format r=MarinPostma a=MarinPostma
This PR refactor some multiple components of meilisearch, and introduce the obkv document format to meilisearch
- [x] Split meilisearch-http and meilisearch-lib
- [x] Replace `IndexActor` and `UuidResolver` with `IndexResolver`
- [x] Remove mentions to Actor
- [x] Remove Actor traits to simplify code
- [x] Integrate obkv document format
- [x] Remove `Data`
- [x] Restore all route
- [x] Replace `Box<dyn error>` with `anyhow::Error`
- [x] Introduce update file store
- [x] Update file store error handling
- [x] Fix dumps
- [x] Fix snapshots
- [x] Fix tests
- [x] Update module documentation
- [x] add csv suppport (feat `@ManyTheFish` #1729 )
- [x] add jsonl support
- [x] integrate geosearch (feat `@irevoire` #1725 )
partially implements #1691 and #1690 . The error handling is very basic now, I will finish it in the next pr.
Some unit tests have been disabled, I will re-enable them ASAP, but they need a bit more work.
close #1531
P.S: sorry for this monstrous PR :'(
Co-authored-by: mpostma <postma.marin@protonmail.com>
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: many <maxime@meilisearch.com>
2021-09-29 14:38:55 +00:00
mpostma
311933614e
bump milli to v0.17.0
2021-09-29 15:44:54 +02:00
mpostma
1f537e1b60
jsonl support
2021-09-29 11:28:02 +02:00
mpostma
6e8a3fe8de
move csv parsing to document_formats
2021-09-28 22:58:48 +02:00
many
2a14948123
Use an existing revision of milli
2021-09-28 22:30:34 +02:00
many
d30830a55c
Add csv deserializer for documents
2021-09-28 22:28:13 +02:00
mpostma
5fa9bc67d7
remove unused dependencies
2021-09-28 22:16:18 +02:00
mpostma
3503fbf7fe
re-export milli from meilisearch_lib
2021-09-28 22:08:03 +02:00
mpostma
692c676625
fix tests
2021-09-28 18:57:36 +02:00
Tamo
654f49ccec
[WIP] put milli on branch main
2021-09-28 14:50:26 +02:00
Tamo
c1376a9f2a
add the geosearch to Meilisearch
2021-09-28 14:50:26 +02:00
mpostma
5353be74c3
refactor index actor
2021-09-22 15:07:04 +02:00
Tamo
2e99fa8251
remove the cargo.lock again
2021-09-22 11:30:33 +02:00
mpostma
aa6c5df0bc
Implement documents format
...
document reader transform
remove update format
support document sequences
fix document transform
clean transform
improve error handling
add documents! macro
fix transform bug
fix tests
remove csv dependency
Add comments on the transform process
replace search cli
fmt
review edits
fix http ui
fix clippy warnings
Revert "fix clippy warnings"
This reverts commit a1ce3cd96e603633dbf43e9e0b12b2453c9c5620.
fix review comments
remove smallvec in transform loop
review edits
2021-09-21 16:58:33 +02:00
mpostma
60518449fc
split meilisearch-http and meilisearch-lib
2021-09-21 13:23:22 +02:00
mpostma
e14640e530
refactor meilisearch
2021-09-20 14:54:20 +02:00
happysalada
770b6d25ae
deps: unify pest dependency
2021-09-15 12:15:44 +09:00
Felix Yan
a57943b77e
Use tikv-jemallocator instead of jemallocator
...
`jemallocator` has been abandoned for nearly two years, and `rustc`
itself moved to use `tikv-jemallocator` instead:
3965773ae7
2021-09-14 18:30:24 +03:00
mpostma
0f7625e29a
bump dependencies
2021-09-13 15:17:08 +02:00
Clémentine Urquizar
6d2f7af642
Bump milli: fix some crashes
2021-09-13 12:14:54 +02:00
Tamo
49c918defa
bump milli
2021-09-08 17:41:47 +02:00
Clémentine Urquizar
33514b28be
Merge pull request #1588 from meilisearch/test-new-indexer
...
Integrate the new indexer
2021-09-06 10:21:42 +02:00
Clément Renault
7e80337e5b
Bump milli to v0.12.0
2021-09-02 18:19:12 +02:00
many
8d4723d91b
Update lock file
2021-09-02 18:19:12 +02:00
Clémentine Urquizar
63e67f72e3
Update tokenizer and new milli version
2021-09-02 18:19:00 +02:00
Kerollmops
0cd66c3a89
Bump the milli version
2021-09-02 18:19:00 +02:00
Clément Renault
b092a624ed
Introduce the MaxMemory struct that defaults to 2/3 of the available memory
2021-09-02 18:18:59 +02:00
Clémentine Urquizar
24e84d7ca1
Test new indexer
2021-09-02 18:11:20 +02:00
Clémentine Urquizar
90116155b4
Update version for the next release (v0.22.0)
2021-09-01 12:33:30 +02:00
bors[bot]
19c5c74291
Merge #1652 #1654 #1657
...
1652: Remove dependabot r=MarinPostma a=curquiza
Fixes #1649
Dependabot for vulnerability and security updates is still activated.
1654: Add Script for Windows r=MarinPostma a=singh08prashant
fixes #1570
changes:
1. added script for detecting windows os running git bash
2. appended `.exe` to `$release_file` for windows as listed [here](https://github.com/meilisearch/MeiliSearch/releases/ )
3. removed global `$BINARY_NAME='meilisearch'` as windows require `.exe` file
1657: Bring vergen hotfix from `stable` to `main` r=MarinPostma a=curquiza
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
Co-authored-by: singh08prashant <singh08prashant@gmail.com>
Co-authored-by: Kerollmops <clement@meilisearch.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
2021-08-31 14:31:42 +00:00
Kerollmops
b8c954eb3f
Bump the MeiliSearch version to v0.21.1
2021-08-30 17:41:25 +02:00
Kerollmops
1782753387
Bump vergen and remove unused build feature
2021-08-30 15:03:45 +02:00
Kerollmops
51387b2c80
Introduce the new invalid sortable error codes
2021-08-25 17:29:30 +02:00
Irevoire
672fcee8aa
remove sentry
2021-08-24 12:38:31 +02:00
Clémentine Urquizar
149f46c184
Fix pacic when decoding
2021-08-23 12:37:51 +02:00
Clémentine Urquizar
38fc876704
Update tokenizer and new milli version with new tags
2021-08-18 14:55:10 +02:00
Clémentine Urquizar
39d5a99095
Update milli and tokenizer version
2021-08-18 12:09:34 +02:00
bors[bot]
2beb306834
Merge #1577
...
1577: Update milli dependency: fix facet values bugs r=Kerollmops a=curquiza
Fixes #1576
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-08-16 16:13:42 +00:00
Clémentine Urquizar
f3e595e2f0
Update milli dependency
2021-08-16 13:36:42 +02:00
Clémentine Urquizar
7531280764
Update tokenizer version to v0.2.4
2021-08-12 13:55:47 +02:00
bors[bot]
7e3b2ddff2
Merge #1554
...
1554: Fix dump v1 (attributesForFaceting, and criteria) r=curquiza a=MarinPostma
close #1553
Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-08-05 19:45:52 +00:00
mpostma
8f05d8d546
fix clippy warnings
2021-08-05 16:00:47 +02:00
Clémentine Urquizar
31fe263356
Update milli to v0.9.0
2021-08-05 13:08:27 +02:00
bors[bot]
9810f6b695
Merge #1540
...
1540: Update milli to version 0.8.1 r=curquiza a=curquiza
Integrates this fix into MeiliSearch https://github.com/meilisearch/milli/pull/296
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-07-29 17:15:52 +00:00
mpostma
53b4b2fcbc
Use serdeval for validating json format.
2021-07-29 18:02:54 +02:00
Clémentine Urquizar
9a8629a6a9
Update milli
2021-07-29 17:45:31 +02:00
Clémentine Urquizar
e5dabf265a
Update milli version to v0.8.0
2021-07-28 10:52:47 +02:00
Tamo
0038b3848a
add a simple github cache
2021-07-26 15:31:26 +02:00
Tamo
f79b8287f5
update vergen
2021-07-26 15:25:30 +02:00
Kerollmops
0353fbb5df
Bump the tokenizer version to v0.2.4
2021-07-22 17:14:45 +02:00
Kerollmops
838ed1cd32
Use an u16 field id instead of one byte
2021-07-06 11:58:03 +02:00
Kerollmops
91c5d0c042
Use the AlwaysFreePages flag when opening an index
2021-07-05 16:36:13 +02:00
Tamo
9949a2a930
bump milli to 0.7.2
2021-07-05 12:19:27 +02:00
Kerollmops
a6b4069172
Bump to v0.7.2
2021-07-05 10:54:53 +02:00
Tamo
fc09d77e89
fix the benchmarks dependcies
2021-07-01 11:38:30 +02:00
Clémentine Urquizar
3c149d8a43
Update tokenizer version to v0.2.3
2021-06-30 18:41:35 +02:00
Clémentine Urquizar
4f0d3b065f
Update milli
2021-06-30 18:39:06 +02:00
Clémentine Urquizar
01de7f9e36
Update version
2021-06-30 15:59:59 +02:00
Clémentine Urquizar
b489515f4d
Update milli version to v0.7.1
2021-06-30 13:52:46 +02:00
Tamo
ec809ca487
use rustls instead of openssl and remove all default-features of reqwest
2021-06-29 13:07:40 +02:00
Clémentine Urquizar
c09e610bb5
Update heed and milli
2021-06-29 10:25:47 +02:00
Clément Renault
80c6aaf1fd
Bump milli to 0.7.0
2021-06-28 18:31:56 +02:00
Clément Renault
bdc5599b73
Bump heed to use the git repo with v0.12.0
2021-06-28 18:26:20 +02:00
Tamo
a59f437ee3
use only half of the computer threads for the indexation by default
2021-06-28 14:35:50 +02:00
marin postma
74bb748a4e
bump milli to 0.6.0
2021-06-23 18:40:19 +02:00
Kerollmops
98285b4b18
Bump milli to 0.6.0
2021-06-23 17:30:26 +02:00
Clémentine Urquizar
9885fb4159
Update version for the next release (v0.5.1)
2021-06-23 14:05:20 +02:00
Clémentine Urquizar
0de696feaf
Update version for alpha 6
2021-06-22 18:40:51 +02:00
Clémentine Urquizar
e2844f3a92
Update tokenizer version to v0.2.3
2021-06-22 15:57:47 +02:00
Clémentine Urquizar
2e3d85c31a
Update milli version to v0.5.0
2021-06-22 15:57:46 +02:00
Tamo
77eb37934f
add jemalloc to http-ui and the benchmarks
2021-06-22 14:17:56 +02:00
Clémentine Urquizar
320670f8fe
Update version for the next release (v0.5.0)
2021-06-21 15:59:17 +02:00
marin postma
0bfdf9a785
bump milli
2021-06-21 12:11:09 +02:00
marin postma
abdf642d68
integrate milli errors
2021-06-21 12:11:08 +02:00
Clémentine Urquizar
35fcc351a0
Update version for the next release (v0.4.2)
2021-06-20 17:37:24 +02:00
Kerollmops
ccd6f13793
Update version to the next release (0.4.1)
2021-06-17 15:01:20 +02:00
bors[bot]
2f2484e186
Merge #212
...
212: bump milli to 0.4.0 r=MarinPostma a=MarinPostma
Co-authored-by: marin postma <postma.marin@protonmail.com>
2021-06-16 15:42:34 +00:00
bors[bot]
2062b10b79
Merge #209
...
209: Integrate amplitude r=MarinPostma a=irevoire
And merge the sentry and amplitude usage under one “Enable analytics” flag
closes #180
Co-authored-by: Tamo <tamo@meilisearch.com>
Co-authored-by: Irevoire <tamo@meilisearch.com>
2021-06-16 15:25:31 +00:00
marin postma
5a47cef9a8
bump milli to 0.4.0
2021-06-16 17:15:56 +02:00
Clémentine Urquizar
f5ff3e8e19
Update version for the next release (v0.4.0)
2021-06-16 14:01:05 +02:00
Tamo
def1596eaf
Integrate amplitude
...
And merge the sentry and amplitude usage under one “Enable analytics”
flag
2021-06-15 15:36:30 +02:00
Clémentine Urquizar
fe5a494035
Update alpha for the next release
2021-06-14 17:55:04 +02:00
Kerollmops
312c2d1d8e
Use the Error enum everywhere in the project
2021-06-14 16:58:38 +02:00
bors[bot]
b119bb4ab0
Merge #197
...
197: Update milli (v0.3.1) with filterable attributes r=MarinPostma a=curquiza
Fixes #187 and #70
Also fixes #195
Co-authored-by: Clémentine Urquizar <clementine@meilisearch.com>
2021-06-14 12:19:42 +00:00
Clémentine Urquizar
9996c59183
Update with milli 0.3.1
2021-06-14 13:20:43 +02:00
Clémentine Urquizar
aa04124bfc
Add changes according to milli update
2021-06-14 13:20:37 +02:00
marin postma
0f767e3743
conccurrent update run loop
2021-06-14 10:57:14 +02:00
bors[bot]
1e659bb17b
Merge #194
...
194: Bump sentry version r=MarinPostma a=irevoire
closes #102
Co-authored-by: tamo <tamo@meilisearch.com>
2021-06-14 08:34:04 +00:00
Clémentine Urquizar
7d5395c12b
Update Tokenizer version to v0.2.3
2021-06-10 17:00:04 +02:00
Clémentine Urquizar
dc64e139b9
Update version for the next release (v0.3.1)
2021-06-09 14:39:21 +02:00
Kerollmops
103dddba2f
Move the UpdateStore into the http-ui crate
2021-06-08 17:59:51 +02:00
Kerollmops
29824d05ab
Reduce the length of the benchmarks names
2021-06-03 15:59:43 +02:00
tamo
4536dfccd0
add a way to provide primary_key or autogenerate documents ids
2021-06-02 11:13:20 +02:00
tamo
dffbaca63b
bump sentry version
2021-05-31 13:59:31 +02:00
tamo
26dcb9e66d
bump milli version and fix a performance issue for large dumps
2021-05-10 20:25:12 +02:00
Clémentine Urquizar
1e11578ef0
Update version for the next release (v0.2.1)
2021-05-05 14:57:34 +02:00
Clémentine Urquizar
eb03a3ccb1
Upgrade Milli and Tokenizer
2021-05-04 17:56:19 +02:00
Marin Postma
928fb34eff
bump milli and fix tests
2021-05-04 15:10:22 +02:00
Clémentine Urquizar
a8680887d8
Upgrade Milli version (v0.2.0)
2021-05-03 14:50:47 +02:00
Clémentine Urquizar
34e02aba42
Upgrade Tokenizer version (v0.2.2)
2021-05-03 10:55:55 +02:00
Marin Postma
78217bcf18
Fix cors authentication issue
2021-04-29 16:28:12 +02:00
Marin Postma
a961f0ce75
fix clippy warnings
2021-04-27 18:28:46 +02:00
Marin Postma
4fe2a13c71
rewrite update store
2021-04-27 15:20:52 +02:00
Marin Postma
5a38f13cae
multi_index udpate store
2021-04-27 15:07:13 +02:00
many
0d7d3ce802
Update roaring package
2021-04-27 14:39:53 +02:00
many
71740805a7
Fix forgotten typo tests
2021-04-27 14:39:53 +02:00
Kerollmops
0f4c0beffd
Introduce the Attribute criterion
2021-04-27 14:25:34 +02:00
Clémentine Urquizar
f3b6bf55a6
Update version for the next release (alpha4)
2021-04-26 19:05:16 +02:00
Marin Postma
e4bd1bc5ce
update actix-web-static-file rev
2021-04-22 11:42:41 +02:00
Marin Postma
6bcf20c70e
serve static site
2021-04-22 10:26:54 +02:00
Marin Postma
bb79695e44
load mini-dashboard assets
2021-04-22 10:26:54 +02:00
Marin Postma
ea5517bc8c
add mini-dashboard feature
2021-04-22 10:26:54 +02:00
Marin Postma
c6bb36efa5
implement _formated
2021-04-20 21:23:28 +02:00
Clémentine Urquizar
b8e535579f
Update version for the next release (alpha3)
2021-04-20 16:11:07 +02:00
Marin Postma
f8c338e3a7
add test for dedicated distinct route
2021-04-20 15:49:17 +02:00
mpostma
ec230c2835
enable distinct
2021-04-20 11:29:06 +02:00
Marin Postma
45665245dc
bump actix
2021-04-20 11:07:23 +02:00
Kerollmops
c9b2d3ae1a
Warn instead of returning an error when a conversion fails
2021-04-20 10:23:31 +02:00
Kerollmops
51767725b2
Simplify integer and float functions trait bounds
2021-04-20 10:23:31 +02:00
Kerollmops
efbfa81fa7
Merge the Float and Integer enum variant into the Number one
2021-04-20 10:23:30 +02:00
Clémentine Urquizar
2c380731b9
Update milli version to v0.1.1
2021-04-19 16:03:39 +02:00
Clémentine Urquizar
127d3d028e
Update version for the next release (v0.1.1)
2021-04-19 14:48:13 +02:00
Clémentine Urquizar
b0717b75d9
Update tokenizer to v0.2.1
2021-04-14 19:06:18 +02:00
Clémentine Urquizar
2c5c79d68e
Update Tokenizer version to v0.2.1
2021-04-14 18:54:04 +02:00
Alexey Shekhirin
3af8fa194c
test(http): combine settings assert_(ser|de)_tokens into 1 test
2021-04-10 12:13:59 +03:00
Alexey Shekhirin
ae1655586c
fixes after review
2021-04-09 14:40:48 +03:00
Alexey Shekhirin
dd9eae8c26
feat(http): stats route
2021-04-09 14:34:42 +03:00
Alexey Shekhirin
51ba1bd7d3
fix(http, index): init analyzer with optional stop words
...
Next release
update tokenizer
2021-04-08 17:16:13 +03:00
Alexey Shekhirin
84c1dda39d
test(http): setting enum serialize/deserialize
2021-04-08 17:03:40 +03:00
Alexey Shekhirin
dc636d190d
refactor(http, update): introduce setting enum
2021-04-08 17:03:40 +03:00
tamo
40ef9a3c6a
push a first implementation of the stop_words
2021-04-06 16:29:04 +02:00
Clémentine Urquizar
4ee6ce7871
Next release
2021-04-01 17:16:16 +02:00
tamo
62a8f1d707
bump the version of the tokenizer
2021-04-01 13:49:22 +02:00
tamo
73dcdb27f6
select a specific release of the tokenizer instead of using the latests git commit
2021-03-25 15:00:18 +01:00
mpostma
46293546f3
add tests and mocks
2021-03-24 12:38:59 +01:00
mpostma
8b99860e85
use write sender for updates
2021-03-18 08:32:05 +01:00
bors[bot]
ca3b343b1f
Merge #96
...
96: Check json payload on document addition r=curquiza a=MarinPostma
Check if the json payload in updates is valid. It uses a json validator to avoid allocation, and only serializes the json in case of error, to return a pretty message.
Co-authored-by: mpostma <postma.marin@protonmail.com>
2021-03-16 17:20:44 +00:00
mpostma
204c743bcc
add json payload check on document addition
2021-03-16 14:28:13 +01:00
mpostma
6a742ee62c
restore version route
2021-03-15 19:11:27 +01:00
mpostma
c29b86849b
use actix cors git dependency
2021-03-15 17:40:20 +01:00
mpostma
f727dcc8c6
update milli
2021-03-15 14:26:59 +01:00
mpostma
80d0f9c49d
methods to update index time metadata
2021-03-15 14:05:47 +01:00
mpostma
adc71a70ce
fix displayed attributes in document retrieval
2021-03-15 10:17:41 +01:00
mpostma
3f68460d6c
fix update dedup
2021-03-11 20:58:51 +01:00
mpostma
79a4bc8129
use meta from milli
2021-03-11 19:40:18 +01:00
mpostma
a56e8c1a0c
fix tests
2021-03-10 14:47:04 +01:00
mpostma
5ecf514d28
restructure project
2021-03-10 13:46:49 +01:00
mpostma
562da9dd3f
fix test compilation
2021-03-10 11:56:51 +01:00
Clément Renault
b18ec00a7a
Add a logging_timer macro to te criterion next methods
2021-03-08 16:12:06 +01:00
Kerollmops
636a9df177
Temporarily fix the tinytemplate doc hidden issue
2021-03-08 15:57:45 +01:00
mpostma
7d28f8cff0
implement get single udpate
2021-03-06 10:51:52 +01:00
mpostma
f090f42e7a
multi index store
...
create two channels for Index handler, one for writes and one for reads,
so write are processed one at a time, while reads are processed in
parallel.
2021-03-04 19:18:01 +01:00
Kerollmops
07784c8990
Tune the words prefixes threshold to compute for 1/1000 instead
2021-03-03 15:51:28 +01:00
Kerollmops
79a143b32f
Introduce the query tree data structure
2021-03-03 13:40:18 +01:00
mpostma
62532b8f79
WIP concurent index store
2021-03-02 14:05:03 +01:00
Clément Renault
9423310816
Introduce an helpers crate that export the database to stdout
2021-03-01 19:55:04 +01:00
mpostma
61ce749122
update tokio and disable all routes
2021-02-26 09:10:04 +01:00
Kerollmops
519b1cb5c9
Update dependencies
2021-02-21 10:26:04 +01:00
mpostma
91d6e90d5d
enable faceted searches
2021-02-16 19:20:39 +01:00
Clément Renault
fecf3d6fc1
Move the command lines helpers into different crates
2021-02-14 18:55:15 +01:00
Clément Renault
d8f3421608
Update the dependencies and remove the unused ones
2021-02-14 18:32:46 +01:00
Clément Renault
e8639517da
Change the project to become a workspace with milli as a default-member
2021-02-12 16:15:09 +01:00
mpostma
f8f02af23e
incorporate review changes
2021-02-04 13:21:15 +01:00
mpostma
9af0a08122
post review fixes
2021-02-02 17:34:06 +01:00
mpostma
17c463ca61
remove unused deps
2021-02-01 13:32:21 +01:00
mpostma
e9c95f6623
remove useless files
2021-01-28 19:43:54 +01:00
mpostma
6c63ee6798
implement list all indexes
2021-01-28 18:32:24 +01:00
mpostma
74410d8c6b
architecture rework
2021-01-28 14:12:34 +01:00
Clément Renault
433ac8c38a
Remove the ordered-float serde feature
2021-01-27 14:11:10 +01:00
Kerollmops
61dbcfa44a
Bump the roaring to 0.6.4
2021-01-26 14:38:43 +01:00
Clément Renault
51a37de885
Introduce the FacetValue enum type
2021-01-26 14:09:09 +01:00
mpostma
87a56d2bc9
Fix settings bug
...
replace ids with str in settings
This allows for better maintainability of the settings code, since
updating the searchable attributes is now straightforward.
criterion use string
fix reindexing fieldid remaping
add tests for primary_key compute
fix tests
fix http-ui
fixup! add tests for primary_key compute
code improvements settings
update deps
fixup! code improvements settings
fixup! refactor settings updates and fix bug
fixup! Fix settings bug
fixup! Fix settings bug
fixup! Fix settings bug
Update src/update/index_documents/transform.rs
Co-authored-by: Clément Renault <clement@meilisearch.com>
fixup! Fix settings bug
2021-01-26 13:53:08 +01:00
mpostma
6a3f625e11
WIP: refactor IndexController
...
change the architecture of the index controller to allow it to own an
index store.
2021-01-16 15:09:48 +01:00
mpostma
686f987180
fix compile errors
2021-01-14 11:27:07 +01:00
mpostma
d22fab5bae
implement open index
2021-01-13 18:20:14 +01:00
mpostma
ddd7789713
WIP: IndexController
2021-01-13 17:50:36 +01:00
mpostma
4f7f7538f7
highlight with new tokenizer
2021-01-11 21:59:37 +01:00
mpostma
1ae761311e
integrate with meilisearch tokenizer
2021-01-07 16:14:27 +01:00
mpostma
b4d447b5cb
temp
2021-01-01 16:59:49 +01:00
mpostma
d1e9ded76f
setting builder takes ownership
2020-12-31 00:50:30 +01:00
mpostma
d9dc2036a7
support error & return document count on addition
2020-12-30 18:44:33 +01:00
mpostma
54861335a0
retrieve update status
2020-12-30 18:16:07 +01:00
mpostma
0cd9e62fc6
search first iteration
2020-12-24 12:58:34 +01:00
mpostma
1a38bfd31f
data add documents
2020-12-23 13:52:28 +01:00
mpostma
55e1552957
update queue refactor, first iteration
2020-12-22 17:13:50 +01:00
mpostma
7c9eaaeadb
clean code, and fix errors
2020-12-22 14:02:41 +01:00
Kerollmops
77e951e933
Use the byte-unit crate to ease library usage
2020-12-20 12:00:37 +01:00
Clément Renault
e7f2ab9138
Bump grenad to fix an indexing bug
2020-12-05 16:39:15 +01:00
Clément Renault
0959e1501f
Introduce the FacetRevRange Iterator struct
2020-12-04 12:02:23 +01:00
Clément Renault
61b383f422
Introduce the criteria update setting
2020-12-04 12:02:22 +01:00
Clément Renault
a0adfb5e8e
Introduce a real pest parser and support every facet filter conditions
2020-11-23 16:43:55 +01:00
Clément Renault
07a0c82790
Bump heed to 0.10.4 to use be able to lazily decode roaring bitmaps
2020-11-23 16:43:53 +01:00
Clément Renault
38c76754ef
Make the facet level search system generic on f64 and i64
2020-11-23 16:43:52 +01:00
Clément Renault
b255be93fa
Bump heed to 0.10.3
2020-11-23 16:43:49 +01:00
Clément Renault
a18d9a1f87
Parse and store the faceted fields
2020-11-13 16:13:51 +01:00
Clément Renault
640c7d748a
Modify the highlight function to support any JSON type
2020-11-05 13:59:32 +01:00
Clément Renault
0408c9d66a
Move the http server into its own sub-module
2020-11-05 11:16:39 +01:00
Clément Renault
4fded5bd0e
Bump heed to be able to reference a RoTxn from multiple threads
2020-11-02 12:49:23 +01:00
Clément Renault
f0d028d3a4
Update the Transform struct to support JSON updates
2020-10-31 20:52:49 +01:00
Clément Renault
9d47ee52b4
Generate a uuid v4 based document id when missing
2020-10-31 15:11:06 +01:00
Clément Renault
085d3b9d94
Update heed to 0.10.0
2020-10-30 11:42:00 +01:00
Clément Renault
b5d52b6b45
Prefer using a smallstr instead of a real String to reduce allocations
2020-10-29 14:32:32 +01:00
Clément Renault
98fc24cbdf
Bump heed to fix a prefix iter bug
2020-10-28 10:55:21 +01:00
Clément Renault
b44b04d25b
Serialize the CSV record values as JSON strings
2020-10-24 14:43:46 +02:00
Clément Renault
802e925fd7
Switch to a JSON protocol for the front page
2020-10-21 18:26:29 +02:00
Clément Renault
2210818114
Introduce the obkv heed codec
2020-10-21 15:51:48 +02:00
Clément Renault
f948a03be2
Optimise the merge functions to avoid allocations
2020-10-20 16:40:50 +02:00
Clément Renault
cde8478388
Replace the panic in the merge function by actual errors
2020-10-20 16:19:07 +02:00
Clément Renault
35c9a3c558
Brodacast the updates infos to every ws clients
2020-10-20 11:19:34 +02:00
Clément Renault
871222aebd
Introduce some new routes to handle live indexing
2020-10-19 16:06:43 +02:00
Clément Renault
65e32fecb1
Move the binaries into one with subcommands
2020-10-19 13:44:17 +02:00
Clément Renault
ff389f1270
Update heed-types to 0.7.1
2020-10-19 11:52:59 +02:00
Clément Renault
eca49e3a03
Introduce a notification channel for the UpdateStore
2020-10-18 16:37:37 +02:00
Clément Renault
83c1db8763
Introduce the UpdateStore
2020-10-18 15:26:57 +02:00
Clément Renault
9021b2dba6
Introduce the enable-chunk-fusing flag
2020-10-14 18:44:59 +02:00
Kerollmops
f980422c57
Move from oxidized-mtbl to grenad
2020-10-14 12:47:32 +02:00
Kerollmops
4e9bd1fef5
Bump oxidized-mtbl
2020-10-07 14:23:22 +02:00
Kerollmops
433d9bbc6e
Use CompressionType::from_str rather than a custom function
2020-10-06 13:50:34 +02:00
Kerollmops
4b819457c9
Enable the strucopt/clap warp help feature
2020-10-06 13:06:22 +02:00
Clément Renault
770f29fd05
Bump the oxidized-mtbl dependency
2020-10-04 17:04:33 +02:00
Kerollmops
68f4af7d2e
Improve the display of the number of processed documents
2020-09-29 16:08:58 +02:00
Clément Renault
ed05999f63
Replace the arc cache by a simple linked hash map
2020-09-23 14:50:52 +02:00
Clément Renault
d6fa9c0414
Index the intra documents word pair proximities
2020-09-22 14:04:33 +02:00
Kerollmops
3ded98e5fa
Bump the roaring version that fix a deserialization bug
2020-09-10 22:37:51 +02:00
Kerollmops
d5e5baa20f
Bump the oxidized-mtbl dependency
2020-09-10 13:29:12 +02:00
Kerollmops
0fb086f241
Use the crates.io raoring library
2020-09-08 15:16:04 +02:00
Clément Renault
bb1ab428db
Use another function to define the proximity
2020-09-06 17:55:07 +02:00
Clément Renault
f928b91e9d
Specify the exact rev for the near-proximity dep
2020-09-06 17:21:38 +02:00
Clément Renault
1c504471d3
Introduce the plane-sweep algorithm
2020-09-05 18:25:27 +02:00
Clément Renault
dc88a86259
Store the word positions under the documents
2020-09-05 18:03:06 +02:00
Kerollmops
580ed1119a
Make the engine to return csv string records as documents and headers
2020-08-31 19:02:00 +02:00
Clément Renault
bad0663138
Come back to the old tokenizer
2020-08-31 13:34:38 +02:00
Clément Renault
3fe497e129
Improve the Mtbl heed codec to only encode MTBL databases
2020-08-29 11:20:39 +02:00
Clément Renault
d19f394630
Make the indexer support gzipped CSV as input
2020-08-21 18:10:24 +02:00
Clément Renault
ff479c865d
Replace pipe by ringtail to improve stdin read performances
2020-08-21 17:45:52 +02:00
Clément Renault
8806fcd545
Introduce a better query and document lexer
2020-08-16 14:36:54 +02:00
Clément Renault
1e358e3ae8
Introduce the AstarBagIter that iterates through best paths
2020-08-15 16:24:06 +02:00
Clément Renault
fae694a102
Put the documents into an MTBL database
2020-08-07 12:14:40 +02:00
Clément Renault
405a71d3a4
Accept csv from stdin
2020-08-06 13:38:21 +02:00
Clément Renault
6508d497ce
Replace the regex highlighting by a simple algorithm
2020-08-05 13:52:27 +02:00
Clément Renault
bd4b18541c
Introduce a new indexer which uses an MTBL sorter
2020-08-04 15:44:37 +02:00
Kerollmops
085c376655
Use the regex crate to highlight "hello"
2020-07-14 11:28:40 +02:00
Kerollmops
12358476da
Use the log crate instead of stderr
2020-07-12 10:55:09 +02:00
Kerollmops
2c62eeea3c
Rename the project milli
2020-07-12 00:16:41 +02:00
Kerollmops
f6eae91c7d
Pretty print the new dashboard numbers
2020-07-11 14:17:37 +02:00
Kerollmops
11c7fef80a
Implement a memory dumper
...
It moves the in memory HashMaps used when indexing to a disk based MTBL file
2020-07-07 16:48:49 +02:00
Kerollmops
7178b6c2c4
First basic version using MTBL again
2020-07-07 11:32:33 +02:00
Kerollmops
2a3b03138b
Use heed 0.8.1 with the RwIter append method
2020-07-05 19:50:28 +02:00
Kerollmops
46ced5c828
Introduce the RwIter append heed API
2020-07-04 12:34:10 +02:00
Kerollmops
2ae3f40971
Make the indexer ignore certain words
...
This is a preparation for making the indexing fully parallel by making the
indexer only be aware of certain words for each threads to avoid postings lists
conflicts for each words
2020-07-01 17:49:46 +02:00
Kerollmops
f98b615bf3
Replace the LRU by an Arc cache
2020-06-29 20:48:57 +02:00
Kerollmops
07abebfc46
Introduce a (too big) LRU cache
2020-06-29 18:15:03 +02:00
Kerollmops
5f0088594b
Index by writing directly into LMDB
2020-06-29 13:54:47 +02:00
