searchengines-web/MeiliSearch
searchengines-web/MeiliSearch
1
0
Fork 0
mirror of https://github.com/meilisearch/MeiliSearch synced 2024-11-05 04:28:55 +01:00
Code Issues Releases Wiki Activity

Merge #169

Browse Source 
169: Optimize roaring codec r=Kerollmops a=MarinPostma

Optimize the `BoRoaringBitmapCodec` by preventing it from emiting useless error that caused allocation. On my flamegraph, the byte_decode function went from 4.13% to  1.70% (of transplant graph).

This may not be the greatest optimization ever, but hey, this was a low hanging fruit.

before:
![image](https://user-images.githubusercontent.com/28804882/116241125-17018880-a754-11eb-9f9d-a67418d100e1.png)
after:
![image](https://user-images.githubusercontent.com/28804882/116241167-21bc1d80-a754-11eb-9afc-d9d72727477c.png)



Co-authored-by: Marin Postma <postma.marin@protonmail.com>
This commit is contained in:
bors[bot] 2021-06-01 06:30:25 +00:00 committed by GitHub
commit 2f9f6a1f21
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 16 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
milli/src/heed_codec/roaring_bitmap/bo_roaring_bitmap_codec.rs
View File

@ -1,5 +1,7 @@
use std::borrow::Cow;
use byteorder::{NativeEndian, ReadBytesExt, WriteBytesExt};
use std::convert::TryInto;
use std::mem::size_of;
use roaring::RoaringBitmap;
pub struct BoRoaringBitmapCodec;
@ -7,11 +9,14 @@ pub struct BoRoaringBitmapCodec;
impl heed::BytesDecode<'_> for BoRoaringBitmapCodec {
type DItem = RoaringBitmap;
fn bytes_decode(mut bytes: &[u8]) -> Option<Self::DItem> {
fn bytes_decode(bytes: &[u8]) -> Option<Self::DItem> {
let mut bitmap = RoaringBitmap::new();
while let Ok(integer) = bytes.read_u32::<NativeEndian>() {
bitmap.insert(integer);
for chunk in bytes.chunks(size_of::<u32>()) {
let bytes = chunk.try_into().ok()?;
bitmap.push(u32::from_ne_bytes(bytes));
}
Some(bitmap)
}
}
@ -20,10 +25,12 @@ impl heed::BytesEncode<'_> for BoRoaringBitmapCodec {
type EItem = RoaringBitmap;
fn bytes_encode(item: &Self::EItem) -> Option<Cow<[u8]>> {
let mut bytes = Vec::with_capacity(item.len() as usize * 4);
for integer in item.iter() {
bytes.write_u32::<NativeEndian>(integer).ok()?;
}
Some(Cow::Owned(bytes))
let mut out = Vec::with_capacity(item.len() as usize * size_of::<u32>());
item.iter()
.map(|i| i.to_ne_bytes())
.for_each(|bytes| out.extend_from_slice(&bytes));
Some(Cow::Owned(out))
}
}