Just stumble across Why we didn't rewrite our feed handler in Rust from databento
And they say they cannot share a buffer during a loop. What would be the rust way to fix this without making copies of data? I made a minimal example
struct Source {
id: usize,
data: Vec<u8>,
}
impl Source {
fn fetch_data(&self) -> Vec<u8> {
println!("Fetching data from source {}", self.id);
self.data.clone()
}
}
const SPLITTER: u8 = b',';
fn process_data(chunks: &[&[u8]]) {
println!("Processing {} chunks:", chunks.len());
for (i, chunk) in chunks.iter().enumerate() {
println!(" Chunk {}: {:?}", i, String::from_utf8_lossy(chunk));
}
}
fn process_sources(sources: Vec<Source>) {
let mut buffer: Vec<&[u8]> = Vec::new(); // allocate once
for source in sources {
let data: Vec<u8> = source.fetch_data();
buffer.extend(data.split(|b| *b == SPLITTER));
process_data(&buffer);
buffer.clear();
}
}
fn main() {
let sources = vec![
Source {
id: 1,
data: b"hello,world,this,is,source1".to_vec(),
},
Source {
id: 2,
data: b"foo,bar,baz".to_vec(),
},
];
process_sources(sources);
}
The error is:
error[E0597]: `data` does not live long enough
--> src/main.rs:27:23
|
| let data: Vec<u8> = source.fetch_data();
| ---- binding `data` declared here
| buffer.extend(data.split(|b| *b == SPLITTER));
| ------ ^^^^ borrowed value does not live long enough
| |
| borrow later used here
...
| }
| - `data` dropped here while still borrowed