Skip to content

Develop #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 110 commits into from
Jul 8, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
110 commits
Select commit Hold shift + click to select a range
c9e31f9
CHORE: version update to 0.2.5
sander-willems-bruker Apr 5, 2024
26fc5b8
Merge branch 'main' into develop
sander-willems-bruker Apr 5, 2024
0211d13
CHORE: refactored converters to be more explicit
sander-willems-bruker Apr 8, 2024
8c39a9e
CHORE: grouped ms data together
sander-willems-bruker Apr 8, 2024
6d648b4
CHORE: renamed modules
sander-willems-bruker Apr 8, 2024
0a99fe5
CHORE: renamed vec_utils
sander-willems-bruker Apr 8, 2024
7e4ba61
FEAT: crude mgf writer implementation
sander-willems-bruker Apr 8, 2024
56a06a6
CHORE: unified converter setup
sander-willems-bruker Apr 9, 2024
6df48f3
CHORE: version update bigger than patch
sander-willems-bruker Apr 9, 2024
c421d0e
CHORE: cleaned up precursor and frame structs in favor of simplicity
sander-willems-bruker Apr 11, 2024
2262abf
FEAT: Updated frames with ms info and quad settings
sander-willems-bruker Apr 12, 2024
a9064a5
CHORE: partial move of writers to io
sander-willems-bruker Apr 23, 2024
93c87af
CHORE: move of tdf_blobs
sander-willems-bruker Apr 23, 2024
ed8148a
DOCS: updated package to work in modules rather than forwrding structs
sander-willems-bruker Apr 23, 2024
38418d4
CHORE: rustfmt now has width 100
sander-willems-bruker Apr 26, 2024
cbbb37a
CHORE: undo rustfmt
sander-willems-bruker Apr 26, 2024
997385b
FEAT: new sql reader module
sander-willems-bruker Apr 26, 2024
48d18f8
FIX: simplfy rt reading of precursors
sander-willems-bruker Apr 26, 2024
34f5ae6
FEAT: use new sql reader format to read frames
sander-willems-bruker Apr 26, 2024
34f99a8
FEAT: update TdfBlob with error handling and option to use non indexe…
sander-willems-bruker Apr 26, 2024
4ed4557
FEAT: included sql readers in io::common
sander-willems-bruker Apr 26, 2024
544cf6d
FEAT: removed unused frame table
sander-willems-bruker Apr 26, 2024
d15469b
CHORE fully deprecated ms_data_blobs module in favor of TdfBlobs
sander-willems-bruker Apr 26, 2024
36d449e
FEAT: created new frame_reader module
sander-willems-bruker Apr 26, 2024
5c062e1
CHORE: remove unused sql frame table
sander-willems-bruker Apr 26, 2024
c2aada8
FEAT: exposed frame reader to crate
sander-willems-bruker Apr 26, 2024
02bc4e5
FEAT: updated TdfReader to use frameReader instead
sander-willems-bruker Apr 26, 2024
6120d31
CHORE: updated benchmarks
sander-willems-bruker Apr 30, 2024
3032d6d
DOCS: typo
sander-willems-bruker Jun 14, 2024
abf3744
FIX: reading of acquisition software
sander-willems-bruker Jun 14, 2024
e2a145f
FEAT: cleaner reading of frames
sander-willems-bruker Jun 14, 2024
d4de769
FEAT: using iterator to read frames
sander-willems-bruker Jun 14, 2024
ded22c1
FEAT: updated frame reader
sander-willems-bruker Jun 14, 2024
9e4f050
FEAT: github workflow update
sander-willems-bruker Jun 14, 2024
01555f9
FEAT: added intensity factors to frames
sander-willems-bruker Jun 14, 2024
1f221c3
FEAT: added rt converter to tdf reader and metada
sander-willems-bruker Jun 14, 2024
22448c0
CHORE: added type annotations to tdfblobs
sander-willems-bruker Jun 14, 2024
80c47fb
FEAT: cleaned up code
sander-willems-bruker Jun 14, 2024
0ae839d
FEAT: removed old tdf and frame reader
sander-willems-bruker Jun 14, 2024
72217a9
CHORE: renamed io:common to io:file_readers
sander-willems-bruker Jun 14, 2024
87a0b98
FEAT: reformatted sql precursors
sander-willems-bruker Jun 14, 2024
1b817de
CHORE: refactored pasefmsmsframes
sander-willems-bruker Jun 14, 2024
17d2cf8
CHORE: cargo fmt
sander-willems-bruker Jun 14, 2024
3b8a4ed
CHORE: simplificaiton of precursor reader
sander-willems-bruker Jun 14, 2024
4705a58
FIX: diaPASEF type typo
sander-willems-bruker Jun 14, 2024
81cd6ee
FIX: unified MsLevel parsing
sander-willems-bruker Jun 24, 2024
ac6a23d
FEAT: added metadata
sander-willems-bruker Jun 24, 2024
0763df4
CHORE: updated sqlreadable trait name to ReadableSqlTable
sander-willems-bruker Jun 24, 2024
bfe71cc
FEAT: added hashmap reader for sql
sander-willems-bruker Jun 24, 2024
53d4748
FEAT: reamoved option to get connection from sqlreader
sander-willems-bruker Jun 24, 2024
6f4f2a8
FEAT: make sqlhashmap propagate error
sander-willems-bruker Jun 24, 2024
4e0e387
FEAT: implemented metadata reader
sander-willems-bruker Jun 24, 2024
04de7ff
FEAT: fully refactored sql reader to use metadata struct
sander-willems-bruker Jun 24, 2024
1eb57f9
FIX: remove lingering module that was refactored
sander-willems-bruker Jun 24, 2024
cee2b42
FEAT: renamed tdf blobs to refelect reader
sander-willems-bruker Jun 24, 2024
1cc1842
FEAT: refactored precursor reader in the new io structure
sander-willems-bruker Jun 24, 2024
d81d93f
CHORE: cleaned tdf blob reader
sander-willems-bruker Jun 24, 2024
4d0d7ca
FEAT: simplifying spectrum readers
sander-willems-bruker Jun 24, 2024
ad6d2d5
FEAT: simplied old file reader for easier refactoring
sander-willems-bruker Jun 24, 2024
c56ae7d
DOCS: cleaned up io
sander-willems-bruker Jun 25, 2024
e43da82
CHORE: implemented MGFEntry as struct rather than impl for spectrum
sander-willems-bruker Jun 25, 2024
7ec524b
FIX: typos
sander-willems-bruker Jun 25, 2024
3faadc8
FEAT: cleaner mgf writing
sander-willems-bruker Jun 25, 2024
7465040
FIX: typo
sander-willems-bruker Jun 25, 2024
73dfbdc
FEAT: updated parquet reader
sander-willems-bruker Jun 25, 2024
4a25da9
FEAT: updated metadata path and precursor reader
sander-willems-bruker Jun 25, 2024
0389075
FEAT: updated minitdf reader to muse precursor reader properly
sander-willems-bruker Jun 25, 2024
b98b5ab
CHORE: cleaned up minitdf spectrum reader
sander-willems-bruker Jun 25, 2024
45e37d1
CHORE: formatting of code
sander-willems-bruker Jun 25, 2024
5717c6d
FEAT: added dummy code to start refactoring spectrum readers
sander-willems-bruker Jun 25, 2024
5db16b3
CHORE: changed visibility of sub readers
sander-willems-bruker Jun 25, 2024
dd768d3
CHORE: formatting
sander-willems-bruker Jun 25, 2024
f87273d
CHORE: simplify parquet table reading
sander-willems-bruker Jun 25, 2024
76f6f9b
FEATL simplifying frame reader
sander-willems-bruker Jun 26, 2024
e806627
FEAT: fully refactored minitdf spectrum reader
sander-willems-bruker Jun 26, 2024
308b891
FEAT: made extension finder more generic
sander-willems-bruker Jun 26, 2024
1558d0c
FEAT: started on refactoring of TDFSpectrumReader
sander-willems-bruker Jun 26, 2024
6266264
CHORE: unify path reading functions
sander-willems-bruker Jun 26, 2024
d8114ad
FIX: properly process empty frames
sander-willems-bruker Jul 1, 2024
da15c25
CHORE: simplification of vec utils filter with mask
sander-willems-bruker Jul 3, 2024
332e9aa
FEAT: added reading of frame groups and quad settings to sql reader
sander-willems-bruker Jul 3, 2024
cd47e8a
CHORE: udated quad settings types
sander-willems-bruker Jul 3, 2024
f300bb4
FEAT: added frame_group to frames
sander-willems-bruker Jul 3, 2024
4bb2ce6
PERF: removed inline from tdf blobs to let compiler decide
sander-willems-bruker Jul 3, 2024
7d7e40d
DOCS: Updated file readerd docs and removed always calibrate option
sander-willems-bruker Jul 3, 2024
a0e6427
FEAT: Added quad setting (dia) data to frame reader
sander-willems-bruker Jul 3, 2024
9371c8e
FEAT: reformatted spectrum reader (still not done)
sander-willems-bruker Jul 3, 2024
668d74e
FIX: read im values from file rather than hardcoded
sander-willems-bruker Jul 3, 2024
0ddd915
FIX: remove print from metadata
sander-willems-bruker Jul 3, 2024
bdb7eed
FEAT: added CE to spectrum rather than precursor
sander-willems-bruker Jul 3, 2024
bff0a73
FEAT: explicit intensity reader in frame for ICC data
sander-willems-bruker Jul 3, 2024
6d8ab0e
CHORE: technical fix of tdf blob reader without effect
sander-willems-bruker Jul 3, 2024
2207459
CHORE: simplified spectral ordering
sander-willems-bruker Jul 3, 2024
149cfab
FEAT: added timscompressiontype to metadata
sander-willems-bruker Jul 3, 2024
40a39f4
CHORE: made todos for refactoring
sander-willems-bruker Jul 3, 2024
5405b67
CHORE: made todo for refactoring
sander-willems-bruker Jul 3, 2024
a27e455
CHORE: refactored tdf spectrum reader
sander-willems-bruker Jul 3, 2024
ccd8e3a
FEAT: added isolation width and mz to spectra
sander-willems-bruker Jul 3, 2024
c22682a
FEAT: renamed raw spextrum reader
sander-willems-bruker Jul 3, 2024
14783d9
CHORE set reminders to update error handling ands docs
sander-willems-bruker Jul 3, 2024
d24e6b5
DEBUG: updated main to move out of timsrust
sander-willems-bruker Jul 3, 2024
f1b31b7
CHORE: added todo to refactor frame reader
sander-willems-bruker Jul 4, 2024
d49ba77
FEAT: implemented a dia spectrum reader
sander-willems-bruker Jul 5, 2024
cd29750
FEAT: made proper quad_settings_reader that can be recycled
sander-willems-bruker Jul 5, 2024
0eb51dd
FEAT: implemente None option for precursor charge and intensity
sander-willems-bruker Jul 5, 2024
3b0636e
FEAT: removed manual testing main in favor of showing timsrust is a l…
sander-willems-bruker Jul 5, 2024
2a6ff12
FEAT: option to filter spectra by top n peaks
sander-willems-bruker Jul 5, 2024
2ce3547
FIX: top_n for small values
sander-willems-bruker Jul 5, 2024
71a6536
FIX: implementation to take isoalation width and mz into account for …
sander-willems-bruker Jul 8, 2024
38b69d9
FIX: failing tests
sander-willems-bruker Jul 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,7 @@ env:

jobs:
build:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Build
Expand Down
2 changes: 1 addition & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "timsrust"
version = "0.2.4"
version = "0.3.0"
edition = "2021"
description = "A crate to read Bruker timsTOF data"
license = "Apache-2.0"
Expand Down
64 changes: 57 additions & 7 deletions benches/speed_performance.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
use criterion::{black_box, criterion_group, criterion_main, Criterion};
use timsrust::FileReader;

const DDA_TEST: &str =
"/mnt/c/Users/Sander.Willems/Documents/data/tims05_300SPD/20230505_TIMS05_PaSk_MA_HeLa_6min_ddaP_S1-C10_1_2323.d/";
const DIA_TEST: &str =
"/mnt/c/Users/Sander.Willems/Documents/data/20230505_TIMS05_PaSk_SA_HeLa_6min_diaP_8scans_S1-D3_1_2329.d/";
const SYP_TEST: &str =
"/mnt/c/Users/Sander.Willems/Documents/data/20230505_TIMS05_PaSk_SA_HeLa_6min_syP_5scans_30Da_S1-D4_1_2330.d/";

fn read_all_frames(file_reader: &FileReader) {
file_reader.read_all_frames();
}
Expand All @@ -17,27 +24,70 @@ fn read_all_spectra(file_reader: &FileReader) {
file_reader.read_all_spectra();
}

fn criterion_benchmark(c: &mut Criterion) {
fn criterion_benchmark_dda(c: &mut Criterion) {
// c.bench_function("fib 20", |b| b.iter(|| fibonacci(black_box(20))));
let mut group = c.benchmark_group("sample-size-example");
group.significance_level(0.001).sample_size(10);
let d_folder_name: &str = "/home/sander/data/20230505_TIMS05_PaSk_MA_HeLa_6min_ddaP_S1-C10_1_2323.d/";
let d_folder_name: &str = DDA_TEST;
let file_reader: FileReader =
FileReader::new(d_folder_name.to_string()).unwrap();
group.bench_function("read_all_frames 6m dda", |b| {
group.bench_function("DDA read_all_frames 6m", |b| {
b.iter(|| read_all_frames(black_box(&file_reader)))
});
group.bench_function("read_all_ms1_frames 6m dda", |b| {
group.bench_function("DDA read_all_ms1_frames 6m", |b| {
b.iter(|| read_all_ms1_frames(black_box(&file_reader)))
});
group.bench_function("read_all_ms2_frames 6m dda", |b| {
group.bench_function("DDA read_all_ms2_frames 6m", |b| {
b.iter(|| read_all_ms2_frames(black_box(&file_reader)))
});
group.bench_function("read_all_spectra 6m dda", |b| {
group.bench_function("DDA read_all_spectra 6m", |b| {
b.iter(|| read_all_spectra(black_box(&file_reader)))
});
group.finish();
}

criterion_group!(benches, criterion_benchmark);
fn criterion_benchmark_dia(c: &mut Criterion) {
// c.bench_function("fib 20", |b| b.iter(|| fibonacci(black_box(20))));
let mut group = c.benchmark_group("sample-size-example");
group.significance_level(0.001).sample_size(10);
let d_folder_name: &str = DIA_TEST;
let file_reader: FileReader =
FileReader::new(d_folder_name.to_string()).unwrap();
group.bench_function("DIA read_all_frames 6m", |b| {
b.iter(|| read_all_frames(black_box(&file_reader)))
});
group.bench_function("DIA read_all_ms1_frames 6m", |b| {
b.iter(|| read_all_ms1_frames(black_box(&file_reader)))
});
group.bench_function("DIA read_all_ms2_frames 6m", |b| {
b.iter(|| read_all_ms2_frames(black_box(&file_reader)))
});
group.finish();
}

fn criterion_benchmark_syp(c: &mut Criterion) {
// c.bench_function("fib 20", |b| b.iter(|| fibonacci(black_box(20))));
let mut group = c.benchmark_group("sample-size-example");
group.significance_level(0.001).sample_size(10);
let d_folder_name: &str = SYP_TEST;
let file_reader: FileReader =
FileReader::new(d_folder_name.to_string()).unwrap();
group.bench_function("SYP read_all_frames 6m", |b| {
b.iter(|| read_all_frames(black_box(&file_reader)))
});
group.bench_function("SYP read_all_ms1_frames 6m", |b| {
b.iter(|| read_all_ms1_frames(black_box(&file_reader)))
});
group.bench_function("SYP read_all_ms2_frames 6m", |b| {
b.iter(|| read_all_ms2_frames(black_box(&file_reader)))
});
group.finish();
}

criterion_group!(
benches,
criterion_benchmark_dda,
// criterion_benchmark_dia,
// criterion_benchmark_syp
);
criterion_main!(benches);
7 changes: 0 additions & 7 deletions src/acquisition.rs

This file was deleted.

29 changes: 0 additions & 29 deletions src/calibration.rs

This file was deleted.

89 changes: 0 additions & 89 deletions src/converters.rs

This file was deleted.

13 changes: 13 additions & 0 deletions src/domain_converters.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
//! Allows conversions between domains (e.g. Time of Flight and m/z)
mod frame_to_rt;
mod scan_to_im;
mod tof_to_mz;

pub use frame_to_rt::Frame2RtConverter;
pub use scan_to_im::Scan2ImConverter;
pub use tof_to_mz::Tof2MzConverter;

/// Convert from one domain (e.g. Time of Flight) to another (m/z).
pub trait ConvertableDomain {
fn convert<T: Into<f64> + Copy>(&self, value: T) -> f64;
}
19 changes: 19 additions & 0 deletions src/domain_converters/frame_to_rt.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
/// A converter from Frame -> retention time.
#[derive(Debug, Clone)]
pub struct Frame2RtConverter {
rt_values: Vec<f64>,
}

impl Frame2RtConverter {
pub fn from_values(rt_values: Vec<f64>) -> Self {
Self { rt_values }
}
}

impl super::ConvertableDomain for Frame2RtConverter {
fn convert<T: Into<f64> + Copy>(&self, value: T) -> f64 {
let lower_value: f64 = self.rt_values[value.into().floor() as usize];
let upper_value: f64 = self.rt_values[value.into().ceil() as usize];
(lower_value + upper_value) / 2.
}
}
28 changes: 28 additions & 0 deletions src/domain_converters/scan_to_im.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/// A converter from Scan -> (inversed) ion mobility.
#[derive(Debug, Clone)]
pub struct Scan2ImConverter {
scan_intercept: f64,
scan_slope: f64,
}

impl Scan2ImConverter {
pub fn from_boundaries(
im_min: f64,
im_max: f64,
scan_max_index: u32,
) -> Self {
let scan_intercept: f64 = im_max;
let scan_slope: f64 = (im_min - scan_intercept) / scan_max_index as f64;
Self {
scan_intercept,
scan_slope,
}
}
}

impl super::ConvertableDomain for Scan2ImConverter {
fn convert<T: Into<f64> + Copy>(&self, value: T) -> f64 {
let scan_index_f64: f64 = value.into();
self.scan_intercept + self.scan_slope * scan_index_f64
}
}
42 changes: 42 additions & 0 deletions src/domain_converters/tof_to_mz.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
use linreg::linear_regression;

/// A converter from TOF -> m/z.
#[derive(Debug, Clone)]
pub struct Tof2MzConverter {
tof_intercept: f64,
tof_slope: f64,
}

impl Tof2MzConverter {
pub fn from_boundaries(
mz_min: f64,
mz_max: f64,
tof_max_index: u32,
) -> Self {
let tof_intercept: f64 = mz_min.sqrt();
let tof_slope: f64 =
(mz_max.sqrt() - tof_intercept) / tof_max_index as f64;
Self {
tof_intercept,
tof_slope,
}
}

pub fn from_pairs(data: &Vec<(f64, u32)>) -> Self {
let x: Vec<u32> = data.iter().map(|(_, x_val)| *x_val).collect();
let y: Vec<f64> =
data.iter().map(|(y_val, _)| (*y_val).sqrt()).collect();
let (tof_slope, tof_intercept) = linear_regression(&x, &y).unwrap();
Self {
tof_intercept,
tof_slope,
}
}
}

impl super::ConvertableDomain for Tof2MzConverter {
fn convert<T: Into<f64> + Copy>(&self, value: T) -> f64 {
let tof_index_f64: f64 = value.into();
(self.tof_intercept + self.tof_slope * tof_index_f64).powi(2)
}
}
9 changes: 8 additions & 1 deletion src/errors.rs
Original file line number Diff line number Diff line change
@@ -1,9 +1,16 @@
use crate::file_readers;
use crate::{
file_readers,
// io::readers::common::{sql_reader::SqlError, tdf_blobs::TdfBlobError},
};

/// An error that is produced by timsrust (uses [thiserror]).
#[derive(thiserror::Error, Debug)]
pub enum Error {
/// An error to indicate a path is not a Bruker File Format.
#[error("FileFormatError: {0}")]
FileFormatError(#[from] file_readers::FileFormatError),
// #[error("SqlError: {0}")]
// SqlError(#[from] SqlError),
// #[error("BinError: {0}")]
// BinError(#[from] TdfBlobError),
}
Loading
Loading