processing time

[kukretiaditya]

i was practicing unidec pp and took the input data from this article https://doi.org/10.1021/acs.analchem.3c02010?urlappend=%3Fref%3DPDF&jav=VoR&rel=cite-as but it just keeps showing processing have i done any step wrong or it takes a lot of time ?

[michaelmarty]

Hmm. Can you try clicking open combined report or is it frozen? It looks like it got through everything when it says it's processing 114 of 114, but it may be stuck on the last file or something at the end. Also, which version are you using?

[kukretiaditya]

sir i am using version 8 with python latest version , this is what the cmd window of the unidec was showing

[michaelmarty]

Sorry for the delayed response. I've been out of the office for the holidays. I think I see your issue. The file name should point to the data file, not the excel file. You will also need to download all the raw files from MassIVE and update your data folder to match. It's basically not finding the raw data files. Here's the link to the data: https://massive.ucsd.edu/ProteoSAFe/dataset.jsp?accession=MSV000092242. The first image you posted has the right structure; I think you just need to download the data and update the "Data Directory" column.

[kukretiaditya]

Thank you very much for your response, and I sincerely appreciate your time despite the holiday season. I apologize for any inconvenience caused earlier.

[kukretiaditya]

so it is not possible to run it on unidec with just the excel input file , we would need the dataset on the massIVE too.

[kukretiaditya]

Downloaded the BsAb files through the ftp link using winSCP got downloaded in photo viewer format

[kukretiaditya]

worked 😁 ( open file option) , the add file option still gives the value error

[michaelmarty]

Great! Correct, you'll need to have the raw data in some format (.raw, .txt, .mzML). The excel file only points to where to find the raw data. You don't actually need to convert it to mzML. Even if the default program to view .raw files is the windows photo viewer, UniDec will still open them correctly.

I just tested the add file, and it seems to be working fine. For that, you select a data file, and it adds it to the list. It will only let you add certain file types that are recognized as MS data. Try that and let me know if it's still giving you errors. Thanks!

[kukretiaditya]

also an agilent .d folder cannot be uploaded in the upp right ?

[michaelmarty]

We're seeing the same bug here and working on it. Just to confirm, is it working when you are using Python scripting and the source code?

It should be possible to use .d folders in UPP. You cannot use the "Add File" to add it because the wxPython window doesn't recognize it as a file. However, if you edit the excel file, you can add it manually, and it should work. It's working for me on Agilent test data on my computer, but we've definitely got a bug with it.

[michaelmarty]

As for how it converts Agilent data, it's all in the Agilent DLLs. We use pythonnet (https://pypi.org/project/pythonnet/) which uses the common language runtime (import clr) for .NET code. Basically, it imports .NET functions from the DLLs and pretends they are Python functions. I'm not an expert in .NET, but a program called DotPeek is really useful for looking into the functions that are available within the DLLs.

[kukretiaditya]

in version 8.0.0 when i run a .d agilent folder in gui unidec it stops for a minute and then closes on it's own showing the same basetof.dll dependency error ; ;

[kukretiaditya]

We're seeing the same bug here and working on it. Just to confirm, is it working when you are using Python scripting and the source code?

It should be possible to use .d folders in UPP. You cannot use the "Add File" to add it because the wxPython window doesn't recognize it as a file. However, if you edit the excel file, you can add it manually, and it should work. It's working for me on Agilent test data on my computer, but we've definitely got a bug with it.

i get the basetof.dll error when I run the application directly and upload the .d folder in the GUI and it automatically closes

and same error when I run it through source code

to use .d folder through upp I will have to convert them manually to excel file with the required columns like sample name and data directory?

[kukretiaditya]

for agilent .d folder in ver 7.0.2 while converting it the cmd says failed native raw averaging and goes back to python averaging, I think that's bcz the .d folder data is maybe too big. so that error should not be a problem right?

[JoeyGrimes]

Hi Kukretiaditya, Thanks for being patient with us. I Have instructions on ensuring v 8.0.0 works properly.

Here is a recent executable build from dropbox that I have made:
https://www.dropbox.com/scl/fi/e5df1hnbdztgzzqiox4jo/UniDec_Windows.zip?rlkey=rkxbog66y82hzjclrmk66eecw&st=lajevmvp&dl=0

Please ensure you are running Python version 3.11.9, You can verify this by running python --version in command prompt.

You can install version Python 3.11.9 here: https://www.python.org/downloads/release/python-3119/

Once you have the correct version, make sure you have pywin32, you can see your current packages via running 'pip list' in command prompt. If it's not available please run 'pip install pywin32'

After this has been done. Please download the zip from dropbox, before extracting:
right click the zip->properties->unblock->apply

Inside of the extracted zip file UniDec_Windows. There should be a directory called _internal.

Then inside of _internal, there should be a file called register.cmd

Please run this, and it will register all of the agilent DLL's required(You only have to do this once).

You should be able to use agilent after this.

We will integrate this into the github branch soon.

Thank you,
Joe

[kukretiaditya]

it will take time for me to try on 3.11.9 but I did try on 3.12.7 and it did work but the txt file I got from the agilent .d folder (the raw data file) is a bit different from what I am getting in ver 7.0.2. Maybe bcz in in 7.0.2 when I upload my ,d folder it is not using the native raw averaging and pulling back to python averaging.

[kukretiaditya]

i did what you wrote but when i run or open register.cmd it shows

[JoeyGrimes]

Hi,

Can you try open command prompt and try 'pip install pywin32'

And let me know what it shows?

[kukretiaditya]

i also opened the terminal in the folder and installed the requirements .txt , pip install unidec and pip install massql
now when i open the app this comes

and this when i upload a agilent .d folder in ver 8.0.0
note - i am doing it on my other laptop with fresh download of py 3.11.9 in path .

[JoeyGrimes]

It looks like it registered properly. One thing I would test is trying to open other .d files and see if the issue persists.

Is there any way you can send me that .d directory via dropbox? I'd like to see if I can recreate your error.

Appreciate your patience.

[kukretiaditya]

oh no sir i am fine i should thank you for helping me out so much

[kukretiaditya]

meantime while i use version 7.0.2 the GUI makes a raw data txt file of my agilent .d folder , when I upload that raw data txt file in the UPP and hit run after selecting option deconvolve data it opens a HTML webpage and in result I get a plot with mass on x axis but there are 119 peaks detected? doesn't deconvolution shows only one peak that helps us know mass? is this normal ?

and yeah upp is not taking agilent .d folder as whole but I just put the agilent .d folder in an excel file with 2 param sample name and data and then open file and it's running but numpeaks is very high

[kukretiaditya]

[michaelmarty]

It depends entirely on your data and the settings you use. Some data naturally has a bunch of peaks. Other times, if data is noisy, you need to adjust the settings (including data processing, deconvolution, and peak picking) to get what you want. Try playing with it in regular UniDec to get a sense for how your data responds.

[kukretiaditya]

oh yh this was going on in my mind but was just confused because of these multiple peaks thanks for the advice will do exactly that .
Btw why don't you structure the discord server it's pretty good place for conversation and all I see there are a lot of people on there but no one replies when they msg and there is someone spamming fake steam links on your discord server.
I can help with discord if you like I have good experience in discord server community building and am also learning about mass spec ,python and analytics.

[kukretiaditya]

When will the source code for the new 8.0.1 be released? the executable dropbox version you shared with me

[kukretiaditya]

is it possible for you to tell me where is the formula that you have used fr deconvolution in the .exe files there are a lot of headers and stuff

[michaelmarty]

We push all new changes to the v3 branch, so you can check out the latest version there: https://github.com/michaelmarty/UniDec/tree/v3. I'm not sure I understand your second question about the formula. Can you explain more? The sln file is the Visual Studio project that manages compilation. As to the header files, I realize it's not standard C to put all the code there, and I'm hoping to go back and clean it up someday. There are also scripts for linux and mac to compile with.

[kukretiaditya]

i dont know c language thats why I am not able to figure out the deconvolution code written in exe file. so I was trying to deconvolve by making a py script for formula.

[kukretiaditya]

does this formula looks correct if I loop it for each peak I will get mass and z out of the m/z

[michaelmarty]

Your formula is definitely correct, and it will work as long as you have simple spectra where the peaks are all in the same series and there are no additional peaks to confuse it. UniDec uses a completely different approach (see paper). There used to be a feature that let you pick peaks and apply that equation, but I turned it off in recent builds because it was a pain from a UI point of view.

[kukretiaditya]

there are multiple fitting functions in the fitting.py so how does it decide which to use

[kukretiaditya]

if i simplify the data and then apply my formula to it u think it will work that way. for example I tried simplifying the sample data provided in upp paper. the kamal 1 one data.
made a script that loads data and applies savgol filter to it after which peak picking is done not able to understand how will the mass be calculated after this

[michaelmarty]

The smoothing and peak picking on the data prior to deconvolution is optional. The algorithm itself does not need it. The GUI uses a simple Gaussian smooth.

Mass determination is done after deconvolution, on the zero-charge spectrum. It picks peaks based on the apex (mass of the point with the highest intensity), although there are also tools for calculating centroids.

Happy for you to clean up the Discord if it's gotten messed up, but I'm not very active on that platform. Too many other incoming streams to manage at the moment. Hopefully someday!

[kukretiaditya]

By zero charge spectrum it means removing the z from the m/z right? but if we have already removed the z then don't we already have the mass of that sample. I am not able to understand how do I deconvolve the data and get the results like molecular mass like how unidec does in just python script by myself and how do i convert it to zero charge mass spectrum in my colab code.? 😭
and if smoothing and peak picking is optional then I guess the colab automated code I made is useless too

[kukretiaditya]

I tried doing some things i learnt like gaussian fitting but none of the things I got matched to what results i got in this data's HTML file

[michaelmarty]

I must admit that I'm a bit lost on what you are trying to do, but I'll do my best to answer. The zero charge spectrum is indeed pure mass. The exact process of transforming m/z to pure mass is actually fairly complex and took years for us to come up with a way I was happy with. Basically, you have to assign the charge of each peak and then transform the m/z axis into a mass axis by multiplying by the assigned charge. If multiple charge states are assigned, you have to fractionally distribute the intensity. The real challenge becomes that you need to resample the m/z axis to make the mass data points line up and get uniform mass sampling. This can be done with either interpolation (if oversampling in mass) or integration (if undersampling). UniDec now uses a "smart" transform that determines whether it is under or over sampling and then applies the correct method at each data point.

One tip, in your Gaussian fitting, I think you could add an amplitude parameter to help. However, note that UniDec doesn't do any fitting. Each data point is transformed. In that aspect, it is not really a peak assignment tool but instead assigns the charge state distribution of each data point.

[kukretiaditya]

Hi is there some issue with the email addresses I am trying to mail but it's not working and showing this error
mtmarty@email.arizona.edu
mtmarty@arizona.edu

[michaelmarty]

Arizona upgrade their email filters a while ago, and they are now overly aggressive at blocking things. That said, we are also moving to Texas, so you can use mtmarty@utexas.edu going forward. Thanks!

[kukretiaditya]

yes thanks this email seems to be working

[kukretiaditya]

Hi is it just taking forever to read the mzml file ,

[michaelmarty]

Try the latest version. I think there might be bugs that we've since fixed that help speed up the mzML reading. Thanks!

[michaelmarty]

Try removing .d from the folder path name. I think the issue might be that it's getting confused trying to parse the file name

[kukretiaditya]

it did read but ; ; now showing this error for the 2 diff mzml files I have
Problem with preset manager: [WinError 3] The system cannot find the path specified: 'C:\Users\suneel\anaconda3\envs\Unidec_8_0_3\lib\site-packages\unidec\bin\Example Data\UniChrom'
Opening: IgF1 std.mzML
Opening Time: 0.006685599999997294
Reading mzML file: C:\Users\suneel\Desktop\folders\IgF1 std.mzML
Traceback (most recent call last):
File "C:\Users\suneel\anaconda3\envs\Unidec_8_0_3\lib\site-packages\unidec\UniChrom.py", line 60, in on_open
self.open_file(dlg.GetPath())
File "C:\Users\suneel\anaconda3\envs\Unidec_8_0_3\lib\site-packages\unidec\UniChrom.py", line 77, in open_file
load_hdf5 = self.eng.open_chrom(path, refresh=refresh)
File "C:\Users\suneel\anaconda3\envs\Unidec_8_0_3\lib\site-packages\unidec\modules\ChromEng.py", line 48, in open_chrom
load_hdf5 = self.load_mzml(self.path, load_hdf5, refresh=refresh)
File "C:\Users\suneel\anaconda3\envs\Unidec_8_0_3\lib\site-packages\unidec\modules\ChromEng.py", line 94, in load_mzml
self.polarity = self.chromdat.get_polarity()
File "C:\Users\suneel\anaconda3\envs\Unidec_8_0_3\lib\site-packages\unidec\UniDecImporter\MZML\mzML.py", line 279, in get_polarity
spec = self.msrun[scan]
File "C:\Users\suneel\anaconda3\envs\Unidec_8_0_3\lib\site-packages\pymzml\run.py", line 201, in getitem
spectrum = self.info["file_object"][identifier]
File "C:\Users\suneel\anaconda3\envs\Unidec_8_0_3\lib\site-packages\pymzml\file_interface.py", line 106, in getitem
return self.file_handler[identifier]
File "C:\Users\suneel\anaconda3\envs\Unidec_8_0_3\lib\site-packages\pymzml\file_classes\standardMzml.py", line 123, in getitem
spectrum = self._binary_search(identifier)
File "C:\Users\suneel\anaconda3\envs\Unidec_8_0_3\lib\site-packages\pymzml\file_classes\standardMzml.py", line 155, in _binary_search
raise Exception(
Exception: Spectrum ID should be between 2305 and 2279984

[michaelmarty]

Strange. Do you have UV data in your MS file? How many scans should you have? There can sometimes be issues if there's other data stored in the mzML file.

[kukretiaditya]

there is a uv detector in the lcms machine , there must be 16k something scans as far as I remember , the raw mzml file is being read in some other python libraries thou

[michaelmarty]

That makes sense. I just pushed a potential fix to the V3 branch. let me know if you need an updated release as well. Thanks!