With the publication of the beam search variant of BCD supertrees (Fleischauer and Böcker, PeerJ 2018), this project has come to an end. BCD supertrees shows an outstanding performance for a supertree method with guaranteed polynomial running time, and is usually on par or even better than established supertree methods such as MRP or SuperFine. With the beam search, you can trade running time for supertree quality; but for input trees that contain branch lengths, even the “regular” BCD shows excellent performance.
We sincerely hope that someone will continue our work and, in particular, will integrate BCD supertrees into a divide-and-conquer strategy to improve the quality of phylogenetic reconstruction for very large trees. In (Fleischauer and Böcker, Mol Biol Evol 2017) we have shown that this is indeed possible (Fig. 2): Computation with RAxML gets faster and the tree quality is improved. Given BCDs fast and guaranteed running times, this should be very interesting for large phylogenies with several thousand taxa: BCD requires only hours to compute a supertree with 5000+ taxa and, even more importantly, supertree quality does not deteriorate for such large datasets.
For us, this is it in phylogenetics — at least, for the moment. It has been a great experience with challenging and fascinating combinatorial problems!
ps. We gratefully acknowledge funding by Deutsche Forschungsgemeinschaft.
pps. The BCD code is available on GitHub.
Larson et al. (J. Am. Soc. Mass Spectrom., 2018) have applied CSI:FingerID to dopant-assisted atmospheric pressure chemical ionization (dAPCI) gas chromatography mass spectrometry (GC-MS) data. They identified almost three times as many compounds with SIRIUS and CSI:FingerID as when searching in the NIST spectral library. Find their study here.
On May 9, 2018, the CSI:FingerID web service has passed two million compound queries. Nice.
We have fixed a bug in SIRIUS when analyzing compounds in negative ion mode. These were wrongly treated as intrinsically charged. If you have analyzed negative ion mode data with SIRIUS 4 and CSI:FingerID, you might want to reanalyze the data with the newest version.
Some of you might have experienced problems to reach the CSI:FingerID web service on Thursday, April 12, 2018. The reason is: After a “relatively quiet” Wednesday with “only” 120k compound queries (irony warning), the CSI:FingerID web service had to handle a real-world stress test on Thursday, with 260k compound queries submitted on a single day. That is 3 compound queries per second on average.
This brought our job database into some trouble: The database runs on a redundant twin server and is regularly mirrored between the two servers, in case one of them is hosed. Unfortunately, the log files used for mirroring jobs became too big, stalling the web server.
The problem resided until Friday 13th noon. Bad luck.
We are sorry if this caused any inconvenience. All systems should be up and running again.
As a courtesy towards other users: In case you want to submit complete databases with hundreds of runs, it would be great if you could distribute your computations over the course of several days. Also, you might want to contact us in advance, so that we do not accidentally block you.
By the way: We have obviously passed 1.5 million compound queries after these two days.
The International Max Planck Research School at the MPI for Chemical Ecology in Jena is looking for PhD students, and one of the projects is on “making SIRIUS and CSI:FingerID GCMS-ready”. Only a half position is being paid by the IMPRS, but this can be supplemented by funding from our chair. We are searching for motivated candidates from bioinformatics, cheminformatics and computer science who want to work in this exciting, quickly evolving interdisciplinary field. Please see here for details, and apply here. Application deadline is May 16th, 2018. Contact Sebastian in case your have questions.
With the release of the new SIRIUS version (and, behind the scenes, a new version of CSI:FingerID), we want to share some numbers so you know if it was worth the hassle. We use CASMI 2016 data, to allow you to compare our results against those of other methods. We use the candidate structures provided as part of category 2, automated structural identification. See also Schymanski et al. (J Cheminf 2017).
For molecular formula identification, we use both isotope patterns and fragmentation patterns. (Isotope pattern data were released after the contest.) We consider all molecular formulas — we will not get bored to stress that if you limit molecular formulas to those found in some structure database, you will never ever find a new molecular formula. We find that SIRIUS 4 identifies the correct molecular formula for 91.3% of the challenges.
Next, we use CSI:FingerID to identify the compound structures. During the last year, the CASMI 2016 data have found their way into the CSI:FingerID training data. We know that CSI:FingerID has excellent identification performance if a spectrum for this structure is present in the training data (expect anything between 70% and 95%). But this is not challenging, and also does not tell us how good SIRIUS and CSI:FingerID can identify truly novel compounds.
To this end, we excluded all structures from CASMI 2016 from the training data. Hence, any structure is novel, in the sense that CSI:FingerID has never before seen any MS/MS data for this structure. SIRIUS 4 and the new CSI:FingerID reach 37.8% correct identifications for novel structures and positive ion mode; this is significantly better than the 27.6% reported in the CASMI paper (Schymanski et al., J Cheminf 2017). In addition, we can now also process challenges in negative ion mode, thanks to the training data available in NIST; here, we reach 28.4% correct identifications for novel structures.
These numbers are for “unambiguously correctly” identified structures: Sometimes, two candidate structures have exactly the same molecular fingerprint and are scored with exactly the same score. If we include these “ambiguously correctly” identified structures, numbers increase to 40.2% and 30.9%, respectively.
By the way: The idea of a challenge is to be challenging. That is why category 2 of CASMI 2016 uses candidate lists directly extracted from ChemSpider. In application, you will probably use a smaller candidate lists, which will make identification easier and improve identification rates: For example, unambiguous correct identifications for novel structures and positive ion mode increase to 71.7% if we search in a biomolecular structure database with “only” 0.5 million structures.
To cut a long story short: SIRIUS 4 and CSI:FingerID provide outstanding performance for molecular formula and structure identification. As mentioned in the release news, it is also much faster than before: On a set of 1533 GNPS compounds, we observed a 36-fold speedup.
Update: Two bugs had to be corrected in our evaluation. Minor: We chose the wrong parameter set (TOF vs. Orbitrap), resulting in small ID rate changes. Major: We did not search in the biomolecule DB but rather in “biomolecules plus MINEs”. Fixing this resulting in pretty dramatic changes.
The CSI:FingerID web service has now processed more than one million compound queries. (In fact, we are already 50k queries beyond that.) We are thrilled, and hope that you made some interesting discoveries. 😉
During the Dagstuhl seminar on Computational Metabolomics a few days ago, there was a session on “bridging the gap” between methods developers and experimentalists, which resulted in a long list of what method developers should do to bridge the gap (develop GUIs, provide example data, manuals, tutorials, etc) but rather little that experimentalists can do.
Now, let us turn the tables: SIRIUS and CSI:FingerID come with a nice GUI, manual etc. But undoubtedly the best to explain how to use our software in practice, are you, the users! To this end, we ask you to send us any tutorial material you have (maybe, a video tutorial?) that may help fellow experimentalists to get acquainted to the software more quickly; you may also point out what the software cannot do, and how you deal with it; and so on.
Please leave a comment below with the link (I hope that works) or, even better, send us a link ( ed.an1537561812ej-in1537561812u@sui1537561812ris1537561812) that we can put on our soon-to-come training material web page!
ps. The first video tutorial is already out there: Louis-Felix Nothias-Scaglia (UCSD) has prepared this video tutorial on YouTube that explains how to combine Optimus (OpenMS), SIRIUS plus CSI:FingerID and GNPS. Many thx to him! The tutorial is potentially already outdated as development is progressing fast, but isn’t that nice?!
CASMI (Critical Assessment of Small Molecule Identification) 2017 results are finally out! Kai participated with SIRIUS/CSI:FingerID, and won categories 1 (Best Structure Identification on Natural Products), 2 (Best Automatic Structural Identification – In Silico Fragmentation Only), and 4 (Best Automatic Candidate Ranking). He did not participate in category 3.
For category 4 (in silico methods for searching in molecular structure databases), CSI:FingerID correctly identified 66 of 198 compounds (33.3%). This is more than 6-fold of what the best non-CSI:FingerID contestant reached. (Submissions containing “IOKR” in their name, correspond to different variants of the Input Output Kernel Regression version of CSI:FingerID.)
Unfortunately, CASMI 2017 data did not include isotope patterns. It appears that in many cases, SIRIUS was not able to find the correct molecular formula among the top candidate. This resulted in several cases where the correct structure was excluded due to the “wrong” molecular formula. We will investigate how many compounds would have been correctly identified if isotope pattern data would have been available.
The CSI:FingerID web service has just passed the mark of processing data from 500,000 query compounds — congratulations to CSI:FingerID, and thank you for your interest in our tools! (Be reminded that CSI:FingerID should be accessed via the SIRIUS application, not via the web page.)
Since version 3.4, SIRIUS is licensed under the GNU General Public License (GNU GPL). If you need SIRIUS under a different license, please contact us.
Today, Kerstin was awarded the “Wissenschaftspreis für anwendungsorientierte Abschlussarbeit” (scientific award for application-oriented thesis) from the Wirtschaftsförderungsgesellschaft mbH and the FSU Jena, for her PhD thesis “Small molecules: From mass spectral fragmentation data to structural elucidation”. Congratulations!
Sebastian will give a talk on CSI:FingerID and SIRIUS at the OpenMS user meeting 2016 in Tübingen, 21-23 September 2016.