Eva Kohnert (IMBI): Advancing the Principles of Benchmark Studies: Validating Experimental Results with Synthetic Data for 16S Microbiome Data.

Abstract

In computational life sciences, selecting the optimal analysis method such as for high-throughput data remains a challenging task due to the lack of clear guidelines and rules. Benchmark studies have proven to be valuable tools for evaluating the performance and applicability of bioinformatics analysis methods. However, these studies are often subject to methodological limitations and deficiencies, leading to potentially biased results.

To overcome these issues, we propose that benchmark studies should be planned and conducted following a clinical trial-inspired framework. In this work, we present a study protocol for a benchmark study evaluating differential abundance analyses for microbiome data.

The primary objective is to validate the conclusions of a previous benchmark study that relied solely on experimental data by using simulated microbiome data. Our study also demonstrates the utility, applicability, and limitations of using simulated data within the context of benchmarking.