Background: Most ordinary differential equation (ODE) based modeling studies in systems biology involve a hold-out validation step for model validation. In this framework a pre-determined part of the data is used as validation data and, therefore it is not used for estimating the parameters of the model. The model is assumed to be validated if the model predictions on the validation dataset show good agreement with the data. Model selection between alternative model structures can also be performed in the same setting, based on the predictive power of the model structures on the validation dataset. However, drawbacks associated with this approach are usually under-estimated.
Results: We have carried out simulations by using a recently published High Osmolarity Glycerol (HOG) pathway from S.cerevisiae to demonstrate these drawbacks. We have shown that it is very important how the data is partitioned and which part of the data is used for validation purposes. The hold-out validation strategy leads to biased conclusions, since it can lead to different validation and selection decisions when different partitioning schemes are used. Furthermore, finding sensible partitioning schemes that would lead to reliable decisions are heavily dependent on the biology and unknown model parameters which turns the problem into a paradox. This brings the need for alternative validation approaches that offer flexible partitioning of the data. For this purpose, we have introduced a stratified random cross-validation (SRCV) approach that successfully overcomes these limitations.
Conclusions: SRCV leads to more stable decisions for both validation and selection which are not biased by underlying biological phenomena. Furthermore, it is less dependent on the specific noise realization in the data. Therefore, it proves to be a promising alternative to the standard hold-out validation strategy.
Increase the total number of rows showing on this page using the pull-down located below the table, or use the page scroll at the table's top right to browse through the table's pages; use the arrows to the right of a column header to sort by that column; filter the table using the "Filter" box at the top of the table.
Evidence ID | Analyze ID | Gene/Complex | Systematic Name/Complex Accession | Qualifier | Gene Ontology Term ID | Gene Ontology Term | Aspect | Annotation Extension | Evidence | Method | Source | Assigned On | Reference |
---|
Increase the total number of rows showing on this page using the pull-down located below the table, or use the page scroll at the table's top right to browse through the table's pages; use the arrows to the right of a column header to sort by that column; filter the table using the "Filter" box at the top of the table; click on the small "i" buttons located within a cell for an annotation to view further details.
Evidence ID | Analyze ID | Gene | Gene Systematic Name | Phenotype | Experiment Type | Experiment Type Category | Mutant Information | Strain Background | Chemical | Details | Reference |
---|
Increase the total number of rows showing on this page using the pull-down located below the table, or use the page scroll at the table's top right to browse through the table's pages; use the arrows to the right of a column header to sort by that column; filter the table using the "Filter" box at the top of the table.
Evidence ID | Analyze ID | Gene | Gene Systematic Name | Disease Ontology Term | Disease Ontology Term ID | Qualifier | Evidence | Method | Source | Assigned On | Reference |
---|
Increase the total number of rows displayed on this page using the pull-down located below the table, or use the page scroll at the table's top right to browse through the table's pages; use the arrows to the right of a column header to sort by that column; to filter the table by a specific experiment type, type a keyword into the Filter box (for example, “microarray”); download this table as a .txt file using the Download button or click Analyze to further view and analyze the list of target genes using GO Term Finder, GO Slim Mapper, SPELL, or YeastMine.
Evidence ID | Analyze ID | Regulator | Regulator Systematic Name | Target | Target Systematic Name | Direction | Regulation of | Happens During | Regulator Type | Direction | Regulation Of | Happens During | Method | Evidence | Strain Background | Reference |
---|
Increase the total number of rows showing on this page by using the pull-down located below the table, or use the page scroll at the table's top right to browse through its pages; use the arrows to the right of a column header to sort by that column; filter the table using the "Filter" box at the top of the table.
Site | Modification | Modifier | Source | Reference |
---|
Increase the total number of rows showing on this page by using the pull-down located below the table, or use the page scroll at the table's top right to browse through the table's pages; use the arrows to the right of a column header to sort by that column; filter the table using the "Filter" box at the top of the table; click on the small "i" buttons located within a cell for an annotation to view further details about experiment type and any other genes involved in the interaction.
Evidence ID | Analyze ID | Interactor | Interactor Systematic Name | Interactor | Interactor Systematic Name | Allele | Assay | Annotation | Action | Phenotype | SGA score | P-value | Source | Reference | Note |
---|
Increase the total number of rows showing on this page by using the pull-down located below the table, or use the page scroll at the table's top right to browse through the table's pages; use the arrows to the right of a column header to sort by that column; filter the table using the "Filter" box at the top of the table; click on the small "i" buttons located within a cell for an annotation to view further details about experiment type and any other genes involved in the interaction.
Evidence ID | Analyze ID | Interactor | Interactor Systematic Name | Interactor | Interactor Systematic Name | Assay | Annotation | Action | Modification | Source | Reference | Note |
---|
Increase the total number of rows showing on this page by using the pull-down located below the table, or use the page scroll at the table's top right to browse through its pages; use the arrows to the right of a column header to sort by that column; filter the table using the "Filter" box at the top of the table.
Complement ID | Locus ID | Gene | Species | Gene ID | Strain background | Direction | Details | Source | Reference |
---|
Increase the total number of rows displayed on this page using the pull-down located below the table, or use the page scroll at the table's top right to browse through the table's pages; use the arrows to the right of a column header to sort by that column; filter the table using the "Filter" box at the top of the table; download this table as a .txt file using the Download button;
Evidence ID | Analyze ID | Dataset | Description | Keywords | Number of Conditions | Reference |
---|
Increase the total number of rows displayed on this page using the pull-down located below the table, or use the page scroll at the table's top right to browse through the table's pages; use the arrows to the right of a column header to sort by that column; filter the table using the "Filter" box at the top of the table; download this table as a .txt file using the Download button;
Evidence ID | Analyze ID | File | Description |
---|