[Paper Review] Annotation Enrichment Analysis: An Alternative Method for Evaluating the Functional Properties of Gene Sets

Kimberly Glass, Michelle Girvan|arXiv (Cornell University)|Aug 20, 2012

Bioinformatics and Genomic Networks66 references4 citations

TL;DR

This paper introduces Annotation Enrichment Analysis (AEA), a novel method that corrects for biases in traditional functional enrichment analysis by accounting for non-uniform gene annotation distributions in databases like Gene Ontology. Unlike Fisher’s Exact Test (FET), which overestimates significance when gene sets contain highly annotated genes, AEA evaluates overlap in functional annotations rather than gene overlap, revealing biologically meaningful enrichments previously obscured by false positives.

ABSTRACT

Gene annotation databases (compendiums maintained by the scientific community that describe the biological functions performed by individual genes) are commonly used to evaluate the functional properties of experimentally derived gene sets. Overlap statistics, such as Fisher's Exact Test (FET), are often employed to assess these associations, but don't account for non-uniformity in the number of genes annotated to individual functions or the number of functions associated with individual genes. We find FET is strongly biased toward over-estimating overlap significance if a gene set has an unusually high number of annotations. To correct for these biases, we develop Annotation Enrichment Analysis (AEA), which properly accounts for the non-uniformity of annotations. We show that AEA is able to identify biologically meaningful functional enrichments that are obscured by numerous false-positive enrichment scores in FET, and we therefore suggest it be used to more accurately assess the biological properties of gene sets.

Motivation & Objective

• To identify and correct for biases in traditional functional enrichment analysis caused by non-uniform annotation distributions in gene databases.
• To address the overestimation of significance in Fisher’s Exact Test (FET) when gene sets contain an unusually high number of highly annotated genes.
• To develop a method that evaluates functional enrichment based on annotation overlap rather than gene overlap, thereby reducing false-positive results.
• To demonstrate that AEA identifies biologically meaningful functional enrichments that are masked by FET’s statistical bias.
• To provide a practical, analytically approximated alternative to FET that accounts for the structural properties of the Gene Ontology DAG.

Proposed method

• Propose Annotation Enrichment Analysis (AEA), which computes overlap between a gene set and the set of functional terms in a GO branch, rather than between genes and terms.
• Define a GO branch as a parent term and all its descendants, with the total number of unique genes annotated to the branch equal to the number annotated to the parent term.
• Construct biased random gene sets with fixed total annotation counts (Mg) but varying average gene annotation degrees (k_avg) by iteratively swapping genes to match target annotation levels.
• Generate random GO branches by selecting random terms in a shuffled order until the number of unique genes annotated to the selected terms closely matches a target k_t (within 1%).
• Use AEA to compute p-values based on annotation overlap, with a theoretical approximation to reduce computational cost.
• Compare AEA results to FET and FDR-corrected FET on both random and real gene signatures to assess bias and statistical power.

Experimental results

Research questions

• RQ1 Does the number of annotations per gene in a gene set bias the significance of functional enrichment results when using Fisher’s Exact Test?
• RQ2 Can annotation non-uniformity in the Gene Ontology lead to false-positive enrichment signals in standard functional analysis tools?
• RQ3 Does evaluating overlap at the level of functional annotations rather than genes reduce bias and improve detection of true biological associations?
• RQ4 Can a theoretical approximation of AEA provide a computationally efficient alternative to permutation-based AEA?
• RQ5 Are experimentally derived gene signatures systematically enriched for highly annotated genes, and does this affect their functional enrichment scores?

Key findings

• Fisher’s Exact Test (FET) is strongly biased toward overestimating significance when gene sets contain an unusually high number of highly annotated genes.
• Random gene sets with high average annotation counts (k_avg ≈ 65) produced significantly more false-positive enrichments than random sets with lower annotation counts (k_avg ≈ 21), even when gene set size was held constant.
• Annotation Enrichment Analysis (AEA) effectively eliminates this bias by modeling annotation overlap instead of gene overlap, revealing biologically meaningful enrichments previously obscured by FET.
• Real gene signatures from the Gene Signatures Database (GeneSigDB2012) contain a disproportionate number of highly annotated genes, contributing to inflated FET p-values.
• The AEA method identifies significant functional enrichments in real gene sets that are not detected by FET due to its sensitivity to annotation bias.
• An analytic approximation to AEA is provided that enables faster computation while maintaining accuracy, making it suitable for large-scale functional enrichment analysis.