Motivation
An overriding motivation for employing pathway analysis was to reduce the multiple-testing burden by basing inferences on the combined effects of probes or variants. Most teams used external bioinformatic data to inform pathway construction (see Table 3). However, the use of pathways to clarify biological function or interpret association results, while highly relevant to future applications of these methods, was a secondary concern given the method-development focus of these studies. Of all the studies, Ziyatdinov came closest to this interpretive approach, using relationships in the data to recover the biological processes implicit in the simulation generating model.
Data selection; pathway assembly
All teams used the BP data (real or simulated) provided for GAW19, including systolic (SBP) and diastolic BP (DBP) and hypertension (HT) status. Several teams also calculated the derived phenotype mean arterial pressure (MAP) from these data as (2/3 DBP + 1/3 SBP). All teams using the simulated data chose not to be blind to the generating model (they “used the answers”), which allowed them to estimate power to detect “true” functional variants and associated gene expression phenotypes. Most teams also used the provided null trait Q1 to estimate type 1 error rate. A limitation of this use of the simulated data (as noted by a reviewer of an early draft of this report) is that evaluation of method performance is necessarily sensitive to the assumptions of the generating model.
Three teams (Brunel, Quillen, Ziyatdinov) focused on the gene expression probes, while the remaining teams considered single nucleotide variants (SNVs) drawn from the GWAS genotypes, exome sequence data, or both. Combined analysis of gene expression and association has been considered in previous GAWs (eg, Charlesworth et al. [5]), but only Brunel attempted this combination in a 2-step approach (data reduction of BP-related gene expression phenotypes followed by GWAS).
As noted, most teams made use of the simulated phenotypes as a basis for estimating power and type 1 error. Kos, Quillen, and Valcarcel constructed synthetic pathways (described more fully below) of gene expression probes (Quillen) or SNVs (Kos, Valcarcel) representing predetermined numbers of randomly chosen genes; these synthetic pathways were then characterized by the number of genes that contributed to the simulated phenotypes as a basis for measuring the effect of functional “dosage” on analytical power. One team (Tayo) focused on a single biological pathway with known relevance to hypertension (aldosterone-regulated sodium reabsorption), using both real and simulated BP phenotypes.
Kos, Quillen, and Valcarcel independently conceived of using sets of probes or variants annotated to randomly selected genes to define synthetic pathways (SPs) to test both the power and type 1 error of their proposed methods. SPs that represented genes used in the simulation generating model were, in Quillen’s terminology, “positive controls” to test the sensitivity of tests. SPs lacking such representation provided data on false positive rate, either in place of or supplementary to tests of the null trait Q1. All 3 teams used various metrics—for example, number of genes represented in each SP or number of generating-model genes represented—to further characterize the performance of their analytical approaches.
In addition to the dimensional reduction achieved by grouping probes or variants in pathways, some teams also filtered variants based on predicted function (eg, Kos).
Lo developed biological pathways nominally based on a hypothesis that hypertension could be related to undetected type 2 diabetes (this is known to have elevated prevalence in the Mexican American cohorts from which the GAW19 data were derived, but diabetes status was not reported as part of the provided data set). Pathway construction consisted of choosing a set of diabetes-related genes from the literature and then expanding these “seed” genes with publicly available gene interaction data. In the context of this workshop, this approach simply provided a biologically plausible way to construct pathways reflecting the number of genes and patterns of gene–gene interaction that might be encountered in a “real” study; this was tangential to the primary goal of characterizing performance of these investigators’ bespoke method for analyzing SNP-SNP interaction effects [6] (see below).
As noted, most pathways were user-defined, either based on numbers of genes (to test effects on power and specificity) or on prior biological information, rather than on empirical networks (eg, by weighted correlation networks) [7]. This was consistent with the emphasis on methodological development, especially on data reduction and control of the burden of multiple testing. The known causal structure of the simulated data might have supported a test of the efficacy of an empirical networks approach to recover the generating model from patterns of SNV association, but no team attempted this.
Analytical tools
The focus for methodological innovation in these studies was typically on the construction or definition of pathways. In general, established tools were used for relating pathway-defined probes or variants to the traits of interest, accounting for family data where appropriate: for example, variance components-based analyses implemented in SOLAR (Sequential Oligogenic Linkage Analysis Routines) [8], by Kos and Quillen; adaptation of SKAT (sequence kernel association test) by Valcarcel; burden tests [9]; GenABEL [10] by Brunel; and PLINK for unrelated data [11].
An exception to this pattern was the use by Lo of an analytical tool to detect gene interactions, previously developed by members of this team [6]. In brief, this method involves comparing the joint effect of 2 SNPs on a trait to the marginal effect of each; a ratio of joint to marginal effects significantly greater than 1 (as assessed by permutation) is taken as evidence of SNP–SNP interaction effects on the phenotype [6]. Using pairwise tests of SNPs annotated to genes in a pathway of interest (described above), these authors reported evidence of a joint effect of variants in GCK and PAX4 on hypertension (real data from the unrelated cohort). As noted by the anonymous reviewers of these authors’ report, the nature of this joint effect—whether due to in-sample LD, interaction between the gene products, or some other effect—needs additional investigation.
Gene expression
We now consider selected approaches in more detail. Taking replicate simulations of DBP as the trait of interest, Quillen analyzed the random effect of similarity in gene expression among related individuals, with the goal of identifying pathways for which coexpression accounted for a significant proportion of variance relative to modeling the random effect of kinship alone. This analysis compared the effect of 16 methods for computing similarity matrices from the expression data; of these, simple correlation and extended Jaccard distance [12] outperformed the others and greatly exceeded the performance of single-probe association as a consequence of reduced multiple testing. As expected, performance was dependent on the number of genes in the SPs that contributed by design to the simulated phenotypes [13].
Brunel developed pathways based on probes in the GAW19 data that were annotated to genes related to BP in Gene Ontology, or to genes related to these by published evidence of protein–protein interaction. They then used independent components analysis (ICA) to derive “meta-expression” phenotypes (ICA factors). Unfortunately, the use of phenotypes derived from the real gene expression data, rather than simulated data, prevented estimation of power and type 1 error rate, so it was not possible to determine if any increase in signal from data reduction could overcome the multiple-testing burden inherent in GWAS.
Single nucleotide variants
Valcarcel proposed 2-step association tests of a simulated BP phenotype that consisted of variant aggregation (via either SKAT or burden test) for all variants in a pathway, followed by gene-centric aggregation tests within pathways that passed a chosen significance threshold. Their pathways were synthetic (randomly chosen sets of specified numbers of genes with retention of pathways that contained at least 1 gene that was causal in the simulation). The 2-step approach was more powerful, with reasonable control of type 1 error, than a 1-step gene-centric approach because of the reduction in multiple testing by prescreening of pathways.