Open Access Highly Accessed Open Badges Research

Low concordance of multiple variant-calling pipelines: practical implications for exome and genome sequencing

Jason O'Rawe12, Tao Jiang3, Guangqing Sun3, Yiyang Wu12, Wei Wang4, Jingchu Hu3, Paul Bodily5, Lifeng Tian6, Hakon Hakonarson6, W Evan Johnson7, Zhi Wei4, Kai Wang89* and Gholson J Lyon129*

Author Affiliations

1 Stanley Institute for Cognitive Genomics, Cold Spring Harbor Laboratory, One Bungtown Rd, Cold Spring Harbor, 11724, USA

2 Stony Brook University, 100 Nicolls Rd, Stony Brook, 11794, USA

3 BGI-Shenzhen, Shenzhen 518000, China

4 New Jersey Institute of Technology, Martin Luther King Jr. Blvd, Newark, 07103, USA

5 Brigham Young University, N University Ave, Provo, 84606, USA

6 Children's Hospital of Philadelphia, Civic Center Blvd, Philadelphia, 19104, USA

7 Boston University School of Medicine, E Concord St, Boston, 02118, USA

8 University of Southern California, 1501 San Pablo Street, Los Angeles, 90089, USA

9 Utah Foundation for Biomedical Research, E 3300 S, Salt Lake City, 84106, USA

For all author emails, please log on.

Genome Medicine 2013, 5:28  doi:10.1186/gm432

Published: 27 March 2013



To facilitate the clinical implementation of genomic medicine by next-generation sequencing, it will be critically important to obtain accurate and consistent variant calls on personal genomes. Multiple software tools for variant calling are available, but it is unclear how comparable these tools are or what their relative merits in real-world scenarios might be.


We sequenced 15 exomes from four families using commercial kits (Illumina HiSeq 2000 platform and Agilent SureSelect version 2 capture kit), with approximately 120X mean coverage. We analyzed the raw data using near-default parameters with five different alignment and variant-calling pipelines (SOAP, BWA-GATK, BWA-SNVer, GNUMAP, and BWA-SAMtools). We additionally sequenced a single whole genome using the sequencing and analysis pipeline from Complete Genomics (CG), with 95% of the exome region being covered by 20 or more reads per base. Finally, we validated 919 single-nucleotide variations (SNVs) and 841 insertions and deletions (indels), including similar fractions of GATK-only, SOAP-only, and shared calls, on the MiSeq platform by amplicon sequencing with approximately 5000X mean coverage.


SNV concordance between five Illumina pipelines across all 15 exomes was 57.4%, while 0.5 to 5.1% of variants were called as unique to each pipeline. Indel concordance was only 26.8% between three indel-calling pipelines, even after left-normalizing and intervalizing genomic coordinates by 20 base pairs. There were 11% of CG variants falling within targeted regions in exome sequencing that were not called by any of the Illumina-based exome analysis pipelines. Based on targeted amplicon sequencing on the MiSeq platform, 97.1%, 60.2%, and 99.1% of the GATK-only, SOAP-only and shared SNVs could be validated, but only 54.0%, 44.6%, and 78.1% of the GATK-only, SOAP-only and shared indels could be validated. Additionally, our analysis of two families (one with four individuals and the other with seven), demonstrated additional accuracy gained in variant discovery by having access to genetic data from a multi-generational family.


Our results suggest that more caution should be exercised in genomic medicine settings when analyzing individual genomes, including interpreting positive and negative findings with scrutiny, especially for indels. We advocate for renewed collection and sequencing of multi-generational families to increase the overall accuracy of whole genomes.