skip to main content
research-article
Best Paper

Darwin: A Genomics Co-processor Provides up to 15,000X Acceleration on Long Read Assembly

Published:19 March 2018Publication History
Skip Abstract Section

Abstract

Genomics is transforming medicine and our understanding of life in fundamental ways. Genomics data, however, is far outpacing Moore»s Law. Third-generation sequencing technologies produce 100X longer reads than second generation technologies and reveal a much broader mutation spectrum of disease and evolution. However, these technologies incur prohibitively high computational costs. Over 1,300 CPU hours are required for reference-guided assembly of the human genome, and over 15,600 CPU hours are required for de novo assembly. This paper describes "Darwin" --- a co-processor for genomic sequence alignment that, without sacrificing sensitivity, provides up to $15,000X speedup over the state-of-the-art software for reference-guided assembly of third-generation reads. Darwin achieves this speedup through hardware/algorithm co-design, trading more easily accelerated alignment for less memory-intensive filtering, and by optimizing the memory system for filtering. Darwin combines a hardware-accelerated version of D-SOFT, a novel filtering algorithm, alignment at high speed, and with a hardware-accelerated version of GACT, a novel alignment algorithm. GACT generates near-optimal alignments of arbitrarily long genomic sequences using constant memory for the compute-intensive step. Darwin is adaptable, with tunable speed and sensitivity to match emerging sequencing technologies and to meet the requirements of genomic applications beyond read assembly.

References

  1. Pico computing product brief: M-505-k325t. URL https://goo.gl/poeWUA.Google ScholarGoogle Scholar
  2. TimeLogic Corporation. URL http://www.timelogic.com.Google ScholarGoogle Scholar
  3. S. F. Altschul, W. Gish, W. Miller, E. W. Myers, and D. J. Lipman. Basic local alignment search tool. Journal of molecular biology, 1990.Google ScholarGoogle Scholar
  4. S. F. Altschul, T. L. Madden, A. A. Schaffer, J. Zhang, Z. Zhang, W. Miller, and D. J. Lipman. Gapped blast and psi-blast: a new generation of protein database search programs. Nucleic acids research, 25 (17): 3389--3402, 1997.Google ScholarGoogle Scholar
  5. G. Bejerano, M. Pheasant, I. Makunin, S. Stephen, W. J. Kent, J. S. Mattick, and D. Haussler. Ultraconserved elements in the human genome. Science, 304 (5675): 1321--1325, 2004.Google ScholarGoogle ScholarCross RefCross Ref
  6. K. Berlin, S. Koren, C.-S. Chin, J. P. Drake, J. M. Landolin, and A. M. Phillippy. Assembling large genomes with single-molecule sequencing and locality-sensitive hashing. Nature biotechnology, 33 (6): 623--630, 2015.Google ScholarGoogle Scholar
  7. N. Bray and L. Pachter. Mavid: constrained ancestral alignment of multiple sequences. Genome research, 14 (4): 693--699, 2004.Google ScholarGoogle Scholar
  8. B. Brejová, D. G. Brown, and T. Vinavr. Vector seeds: An extension to spaced seeds. Journal of Computer and System Sciences, 70 (3): 364--380, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. J. Buhler. Efficient large-scale sequence comparison by locality-sensitive hashing. Bioinformatics, 17 (5): 419--428, 2001.Google ScholarGoogle ScholarCross RefCross Ref
  10. B. Buyukkurt and W. A. Najj. Compiler generated systolic arrays for wavefront algorithm acceleration on fpgas. In 2008 International Conference on Field Programmable Logic and Applications, pages 655--658. IEEE, 2008.Google ScholarGoogle ScholarCross RefCross Ref
  11. M. J. Chaisson and G. Tesler. Mapping single molecule sequencing reads using basic local alignment with successive refinement (blasr): application and theory. BMC bioinformatics, 13 (1): 238, 2012.Google ScholarGoogle Scholar
  12. K. Chandrasekar, C. Weis, Y. Li, B. Akesson, N. Wehn, and K. Goossens. Drampower: Open-source dram power & energy estimation tool. URL: http://www. drampower. info.Google ScholarGoogle Scholar
  13. K.-M. Chao, W. R. Pearson, and W. Miller. Aligning two sequences within a specified diagonal band. Computer applications in the biosciences: CABIOS, 8 (5): 481--487, 1992.Google ScholarGoogle Scholar
  14. P. Chen, C. Wang, X. Li, and X. Zhou. Accelerating the next generation long read mapping with the fpga-based system. IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB), 11 (5): 840--852, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Y.-T. Chen, J. Cong, J. Lei, and P. Wei. A novel high-throughput acceleration engine for read alignment. In Field-Programmable Custom Computing Machines (FCCM), 2015 IEEE 23rd Annual International Symposium on, pages 199--202. IEEE, 2015. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. S. L. Clarke, J. E. VanderMeer, A. M. Wenger, B. T. Schaar, N. Ahituv, and G. Bejerano. Human developmental enhancers conserved between deuterostomes and protostomes. PLoS genetics, 8 (8): e1002852, 2012.Google ScholarGoogle Scholar
  17. W. J. Dally and B. P. Towles. Principles and practices of interconnection networks. Elsevier, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. A. Döring, D. Weese, T. Rausch, and K. Reinert. Seqan an efficient, generic c+ library for sequence analysis. BMC bioinformatics, 9 (1): 11, 2008.Google ScholarGoogle Scholar
  19. Edico Genome. Dragen bio-it platform. URL http://edicogenome.com/dragen-bioit-platform/.Google ScholarGoogle Scholar
  20. J. Eid, A. Fehr, J. Gray, K. Luong, J. Lyle, G. Otto, P. Peluso, D. Rank, P. Baybayan, B. Bettman, et al. Real-time dna sequencing from single polymerase molecules. Science, 323 (5910): 133--138, 2009.Google ScholarGoogle ScholarCross RefCross Ref
  21. M. Eisenstein. Oxford nanopore announcement sets sequencing sector abuzz. Nature biotechnology, 30 (4): 295--296, 2012.Google ScholarGoogle Scholar
  22. P. Faes, B. Minnaert, M. Christiaens, E. Bonnet, Y. Saeys, D. Stroobandt, and Y. Van de Peer. Scalable hardware accelerator for comparing dna and protein sequences. In Proceedings of the 1st international conference on Scalable information systems, page 33. ACM, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. S. Goodwin, J. Gurtowski, S. Ethe-Sayers, P. Deshpande, M. C. Schatz, and W. R. McCombie. Oxford nanopore sequencing, hybrid error correction, and de novo assembly of a eukaryotic genome. Genome research, 25 (11): 1750--1756, 2015.Google ScholarGoogle Scholar
  24. D. Gordon, J. Huddleston, M. J. Chaisson, C. M. Hill, Z. N. Kronenberg, K. M. Munson, M. Malig, A. Raja, I. Fiddes, L. W. Hillier, et al. Long-read sequence assembly of the gorilla genome. Science, 352 (6281): aae0344, 2016.Google ScholarGoogle Scholar
  25. O. Gotoh. An improved algorithm for matching biological sequences. Journal of molecular biology, 162 (3): 705--708, 1982.Google ScholarGoogle Scholar
  26. D. Greaves, S. Sanyal, and S. Singh. Synthesis of a parallel smith-waterman sequence alignment kernel into fpga hardware. URL http://www.cl.cam.ac.uk/ djg11/pubs/mrsc09a.pdf.Google ScholarGoogle Scholar
  27. D. Gusfield. Algorithms on strings, trees and sequences: computer science and computational biology. Cambridge university press, 1997. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. M. A. Hamburg and F. S. Collins. The path to personalized medicine. New England Journal of Medicine, 363 (4): 301--304, 2010.Google ScholarGoogle ScholarCross RefCross Ref
  29. J. Handelsman. Metagenomics: application of genomics to uncultured microorganisms. Microbiology and molecular biology reviews, 68 (4): 669--685, 2004.Google ScholarGoogle Scholar
  30. R. S. Harris. Improved pairwise alignment of genomic DNA. ProQuest, 2007.Google ScholarGoogle Scholar
  31. M. Hiller, B. T. Schaar, V. B. Indjeian, D. M. Kingsley, L. R. Hagey, and G. Bejerano. A "forward genomics" approach links genotype to phenotype using independent phenotypic losses among related species. Cell reports, 2 (4): 817--823, 2012.Google ScholarGoogle Scholar
  32. D. S. Hirschberg. A linear space algorithm for computing maximal common subsequences. Communications of the ACM, 18 (6): 341--343, 1975. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Illumina. Illumina hiseq x series of sequencing systems: Specification sheet. URL https://goo.gl/paq2X5.Google ScholarGoogle Scholar
  34. Intel. Intel arria 10 device overview. URL https://www.altera.com/en_US/pdfs/literature/hb/arria-10/a10_overview.pdf.Google ScholarGoogle Scholar
  35. Intel. Intel pcm power utility. URL https://goo.gl/4KumhA.Google ScholarGoogle Scholar
  36. X. Jiang, X. Liu, L. Xu, P. Zhang, and N. Sun. A reconfigurable accelerator for smith--waterman algorithm. IEEE Transactions on Circuits and Systems II: Express Briefs, 54 (12): 1077--1081, 2007.Google ScholarGoogle ScholarCross RefCross Ref
  37. L. S. Johnson, S. R. Eddy, and E. Portugaly. Hidden markov model speed heuristic and iterative hmm search procedure. BMC bioinformatics, 11 (1): 431, 2010.Google ScholarGoogle Scholar
  38. U. Keich, M. Li, B. Ma, and J. Tromp. On spaced seeds for similarity search. Discrete Applied Mathematics, 138 (3): 253--263, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. W. J. Kent. BLAT - the BLAST-like alignment tool. Genome research, 2002.Google ScholarGoogle Scholar
  40. Y. Kim, W. Yang, and O. Mutlu. Ramulator: A fast and extensible dram simulator. IEEE Computer Architecture Letters, 15 (1): 45--49, 2016. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. K.-P. Koepfli, B. Paten, G. K. C. of Scientists, and S. J. O'Brien. The genome 10k project: a way forward. Annu. Rev. Anim. Biosci., 3 (1): 57--111, 2015.Google ScholarGoogle ScholarCross RefCross Ref
  42. S. Koren and A. M. Phillippy. One chromosome, one contig: complete microbial genomes from long-read sequencing and assembly. Current opinion in microbiology, 23: 110--120, 2015.Google ScholarGoogle Scholar
  43. S. Koren, B. P. Walenz, K. Berlin, J. R. Miller, and A. M. Phillippy. Canu: scalable and accurate long-read assembly via adaptive k-mer weighting and repeat separation. bioRxiv, page 071282, 2016.Google ScholarGoogle Scholar
  44. N. Krumm, T. N. Turner, C. Baker, L. Vives, K. Mohajeri, K. Witherspoon, A. Raja, B. P. Coe, H. A. Stessman, Z.-X. He, et al. Excess of rare, inherited truncating mutations in autism. Nature genetics, 47 (6): 582--588, 2015.Google ScholarGoogle Scholar
  45. T. W. Lam, W.-K. Sung, S.-L. Tam, C.-K. Wong, and S.-M. Yiu. Compressed indexing and local alignment of dna. Bioinformatics, 24 (6): 791--797, 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. E. S. Lander, L. M. Linton, B. Birren, C. Nusbaum, M. C. Zody, J. Baldwin, K. Devon, K. Dewar, M. Doyle, W. FitzHugh, et al. Initial sequencing and analysis of the human genome. Nature, 409 (6822): 860--921, 2001.Google ScholarGoogle ScholarCross RefCross Ref
  47. H. Li. Aligning sequence reads, clone sequences and assembly contigs with bwa-mem. arXiv preprint arXiv:1303.3997, 2013.Google ScholarGoogle Scholar
  48. H. Li and R. Durbin. Fast and accurate long-read alignment with burrows--wheeler transform. Bioinformatics, 26 (5): 589--595, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. J. Z. Li, D. M. Absher, H. Tang, A. M. Southwick, A. M. Casto, S. Ramachandran, H. M. Cann, G. S. Barsh, M. Feldman, L. L. Cavalli-Sforza, et al. Worldwide human relationships inferred from genome-wide patterns of variation. Science, 319 (5866): 1100--1104, 2008.Google ScholarGoogle ScholarCross RefCross Ref
  50. Z. Li, Y. Chen, D. Mu, J. Yuan, Y. Shi, H. Zhang, J. Gan, N. Li, X. Hu, B. Liu, et al. Comparison of the two major classes of assembly algorithms: overlap--layout--consensus and de-bruijn-graph. Briefings in functional genomics, 11 (1): 25--37, 2012.Google ScholarGoogle Scholar
  51. R. J. Lipton and D. P. Lopresti. Comparing long strings on a short systolic array. Princeton University, Department of Computer Science, 1986.Google ScholarGoogle Scholar
  52. C. López-Ot'ın, M. A. Blasco, L. Partridge, M. Serrano, and G. Kroemer. The hallmarks of aging. Cell, 153 (6): 1194--1217, 2013.Google ScholarGoogle ScholarCross RefCross Ref
  53. A. Marcovitz, Y. Turakhia, M. Gloudemans, B. A. Braun, H. I. Chen, and G. Bejerano. A novel unbiased test for molecular convergent evolution and discoveries in echolocating, aquatic and high-altitude mammals. bioRxiv, page 170985, 2017.Google ScholarGoogle Scholar
  54. C. Y. McLean, P. L. Reno, A. A. Pollen, A. I. Bassan, T. D. Capellini, C. Guenther, V. B. Indjeian, X. Lim, D. B. Menke, B. T. Schaar, et al. Human-specific loss of regulatory dna and the evolution of human-specific traits. Nature, 471 (7337): 216--219, 2011.Google ScholarGoogle ScholarCross RefCross Ref
  55. J. D. Merker, A. M. Wenger, T. Sneddon, M. Grove, Z. Zappala, L. Fresard, D. Waggott, S. Utiramerur, Y. Hou, K. S. Smith, et al. Long-read genome sequencing identifies causal structural variation in a mendelian disease. Genetics in medicine: official journal of the American College of Medical Genetics, 2017.Google ScholarGoogle Scholar
  56. G. Myers. A fast bit-vector algorithm for approximate string matching based on dynamic programming. Journal of the ACM (JACM), 46 (3): 395--415, 1999. Google ScholarGoogle ScholarDigital LibraryDigital Library
  57. G. Myers. Efficient local alignment discovery amongst noisy long reads. In International Workshop on Algorithms in Bioinformatics, pages 52--67. Springer, 2014.Google ScholarGoogle ScholarCross RefCross Ref
  58. G. Navarro. A guided tour to approximate string matching. ACM computing surveys (CSUR), 33 (1): 31--88, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  59. Z. Nawaz, M. Nadeem, H. van Someren, and K. Bertels. A parallel fpga design of the smith-waterman traceback. In Field-Programmable Technology (FPT), 2010 International Conference on, pages 454--459. IEEE, 2010.Google ScholarGoogle ScholarCross RefCross Ref
  60. L. Noé and G. Kucherov. Yass: enhancing the sensitivity of dna similarity search. Nucleic acids research, 33 (suppl 2): W540--W543, 2005.Google ScholarGoogle Scholar
  61. C. B. Olson, M. Kim, C. Clauson, B. Kogon, C. Ebeling, S. Hauck, and W. L. Ruzzo. Hardware acceleration of short read mapping. In Field-Programmable Custom Computing Machines (FCCM), 2012 IEEE 20th Annual International Symposium on, pages 161--168. IEEE, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Y. Ono, K. Asai, and M. Hamada. Pbsim: Pacbio reads simulator-toward accurate genome assembly. Bioinformatics, 29 (1): 119--121, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. M. K. Papamichael and J. C. Hoe. Connect: re-examining conventional wisdom for designing nocs in the context of fpgas. In Proceedings of the ACM/SIGDA international symposium on Field Programmable Gate Arrays, pages 37--46. ACM, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. nez, Bignell, et al.}pleasance2010comprehensiveE. D. Pleasance, R. K. Cheetham, P. J. Stephens, D. J. McBride, S. J. Humphray, C. D. Greenman, I. Varela, M.-L. Lin, G. R. Ordó nez, G. R. Bignell, et al. A comprehensive catalogue of somatic mutations from a human cancer genome. Nature, 463 (7278): 191--196, 2010.Google ScholarGoogle Scholar
  65. A. Putnam, A. M. Caulfield, E. S. Chung, D. Chiou, K. Constantinides, J. Demme, H. Esmaeilzadeh, J. Fowers, G. P. Gopal, J. Gray, et al. A reconfigurable fabric for accelerating large-scale datacenter services. In Computer Architecture (ISCA), 2014 ACM/IEEE 41st International Symposium on, pages 13--24. IEEE, 2014. Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. M. Roberts, W. Hayes, B. R. Hunt, S. M. Mount, and J. A. Yorke. Reducing storage requirements for biological sequence comparison. Bioinformatics, 20 (18): 3363--3369, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  67. E. E. Schadt, S. Turner, and A. Kasarskis. A window into third-generation sequencing. Human molecular genetics, 19 (R2): R227--R240, 2010.Google ScholarGoogle Scholar
  68. P. Shivakumar and N. P. Jouppi. Cacti 3.0: An integrated cache timing, power, and area model. Technical report, Technical Report 2001/2, Compaq Computer Corporation, 2001.Google ScholarGoogle Scholar
  69. T. F. Smith and M. S. Waterman. Identification of common molecular subsequences. Journal of molecular biology, 147 (1): 195--197, 1981.Google ScholarGoogle Scholar
  70. M. Sovsić and M. Sikić. Edlib: a c/c+ library for fast, exact sequence alignment using edit distance. Bioinformatics, 2017.Google ScholarGoogle Scholar
  71. I. Sović, M. vSikić, A. Wilm, S. N. Fenlon, S. Chen, and N. Nagarajan. Fast and sensitive mapping of nanopore sequencing reads with graphmap. Nature communications, 7, 2016.Google ScholarGoogle Scholar
  72. O. Spichenok, Z. M. Budimlija, A. A. Mitchell, A. Jenny, L. Kovacevic, D. Marjanovic, T. Caragine, M. Prinz, and E. Wurmbach. Prediction of eye and skin color in diverse populations using seven SNPs. Forensic Science International: Genetics, 5 (5): 472--478, 2011.Google ScholarGoogle ScholarCross RefCross Ref
  73. Z. D. Stephens, S. Y. Lee, F. Faghri, R. H. Campbell, C. Zhai, M. J. Efron, R. Iyer, M. C. Schatz, S. Sinha, and G. E. Robinson. Big data: astronomical or genomical? PLoS biology, 13 (7): e1002195, 2015.Google ScholarGoogle Scholar
  74. Synopsys IC. Compiler user guide, 2013. URL http://www. synopsys. com.Google ScholarGoogle Scholar
  75. Synopsys Inc. Compiler, design and user, rtl and guide, modeling, 2001. URL http://www. synopsys. com.Google ScholarGoogle Scholar
  76. W. Tang, W. Wang, B. Duan, C. Zhang, G. Tan, P. Zhang, and N. Sun. Accelerating millions of short reads mapping on a heterogeneous architecture with fpga accelerator. In Field-Programmable Custom Computing Machines (FCCM), 2012 IEEE 20th Annual International Symposium on, pages 184--187. IEEE, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. J. D. Thompson, D. G. Higgins, and T. J. Gibson. Clustal w: improving the sensitivity of progressive multiple sequence alignment through sequence weighting, position-specific gap penalties and weight matrix choice. Nucleic acids research, 22 (22): 4673--4680, 1994.Google ScholarGoogle Scholar
  78. P. Van Rooyen, R. J. McMillen, and M. Ruehle. Bioinformatics systems, apparatuses, and methods executed on an integrated circuit processing platform, Jan. 5 2016. US Patent App. 14/988,666.Google ScholarGoogle Scholar
  79. J. C. Venter, M. D. Adams, E. W. Myers, P. W. Li, R. J. Mural, G. G. Sutton, H. O. Smith, M. Yandell, C. A. Evans, R. A. Holt, et al. The sequence of the human genome. science, 291 (5507): 1304--1351, 2001.Google ScholarGoogle Scholar
  80. R. H. Waterson, E. S. Lander, R. K. Wilson, et al. Initial sequence of the chimpanzee genome and comparison with the human genome. Nature, 437 (7055): 69, 2005.Google ScholarGoogle ScholarCross RefCross Ref
  81. Y. Yamaguchi, T. Maruyama, and A. Konagaya. High speed homology search with fpgas. In Proceedings of the 7th Pacific Symposium on Biocomputing (PSB'02), pages 271--282, 2001.Google ScholarGoogle ScholarCross RefCross Ref
  82. C. W. Yu, K. Kwong, K.-H. Lee, and P. H. W. Leong. A smith-waterman systolic cell. In New Algorithms, Architectures and Applications for Reconfigurable Computing, pages 291--300. Springer, 2005.Google ScholarGoogle Scholar
  83. G. Zhang. Genomics: Bird sequencing project takes off. Nature, 522 (7554): 34--34, 2015.Google ScholarGoogle ScholarCross RefCross Ref
  84. Z. Zhang, S. Schwartz, L. Wagner, and W. Miller. A greedy algorithm for aligning dna sequences. Journal of Computational biology, 7 (1--2): 203--214, 2000.Google ScholarGoogle Scholar

Index Terms

  1. Darwin: A Genomics Co-processor Provides up to 15,000X Acceleration on Long Read Assembly

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM SIGPLAN Notices
      ACM SIGPLAN Notices  Volume 53, Issue 2
      ASPLOS '18
      February 2018
      809 pages
      ISSN:0362-1340
      EISSN:1558-1160
      DOI:10.1145/3296957
      Issue’s Table of Contents
      • cover image ACM Conferences
        ASPLOS '18: Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems
        March 2018
        827 pages
        ISBN:9781450349116
        DOI:10.1145/3173162

      Copyright © 2018 ACM

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 19 March 2018

      Check for updates

      Qualifiers

      • research-article

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader
    About Cookies On This Site

    We use cookies to ensure that we give you the best experience on our website.

    Learn more

    Got it!