Scripting for large-scale sequencing based on Hadoop

André Schumacher, Luca Pireddu, Aleksi Kallio, Matti Niemenmaa, Eija Korpelainen, Gianluigi Zanetti, Keijo Heljanko

Abstract


The large volumes of data generated by modern sequencing experiments present significant challenges in their manipulation and analysis. Traditional approaches are often found to be complicated to scale. We describe our ongoing work on SeqPig, a tool that facilitates the use of the Pig Latin distributed scripting language to manipulate, analyze and query sequencing data applying the advances motivated by the “big data revolution” in data-intensive activities. SeqPig provides access to popular data formats and implements a number of custom sequencing-specific functions.  Most importantly, it grants users access to the scalable Hadoop platform from a high level scripting language

Keywords


bioinformatics; NGS; data analysis; cloud computing; high-performance computing

Full Text:

PDF


DOI: https://doi.org/10.14806/ej.19.A.628

Refbacks

  • There are currently no refbacks.