Drosophila Melanogaster

Drosophila Melanogaster, the common fruit fly, is a model organism which has been extensively used in entymological research. It is one of the most studied organisms in biological research, particularly in genetics and developmental biology.

When its not being used for scientific research, D. melanogaster is a common pest in homes, restaurants, and anywhere else that serves food. They are not to be confused with Tephritidae flys (also known as fruit flys).

https://en.wikipedia.org/wiki/Drosophila_melanogaster

about the Genome

This genome was first sequenced in 2000. It contains four pairs of chromosomes (2,3,4 and X/Y). More than 60% of the genome appears to be functional non-protein-coding DNA.

The genome is maintained and frequently updated at Flybase. This dataset is sourced from the UCSC Genome Bioinformatics download page. It uses the August 2014 version of the D. melanogaster genome (dm6, BDGP Release 6 + ISO1 MT). http://hgdownload.soe.ucsc.edu/downloads.html#fruitfly

Files were modified by Kaggle to be a better fit for analysis on scripts. This primarily involved turning files into CSV format, with a header row, as well as converting the genome itself from 2bit format into a FASTA sequence file.

Bioinformatics

Genomic analysis can be daunting to data scientists who haven't had much experience with bioinformatics before. We have tried to give basic explanations to each of the files in this dataset, as well as links to further reading on the biological basis for each. If you haven't had the chance to study much biology before, some light reading (ie wikipedia) on the following topics may be helpful to understand the nuances of the data provided here: Genetics, Genomics (Sequencing/Genome Assembly), Chromosomes, DNA, RNA (mRNA/miRNA), Genes, Alleles, Exons, Introns, Transcription, Translation, Peptides, Proteins, Gene Regulation, Mutation, Phylogenetics, and SNPs.

Of course, if you've got some idea of the basics already - don't be afraid to jump right in!

Learning Bioinformatics

There are a lot of great resources for learning bioinformatics on the web. One cool site is Rosalind - a platform that gives you bioinformatic coding challenges to complete. You can use Kaggle scripts on this dataset to easily complete the challenges on Rosalind (and see Myles' solutions here if you get stuck). We have set up Biopython on Kaggle's docker image which is a great library to help you with your analyses. Check out their tutorial here and we've also created a python notebook with some of the tutorial applied to this dataset as a reference.

Files in this Dataset

Drosophila Melanogaster Genome

• genome.fa

The assembled genome itself is presented here in FASTA format. Each chromosome is a different sequence of nucleotides. Repeats from RepeatMasker and Tandem Repeats Finder (with period of 12 or less) are show in lower case; non-repeating sequence is shown in upper case.

meta Information

There are 3 additional files with meta information about the genome.

• meta-cpg-island-ext-unmasked.csv

This file contains descriptive information about CpG Islands in the genome.

https://en.wikipedia.org/wiki/CpG_site

• meta-cytoband.csv

This file describes the positions of cytogenic bands on each chromosome.

https://en.wikipedia.org/wiki/Cytogenetics

• meta-simple-repeat.csv

This file describes simple tandem repeats in the genome.

https://en.wikipedia.org/wiki/Repeated_sequence_(DNA)
https://en.wikipedia.org/wiki/Tandem_repeat

Drosophila Melanogaster mRNA Sequences

Messenger RNA (mRNA) is an intermediate molecule created as part of the cellular process of converting genomic information into proteins. Some mRNA are never translated into proteins and have functional roles in the cell on their own. Collectively, organism mRNA information is known as a Transcriptome. mRNA files included in this dataset give insight into the activity of genes in the organism.

https://en.wikipedia.org/wiki/Messenger_RNA

• mrna-genbank.fa

This file includes all mRNA sequences from GenBank associated with Drosophila Melanogaster.

http://www.ncbi.nlm.nih.gov/genbank/

• mrna-refseq.fa

This file includes all mRNA sequences from RefSeq associated with Drosophila Melanogaster.

http://www.ncbi.nlm.nih.gov/refseq/

Gene Predictions

A gene is a segment of DNA on the genome which, through mRNA, is used to create proteins in the organism. Knowing which parts of DNA are coding (genes) or non-coding is difficult, and a number of different systems for prediction exist. This dataset includes a number of different gene prediction systems applied to the drosophila melanogaster genome.

https://en.wikipedia.org/wiki/Gene_prediction

• genes-augustus.csv

AUGUSTUS is a piece of software that predicts genes ab initio using Hidden Markov Models.
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC441517/

• genes-genscan.csv

GENSCAN is an older ab initio software for predicting genes.
http://genes.mit.edu/GENSCANinfo.html

• genes-ensembl.csv

• ensembl-gtp.csv

• ensembl-pep.csv

• ensembl-source.csv

• ensembl-to-gene-name.csv

Ensembl provides gene annotation generated by their software Genebuild. This process combines automatic annotation alongside manual curation.
http://uswest.ensembl.org/info/genome/genebuild/genome_annotation.html

We have also included some supplementary files for these, including predicted protein peptide sequences for each predicted gene.

• genes-refseq.csv

• genes-xeno-refseq.csv

• refseq-link.csv

• refseq-summary.csv

We have included two RefSeq gene predictions in this dataset. The first is based solely on information from the drosophila melanogaster genome. The second (genes-xeno-refseq.csv) uses genes from other organisms as a basis for predicting genes in drosophila melanogaster.

RefSeq RNAs were aligned against the D. melanogaster genome using blat; those with an alignment of less than 15% were discarded. When a single RNA aligned in multiple places, the alignment having the highest base identity was identified. only alignments having a base identity level within 0.1% of the best and at least 96% base identity with the genomic sequence were kept.

We have also included supplementary files for these which include information about the genes that have been identified.