# Genomic Interval Metadata & Annotation

**URL:** <https://discourse.julialang.org/t/genomic-interval-metadata-annotation/4721>\
**Category:** Biology, Health, and Medicine\
**Tags:** question\
**Created:** [July 7, 2017, 5:25pm UTC](https://discourse.julialang.org/t/genomic-interval-metadata-annotation/4721 "2017-07-07T17:25:35Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![snystrom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/snystrom/32/7026_2.png) [@snystrom](https://discourse.julialang.org/u/snystrom)\
**Post date:** [July 7, 2017, 5:25pm UTC](https://discourse.julialang.org/t/genomic-interval-metadata-annotation/4721/1 "2017-07-07T17:25:35Z")

</div>

Hi,

Pretty new to Julia but looking forward to its continued growth and use for genomic data. Given that the current BioJulia documentation is a bit sparse, I wanted to get the opinion of people who are more familiar with the codebase about whether this need is currently being met.

In most of my current work I use the GenomicRanges R/Bioconductor package for all my genomic interval operations. I notoriously abuse the metadata columns of this data structure to track basically everything about my peaks, using a single large GRanges object as sort of a database backend that I subset from to make plots/do analysis. In the current setup of the GenomicFeatures.Interval, there also exists a metadata-like column. Would the Julia solution to my GRanges database be to use a dataframe for the metadata field? Is there a datastructure that is better suited to this in the BioJulia/Julia universe (ie just a dataframe?)? The issue I sometimes wind up having in R is having to jump things back and forth from dataframe to GRanges to accomplish certain tasks, or having to use non-intuitive lookup tables to link Genomic features to some kind of meaningful data (like a clustered count matrix for heatmaps) so it could be that developing a BioJulia framework that prevents a similar issue could be a useful one.

---

<div class="post-metadata">

**Author:** ![phaverty](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/phaverty/32/1424_2.png) [@phaverty](https://discourse.julialang.org/u/phaverty)\
**Post date:** [July 19, 2017, 9:52pm UTC](https://discourse.julialang.org/t/genomic-interval-metadata-annotation/4721/2 "2017-07-19T21:52:07Z")

</div>

You may find the GenomicTable type in [GenomicVectors.jl](https://github.com/phaverty/GenomicVectors.jl) useful. It is a subtype of AbstractDataTable that has a GenomicRanges or GenomicPositions as a row index. The end result is something much like BioConductor’s GenomicRanges with a DataFrame as metadata.

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [July 19, 2017, 11:09pm UTC](https://discourse.julialang.org/t/genomic-interval-metadata-annotation/4721/3 "2017-07-19T23:09:12Z")

</div>

It sounds like @phaverty 's package might be your best bet, since it’s directly inspired by the R package you’re used to.

But we’ve also got BioJulia, which is undergoing a bit of a reorganization at the moment, so it might be tough to find what you’re looking for. It sounds like you might want [`GenomicFeatures.jl`](http://biojulia.net/GenomicFeatures.jl/latest/intervals/) which has intervals. Eg:

```julia
julia> i = Interval("chr1", 10000, 20000, '+', "some annotation")
GenomicFeatures.Interval{String}:
  sequence name: chr1
  leftmost position: 10000
  rightmost position: 20000
  strand: +
  metadata: some annotation

julia> seqname(i)
"chr1"

julia> leftposition(i)
10000

julia> rightposition(i)
20000

julia> strand(i)
STRAND_POS

julia> metadata(i)
"some annotation"

```

Of note, that metadata field can be of any type, so you could store a `Dict` or even a `DataFrame` in the metadata field if you wanted to. That package also has `IntervalCollection`s, so you can group all of the intervals for a given sequence in an efficient way.

I’m not the author of this package, but the Bio.jl folks typically are trying to write code that takes advantage of julia’s awesomeness in innovative ways. So stuff might be less familiar, but ultimately may be faster than code ported from a different language (I’m not familiar with that R package, so I don’t know if it’s better or worse for what you want). That said, sometimes ease \> speed, so YMMV.

Maybe @bicycle1885 or @Ward9250 might have more to say on the design philosophy for `Intervals`.

---

<div class="post-metadata">

**Author:** ![kevbonham](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kevbonham/32/216165_2.png) [@kevbonham](https://discourse.julialang.org/u/kevbonham)\
**Post date:** [July 19, 2017, 11:29pm UTC](https://discourse.julialang.org/t/genomic-interval-metadata-annotation/4721/4 "2017-07-19T23:29:57Z")

</div>

@synstrom Crap, I see now that my response is basically exactly what you’re asking about :-/. Reading comprehension fail!

In which case I don’t have any relevant info to add, other than my tags of some of the key architects of the Bio package. I’ll see myself out.

---

<div class="post-metadata">

**Author:** ![bicycle1885](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bicycle1885/32/107_2.png) [@bicycle1885](https://discourse.julialang.org/u/bicycle1885)\
**Post date:** [July 20, 2017, 1:24am UTC](https://discourse.julialang.org/t/genomic-interval-metadata-annotation/4721/5 "2017-07-20T01:24:32Z")

</div>

> Would the Julia solution to my GRanges database be to use a dataframe for the metadata field?

`GenomicFeatures.Interval{T}` is parameterized by its metadata type and can store any object you want there, such as `Float64`, `String`, `Dict`, `Vector`, `DataFrame`, etc.

> Is there a datastructure that is better suited to this in the BioJulia/Julia universe (ie just a dataframe?)?

As @phaverty already mentioned, his GenomicVectors.jl package would be a good solution to your problem. Unfortunately, I have no idea about the performance for your tasks.

GenomicFeatures.jl has `IntervalCollection` type, which is conceptually a sequence of `Interval`s with index. `IntervalCollection` is more like a dictionary with interval keys and metadata values while GenomicVectors.jl is more like a data frame. The index data structure is based on the [interval tree](https://en.wikipedia.org/wiki/Interval_tree) built on top of [B+ tree](https://en.wikipedia.org/wiki/B%2B_tree) while GRanges of R uses [NCList](https://doi.org/10.1093/bioinformatics/btl647) by default. Both support overlap query but they are very different data structures and hence the performance characteristics would be very different as well. We’ve [compared these two before](https://github.com/BioJulia/Bio.jl/issues/340#issuecomment-247602325) and concluded that the interval tree is better for our package.

I cannot reach a definitive conclusion of your problem but I hope my comments are helpful to you.

---

<div class="post-metadata">

**Author:** ![Peter\_Haverty](https://avatars.discourse-cdn.com/v4/letter/p/ed8c4c/32.png) [@Peter\_Haverty](https://discourse.julialang.org/u/Peter_Haverty)\
**Post date:** [July 20, 2017, 2:38am UTC](https://discourse.julialang.org/t/genomic-interval-metadata-annotation/4721/6 "2017-07-20T02:38:29Z")

</div>

GenomicVectors.jl has a few relevant types. GenomicRanges is like R’s  
GenomicRanges minus the metadata part (it is a vector). It actually uses  
features from GenomicFeatures.jl to represent intervals and to do overlap  
queries. It behaves like Vector{Interval}, rather than IntervalCollection,  
meaning that it can be set to a specific order (e.g. sorted by some score,  
genome location, etc.)

GenomicTable is a DataTable that uses a GenomicRanges as a row index. I  
like having a proper DataTable so you can do all the usual data frame  
operations. My understanding of the metadata for an IntervalCollection is  
that each Interval has its own metadata slot, so you would have to write  
your own functions to operate on the collection of them.

I expect that the speed of things like overlap queries will be a bit slower  
than GenomicFeatures.jl as it is necessary to build an IntervalCollection  
before doing any queries, but it should be plenty fast relative to R. It’s  
all a bit experimental and I don’t know how the pros and cons of these  
different approaches will balance out. I’d be very interested in any  
feedback you can offer.

Hope that helps.

Pete

---

<div class="post-metadata">

**Author:** ![snystrom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/snystrom/32/7026_2.png) [@snystrom](https://discourse.julialang.org/u/snystrom)\
**Post date:** [July 20, 2017, 3:35pm UTC](https://discourse.julialang.org/t/genomic-interval-metadata-annotation/4721/7 "2017-07-20T15:35:08Z")

</div>

I seem to recall a Github issue you posted saying you guys were converting the Interval type to immutable–this would make the metadata column static as well, correct? Regardless, your comments are very helpful, and I think I’ll give GenomicVectors.jl a shot. For my purposes, I care less about interval operations and more about having a datastructure that lets me track and resort data based on other parameters that are not genomic position. Genomic position is used as a secondary characteristic to retrieve sequence information or interval overlaps later between groups that fit the filtered/sorted characteristics.

---

<div class="post-metadata">

**Author:** ![snystrom](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/snystrom/32/7026_2.png) [@snystrom](https://discourse.julialang.org/u/snystrom)\
**Post date:** [July 20, 2017, 3:41pm UTC](https://discourse.julialang.org/t/genomic-interval-metadata-annotation/4721/8 "2017-07-20T15:41:53Z")

</div>

I’ll give this a shot in the next few days when I’ve got some downtime and let you know.

---

<div class="post-metadata">

**Author:** ![jonathanBieler](https://avatars.discourse-cdn.com/v4/letter/j/82dd89/32.png) [@jonathanBieler](https://discourse.julialang.org/u/jonathanBieler)\
**Post date:** [July 20, 2017, 3:44pm UTC](https://discourse.julialang.org/t/genomic-interval-metadata-annotation/4721/9 "2017-07-20T15:44:26Z")

</div>

If the things you work with are type-like you can also just use a type. That is instead of constructing an interval and putting things into its metadata you do:

```julia
type A
    x
    y
    interval
end

```

(or inherit from Interval).

The advantage is that you can build a more natural and structured grammar using types, i.e. if you have an array of peaks and you want to do the histogram of their height:

```
hist(map(height,peaks))

```

---

<div class="post-metadata">

**Author:** ![bicycle1885](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/bicycle1885/32/107_2.png) [@bicycle1885](https://discourse.julialang.org/u/bicycle1885)\
**Post date:** [July 21, 2017, 2:24am UTC](https://discourse.julialang.org/t/genomic-interval-metadata-annotation/4721/10 "2017-07-21T02:24:45Z")

</div>

> [@snystrom](#):
>
> I seem to recall a Github issue you posted saying you guys were converting the Interval type to immutable–this would make the metadata column static as well, correct?

Yeah, the metadata is also immutable. But if it is a reference value (e.g. `Vector`), you can update the value in-place. Also, if you wrap the metadata with `Ref`, you can still update the value even if it is not a reference value (e.g. `Int`) like this:

```julia
julia> i = Interval("chr1", 1:10, '+', Ref(0))
GenomicFeatures.Interval{Base.RefValue{Int64}}:
  sequence name: chr1
  leftmost position: 1
  rightmost position: 10
  strand: +
  metadata: Base.RefValue{Int64}(0)

julia> i.metadata[] # weird brackets are needed
0

julia> i.metadata[] += 1 # update!
1

julia> i.metadata[]
1

```

So, basically making `Interval` immutable will not limit the functionality.