# Interface for loading GBIF datasets into Flux

**URL:** https://discourse.julialang.org/t/interface-for-loading-gbif-datasets-into-flux/65976
**Category:** Machine Learning
**Created:** [August 7, 2021, 2:35am UTC](https://discourse.julialang.org/t/interface-for-loading-gbif-datasets-into-flux/65976 "2021-08-07T02:35:16Z")
**Posts on this page:** 4
**Page:** 1

<div class="post-metadata">

### Author: ![banksiaboy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/banksiaboy/32/27032_2.png) [@banksiaboy](https://discourse.julialang.org/u/banksiaboy)
#### Post date: [August 7, 2021, 2:35am UTC](https://discourse.julialang.org/t/interface-for-loading-gbif-datasets-into-flux/65976/1 "2021-08-07T02:35:16Z")

</div>

Here is an example of a dataset resulting from my query against the GBIF occurrences database.

> **[Download](https://www.gbif.org/occurrence/download/0333654-200613084148143)**
>
> Global Biodiversity Information Facility. Free and Open Access to Biodiversity Data.

As you can see it has its own DOI, is well-structured, and should be a good candidate for loading as a data source into Flux. I’m interested in applying fluxml.ai and transfer learning.

As I’m new to the Julia/ ML ecosystems could some folks please advise what is the best way to approach a re-useful way of loading this data. Or building some lightweight infrastructure to do so…

---

<div class="post-metadata">

### Author: ![banksiaboy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/banksiaboy/32/27032_2.png) [@banksiaboy](https://discourse.julialang.org/u/banksiaboy)
#### Post date: [August 7, 2021, 5:44am UTC](https://discourse.julialang.org/t/interface-for-loading-gbif-datasets-into-flux/65976/2 "2021-08-07T05:44:10Z")

</div>

Making it easy to get your data into Flux is a great way to build community.  
Jeremy Howard:  
[FastAI.jl Live Q&A (ML Community Call, 2021-08-02) - YouTube](https://youtube.com/watch?v=ZaiLk_ibC-0&feature=share) 44 mins in

---

<div class="post-metadata">

### Author: ![ToucheSir](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/touchesir/32/14411_2.png) [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)
#### Post date: [August 11, 2021, 6:35pm UTC](https://discourse.julialang.org/t/interface-for-loading-gbif-datasets-into-flux/65976/3 "2021-08-11T18:35:50Z")

</div>

I would say the way to think about this is “how do I get this data into Julia”. Unlike some other language ecosystems, Julia’s ML stack doesn’t try to invent a whole new set of data formats for inputs and outputs. So if you can read in that dataset as an array or dataframe, for example, you’ll be able to use it for ML.

---

<div class="post-metadata">

### Author: ![banksiaboy](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/banksiaboy/32/27032_2.png) [@banksiaboy](https://discourse.julialang.org/u/banksiaboy)
#### Post date: [August 20, 2021, 6:47am UTC](https://discourse.julialang.org/t/interface-for-loading-gbif-datasets-into-flux/65976/4 "2021-08-20T06:47:12Z")

</div>

Thanks, and I have loaded data susbsets into dataframes by `hand`. But the schema for GBIF occurrence datasets is fixed, and I’d like to have a standard way of creating training and reference sets from that schema.

I’m sure I’ll work it out…
