# ANN: MLDataPattern.jl
**URL:** https://discourse.julialang.org/t/ann-mldatapattern-jl/3299
**Category:** Machine Learning
**Tags:** package, announcement
**Created:** [April 20, 2017, 12:12pm UTC](https://discourse.julialang.org/t/ann-mldatapattern-jl/3299 "2017-04-20T12:12:47Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@Evizero](https://discourse.julialang.org/u/Evizero)
#### Post date: [April 20, 2017, 12:12pm UTC](https://discourse.julialang.org/t/ann-mldatapattern-jl/3299/1 "2017-04-20T12:12:47Z")
Hello everyone!
I am really happy to _finally_ announce the next JuliaML package reaching a stable state: MLDataPattern.jl
**Github:** [GitHub - JuliaML/MLDataPattern.jl: Utility package for subsetting, resampling, iteration, and partitioning of various types of data sets in Machine Learning](https://github.com/JuliaML/MLDataPattern.jl)
and with it the long overdue update of MLDataUtils.jl, which now uses MLDataPattern as one of its back-ends; thus serving as a meta package. It took us months to finally get here. The last tag of MLDataUtils was right around the 0.5 release. Since then we have completely redesigned the data munging functionality and just recently - because of code complexity - outsourced them into their own package MLDataPattern. With this change, the original package MLDataUtils will now serve as a convenient end-user facing package that reexports all data related functionality of JuliaML
**Github:** [GitHub - JuliaML/MLDataUtils.jl: Utility package for generating, loading, splitting, and processing Machine Learning datasets](https://github.com/JuliaML/MLDataUtils.jl)
## Description
MLDataPattern is a long running effort from a few of us to design and implement a package for common ML data access pattern in a Julian manner. As such you may find it a bit unintuitive at first if you are used to other frameworks from other languages. Yet we think the benefits are worth it. Most notably the package provides a number of pattern for lazy _shuffling_, _partitioning_, and _resampling_ data sets of various types and origin. At its core, the package was designed around the key requirement of allowing any user-defined type to serve as a custom data source and/or access pattern in a first class manner. We tried to accomplish this by designing the package to be as data container agnostic as we could.
Check it out! The documentation is _very_ comprehensive.
**Documentation:** [MLDataPattern.jl’s documentation — MLDataPattern.jl 0.1 documentation](http://mldatapatternjl.readthedocs.io/en/latest/)
## Closing Words
Let me know what you think. Any kind of feedback or criticism is very welcome!
Big thanks to @tbreloff @oxinabox for design and code contributions to the data access pattern!
---
### Author:  [@mkborregaard](https://discourse.julialang.org/u/mkborregaard)
#### Post date: [April 20, 2017, 12:50pm UTC](https://discourse.julialang.org/t/ann-mldatapattern-jl/3299/2 "2017-04-20T12:50:53Z")
This looks really useful. From looking at it, it looks as if much of the functionality could extend way beyond machine learning, as a general data handling functionality. Perhaps by defining this on new user types?
Yet you write that it explicitly belongs under the framework of machine-learning - can you expand a bit on why that is? Useful before I consider applying to another purpose.
Thanks!
---
### Author:  [@Evizero](https://discourse.julialang.org/u/Evizero)
#### Post date: [April 20, 2017, 1:07pm UTC](https://discourse.julialang.org/t/ann-mldatapattern-jl/3299/3 "2017-04-20T13:07:07Z")
Well, really MLDataPattern is about **nesting data sub-setting operations**. This is a common theme in machine learning, which is one of the very few interesting areas that I know something about. I emphasize in the documentation that it is “machine learning specific” to make clear, that MLDataPattern package has nothing to do with any select/groupby/summarize kind of operation. It doesn’t care about the data itself, or even what it represents (other than potential prediction targets). It only cares about sub-setting.
Does that answer your question? If there is some specific use case you are uncertain of, please also feel free to message me personally on gitter.
---
### Author:  [@mkborregaard](https://discourse.julialang.org/u/mkborregaard)
#### Post date: [April 20, 2017, 1:08pm UTC](https://discourse.julialang.org/t/ann-mldatapattern-jl/3299/4 "2017-04-20T13:08:37Z")
Thanks, I’ll catch you on gitter 🙂
---
### Author:  [@mauro3](https://discourse.julialang.org/u/mauro3)
#### Post date: [April 20, 2017, 1:37pm UTC](https://discourse.julialang.org/t/ann-mldatapattern-jl/3299/5 "2017-04-20T13:37:59Z")
(I can’t find a link to the gitter room. Where is it? Maybe worth linking to it too.)
---
### Author:  [@Evizero](https://discourse.julialang.org/u/Evizero)
#### Post date: [April 20, 2017, 1:42pm UTC](https://discourse.julialang.org/t/ann-mldatapattern-jl/3299/6 "2017-04-20T13:42:41Z")
**Gitter:** [JuliaML/chat - Gitter](https://gitter.im/JuliaML/chat)
We do have a section on [“Getting help”](http://mldatapatternjl.readthedocs.io/en/latest/introduction/gettingstarted.html#getting-help) where it is listed. Maybe I should add it to the readme as well
---
### Author:  [@tim.holy](https://discourse.julialang.org/u/tim.holy)
#### Post date: [April 21, 2017, 1:32pm UTC](https://discourse.julialang.org/t/ann-mldatapattern-jl/3299/7 "2017-04-21T13:32:24Z")
This looks like an elegant approach to easing some very common operations. Nice work!