# \[ANN\] DataPipes.jl 0.3.0
**URL:** https://discourse.julialang.org/t/ann-datapipes-jl-0-3-0/60734
**Category:** Package Announcements
**Tags:** data, piping
**Created:** [May 7, 2021, 5:29pm UTC](https://discourse.julialang.org/t/ann-datapipes-jl-0-3-0/60734 "2021-05-07T17:29:26Z")
**Posts on this page:** 8
**Page:** 4
### Author:  [@rafael.guerra](https://discourse.julialang.org/u/rafael.guerra)
#### Post date: [December 13, 2021, 11:03pm UTC](https://discourse.julialang.org/t/ann-datapipes-jl-0-3-0/60734/63 "2021-12-13T23:03:51Z")
> [@Maximilian\_Roos](#):
>
> I see it was created 11 months ago;
See possible [explanation here](https://discourse.julialang.org/t/ann-plutostyles-jl-override-styles-of-pluto-notebooks/64280/26).
---
### Author:  [@Maximilian\_Roos](https://discourse.julialang.org/u/Maximilian_Roos)
#### Post date: [December 13, 2021, 11:04pm UTC](https://discourse.julialang.org/t/ann-datapipes-jl-0-3-0/60734/64 "2021-12-13T23:04:44Z")
Thanks, that makes sense!
@aplavin FWIW I often use the date of the last change to assess if a repo is still being maintained, and skipped this one in favor of others because I thought it wasn’t being changed. Maybe I’m an outlier? If not, it might be worth “rounding” by month / quarter rather than year, or adding a note to the Readme of the date of the latest release, or including the release on GitLab.
---
### Author:  [@aplavin](https://discourse.julialang.org/u/aplavin)
#### Post date: [December 14, 2021, 10:29am UTC](https://discourse.julialang.org/t/ann-datapipes-jl-0-3-0/60734/65 "2021-12-14T10:29:45Z")
Maximilian, indeed the linked explanation is correct. I was also thinking about finer rounding, or just assigning all commit dates to “now” - release times are effectively public anyway. Will probably go with this approach…
Anyway, JuliaHub shows correct release dates at the package page [JuliaHub](https://juliahub.com/ui/Packages/DataPipes/jwfGJ/0.2.1), so maybe I should just link to JuliaHub instead of the repo itself.
---
### Author:  [@aplavin](https://discourse.julialang.org/u/aplavin)
#### Post date: [January 12, 2022, 1:35pm UTC](https://discourse.julialang.org/t/ann-datapipes-jl-0-3-0/60734/66 "2022-01-12T13:35:20Z")
Announcing another update, that should help writing and debugging long pipelines:
## The **`@pDEBUG`** macro
Its intended usage is when your pipeline doesn’t work as expected, possibly throwing an error somewhere:
```julia-auto
julia> @p begin
1:5
map(_ ^ 2)
filter(_ > 3)
only
end
ERROR: ArgumentError: Collection has multiple elements, must contain exactly 1 element
```
Replace (temporarily) `@p` with `@pDEBUG`, and it’ll export all intermediate results till the first error into the `_pipe` variable:
```julia-auto
julia> @pDEBUG begin
1:5
map(_ ^ 2)
filter(_ > 3)
only
end
ERROR: ArgumentError: Collection has multiple elements, must contain exactly 1 element
julia> _pipe
3-element Vector{Any}:
1:5
[1, 4, 9, 16, 25]
[4, 9, 16, 25]
```
`_pipe` is a vector of all intermediate pipe results in their order.
Now, you can clearly see why the error in `only` happened.
Further, all intermediate pipe variables are also exported:
```julia-auto
julia> @pDEBUG begin
1:5
x = map(_ ^ 2)
...
end
julia> x
[1, 4, 9, 16, 25]
```
Note: `@pDEBUG` exports to the **global** scope, so the variables are accessible even if the pipe is inside a function.
---
### Author:  [@tamasgal](https://discourse.julialang.org/u/tamasgal)
#### Post date: [January 12, 2022, 1:52pm UTC](https://discourse.julialang.org/t/ann-datapipes-jl-0-3-0/60734/67 "2022-01-12T13:52:02Z")
Sorry for being off-topic, but consider updating your clock 😃
---
### Author:  [@aplavin](https://discourse.julialang.org/u/aplavin)
#### Post date: [January 12, 2022, 2:03pm UTC](https://discourse.julialang.org/t/ann-datapipes-jl-0-3-0/60734/68 "2022-01-12T14:03:34Z")
That’s my hello from near future (:
What’s a little `floor` vs `ceil` between friends…
---
### Author:  [@aplavin](https://discourse.julialang.org/u/aplavin)
#### Post date: [September 7, 2022, 12:19pm UTC](https://discourse.julialang.org/t/ann-datapipes-jl-0-3-0/60734/69 "2022-09-07T12:19:34Z")
A new version of `DataPipes` is now released: `0.3.0`.
[README](https://gitlab.com/aplavin/DataPipes.jl) and [docs](https://aplavin.github.io/DataPipes.jl/examples/notebook.html) are significantly improved. Unfortunately, the first post in this thread isn’t editable anymore.
**Nothing breaks in the core piping functionality.** The only breaking change in the package is the removal of convenience functions that were previously defined in `DataPipes`: `filtermap`, `mutate`, and a few more.
**Now, `DataPipes` just implements its piping syntax and does nothing else.** There are no dependencies anymore, and the loading time is less than 1 ms. These changes make `DataPipes` itself a no-brainer to include as a dependency, even in very lightweight projects.
Of course, those removed data processing functions are useful by themselves as well. Over time, more and more of such functions gathered in DataPipes, which didn’t really make sense conceptually. I’m making them more general and performant, and plan to release as another package soonish. If you also use them, feel free to stay on `DataPipes@0.2` for now. For now, basically the only difference between versions is removal of those functions in `0.3`.
There are also a couple of minor fixes/improvements in the core piping functionality since the previous announcement here. I haven’t encountered any serious issues for a long time in my pretty heavy usage of `DataPipes`. So, recent changes addressed some remaining corner cases:
- implicit inner pipes (that start with `__`) now work everywhere they make sense, including kwargs
- `@p let ... end` and `@p begin ... end` forms generate corresponding blocks, `let` or `begin`, making variable scoping consistent with plain Julia
- `function (x) ... end` is treated exactly the same as `x -> ...`
- qualified function calls also work without brackets, as in regular pipes: `@p data |> Iterators.flatten`
---
### Author:  [@aplavin](https://discourse.julialang.org/u/aplavin)
#### Post date: [November 23, 2022, 10:39am UTC](https://discourse.julialang.org/t/ann-datapipes-jl-0-3-0/60734/70 "2022-11-23T10:39:37Z")
Let me announce another release of `DataPipes`, `v0.3.5` - already registered in `General`.
The main highlight since the last update is
# Pipe broadcast: `.|>`
Sometimes operations are more natural to write as a `map` call, sometimes as a broadcast. Making this even more convenient, `DataPipes` now supports the broadcasted pipe, `.|>`.
For the regular pipe `|>`, the ` __` placeholder gets replaced with the result of the previous step. Likewise, for the `.|>` broadcasted pipe, `__ ` means a single element of the previous step result.
When this placeholder is not used, `DataPipes` implicitly appends it to the function arguments, same for `|>` and `.|>`.
Some examples:
```julia-auto
julia> @p "1, 2, 3, 4" |> eachmatch(r"(\d)") .|> __.captures[1]
4-element Vector{SubString{String}}:
"1"
"2"
"3"
"4"
```
```julia-auto
julia> @p [[1, 2], [3]] .|> __.+ 1
2-element Vector{Vector{Int64}}:
[2, 3]
[4]
```
```julia-auto
julia> @p 1:10 |> group(_ % 3) .|> map(_ ^ 2)
3-element Dictionaries.Dictionary{Int64, Vector{Int64}}
1 │ [1, 16, 49, 100]
2 │ [4, 25, 64]
0 │ [9, 36, 81]
```
Especially the last case demonstrates that `.|>` is convenient for processing nested datasets in a single line. Alternatives, such as nested maps, are more noisy for simple one-liner pipes.
[Previous page](https://discourse.julialang.org/t/ann-datapipes-jl-0-3-0/60734.md?page=3)