# SampledVectors.jl - vectors of limited capacity and unlimited logical length

**URL:** <https://discourse.julialang.org/t/sampledvectors-jl-vectors-of-limited-capacity-and-unlimited-logical-length/74702>\
**Category:** Package Announcements\
**Tags:** package, announcement, vector\
**Created:** [January 16, 2022, 1:48pm UTC](https://discourse.julialang.org/t/sampledvectors-jl-vectors-of-limited-capacity-and-unlimited-logical-length/74702 "2022-01-16T13:48:14Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![guoyongzhi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/guoyongzhi/32/28306_2.png) [@guoyongzhi](https://discourse.julialang.org/u/guoyongzhi)\
**Post date:** [January 16, 2022, 1:48pm UTC](https://discourse.julialang.org/t/sampledvectors-jl-vectors-of-limited-capacity-and-unlimited-logical-length/74702/1 "2022-01-16T13:48:15Z")

</div>

> **[GitHub - guo-yong-zhi/SampledVectors.jl: vectors of limited capacity and unlimited logical...](https://github.com/guo-yong-zhi/SampledVectors.jl)**
>
> vectors of limited capacity and unlimited logical length

Hello, everyone. I want to announce a simple and handy package - [**SampledVectors.jl**](https://github.com/guo-yong-zhi/SampledVectors.jl). `SampledVector` is of limited capacity, but has unlimited logical length. If a new element pushed leads to the vector exceeding its maximum capacity, automatical downsampling will be triggered. `SampledVector` can be used to record metrics, such as training loss curve in machine learning.

## Installation

```julia

import Pkg; Pkg.add("SampledVectors")

```

## Sampling and Interpolation

There are 1 exported type, `SampledVectors`, and 3 important methods, `push!`, `sampled` and `sampledindexes`.

```julia

using Plots

y = [cos(x^2/900) for x in 1:100]

plot(y, label="original curve")

using SampledVectors

vector = SampledVector{Float64}(20) #20 is the capacity (maximum number of stored elements)

for yy in y

    push!(vector, yy)

end

# For visualization purposes, `sampled(vector)` would be enough when the vector is set at proper capacity.

# And it runs fast.

plot!(collect(sampledindexes(vector)), sampled(vector), color="gray", label="sampling points")

scatter!(collect(sampledindexes(vector)), sampled(vector), color="gray", label=nothing)

# `collect(vector)` can be seen as an interpolation result, but its length may be very large.

@assert length(collect(vector)) == length(y)

plot!(collect(vector), linestyle=:dash, label="basic interpolation")

# We can also use the package `Interpolations` to get a better result.

using Interpolations

sx = 1:step(vector):length(vector)

sy = sampled(vector)[1:length(sx)] #The last point may be lost

itp_cubic = CubicSplineInterpolation(sx, sy, extrapolation_bc=Line())

plot!(1:100, itp_cubic.(1:100), linestyle=:dash, label="better interpolation")

plot!(legend = :bottomleft)

```

 ![sampling_and_interpolation](https://global.discourse-cdn.com/julialang/original/3X/0/8/089d53ad0bf48c0a3d5526fda4197c9bb5afd814.png)

## Filtering

If the original signal contains high frequency components, an anti-aliasing filter may be required.

```julia

using Plots

y = [cos(x^2/90000)+0.6cos(0.75x) for x in 1:1000]

plot(y, label="original curve")

using SampledVectors

vector = SampledVector{Float64}(200)

for yy in y

    push!(vector, yy)

end

# Aliasing occurs

plot!(collect(sampledindexes(vector)), sampled(vector), color="gray", label="sampled & unfiltered")

scatter!(collect(sampledindexes(vector)), sampled(vector), color="gray", label=nothing)

using DSP

using OnlineStats

# In order to prevent aliasing, the original signal should be removed of its high-frequency components before pushed.

# Here, a moving window is used to implement an online filter.

kernel = digitalfilter(Lowpass(0.1), FIRWindow(hanning(25)))

window = MovingWindow(Float64, length(kernel))

fit!(window, repeat([0.], length(kernel))) #zero padding

vector2 = SampledVector{Float64}(200)

for yy in y

    fit!(window, yy)

    push!(vector2, kernel'value(window))

end

plot!(collect(sampledindexes(vector2)), sampled(vector2), color="orange", label="sampled & online filtered")

scatter!(collect(sampledindexes(vector2)), sampled(vector2), color="orange", label=nothing)

# Let's plot the output of the offline filter for comparison.

plot!(filt(kernel, y), color="red", label="unsampled & filtered")

plot!(legend = :bottomleft)

```

 ![unfiltered_vs_filtered](https://global.discourse-cdn.com/julialang/original/3X/6/f/6f7a831e5ec6a99e97b7ec5374076976df0049c5.png)  
THANKS

---

<div class="post-metadata">

**Author:** ![Datseris](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/datseris/32/13406_2.png) [@Datseris](https://discourse.julialang.org/u/Datseris)\
**Post date:** [January 16, 2022, 1:55pm UTC](https://discourse.julialang.org/t/sampledvectors-jl-vectors-of-limited-capacity-and-unlimited-logical-length/74702/2 "2022-01-16T13:55:23Z")

</div>

> [@guoyongzhi](#):
>
> `SampledVector` is of finite length, but elements can be pushed into it infinitely.

So what’s the difference with good old `CircularBuffer` from `DataStructures`?

---

<div class="post-metadata">

**Author:** ![guoyongzhi](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/guoyongzhi/32/28306_2.png) [@guoyongzhi](https://discourse.julialang.org/u/guoyongzhi)\
**Post date:** [January 16, 2022, 2:07pm UTC](https://discourse.julialang.org/t/sampledvectors-jl-vectors-of-limited-capacity-and-unlimited-logical-length/74702/3 "2022-01-16T14:07:40Z")

</div>

In `CircularBuffer`, new items will overwrite the oldest items. But in `SampledVector`, the items will be sampled uniformly. The overall shape of the curve is preserved, not just the latest segment. In fact, the first pushed point will never be deleted in `SampledVector`.