How can I define vectors using struct?

Hello,

I am new to Julia. I want to apply some RL algorithms using POMDPs.jl, but first I need to define my MDP.

The state space are vectors of integer numbers and lenght (let’s suposse 15 but maybe later it will be bigger), so I need to create all the possible combinatios.

The terminal state is any vector which last term is zero (0). I made the following but it does not work).

using POMDPs, POMDPModelTools, QuickPOMDPs, Random
using LinearAlgebra, StaticArrays

#load solver 
using DiscreteValueIteration

#define State data type
mutable struct State 
    arr::MVector{15, Int64}
end

#define State space
null=rand((0:20), 14)
append!(null,0) #Terminal state 

S = [[State(arr) for arr=rand((0:20), 15)]...,null] 

I get:

MethodError: Cannot convert an object of type Int64 to an object of type MVector{15, Int64}

I know null state neither the other states are ok but I do not know how to fix it.

Could you help me?

Thanks!

Do you mean S = State(null) or S = State(rand(0:20, 15)) or do you want to construct a vector of states?

Hello,

It means that is a vector with its first 14 terms ramdom betwen 0 and 20, and the last term (15th) equal to 0.

Thanks!

So S = State((rand(0:20, 14)..., 0))?

By doing that I only get one vector, for example:

State([13, 7, 11, 12, 13, 15, 6, 15, 19, 13, 15, 13, 6, 13, 0])

But the null state is all that vector whose last value is 0, but the first 14 terms can be any, therefore there will be many vectors.

Just to be sure: do you want a vector of State or a State of vectors of MVectors?

I need all the possible vectors with 15 terms and its values going from 0 to 20.
As an example:

State 1([1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
State 2([1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
State 3([1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])

State Last([20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 20, 100])

Considering that al vectors which last term is 0 (as State 1,State 2,State 3…) are considered the termination state.

I guess it is what you mean with a State of vectors of MVectors .

I think you did not grasp the dimension of the problem.

julia> big(20) ^ 15 # minimum size of such vector
32768000000000000000

julia> 128 * big(1024)^3 # probable max memory of your computer
137438953472

The number of all combinations is at least many million times larger than your computer can handle.

1 Like

Nevertheless I tried to find the correct incantation for k - multicombinations in IterTools or Combinatorics and seemingly failed. Any hints? Thanks!

Hello again,

As @Henrique_Becker I reduced the dimension but as @goerch said I still do not know how to do it.

I will explain it better:

I need to create a state space made of many vectors, let’s consider:

([(d,e,f,g,h) for d in 0:10 for e in 0:10 for f in 0:10 for g in 0:10 for h in 0:10])

In this state space, there are several vectors that determine a terminal state, those whose last term is equal to zero:

([(x,y,z,w,0) for x in 0:10 for y in 0:10 for z in 0:10 for w in 0:10])

I have tried:

using POMDPs, POMDPModelTools, QuickPOMDPs, Random
using LinearAlgebra, StaticArrays

#load solver 
using DiscreteValueIteration

#define State data type
mutable struct State 
    a
end

#define State space
null = State([(x,y,z,w,0) for x in 0:10 for y in 0:10 for z in 0:10 for w in 0:10])
S = [[State([(d,e,f,g,h) for d in 0:10 for e in 0:10 for f in 0:10 for g in 0:10 for h in 1:10])...,null]]```

For null I get:

State([(0, 0, 0, 0, 0), (0, 0, 0, 1, 0), (0, 0, 0, 2, 0), (0, 0, 0, 3, 0), (0, 0, 0, 4, 0), (0, 0, 0, 5, 0), (0, 0, 0, 6, 0), (0, 0, 0, 7, 0), (0, 0, 0, 8, 0), (0, 0, 0, 9, 0) … (10, 10, 10, 1, 0), (10, 10, 10, 2, 0), (10, 10, 10, 3, 0), (10, 10, 10, 4, 0), (10, 10, 10, 5, 0), (10, 10, 10, 6, 0), (10, 10, 10, 7, 0), (10, 10, 10, 8, 0), (10, 10, 10, 9, 0), (10, 10, 10, 10, 0)])

For S I get:
ERROR: MethodError: no method matching iterate(::State)

And when i do:

using POMDPs, POMDPModelTools, QuickPOMDPs, Random
using LinearAlgebra, StaticArrays

#load solver 
using DiscreteValueIteration

#define State data type
mutable struct State 
   arr::MVector{5, Int64}
end

#define State space
null = State([(x,y,z,w,0) for x in 0:10 for y in 0:10 for z in 0:10 for w in 0:10])
S = [[State([(d,e,f,g,h) for d in 0:10 for e in 0:10 for f in 0:10 for g in 0:10 for h in 1:10])...,null]]

For null I get:
ERROR: DimensionMismatch("expected input array of length 5, got length 14641")

For S I get:
ERROR: DimensionMismatch("expected input array of length 5, got length 146410")

None of them works because I do not need a “big vector”, I need a group of vectors. I hope this makes the problem clear. Can you help me?

Found it. The following seems to work for me

using StaticArrays
using Combinatorics

#define State data type
mutable struct State 
    arr::MVector{5, Int64}
end

SS = [State((arr...,)) for arr in with_replacement_combinations(1:10, 5)]
1 Like

Hi!

Thank you so much but with that I get all the states together, how can I get the null vectors separately from the non-null vectors?

SS0 = @show [State((arr..., 0)) for arr in with_replacement_combinations(1:10, 4)]
SSn = @show [State((arr...,)) for arr in with_replacement_combinations(1:10, 5) if arr[5] != 0]

?

1 Like

Hello!

It worked, finally I did:

#load packages
using POMDPs, POMDPModelTools, QuickPOMDPs, Random
using LinearAlgebra, StaticArrays, Combinatorics
using CSV, Tables

#load solver 
using DiscreteValueIteration

#define State data type
mutable struct State 
    arr::MVector{5, Int64}
end

#define State space
null = [State((arr..., 0)) for arr in with_replacement_combinations(0:10, 4)]
SS = [State((arr...,)) for arr in with_replacement_combinations(0:10, 5) if arr[5] != 0]
#check State space
CSV.write("FileNameNull.csv",  Tables.table(null), writeheader=false)
CSV.write("FileNameStates.csv",  Tables.table(SS), writeheader=false)

I am sorry but checking have been the code and it’s not complety right.

For null we get:

State([0, 0, 0, 0, 0])
State([0, 0, 0, 1, 0])
State([0, 0, 0, 2, 0])
State([0, 0, 0, 3, 0])
State([0, 0, 0, 4, 0])
State([0, 0, 0, 5, 0])
State([0, 0, 0, 6, 0])
State([0, 0, 0, 7, 0])
State([0, 0, 0, 8, 0])
State([0, 0, 0, 9, 0])
State([0, 0, 0, 10, 0])
State([0, 0, 1, 1, 0])

But never State([0, 0, 1, 0, 0]) or State([0, 1, 0, 0, 0]) and similar for SS.

I have checked the documentation Combinatorics.jl but I did not find the solution.

Do you know how to fix it?

Hm, you lost me. For

using StaticArrays
using Combinatorics

#define State data type
mutable struct State 
    arr::MVector{5, Int64}
end

display([State((arr..., 0)) for arr in with_replacement_combinations(0:10, 4)])

I get

1001-element Vector{State}:
 State([0, 0, 0, 0, 0])
 State([0, 0, 0, 1, 0])
 State([0, 0, 0, 2, 0])
 State([0, 0, 0, 3, 0])
 State([0, 0, 0, 4, 0])
 State([0, 0, 0, 5, 0])
 State([0, 0, 0, 6, 0])
 State([0, 0, 0, 7, 0])
 State([0, 0, 0, 8, 0])
 State([0, 0, 0, 9, 0])
 State([0, 0, 0, 10, 0])
 ⋮
 State([8, 8, 10, 10, 0])
 State([8, 9, 9, 9, 0])
 State([8, 9, 9, 10, 0])
 State([8, 9, 10, 10, 0])
 State([8, 10, 10, 10, 0])
 State([9, 9, 9, 9, 0])
 State([9, 9, 9, 10, 0])
 State([9, 9, 10, 10, 0])
 State([9, 10, 10, 10, 0])
 State([10, 10, 10, 10, 0])

Hello,

With that you never get the State([0, 0, 1, 0, 0]) or State([0, 1, 0, 0, 0]) for example, finally I did:

null =[State(α,β,γ,δ,ζ) for α in 0:5 for β in 0:5 for γ in 0:5 for δ in 0:5 for ζ in 0:0] #terminal states 
non_null =[State(α,β,γ,δ,ζ) for α in 0:5 for β in 0:5 for γ in 0:5 for δ in 0:5 for ζ in 1:5] #non terminal states
SS = vcat(null, non_null) #state space = concatenate null and non_null`
``

Yeah, you are right. The vector should contain 10x more elements. Bug in with_replacement_combinations or me misunderstanding something (wouldn’t be the first time)?

1 Like

You people are after combinations or permutations? These two are different. Combinations does not care about position and four 0 and one 1 will appear a single time, permutations will have the five versions with 1 in each position.

1 Like

Sorry for the confusion, this was my mistake, which I try to correct…

using StaticArrays

#define State data type
mutable struct State 
    arr::MVector{5, Int64}
end

SS0 = @show [State((arr..., 0)) for arr in [Iterators.product(ntuple(_ -> 0:10, 4)...)...]]
SSn = @show [State((arr...,)) for arr in [Iterators.product(ntuple(_ -> 0:10, 5)...)...] if arr[5] != 0]
1 Like