# Tanslate code from PooledDataArray to CategoricalArray

**URL:** <https://discourse.julialang.org/t/tanslate-code-from-pooleddataarray-to-categoricalarray/8646>\
**Category:** General Usage\
**Created:** [January 27, 2018, 10:56pm UTC](https://discourse.julialang.org/t/tanslate-code-from-pooleddataarray-to-categoricalarray/8646 "2018-01-27T22:56:34Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![matthieu](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@matthieu](https://discourse.julialang.org/u/matthieu)\
**Post date:** [January 27, 2018, 10:56pm UTC](https://discourse.julialang.org/t/tanslate-code-from-pooleddataarray-to-categoricalarray/8646/1 "2018-01-27T22:56:34Z")

</div>

I have this code in [FixedEffectModels](https://github.com/matthieugomez/FixedEffectModels.jl/blob/master/src/utils/group.jl) where a function takes a Dataframe as argument and returns a PooledDataArray. The goal of this function is to create a vector that indexes groups defined by different combination of vector values. For observations where one of the value was missing, the corresponding vector has a missing value. So for instance, we have

```julia
group([1, 2, 3], [1, 2, 3]) = [1, 2, 3]
group([4, 5, 6], [7, 8, 9]) = [1, 2, 3]
group([4, 5, 6], [NA, 8, 9]) = [NA, 1, 2]

```

I don’t know how to translate the code to CategoricalArray. Could someone help me to translate it?

```julia

function group(x::AbstractVector) 
	v = PooledDataArray(x)
	PooledDataArray(RefArray(v.refs), collect(1:length(v.pool)))
end

function pool_combine!(x::Array{UInt64, T}, dv::PooledDataVector, ngroups::Integer) where {T}
	@inbounds for i in 1:length(x)
	    # if previous one is NA or this one is NA, set to NA
	    x[i] = (dv.refs[i] == 0 || x[i] == zero(UInt64)) ? zero(UInt64) : x[i] + (dv.refs[i] - 1) * ngroups
	end
	return(x, ngroups * length(dv.pool))
end

function group(df::AbstractDataFrame) 
	isempty(df) && throw("df is empty")
	ncols = size(df, 2)
	v = df[1]
	ncols = size(df, 2)
	ncols == 1 && return(group(v))
	if typeof(v) <: PooledDataVector
		x = convert(Array{UInt64}, v.refs)
	else
		v = PooledDataArray(v, v.na, UInt64)
		x = v.refs
	end
	ngroups = length(v.pool)
	for j = 2:ncols
		v = PooledDataArray(df[j])
		(x, ngroups) = pool_combine!(x, v, ngroups)
	end
	return(factorize!(x))
end

function reftype(sz) 
	sz <= typemax(UInt8) ? UInt8 :
	sz <= typemax(UInt16) ? UInt16 :
	sz <= typemax(UInt32) ? UInt32 :
	UInt64
end

function factorize!(refs::Array)
	uu = unique(refs)
	sort!(uu)
	has_na = uu[1] == 0
	T = reftype(length(uu)-has_na)
	dict = Dict{eltype(refs), T}(zip(uu, (1-has_na):convert(T, length(uu)-has_na)))
	@inbounds @simd for i in 1:length(refs)
		 refs[i] = dict[refs[i]]
	end
	PooledDataArray(RefArray(refs), collect(1:(length(uu)-has_na)))
end

```

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [January 28, 2018, 9:58am UTC](https://discourse.julialang.org/t/tanslate-code-from-pooleddataarray-to-categoricalarray/8646/2 "2018-01-28T09:58:39Z")

</div>

Have you faced any particular difficulties?

In `group`, you could replace the `PooledDataArray` constructor call with:

```julia
CategoricalArray{Int,1}(v.refs, CategoricalPool(collect(1:length(levels(v)))))

```

Then the main differences are that you should use `length(levels(dv))` instead of `length(dv.pool)`, and `CategoricalArrays.index(dv.pool)[dv.refs[i]]` instead of `dv.refs[i]`.

---

<div class="post-metadata">

**Author:** ![matthieu](https://avatars.discourse-cdn.com/v4/letter/m/da6949/32.png) [@matthieu](https://discourse.julialang.org/u/matthieu)\
**Post date:** [January 29, 2018, 3:38pm UTC](https://discourse.julialang.org/t/tanslate-code-from-pooleddataarray-to-categoricalarray/8646/3 "2018-01-29T15:38:06Z")

</div>

Thanks a lot. Could you succintly give me the reasons for the last two changes? I have a hard time understanding the source code of CategoricalArray.

---

<div class="post-metadata">

**Author:** ![nalimilan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nalimilan/32/147_2.png) [@nalimilan](https://discourse.julialang.org/u/nalimilan)\
**Post date:** [January 29, 2018, 4:21pm UTC](https://discourse.julialang.org/t/tanslate-code-from-pooleddataarray-to-categoricalarray/8646/4 "2018-01-29T16:21:54Z")

</div>

That’s because values in `dv.refs` do not follow the ordering of the levels, they just follow the order in which each level has been created originally. That allows reordering the levels without adapting the references.