Updating a vector inside a for loop crashes the script

General Usage
1

I converted this .m file into Julia. When a vector gets updated inside a for loop and Julia crashes with this (incorrect) error message:

ERROR: LoadError: MethodError: no method matching diagm(::Array{Float64,2})
Closest candidates are:
diagm(::Union{BitArray{1},BitArray{2}}) at linalg/bitarray.jl:90
diagm{Tv,Ti}(::SparseMatrixCSC{Tv,Ti}) at sparse/sparsematrix.jl:3571
diagm{T}(::AbstractArray{T,1}) at linalg/dense.jl:121
…
in macro expansion; at /home/dean/src/ml/logistic.regression/ng/ex.04/ex4.jl:27 [inlined]
in anonymous at ./:?
in include_from_node1(::String) at ./loading.jl:488
while loading /home/dean/src/ml/logistic.regression/ng/ex.04/ex4.jl, in expression starting on line 17

The line that causes the problem is

theta = theta - H\grad

If I comment this line out the code runs. If I remove the loop and just execute the code in the loop once it runs. It’s not a problem with the diagm() function

I’m using Julia 0.5.1.

Have I done something wrong, or is this a bug? Here’s the code and data files:

x = readdlm(“ex4x.dat”)
y = readdlm(“ex4y.dat”)
xWithInt = hcat(ones(80), x)
nrows,ncols = size(xWithInt)
theta = zeros(ncols)
g(z) = 1.0 ./ (1.0 + exp(-z))

MAX_ITR = 7
J = zeros(MAX_ITR)

for i = 1:MAX_ITR

hypothesis

z = xWithInt * theta
h = g(z)

grad = (1/nrows) .* xWithInt’ * (h-y)
H = (1/nrows) .* xWithInt’ * diagm(h) * diagm(1-h) * xWithInt

Calculate J (for testing convergence)

J[i] =(1/nrows)sum(-y . log(h) - (1-y) .* log(1-h))
#J[1] =(1/nrows)sum(-y . log(h) - (1-y) .* log(1-h))

backslash operator is left division

theta = theta - H\grad

end

ex4x.dat

5.5500000e+01 6.9500000e+01
4.1000000e+01 8.1500000e+01
5.3500000e+01 8.6000000e+01
4.6000000e+01 8.4000000e+01
4.1000000e+01 7.3500000e+01
5.1500000e+01 6.9000000e+01
5.1000000e+01 6.2500000e+01
4.2000000e+01 7.5000000e+01
5.3500000e+01 8.3000000e+01
5.7500000e+01 7.1000000e+01
4.2500000e+01 7.2500000e+01
4.1000000e+01 8.0000000e+01
4.6000000e+01 8.2000000e+01
4.6000000e+01 6.0500000e+01
4.9500000e+01 7.6000000e+01
4.1000000e+01 7.6000000e+01
4.8500000e+01 7.2500000e+01
5.1500000e+01 8.2500000e+01
4.4500000e+01 7.0500000e+01
4.4000000e+01 6.6000000e+01
3.3000000e+01 7.6500000e+01
3.3500000e+01 7.8500000e+01
3.1500000e+01 7.2000000e+01
3.3000000e+01 8.1500000e+01
4.2000000e+01 5.9500000e+01
3.0000000e+01 6.4000000e+01
6.1000000e+01 4.5000000e+01
4.9000000e+01 7.9000000e+01
2.6500000e+01 6.4500000e+01
3.4000000e+01 7.1500000e+01
4.2000000e+01 8.3500000e+01
2.9500000e+01 7.4500000e+01
3.9500000e+01 7.0000000e+01
5.1500000e+01 6.6000000e+01
4.1500000e+01 7.1500000e+01
4.2500000e+01 7.9500000e+01
3.5000000e+01 5.9500000e+01
3.8500000e+01 7.3500000e+01
3.2000000e+01 8.1500000e+01
4.6000000e+01 6.0500000e+01
3.6500000e+01 5.3000000e+01
3.6500000e+01 5.3500000e+01
2.4000000e+01 6.0500000e+01
1.9000000e+01 5.7500000e+01> 5.5500000e+01 6.9500000e+01
4.1000000e+01 8.1500000e+01
5.3500000e+01 8.6000000e+01
4.6000000e+01 8.4000000e+01
4.1000000e+01 7.3500000e+01
5.1500000e+01 6.9000000e+01
5.1000000e+01 6.2500000e+01
4.2000000e+01 7.5000000e+01
5.3500000e+01 8.3000000e+01
5.7500000e+01 7.1000000e+01
4.2500000e+01 7.2500000e+01
4.1000000e+01 8.0000000e+01
4.6000000e+01 8.2000000e+01
4.6000000e+01 6.0500000e+01
4.9500000e+01 7.6000000e+01
4.1000000e+01 7.6000000e+01
4.8500000e+01 7.2500000e+01
5.1500000e+01 8.2500000e+01
4.4500000e+01 7.0500000e+01
4.4000000e+01 6.6000000e+01
3.3000000e+01 7.6500000e+01
3.3500000e+01 7.8500000e+01
3.1500000e+01 7.2000000e+01
3.3000000e+01 8.1500000e+01
4.2000000e+01 5.9500000e+01
3.0000000e+01 6.4000000e+01
6.1000000e+01 4.5000000e+01
4.9000000e+01 7.9000000e+01
2.6500000e+01 6.4500000e+01
3.4000000e+01 7.1500000e+01
4.2000000e+01 8.3500000e+01
2.9500000e+01 7.4500000e+01
3.9500000e+01 7.0000000e+01
5.1500000e+01 6.6000000e+01
4.1500000e+01 7.1500000e+01
4.2500000e+01 7.9500000e+01
3.5000000e+01 5.9500000e+01
3.8500000e+01 7.3500000e+01
3.2000000e+01 8.1500000e+01
4.6000000e+01 6.0500000e+01
3.6500000e+01 5.3000000e+01
3.6500000e+01 5.3500000e+01
2.4000000e+01 6.0500000e+01
1.9000000e+01 5.7500000e+01
3.4500000e+01 6.0000000e+01
3.7500000e+01 6.4500000e+01
3.5500000e+01 5.1000000e+01
3.7000000e+01 5.0500000e+01
2.1500000e+01 4.2000000e+01
3.5500000e+01 5.8500000e+01
2.6500000e+01 6.8500000e+01
2.6500000e+01 5.5500000e+01
1.8500000e+01 6.7000000e+01
4.0000000e+01 6.7000000e+01
3.2500000e+01 7.1500000e+01
3.9000000e+01 7.1500000e+01
4.3000000e+01 5.5500000e+01
2.2000000e+01 5.4000000e+01
3.6000000e+01 6.2500000e+01
3.1000000e+01 5.5500000e+01
3.8500000e+01 7.6000000e+01
4.0000000e+01 7.5000000e+01
3.7500000e+01 6.3000000e+01
2.4500000e+01 5.8000000e+01
3.0000000e+01 6.7000000e+01
3.3000000e+01 5.6000000e+01
5.6500000e+01 6.1000000e+01
4.1000000e+01 5.7000000e+01
4.9500000e+01 6.3000000e+01
3.4500000e+01 7.2500000e+01
3.2500000e+01 6.9000000e+01
3.6000000e+01 7.3000000e+01
2.7000000e+01 5.3500000e+01
4.1000000e+01 6.3500000e+01
2.9500000e+01 5.2500000e+01
2.0000000e+01 6.5500000e+01
3.8000000e+01 6.5000000e+01
1.8500000e+01 7.4500000e+01
1.6000000e+01 7.2500000e+01
3.3500000e+01 6.8000000e+01
3.4500000e+01 6.0000000e+01
3.7500000e+01 6.4500000e+01
3.5500000e+01 5.1000000e+01
3.7000000e+01 5.0500000e+01
2.1500000e+01 4.2000000e+01
3.5500000e+01 5.8500000e+01
2.6500000e+01 6.8500000e+01
2.6500000e+01 5.5500000e+01
1.8500000e+01 6.7000000e+01
4.0000000e+01 6.7000000e+01
3.2500000e+01 7.1500000e+01
3.9000000e+01 7.1500000e+01
4.3000000e+01 5.5500000e+01
2.2000000e+01 5.4000000e+01
3.6000000e+01 6.2500000e+01
3.1000000e+01 5.5500000e+01
3.8500000e+01 7.6000000e+01
4.0000000e+01 7.5000000e+01
3.7500000e+01 6.3000000e+01
2.4500000e+01 5.8000000e+01
3.0000000e+01 6.7000000e+01
3.3000000e+01 5.6000000e+01
5.6500000e+01 6.1000000e+01
4.1000000e+01 5.7000000e+01
4.9500000e+01 6.3000000e+01
3.4500000e+01 7.2500000e+01
3.2500000e+01 6.9000000e+01
3.6000000e+01 7.3000000e+01
2.7000000e+01 5.3500000e+01
4.1000000e+01 6.3500000e+01
2.9500000e+01 5.2500000e+01
2.0000000e+01 6.5500000e+01
3.8000000e+01 6.5000000e+01
1.8500000e+01 7.4500000e+01
1.6000000e+01 7.2500000e+01> 5.5500000e+01 6.9500000e+01
4.1000000e+01 8.1500000e+01
5.3500000e+01 8.6000000e+01
4.6000000e+01 8.4000000e+01
4.1000000e+01 7.3500000e+01
5.1500000e+01 6.9000000e+01
5.1000000e+01 6.2500000e+01
4.2000000e+01 7.5000000e+01
5.3500000e+01 8.3000000e+01
5.7500000e+01 7.1000000e+01
4.2500000e+01 7.2500000e+01
4.1000000e+01 8.0000000e+01
4.6000000e+01 8.2000000e+01
4.6000000e+01 6.0500000e+01
4.9500000e+01 7.6000000e+01
4.1000000e+01 7.6000000e+01
4.8500000e+01 7.2500000e+01
5.1500000e+01 8.2500000e+01
4.4500000e+01 7.0500000e+01
4.4000000e+01 6.6000000e+01
3.3000000e+01 7.6500000e+01
3.3500000e+01 7.8500000e+01
3.1500000e+01 7.2000000e+01
3.3000000e+01 8.1500000e+01
4.2000000e+01 5.9500000e+01
3.0000000e+01 6.4000000e+01
6.1000000e+01 4.5000000e+01
4.9000000e+01 7.9000000e+01
2.6500000e+01 6.4500000e+01
3.4000000e+01 7.1500000e+01
4.2000000e+01 8.3500000e+01
2.9500000e+01 7.4500000e+01
3.9500000e+01 7.0000000e+01
5.1500000e+01 6.6000000e+01
4.1500000e+01 7.1500000e+01
4.2500000e+01 7.9500000e+01
3.5000000e+01 5.9500000e+01
3.8500000e+01 7.3500000e+01
3.2000000e+01 8.1500000e+01
4.6000000e+01 6.0500000e+01
3.6500000e+01 5.3000000e+01
3.6500000e+01 5.3500000e+01
2.4000000e+01 6.0500000e+01
1.9000000e+01 5.7500000e+01
3.4500000e+01 6.0000000e+01
3.7500000e+01 6.4500000e+01
3.5500000e+01 5.1000000e+01
3.7000000e+01 5.0500000e+01
2.1500000e+01 4.2000000e+01
3.5500000e+01 5.8500000e+01
2.6500000e+01 6.8500000e+01
2.6500000e+01 5.5500000e+01
1.8500000e+01 6.7000000e+01
4.0000000e+01 6.7000000e+01
3.2500000e+01 7.1500000e+01
3.9000000e+01 7.1500000e+01
4.3000000e+01 5.5500000e+01
2.2000000e+01 5.4000000e+01
3.6000000e+01 6.2500000e+01
3.1000000e+01 5.5500000e+01
3.8500000e+01 7.6000000e+01
4.0000000e+01 7.5000000e+01
3.7500000e+01 6.3000000e+01
2.4500000e+01 5.8000000e+01
3.0000000e+01 6.7000000e+01
3.3000000e+01 5.6000000e+01
5.6500000e+01 6.1000000e+01
4.1000000e+01 5.7000000e+01
4.9500000e+01 6.3000000e+01
3.4500000e+01 7.2500000e+01
3.2500000e+01 6.9000000e+01
3.6000000e+01 7.3000000e+01
2.7000000e+01 5.3500000e+01
4.1000000e+01 6.3500000e+01
2.9500000e+01 5.2500000e+01
2.0000000e+01 6.5500000e+01
3.8000000e+01 6.5000000e+01
1.8500000e+01 7.4500000e+01
1.6000000e+01 7.2500000e+01
3.3500000e+01 6.8000000e+01
3.3500000e+01 6.8000000e+01

ex4y.dat

1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
1.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00
0.0000000e+00

2

I haven’t gone through your code in detail, but I think the error message is telling you that at some point a variable that you think is a vector has become a matrix. You should double check the dimensions of each of your variables to make sure it matches your expectation.

1 Like
3

You’re correct. My problem is the line

y = readdlm(“ex4y.dat”)

readdlm creates a 3x1 Array of 80 elements instead of an 80-element array (vector). This coerces other vectors into being 3 x 1 arrays instead of vectors. The fix is to use

y = vec(readdlm(“ex4y.dat”))

which forces y to be a vector instead of a 3 x 1 Array.

The difference between a 3x1 matrix and a vector is subtle but important. Do you know if it is documented anywhere?

Thanks for your help.

4

The documentation for readdlm is here: https://docs.julialang.org/en/stable/stdlib/io-network/#Base.readdlm

I think it could definitely be made more clear that readdlm always returns a Matrix even when there is only one column. If you’d like to improve the documentation, the relevant file to change is:

https://github.com/JuliaLang/julia/blob/5482f911c3d2f8c0dfd7102932cfece31fc4fd24/base/datafmt.jl#L41-L113

I think a pull request that improves the wording here would be well received.