Benchmark MATLAB & Julia for Matrix Operations

General Usage Performance
121

This is your Julia code. The result: 970 microSeconds (@benchmark answer), 1550 microSeconds, 1090 microSeconds.
This is for a big matrix with the size of 500. If you decrease the matrix size, the noise will be much more!
In your validation example, you combined three operations like addition, svd, and eigen so the noise gets less apparent.

You are using seconds as the unit and linear plots, and that’s why you don’t notice the noise.


using BenchmarkTools;
using LinearAlgebra;
using Statistics;

function SimpleMatrixOperation( mX , mY )
    mA = (0.2 .* mX) .+ (0.6 .* mY);
end

function MeasureTimeIterations( mX , mY, numIterations )
    vRunTime = zeros(numIterations);

    for ii = 1:numIterations
        vRunTime[ii] = @elapsed begin
            mA = SimpleMatrixOperation(mX , mY);
        end
    end

    runTime = minimum(vRunTime);

    return runTime;

end

function MeasureTimeBelapsed( mX , mY )

    runTime = @belapsed SimpleMatrixOperation(mX , mY);

    return runTime;

end


function MeasurTimeBenchmark(mX , mY)

    runTime = @benchmark SimpleMatrixOperation(mX , mY)

end

function directMeasurement( mX , mY)

    runTime = @elapsed begin

        mA = (0.2 .* mX) .+ (0.6 .* mY);
    end

    return runTime
end

function forDirectMeasurement(mX , mY)

    vRunTime=Array{Float64}(undef,numIterations)
    for ii = 1:numIterations
        vRunTime[ii] = directMeasurement(mX , mY);
    end
    runTime = minimum(vRunTime);
end
const matrixSize = 500;
const numIterations = 7; 

const mX = randn(matrixSize, matrixSize);
const mY = randn(matrixSize, matrixSize);

runTimeA = MeasureTimeIterations(mX , mY, numIterations);
# runTimeB = MeasureTimeIterations(mX , mY, 10 * numIterations);
runTimeD = MeasurTimeBenchmark(mX , mY);
runTimeE = forDirectMeasurement(mX , mY)

# @btime SimpleMatrixOperation($mX , $mY)
@benchmark SimpleMatrixOperation($mX , $mY)

But this is for Julia which has a very good compiler. For Matlab, it is even worse. See the big flactuation in the direct measurement result!


% validation
matrixSize=500;
numIterations=7; % 700

mX = randn(matrixSize, matrixSize);
mY = randn(matrixSize, matrixSize);

%%
mRunTime=zeros(1,numIterations);
for kk = 1:numIterations

        fun=@() SimpleMatrixOperation(mX, mY);
        mRunTime(kk) = timeit(fun)*1e6; % computes median of bench times

end
timeItResult=min(mRunTime);

disp(timeItResult)

%%
mRunTime=zeros(1,numIterations);
for kk = 1:numIterations

        [mA, mRunTime(kk)]=DirectSimpleMatrixOperation( mX , mY );

end
directResult=min(mRunTime);

disp(directResult)
%%
function [mA, runtime]=DirectSimpleMatrixOperation( mX , mY )

tic();
mA = (0.2 * mX) + (0.6 * mY);
runtime=toc();    

end
%%  
function mA=SimpleMatrixOperation( mX , mY )
    mA = (0.2 * mX) + (0.6 * mY);
end

% using Numiterations 7:
% Results for different runs:
>> validation
       642.38
        530.5
>> validation
       648.18
        721.6
>> validation
       647.03
        583.9
% using numIterations 700:
>> validation
        642.2
        457.5
>> validation
       644.03
      437

Also, median is a better statistical measure than minimum, since it shows what happens most of the time.

1 Like