19 views (last 30 days)

Show older comments

I have a large sparse matrix for which I am attempting to assign certain segments (simplified example as per below snippet) to some identity matrices. The assignment method below is extremely inefficient, taking up a large amount of memory and too much time (10sec for example below on my server, but my real dataset is much bigger). Looking at the screenshot from taskmanager, we can see clearly a large spike in memory usage while the operation is being carried out.

For somebody used to dealing with large sparse matrices, there should be a more efficient way of carrying out such an assignment operation. Probably a pretty simple solution but I have little experience with sparse-matrix syntax.

A = sparse(20000,60000);

A(10001:end,20001:50000) = ([speye(10000,10000),-speye(10000,10000),speye(10000,10000)]);

the cyclist
on 2 Apr 2013

Edited: the cyclist
on 2 Apr 2013

That operation took less than 3 milliseconds on my machine.

The resulting array is 960,008 bytes.

Sean de Wolski
on 3 Apr 2013

Hi Mark,

I can run the timings for you on any and all releases when I return to Natick on Friday.

Cedric Wannaz
on 2 Apr 2013

Edited: Cedric Wannaz
on 2 Apr 2013

The structure of sparse matrices in memory makes this kind of indexing operations slower than building the sparse matrix directly using vectors of row/col IDs and values.

I = [10001:20000, 10001:20000, 10001:20000] ;

J = [20001:50000] ;

V = [ones(1, 1e4), -ones(1, 1e4), ones(1, 1e4)] ;

S = sparse(I, J, V, 2e4, 6e4) ;

spy(S) ; % Check structure.

You can easily make it more flexible/concise. I can't test it now though.

Cedric Wannaz
on 3 Apr 2013

Teja Muppirala
on 3 Apr 2013

The performance of sparse matrix indexing was enhanced in R2011a.

If you can't upgrade, as somewhat of a workaround, you should be able to get away with something like this:

A = sparse(20000,60000);

A = spreplace(A,10001:20000,20001:50000,[speye(10000,10000),-speye(10000,10000),speye(10000,10000)]);

Where SPREPLACE is the following general purpose function:

function A = spreplace(A,I,J,B)

% Equivalent to

% >> A(I,J) = B

% But does not support "end" indexing in I and J

I = I(:);

J = J(:);

[iA,jA,sA] = find(A);

[iB,jB,sB] = find(B);

trimA = ~(ismember(iA,I) & ismember(jA,J));

A = sparse([iA(trimA); I(iB)],...

[jA(trimA); J(jB)],...

[sA(trimA); sB],...

size(A,1),size(A,2));

When I try this in R2007a, Your original code takes 15 seconds, The workaround above gets it done in about 5 milliseconds.

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!