Thank you Jed for the hint. 

So just to understand it with an example. Say I have this matrix here, which has 4 3x3 blocks 

1 2 0 | 0 5 0 |
0 2 3 | 0 0 1 |       <---- Proc 1
0 0 1 | 0 2 2 |    
--------|--------|
1 2 0 | 0 5 0 |
0 2 0 | 0 0 1 |       <---- Proc 2
0 0 1 | 0 0 2 |    
-------|---------| 

This can be represented as a collection of submatrices like: 

A  B

C  D

A and D are the diagonal blocks, while B and C are the off-diagonal ones. How should I set d_nnz and o_nnz in this case?