No, you can't remove this because it initializes the device etc. You can't have all the threads trying to initialize the device.

OK, I am now running everything once w/o threads so it is fine for me, but I added it back in.

Also, I see there are a lot of cached handles in aijcusparse.cu. If they get set w/o threads (thread 0) and then get used later in OMP that could be a problem. My code runs fine and it first runs without threads and then runs with 10 threads/solves using cuSparse ILU solves. So it seems OK.