Hi Darius,
As you know Nemesis has LMT mechanism that has very useful "cookie" feature, but we have some difficulties to use it.
First of all when we send LMT RTS/CTS/COOKIE packet it may be enqueued by MPIDI_CH3_iStartMsgv routine, but this routine stores only pointer to data (in our case pointer to cookie) and if cookie is on the stack it will be invalid when we return from our routine. Thus packet with invalid cookie data may be sent.
As I can see LMT SHM and VMsplice (mpid_nem_lmt_shm.c, mpid_nem_lmt_vmsplice.c) modules are not affected, because they store their cookies in global structures, only LMT DMA module (mpid_nem_lmt_dma.c) is affected.
In our case the issue appeared under Windows 2008 with our internal LMT module on "isndrcv" test from MPICH1 package.
As workaround, in our LMT module I implemented mechanism that stores cookies while they are not completely sent (for example while responding packet is not arrived)
Probably this is not a bug and "cookies" must not be stored on the stack.
Also MPID_nem_lmt_send_COOKIE routine can handle only MPIDI_REQUEST_TYPE_RECV and MPIDI_REQUEST_TYPE_SEND requests, but for example HPL application can also create MPIDI_REQUEST_TYPE_SSEND requests that cause internal error.
Regards,
Alexey
--------------------------------------------------------------------
Closed Joint Stock Company Intel A/O
Registered legal address: Krylatsky Hills Business Park,
17 Krylatskaya Str., Bldg 4, Moscow 121614,
Russian Federation
This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
