Just to make sure, all of the "sinkhole" hits are hit-nxdomain.opendns.com. Is this not DNS lookups? -- Craig
-----Original Message----- From: [email protected] [mailto:core-sysadmins- [email protected]] On Behalf Of Gene Rackow Sent: Friday, February 14, 2014 10:46 AM To: [email protected] Cc: Rackow, Eugene A. Subject: Network investigation
We got notice from DOE-JC3 about some odd traffic they are investigating. It's tied to a serious breakin problem elsewhere. While the traffic patterns do not completely line up, it's odd enough to cause concern. This dates back to the beginning of Jan, 2014. It appears the traffic stopped in Jan. I don't see it in Feb.
One of the indicators is that the infected machine pokes at a sinkhole server at 67.215.65.132.
One of the hosts doing this is 140.221.11.215.
What we see is Start End Sif SrcIPaddress SrcP DIf DstIPaddress DstP P Fl Pkts Octets 0106.08:22:51.748 0106.08:22:51.748 6 140.221.11.215 58485 3 130.202.101.12 993 6 4 1 40 0106.08:24:18.281 0106.08:24:18.281 6 131.193.77.254 53 3 140.221.11.215 59737 17 0 2 598 0106.08:24:18.284 0106.08:24:18.284 6 140.221.11.215 59737 3 131.193.77.254 53 17 0 2 144 0106.08:24:18.498 0106.08:25:25.498 6 140.221.11.215 58499 3 67.215.65.132 9036 6 2 22 1280 Sinkhole 0106.08:24:24.679 0106.08:25:31.679 6 140.221.11.215 58500 3 67.215.65.132 9034 6 2 22 1280 Sinkhole 0106.08:24:26.740 0106.08:25:33.740 6 140.221.11.215 58501 3 67.215.65.132 9035 6 2 22 1280 Sinkhole 0106.08:25:32.662 0106.08:26:39.662 6 140.221.11.215 58505 3 67.215.65.132 9036 6 2 22 1280 Sinkhole 0106.08:25:33.520 0106.08:25:33.520 6 131.193.77.254 53 3 140.221.11.215 55881 17 0 2 608 0106.08:25:33.524 0106.08:25:33.524 6 140.221.11.215 55881 3 131.193.77.254 53 17 0 2 154 0106.08:25:39.717 0106.08:25:39.717 6 131.193.77.254 53 3 140.221.11.215 61555 17 0 2 602 0106.08:25:39.720 0106.08:25:39.720 6 140.221.11.215 61555 3 131.193.77.254 53 17 0 2 148 0106.08:25:39.871 0106.08:26:46.871 6 140.221.11.215 58506 3 67.215.65.132 9034 6 2 22 1280 Sinkhole 0106.08:25:41.773 0106.08:25:41.773 6 140.221.11.215 52959 3 131.193.77.254 53 17 0 2 154 0106.08:25:41.775 0106.08:25:41.775 6 131.193.77.254 53 3 140.221.11.215 52959 17 0 2 608 0106.08:25:41.932 0106.08:26:48.932 6 140.221.11.215 58507 3 67.215.65.132 9035 6 2 22 1280 Sinkhole 0106.08:25:58.273 0106.08:25:58.273 6 131.193.77.254 53 3 140.221.11.215 56614 17 0 2 630 0106.08:25:58.283 0106.08:25:58.283 6 140.221.11.215 56614 3 131.193.77.254 53 17 0 2 120 0106.08:26:50.850 0106.08:26:50.850 6 131.193.77.254 53 3 140.221.11.215 55847 17 0 2 598 0106.08:26:50.859 0106.08:26:50.859 6 140.221.11.215 55847 3 131.193.77.254 53 17 0 2 144 0106.08:26:50.951 0106.08:27:57.951 6 140.221.11.215 58508 3 67.215.65.132 9036 6 2 22 1280 Sinkhole 0106.08:26:58.169 0106.08:28:05.169 6 140.221.11.215 58509 3 67.215.65.132 9034 6 2 22 1280 Sinkhole 0106.08:27:00.224 0106.08:28:07.224 6 140.221.11.215 58510 3 67.215.65.132 9035 6 2 22 1280 Sinkhole 0106.08:27:31.898 0106.08:43:19.898 6 140.221.11.215 58511 3 74.125.193.109 993 6 3 329 23194 0106.08:27:31.898 0106.08:43:19.898 6 74.125.193.109 993 3 140.221.11.215 58511 6 3 333 211636
There isn't traffic back from the 67.address. I don't know what it may be attempting to get to on port 9034-9036.
It's also the case we don't have detail on what "domain" it may be attempting to look up as it's doing so from the EVL server. Another question is how the ns3.evl.uic.edu server is configured. Does it use OpenDNS as it's source?
These next 2 are visitor machines on your wired net. I'm not sure who they may be.. ========================== Another machine was 130.202.17.10 On Jan 10. Start End Sif SrcIPaddress SrcP DIf DstIPaddress DstP P Fl Pkts Octets 0110.17:11:17.943 0110.17:11:17.943 6 130.202.17.10 54438 3 208.67.222.222 53 17 0 2 124 0110.17:11:17.951 0110.17:11:17.951 6 208.67.222.222 53 3 130.202.17.10 54438 17 0 2 156 0110.17:11:18.449 0110.17:11:28.449 6 130.202.17.10 737 3 67.215.65.132 111 17 0 20 1680 Sinkhole 0110.17:11:24.049 0110.17:12:30.049 5 130.202.17.10 62924 3 17.172.208.29 443 6 3 34 6251 0110.17:11:24.403 0110.17:11:24.403 6 130.202.17.10 63814 3 208.67.222.222 53 17 0 2 156 0110.17:11:24.407 0110.17:11:24.407 6 208.67.222.222 53 3 130.202.17.10 63814 17 0 2 188 0110.17:11:25.085 0110.17:12:30.085 5 17.172.208.29 443 3 130.202.17.10 62924 6 3 27 15876 0110.17:11:30.066 0110.17:11:39.066 6 130.202.17.10 997 3 67.215.65.132 111 17 0 18 1224 Sinkhole This machine is attempting to do SunRPC to somewhere. Not getting a response on this. The machine IS using openDNS as it's DNS server.
========================== Another machine is on Jan 15, 130.202.17.158 Start End Sif SrcIPaddress SrcP DIf DstIPaddress DstP P Fl Pkts Octets 0115.18:10:54.684 0115.18:10:54.684 6 173.194.46.46 443 3 130.202.17.158 44628 6 4 2 80 0115.18:10:54.684 0115.18:10:54.684 6 173.194.46.46 443 3 130.202.17.158 44628 6 4 2 80 0115.18:10:54.684 0115.18:10:54.684 6 173.194.46.46 443 3 130.202.17.158 44628 6 5 2 92 0115.18:10:55.113 0115.18:10:55.113 6 173.194.46.46 443 3 130.202.17.158 44628 6 4 1 40 0115.18:10:55.115 0115.18:10:55.115 6 130.202.17.158 44628 3 173.194.46.46 443 6 1 1 52 0115.18:10:59.353 0115.18:10:59.353 6 130.202.17.158 36853 3 67.215.65.132 443 6 4 1 40 Sinkhole 0115.18:10:59.359 0115.18:10:59.359 6 67.215.65.132 443 3 130.202.17.158 36853 6 1 2 141 Sinkhole 0115.18:10:59.419 0115.18:10:59.419 6 130.202.17.158 36853 3 67.215.65.132 443 6 3 13 1264 Sinkhole 0115.18:10:59.420 0115.18:10:59.420 6 130.202.17.158 36853 3 67.215.65.132 443 6 4 2 80 Sinkhole 0115.18:10:59.420 0115.18:10:59.420 6 130.202.17.158 36853 3 67.215.65.132 443 6 5 2 92 Sinkhole 0115.18:10:59.420 0115.18:10:59.420 6 67.215.65.132 443 3 130.202.17.158 36853 6 3 16 6949 Sinkhole 0115.18:11:00.058 0115.18:15:20.058 5 130.202.17.158 44091 3 108.160.162.102 80 6 2 18 4768 0115.18:11:00.324 0115.18:11:00.324 5 108.160.162.102 80 3 130.202.17.158 44091 6 2 4 224 0115.18:11:14.033 0115.18:11:14.033 5 146.137.81.40 443 3 130.202.17.158 34060 6 0 2 300
I don't know what caused this traffic. According to DNS logs, it appears the only things it's looking up are tied to daisy.ubuntu.com or onmiture.nvidia.com. This would indicate these hits to the sinkhole are something they had cached locally or whatever is attempting a direct connect. ==========================
--Gene