Auditd notes
Question
my machine is continously making udp dns traffic request. what i need to know is the PID of the process generating this traffic.
The normal way in TCP connection is to use netstat/lsof and get the process associated at the pid.
Is UDP the connection is stateles, so, when i call netastat/lsof i can see it only if the UDP socket is opened and it's sending traffic.
I have tried with lsof -i UDP and with nestat -anpue but i cant be able to find wich process is doing that request because i need to call lsof/netstat exactly when the udp traffic is sended, if i call lsof/netstat before/after the udp datagram is sended is impossible to view the opened UDP socket.
call netstat/lsof exactly when 3/4 udp packet is sended is IMPOSSIBLE.
how i can identify the infamous process ? I have already inspected the traffic to try to identify the sended PID from the content of the packet, but is not possible to identify it from the contect of the traffic.
anyone can help me ?
I'm root on this machine FEDORA 12 Linux noise.company.lan 2.6.32.16-141.fc12.x86_64 #1 SMP Wed Jul 7 04:49:59 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux
Answer
Linux auditing can help. It will at least locate users and processes making datagram network connections. UDP packets are datagrams.
First, install the auditd framework on your platform and ensure that auditctl -l returns something, even if it says that no rules are defined.
Then, add a rule to watch the system call socket() and tag it for easy finding later (-k). I need to assume that you are on a 64-bit architecture, but you can substitute b32 in place of the b64 if you aren't.
auditctl -a exit,always -F arch=b64 -F a0=2 -F a1=2 -S socket -k SOCKET
You have to pick through man pages and header files to build this, but what it captures is essentially this system call: socket(PF_INET, SOCK_DGRAM, X) where the 3rd parameter is unspecified, but frequently zero. PF_INET is 2 and SOCK_DGRAM is 2. TCP connections would use SOCK_STREAM which would set a1=1. The -k SOCKET is our keyword we want to use when searching audit trails later. It can be anything, but I like to keep it simple.
Let a few moments go by and review the audit trails. Optionally, you could force a couple of packets by pinging a host out on the net, which will cause a DNS lookup to occur, which uses UDP, which should trip our audit alert.
ausearch -i -ts today -k SOCKET
And output similar to the section below will appear. I'm abbreviating it to highlight the important parts
type=SYSCALL ... arch=x86_64 syscall=socket success=yes exit=1 a0=2 a1=2 ... pid=14510 ... auid=zlagtime uid=zlagtime ... euid=zlagtime ... comm=ping exe=/usr/bin/ping key=SOCKET
In the above output, we can see that the ping command caused the socket to be opened. I could then run strace -p 14510 on the process, if it was still running. The ppid (parent process ID) is also listed in case it is a script that spawns the problem child a lot.
Now, if you have a lot of UDP traffic, this isn't going to be good enough and you'll have to resort to OProfile or SystemTap, both of which are currently beyond my expertise.
This should help narrow things down in the general case.
When you are done, remove the audit rule by using the same line you used to create it, only substitute -a with -d.
auditctl -d exit,always -F arch=b64 -F a0=2 -F a1=2 -S socket -k SOCKET
