How to find out the process(es) that forks a lot?

If you run an OS that supports dtrace, this script will help you identifying what processes are launching short lived processes:

#!/usr/sbin/dtrace -qs

proc:::exec
{
  self->parent=stringof((unsigned char*)curpsinfo->pr_psargs);
}

proc:::exec-success
/self->parent != NULL/
{
  printf("%s -> %s\n",self->parent,curpsinfo->pr_psargs);
  self->parent=NULL;
}

If you are on an OS without dtrace support, have a look to alternatives, e.g. systemtap or sysdig with Linux, ProbeView with AIX.

Here is a sysdig script that will show all commands launch and exit times with their pid and ppid:

sysdig -p"*%evt.time %proc.pid %proc.ppid %evt.dir %proc.exeline" \
    "( evt.dir=< and evt.type=execve ) or evt.type=procexit"

Another method would be to enable process accounting with your OS (if available, commonly the acct package under Linux) and have a look to the generated logs. There is also a top like program that leverage process accounting: atop.


Try top -Sd1. This will show the cumulative time of each process instead of just its own. Cumulative here stands for the time a process has consumed, plus the time its children that don't exist any more consumed.

Normally, the process that goes up faster (and probably the one that's already high enough) is the one you're looking for. After that you can strace the process to verify your suspicion.