Wednesday, February 3, 2010

linux:debugging with gdb uses sigtrap internally

SIGTRAP is used as a mechanism for a debugger to be notified when the
process it's debugging hits a breakpoint.

A typical way for something like GDB to use it would be something like
this:

- The user asks gdb to set a breakpoint at a certain address in the
  target process.  gdb uses ptrace to replace the instruction at that
  address with the "int3" instruction, which generates a debug
  exception.  It also uses ptrace to ask that the process be stopped
  when SIGTRAP is raised.
- When the target process hits that address, the exception is
  generated.  The kernel treats this as raising a SIGTRAP signal.  The
  process is stopped and gdb is notified.
- gdb lets the user examine the state of the target process.  When the
  user is ready to continue, gdb replaces the int3 with the instruction
  that had originally been there, and uses ptrace to tell the kernel to
  restart the target process from that instruction.  AFAIK it would also
  normally tell the kernel not to deliver the SIGTRAP signal to the
  process, since by default that would kill it.  
So it would normally be
  irrelevant how you are handling SIGTRAP (SIG_IGN or SIG_DFL or a
  handler) because the target will never know it occurred.

(Discovered this information thanks to bug 31715)

source

No comments: