Erasing after drawing with CGContext

I also needed erasing functionality. Based on @Jeremy's answer, here is what worked for me:

CGContextRef cgref = UIGraphicsGetCurrentContext();

if(erase == TRUE) // Erase to show background
{
    CGContextSetBlendMode(cgref, kCGBlendModeClear);
}
else // Draw with color
{
    CGContextSetBlendMode(cgref, kCGBlendModeNormal);
}

Display the user's drawing in a layer above the image. Then erasing is as simple as drawing a transparent patch on the drawing layer in order to let the image pixels below it show through.