AdSense invalid traffic warning what to do

I woke up to a scary email from AdSense about invalid traffic on my site. My account is at risk and I am really worried about getting banned. Should I just remove the ad codes for a few days to be safe?