Jamison, I would try a pencil eraser........and some effort.

I don't recommend acetone.......it will do bad things to chips......I soaked a Luxor Chip in acetone for a couple days once and it pretty much bleached it white (This was done to see if you could melt a chip out of a Lucite block without damage to the chip)