They told us that silver badges are okay, but gold badges are bad. I don't know if that's true, and when I worked in a tax office no one else seemed to know anything. We did have a policy, though, of not allowing audits to be conducted at the client's home or place of business - only at the IRS.

It sounds like you followed the best advice: don't talk, and get your lawyer involved.