You could always take a large piece of paper and then easily fold it so it completely covers all of the old mailer, and then tape it. (You could use the blank side of the post office's large Express Mail or Priority Mail plastic-paper mailers, even used ones, for the cover-up paper.)
Robert