Personally, the only time I take matches out is when we have more than one book. I think my husband and I are in the minority, but we'd rather have one or two or three matches than no matches.

If we have two or more books, sometimes I'll strip the matches from one (unless it's a feature book, of course).