Assuming you don't want to burn it, or stab it with a hot needle, you could measure the specific gravity and compare to celluloid. Not perfect, but you could certainly rule out a lot of possibilities.

White Cellulose Nitrate should be close to 1.5 g/cm^3. Red CN is a little bit less (white pigment is heavy), say 1.46 for red.

Newer cellulose acetate dice are 1.27-1.28 g/cm^3. So if somebody's selling supposedly old dice, that should be Cellulose Nitrate based on age, but the SG doesn't check out, they must be counterfeit.

If it floats in water, its definitely not celluloid, since water is SG = 1.0g/cm^3.

I'm not sure about other vintage plastics, but you could look them up. I don't think there's such a thing as white bakelite though? Vinyl didn't exist yet.