You probably travelled from Venice Mestre station. That is on the mainland. Maybe half an hour taxi from there.
Venice Mestre to the island via the causeway can take anything from 5 minutes to an hour with traffic.
The bus is a much better option. 2-1/2 euros flat fare anywhere and as quick as the taxi.