Elvis' musicians and close buddies were known as the "Memphis Mafia", so when Elvis "left the building", the Mafia left with him.
I guess that's a stretch, huh?