Sometimes silence is all you need