take someone at their word

(transitive) To take someone literally even though they may not have been serious; to take someone seriously even though they were joking; to take up a challenge that was initially meant as a joke.