give a fig

(informal, negative) To care what someone else thinks or does, particularly in relation to what they think or say about the speaker.