get someone's goat

To make someone annoyed or angry: “Gavin may seem unflappable, but I know a way to get his goat.” This expression comes from a tradition in horse racing. Thought to have a calming effect on high-strung thoroughbreds, a goat was placed in the horse's stall on the night before the race. Unscrupulous opponents would then steal the goat in an effort to upset the horse and cause it to lose the race.

Annoy or anger someone, as in By teasing me about that article I wrote, he's trying to get my goat, but I won't let him. The origin of this expression is disputed. H.L. Mencken held it came from using a goat as a calming influence in a racehorse's stall and removing it just before the race, thereby making the horse nervous. However, there is no firm evidence for this origin. [c. 1900]