"It's coming out of your tip." is never ever appropriate.
Of course a server's actions and competence can affect their tip. Obviously. But do not, by any means, ever hold that over your servers head. You'll sound like a total dick.
Last night an older man and his date were at the bar and they were rude from minute one. Cut to the end of their night with us, and the man points out that the receipt that he had showed a different total than his credit card was run for. I apologized and then immediately realized that that was the receipt from before the last round of beers. It was printed an hour and ten minutes earlier than he cashed out and the difference was exactly the cost of two beers. I explained that to him and said I would get him a copy of his updated itemized receipt, to which he instantly said, "well it's coming out of your tip then."
What? What is? We did nothing wrong, he was just looking at an outdated reciept. We keep updated reciepts in front of every customer, he happened to hold onto the older one. We did nothing wrong. Douchebag Grouchypants left us no tip on a $78 tab. Fuck you, Mr Grouchypants, and good luck to you if you try to come back to my bar.
Protip: if you pay with a credit card, we will remember your name forever. :)
