In the grammar lecture, the sequence you understood was analyzed as a noun referring to the unstated subject of a command, where the subject is assumed to be the person addressed.

When she pointed out that patience pays off, you said it, and everyone laughed in agreement.

You walked past the 'Wet Paint' sign and sat on the bench—are you blind?

Excuse you! You just cut in line without apologizing.

This homemade smoothie is better for you than the store-bought one.

Of all the options, the grilled salmon is best for you.

Oh, stuff you — I don't have to take orders from you.

When I handed her the breakup letter, I wrote it's not you, it's me because I didn't want her to blame herself for our problems.

The word screw-you is not something I want to hear from a coworker.

Thanks for the compliment, back at you.