I think the punching analogy is a good and simple one to use. It’s the grownup version of “pick on someone your own size.” It’s a very privileged position to feel you can punch in all directions - it can be very risky for someone low down on the pyramid to offend someone above them. Meanwhile, it’s oppressive, cruel, and just plain rude for the larger child to beat up the smaller one.
The offensiveness of a joke or statement is less the content and more who’s aiming it at who. For one Black person to drop an N-bomb on another one is between them to sort out. For a white person, it’s unacceptable regardless.
Context is - as always - everything.