it's certainly possible to argue that allowing the rich to give their offspring advantages is unfair and exacerbates inequality, but the fact remains that these actions don't directly harm anyone else, and it is unfair to restrict one's freedom in the name of systemic effects.
this q is identical to the "is speech violence" one, in that there are people who seek to restrict freedom of speech for reasons of how speech acts can, thru long chains of mediation, cause physical harm. and here too, the correct answer here is that responsibility is laundered.