I've done what you said, and imagined it.
No way in the world it's going to break my neck. Because, I'm moving too. And moving in the exact same direction, as the horse behind me. And I dare say, and hope, that it would be a damn sight quicker than 20k's an hour!
Now time for others to imagine something.
Imagine if the horse being punched, broke, went into a mad gallop, crashed, and brought half the field down. There have been multiple incidents on race tracks in the last 6 months where horses have hit the deck, causing horses in behind to fall, with jockeys going to ground and getting instantly killed. Helmets and vests didn't save them, and they were taken far, far too soon from their friends, family and loved ones.
That is just one reason, and possible scenario, as to why we can't have people punching horses during the running of races where they can be going in excess of 50km per hour, and as you've rightly pointed out, it's 500kg of meat, combined with bits of metal, it may not end well. And that's an understatement.