I agree with you, Trolls and Ogres (as well as Goliaths Firbolgs etc) are kept Medium for Meta reason, to avoid unbalancing the game.
For the same Meta reason, Goblins could keep a d8 Hit Dice (or a D6+ Tough Skin, whatever) to avoid this permanent advantage that unbalances the game.
But this balance concern should work for all origins.
Yeah. When I first read about Hit Dice linked to your size I thought it sounded interesting, but then when in the end every Origin is medium sized except goblins then they are just the odd one out. I also have concerns considering balance, but my knowledge of probability statistics isn't good enough to evaluate it.
I mean, all classes have a D8 Hit Dice, and almost all Origins have a Hit Dice of D8, so why not just go with class based Hit Dice when it's essentially the same for everyone?