Being weary doesn't mean, you're out of combat, it just means that 1s, 2s, and 3s on the d6 don't count. And if you rolled a lot of them during an attack, chances are high, that you'd miss, even if you're not weary. On the other hand, two piercing blows and you are dead. So I'd say, better weary and still able to fight, than dead.
This ^^^
The weary effect on the d6 is one of my favourite rules in the One Ring. Being in a fight is a tiring affair, doing it in a coat of mail even more so, but a skilled warrior will still be landing blows even when close to exhaustion - ie has a lot more d6 to roll.
given that you basically have free choice to start your character with any kind of armor you wish, you almost have to make weariness a close thing for maxed out armor to come into effect in a somewhat balanced fashion. Of course it may depend on how prolonged your combat sessions are, if that feels to strongly or not.
But I generally agree on that is a nice mechanic and it should have an effective place for heavily armored people.
On the topic of "donning armor", a thing I probably would rule is, that putting on armor takes time, which would at least matter if your are being surprised with combat.
I probably would use something that scales along the protection value of your main armor (not counting shields or helms).
Let's say it takes the protection value in actions (for simplicity counting main,secondary and opening volley rounds the same), one of those for each value of protection and your armor only counts once it is completed. You can stretch it out over more turns doing some actions in between or doing it all up front. That would mean a 1d armor could count after a volley at the beginning of the first close combat round, while the minimum time for a Coat of Mail would be 4 "actions", lets say a standard opening volley and and 1 and a half turns.
If you have time out of the combat sequence to prep, all that wouldn't come in to play at all. Of course forgoing the armor to fight directly is always an option.
Also I might consider a 1d Leather(under-)shirt as part of all armor so that you could put on the first bit (or still have it on) and bail on the rest for this combat instance.
Of course you may ask, why would I ever be caught out of armor? Well,
- given that pack horses are a thing, I would say packing (parts of) you armor for lighter travel on your beast would work, and might be at least a good RP reason.
- and I wouldn't give a full rest restoration sleeping in full armor, since Strength already plays into resting, I would say that if your main armor load value exceeds your Strength score then apply the prolonged rest rules for wounded - which would mean depending on your strength the first two armor levels might be good for restful sleep.