I considered that, both conditions cause join damage, hence the dexterity loss, but what about movement loss and perhaps chronic joint pain so they have to keep using poppy juice. Perhaps random slowness like effects when the joints become too inflamed.

Also, there are some issues with being that too large for giants, such as unable to enter places and gnomes who are now easy to bash.

Perhaps limit Giant sized races to dwarfism only, and gnome sized classes to giantism only. Everyone else inbetween can take either.

I think it could be balanced to where it is in fact an overall flaw rather than an edge.