Going from young to mature often goes unnoticed since it only raises stats which at that time are often already maxed anyways, so it isn't surprising you didn't notice it until you became middle-aged.

I don't know if you die if aging drops your con to three.