I thought of that too, hence the reason it would be limited to things that require TRUST. The last thing I want is for Villagers to get emetic'd for some jackass level 1 mage casting "invis" on them. However, can you think of a reason why a Villager would need to walk around with TRUSTALL up all the time? I can't.

EDIT: Also, just so you know (although I'm sure you probably already do), TRUSTALL is off by default. So you'd have to specifically turn it on for people to abuse this to ruin your day, and as a Villager, it seems like a damn rare situation (how often is your dude really visiting a shrine of the gods to let someone commune on them?) Rare enough, at least, that you'd probably remember to shut it off.