1. Some portals may not be magical. If you consider all the crap needed to make portals at the Planar Sanctum, it's possible that some portals are just naturally occurring phenomena. Portals may be part of Thera's natural world.

2. Crossing though a portal does not necessarily mean coming in contact with magic. I doubt most villagers are licking the edges of the portal, they're probably just holding their breath and jumping through.

3. Portals are innoccuous enough that their use is justified. I'm pretty sure that if you told an Amish guy that unless he turned on the electric fence a child-murderer would surely escape you'd see him flip the switch with righteous quickness.

4. Funstick. You can't get in/out of certain areas without portals.