They can be hard to get. Try getting one if you are wanted or have a crappy luck stat.

However, I have two experiences where orbs didn't work.

One was in the upper level of high lords keep. I was getting pounded on by that ap mob and held the orb. It didn't transport me. It was just in my hands. I tried multiple times and it wouldn't do anything.

I had the same thing happen in ysigrath when I held it by mistake.

So I don't know how they work but I so know they are not a guaranteed escape.