It's probably oversight. Neither the mirror, a portal, or totems should work while carrying a cabal item.