You had a chance of getting away. You just didn't, this time. A chance doesn't mean a 100% chance. :P

Generally speaking, I think people who have played RBW enough to have a very keen understanding of their limitations and weak points can fight them as an A-P, and yes, even without barrier. This is a kind of instinctive understanding that I don't think anyone trying to explain it to you or theorycrafting can ever really provide.

As a midlevel A-P, RBWs are both your most reliable source of charge-worthy kills and the most dangerous thing to you. It can be wise to avoid them but it can also be the quickest way to build up some steam.