I've never played an AP, but I've died to plenty of them. Excluding the ones with lots of charges, the two most common ways I die are:

1. I'm a combo without bash protection that doesn't match up well against an AP and get bashed down, or

2. I'm not carrying enough gold/potions/pills to survive getting slept and vaulted, and I get slept and vaulted.

At mid-ranks I personally find it pretty difficult to survive being slept by an AP. You're looking at being cursed, blind, poisoned, plagued, bleeding from the vault, and at about half-health after the vault damage + a few rounds of combat. Even if you manage to flee and teleport, you probably die anyway, which still gives the AP "credit" with respect to charges.