You need them to seriously neglect their saves for it to even be possible, and in that case you probably can just run them out of con the old fashioned way.

I've been withered hundreds if not thousands of times across many characters over the years, and I don't think I've ever been aged by wither.

P.S. I know you're jonesing for a second run at Daevryn shaman.