In judging the role contest, I go purely on the role. While I am more apt to watch someone who has an interesting role (because I want to see the rp in action), it really isn't feasible to do more than that.

By that I mean, I try and run a very tight schedule in terms of the contest. Ideally, I open/announce for one week, and then within three day of closing want to have it decided, and awards given out (so that you wacky kids don't have time to start deleting). If I placed a requirement to supervise all the semi-finalists, I'd probably not be able to sleep/go to work, and in many cases would penalize those who aren't on 6 hours a day since I might not find them before I need my decisions.

That being said, I firmly believe that most people who put effort into a unique and well-written role intend to rp it. So, long story short, just on the role, excluding anything else.