I recall fondly the days when Arbiters would referee in-town-duels. There was nothing better than beating someones ass in market square while people are cheering for you and stuff. There could be a mob in town that you could make challanges to, and he would inform the challangee and ask if he/she is willing to accept or something. Have a few options as far as dueling conditions, results, etc.

Abthalok