If the wits portion does eventually happen, I'd suggest that the questions be made a little more difficult than the last wits contest that took place in the Inn. It ended up being a contest of who could type the fastest, which is sort of silly. With each question, probably 3/4 of the people knew the answer almost immediately, so it was just a race to see who could get it out first.

If you get away from trivia into something more "riddle-ish", make sure the riddles aren't well-known enough that people can just google the answers.