I got attacked in the Inn when I sent a reply to someone I was talking to at the Inn. But did not realize it will go back to the Golem for the Prisons. So, I was immediately was attacked by Olin & the Bouncer. So, I tested it again in Halfling Land and it showed up.

<>
Gelthmore Road
You tell the emerald golem 'What the hell is this ####?'
You yell 'Help! I am being attacked by a border guard!'
You parry a border guard's slice.
A border guard is in perfect health.
<>

-------------------------------------------------------------

Secondly, I did walk into the Golems few times and the Quest shows up every time I walked into it. I am think it’s a Bug.

Quests In Progress:
<35> Prison of Glymarach
+ Speak aloud the answer to the golems' riddle
<35> Prison of Glymarach
+ Speak aloud the answer to the golems' riddle
<35> Prison of Glymarach
+ Speak aloud the answer to the golems' riddle
<35> Prison of Glymarach
+ Speak aloud the answer to the golems' riddle
<35> Prison of Glymarach
+ Speak aloud the answer to the golems' riddle
<35> Feanwyyn Weald: Margoesh the Poor Adventurer
+ Answer Margoesh's question 'Would you buy me some supplies to get me started with?'

<>