I don't think resilience is blocking any poison/disease damage or helping with any saves.

I spent some time chugging poisoned water in Azreth which I noticed didn't drop strength like it normally does but was still doing damage every tick. No improvements to the skill at all.

Attempted to eat some devil's backbone later and hemlock, no poison or disease there, so far so good.

But then I went to fight a necromancer mob later and crimson scourge seemed to be doing full damage.

Can someone check or clarify if this skill is working and how difficult the skill checks are supposed to be? I don't mind if the strength thing and the saving against level 60 prep spells are normal and in line with the rest but it seemed odd that I could sit in ysigrath for a whole hour and not only not see any improvement but the damage from the poison looked normal for a level 23 character. I had no echoes when resilience was working, nor did I get any improvements.

It's working and can be improved correctly. The skill comes into effect when you make a saving throw to avoid becoming poisoned or diseased, not when you take poison or disease damage. By the time crimson scourge is actually on you that ship has already sailed.

You don't get an echo when it works.