Servitors can and do turn even when they follow you quick. Yes it's true that sometimes they follow slower when mad, but it isn't always a true indication. Perhaps even indicate how strong the binding is holding on them would be nice. I dunno, just seeing that they are bound makes detect charm a rather useless spell. It should tell you a little bit more I wish.