Maybe you missed the part where BATTLE was raiding him 24/7/365, and thus was engaged in a huge volume of group-vs-group fights.

Or perhaps the fact that his only useful cabal power, bond, (Since vanguard is less than useful against all of these true neutral villagers...) requires two people to be more or less attached to the hip, and since villagers are pretty much nonstop kamikaze-ing themselves on you there is very little incentive to breakbond since the next raid is always on the horizon.