I think you are right, it will crumble just like Pablo Escobar's empire
I disagree. Practically every community we've seen so far has a group of people plotting against their leader.
This episode may have expose All The Descent in Negan's group, but it also showed all the amazing things he does right.
If you were willing to work hard and keep your head down the savior's offer you a far better life than anywhere else in the apocalypse. You probably won't even have to kill anyone. Sure, you got to see people get beat with bats and thrown into furnace is, but how is that any worse than the horrors you have already experienced up to this point?