There's a wonderful graph called Gradually, Then Suddenly by CB Insights that I like to wheel out from time to time.

That's when it's time to argue with the boss.

It doesn't matter if the boss's ego is bruised, it doesn't matter if they retaliate. Act now, or we're all out of a job anyway.

I've seen it go either way, but what has never happened is that the boss ignored the situation and kept his/her job.