I've known gardens have a lot of rats around the bird feeding stations seemingly without any effect on the birds, so I'm not sure that would be the reason. However, in my own garden I happened to see a sparrowhawk attack a large flock of starlings, taking a juvenile, and it was a couple of months before a few cautiously reappeared, and they've still not returned in the numbers they had previously, so perhaps it could be a predator has been seen and spooked them.