4 dead baby Swallows

Two days ago I found a dead baby swallow on our drive below the nest. Then today I found another baby swallow on our drive but further away from the nest (it can’t have fallen out as it was about 4m away from the nest). it was alive still so, rightly or wrongly, I put it back in the nest. Then, an hour later I looked outside to check on the nest and this time there were three baby swallows on the floor below the nest. Two alive and one sadly dead.

I can’t understand what’s happened here? It’s so sad to see. Does anyone know why this might have happened? The mum is very attentive to the nest but there is another swallow hanging around (looks to be a male).
I watched the nest for a while and noticed the male remove a feather from the nest. 
 Would a swallow remove babies from another swallow’s nest?! Very sad Pensive