Roosting in bird box - when to clean it?

When do you clean out bird boxes if birds roost in them over winter? I have 3 wrens roosting in a bird box which is full of nest material from last year. The wrens built a lovely mossy nest last spring and then disappeared. Then a hornet took over for the rest of summer. The wrens came back in the winter, pulled out the hornet nest to make space and have been there overnight ever since. I’m not sure whether to remove the old nest material or not. It’s only a small birdbox and the nest completely fill it. The grass & moss comes all the way up to the roof so you can’t see anything if you lift the lid.