Help - advice re robins' nest

Today, when moving some empty plant pots I found a robin sitting on a nest underneath. The robin flew off and I could see at least two babies and one unhatched egg - the babies looked extremely young. I replaced the plant pots as I found them and I believe that the robins returned to feeding the babies. The nest is in a plastic sack which is full of wood pieces that I have saved. I would like advice as to whether I can remove the nest once the babies have left or whether I should leave it for them to use again. Problem is I would like to use the plant pots and also am in the process of making over the garden and wish to start on this area soon. I am thrilled to have robins nesting but it's not in a great place. Any advice would be appreciated. Thank you.