Unknown bird of prey?

Hi, I live in on the edge of Cambridge and my garden backs on to allotments, bordering a park which has lots of different trees.  Over the last few weeks I have noticed a bird which regularly perches on top of a tall conifer tree.  At first I thought it was just a pigeon, but on looking more closely via my little NT binoculars, I think it is a  small falcon of some sort, or possibly even an owl.  When it flies off, I get the general impression of red-ish brown wings.  I suspect that it's a Kestrel, except for the fact I haven't seen it hovering at any point.  I generally see it early morning or late pm.  I have now borrowed some stronger binoculars, but it still  isn't quite near enough for me to identify it clearly.   I also wondered if it could possibly be a Merlin, but I don't think they are seen in Cambs are they?  Any ideas?  Thanks