Can you tell me more about your finches? Are they zebras? Zebra finches can breed rapidly and it is probably best to remove all nests. You can also get dummy eggs or hard boil the eggs she laid and put the dead eggs back in the nest until she slows down, I would mark the eggs you boil so that you can keep track. You could consider removing the nest now, I know your birds will be stressed and it isn't ideal. If you wait until your hen is done laying and starts incubation it may even be more stressful. Only you can decide what your plan is.
If you are planning to remove the nest I might try and get your finches out for a free fly and redo their entire cage. Take the nest out, reorganise the perches, get a few new perches/toys try and make it all seem a bit different. Then when your birds are reintroduced it will be a whole new setting and they won't be as transfixed on the lost nest. Unfortunately they will still experience some stress, there is no easy way to deal with this without causing some stress.