How long has he been with you? He is not enjoying being in the cage. If he came to you hand fed and tamed then keeping him locked up is not doing your relationship any favours. The more he's locked in the less he'll want to go back in his cage - you're already experiencing this. If he's breaking the bars then he's getting desperate.
If you do decide to let him out make sure the room is safe for him and no food outside of the cage. Is he stepping up? That would be a great way to bring him out, doing something nice for him. If you have the time don't close the cage as soon as he goes back in, let him eat and come back out for a second time. He already knows there is food in there, you just need to show him it's not a trap. When he's out stay away from the cage.
Any birb getting that freedom for the first time will not want to go back in. In the end an empty crop will make it necessary for them to go back in on their own.
When you get a good daily routine like this he'll soon settle when he's in the cage, because he'll know that he will be coming out again.