Extremely anxious and skittish outside of cage!

[Modern]

Hello, my first ever post! Let me know if there is anything I can improve based on format of my first thread post.
Anyways, to get to the point.
My young cockatiel who I have had for almost a year now is very sweet. He is loving and is not afraid of my hands or my face. He sings and does not pluck his feathers. His cage isn't as large as I would like it to be, I'd say it is on the larger side though. I also provide many different enrichment toys for him to use. He also has a bird buddy who is another cockatiel, but he bonded with me instead. Also, he is free flight and has his flight feathers fully grown in.
The issue here: Whenever I take my sweet boy out of his cage suddenly everything is so scary to him! If there's a person in the background he flies off or a slight noise he flies off. The issue here is that sometimes when he flies he'll straight up keep going around and around until he crashes into something. He's lost feathers over these crashings. I usually hold him in my arms and tell him it'll be okay, but I can hear his little heart beating like crazy. And what's worse is that after these freakouts he usually has them more frequently the longer he is outside of the cage! Even opening the door nearby or hearing someone else's voice is enough for him to just TAKE OFF!! I hate when he does because he never comes back down and keeps going until he gets himself hurt. The result of this hurts me because I no longer have him outside of the cage as much as I used to. I want him to fly and I want him to enjoy being outside of the cage on my shoulder and seeing other people. Sadly, this cannot happen since as soon as he leaves that cage it's only a matter of time he'll get scared of something so insignificant and crash himself.

I really hope there is a solution or a better way to handle this. I miss spending outside cage time with him and I can't even put him in a private section of the house to free fly in (safely) because even opening the door (it creaks) makes him get so scared he just bolts.
I don't know what has him so scared as nothing has ever really hurt him.

I am a young college student who has a pretty big family. There are children living here as well which is why it can be noisy sometimes. Therefore, having the house completely quiet when he is outside of the cage is not an option.

Please, if there is any advice or any explanation really...anything helps. I hate seeing him so scared. But, I know spending so much time in that cage is bad for him.

 

[Zara]

Hello, Welcome to the Avenue

When you get him out of his cage, do you put your hand in and bring him out? Or put you hand in front of door and he steps out onto your hand then you take him away? Or do you just open the door and let him go out on his own?

Are there any external perches attached to his cage?

he is free flight

*fully flighted (Free flighted means they fly free outdoors)

 

[Tiel Feathers]

My cockatiels used to behave similarly to this, but have calmed down as they have gotten older. I always have some background noise going like the radio, so other sudden noises don’t seem so loud. Having a loud house isn’t so bad, it’s when it’s quiet and sudden noises happen that can be startling. If there is something you can fix, fix it to minimize his reactions, like putting a little oil on the creaking door. Maybe ask people not to barge in when he is out. The more he is out, the less he should get so startled, so keeping him in his cage more isn’t a good idea, and make sure there are no windows he can fly into. Having some safe landing areas around and perches outside his door are good ideas. Maybe try taking him through the room and show him things and touch things so he knows they aren’t scary. Can you let him out in a room where people aren’t coming and going, like your bedroom?