I left the food in. What you mentioned is true also, and they seemed to be not fed actual food when I adopted them. So they must be really hungry. I have my hand near the door, not in the middle of the cage, with millet. They peck at it, though not getting on my hand just yet. I'm only planning to do that once a day, or skip a day. Depending on how they react to it.
Not sure if I mentioned this already, but I took the coconut out. They seem to be getting used to me a lot quicker without that thing in the cage. Right now they would still be eating their food when I'm in the room. When I adopted them they would stay silent and not eat. So far it looks like they're used to a clean house with fresh food and everything, including my presence.