We were advised at oath ceremony that we could bring family and small children. However, if during the ceremony, children made noise, they would be asked to leave the room. During the ceremony, no one from outside is allowed in, so once you had to leave the room due to your child, you wouldn't be able to get back in. That would mean your oath ceremony would not be completed.
However understand it's a difficult situation especially when your baby is 5 months old.