Taking the newborn to Passport Office is not a must, but it's better. In my case I took my newborn but the Officer was very courteous and said with a nice smile 'it would have been better if you brought sweets instead of the newborn' :D a rare wonderful experience from a BD Govt. officer.
rest...