CBSA will not give you back your husband passport once he is on a removal order. If your application was received at the CIC office before CBSA made the decision to deport him then you should not have to worry. If they received your application after making their decision to remove him then he will have no choice but to leave.
If they received your application after they made their decision to remove him, my suggestion would be to withdraw the inland application and put in an outland one. Get all the documents and applications and send it in ASAP. Getting a lawyer might be a good idea as well.