Passport is a travel document and supposed to be with the bearer always while traveling abroad, no exceptions for this universal rule.
Even if you manage to get the passports to CIC Ottawa, they may question you about the missing entry/ exit stamps in the passport.