I was able to get my passport back after following these instructions I received from IRCC:
if you wish to have your foreign national passport or travel document returned, the responsible office will keep it until it expires or if you become a Canadian citizen.
If you require your foreign national passport or travel document prior to its expiry, you must return the Canadian travel document in exchange.
We invite you to mail back the Certificate of Identity or Refugee travel document that was issued to you along with a letter of explanation to the following address:
OSC PPT-COI
PO BOX 8783 STN T CSC
Ottawa ON K1G 5G8
In your letter of explanation, please include a confirmation of the mailing address where you would like to receive your passport.
If you require your passport urgently, please also:
- Write URGENT on the envelope, and
- Include the envelope tracking number.