Posting my experience with travel from USA to Canada, I flew with my spouse and kid last week to Calgary. I was holding work permit and my dependants with visitor visa. Since we traveled from USA no authorization letter was required. I confirmed the same by calling IRCC and referring the website. It was smooth