The only issue for you is your Canadian tax status while you were in Taiwan. If you had no major financial ties with Canada (no home, no wife or children living in Canada, and you did not spend 183+ days in a calendar year in Canada), you should have no difficulty explaining to CRA (in a cover letter to your next tax return) why you did not file tax returns in Canada for those five years.
I don’t know that showing Taiwan tax returns is necessary, but they would certainly be convincing proof that you were residing out of Canada, and had superior financial ties to Taiwan. There are other proofs too.
It’s too bad you did not declare yourself a non-taxpayer in your tax return 5 years ago, but the fact that you were residing in Taiwan cannot be disputed, so you’re OK. The worst case is if CRA insists you were a Canadian taxpayer all that time. Research whether there is a tax treaty between Canada and Taiwan, and then calculate the taxes you would have had to pay Canada on your Taiwan and worldwide income. Then subtract the taxes you did pay. If there is any debt left, that’s what you’d have to pay Canada.
But again, no reasonable interpretation of the Tax Act would deem you a Canadian taxpayer fr those years.