Again, if ever audited in the future this would be entirely discretionary depending on the feeling of the auditor reviewing your case. The tax folio linked to above, specifically items 1.14 - 1.17, explain this in detail.You don't need to close your accounts or cancel your DL. You will not be declared a resident for tax purposes based only on those small secondary ties, especially if you aren't even using the bank accounts while you are outside of Canada.
True a few secondary ties will not matter at all. However one should make sure they are not actively using the Canadian accounts while a non-resident, and don't go crazy with 10 or so secondary ties at various places. There is a point where too many secondary ties will turn into a problem, and you don't want to get close to whatever that number is.