Bro, you need to bear in mind the following things:
a- check fee refund policy of your university, usually administrative charges are deducted and the total amount of deductions varies with the amount you paid to the university in advance and there's also minimum amount of deductions set by the university. If you paid for health insurance to your university in advance, the amount of HI will not be refunded (personal experience). Admission fee is non-refundable anyway.
b- you must have a letter of acceptance from the college you will be switching to and you must study at post-secondary level otherwise you will have to notify CHC about your study plans and even file a new study permit application.
c- the process can become a pain if your university is not satisfied with your plans and thus the only way to convince them is to obtain a letter of acceptance from the institution you'll be switching to and explain how you will benefit from this move. Obviously the institution you mentioned on your visa application would be interested to find out why you want to leave them after getting visa.
d- whatever you do just make sure that you don't waste time because hanging in the middle (i.e not starting your course on time and not switching the institution either) is the worst thing to do. It's obvious that your visa will expire before your course finishes when you start your course in the next semester . Getting visa extension will be a problem as you'll have to seriously convince CHC as to why you failed to start studies in Canada on time. So do you homework before you think about switching institutions.
the process is not too tedious providing that you meet the deadlines and plan ahead.
Good luck !
Cheers
