Ideally you should apply on your first visit and soon after your arrival. If you are able to return home multiple times it doesn't really give the impression that you are in fear for your life.
Agreed with the above. If you don't claim on your first visit, it will likely make it harder to convince CIC you are a genuine refugee since you have willingly returned to your home country.
And it would only affect your refugee claim if the risk/threat was in place before your first trip to Canada. However if the risk appeared after any of your previous visits then you shouldn’t have any issues.