Hi Neilmax,
When I lived in the Greater Toronto Area (which includes Scarborough) I quickly learned that it is a unique city because every neighbourhood is very different. Crime rates on one street may be high, and then four or five streets away they are very low. As the other posters mention, if you are looking for a particular field of study in Ontario, that should be the biggest influence for your decision on where to live. For example, my Immigration and Settlement MA program was in downtown Toronto, and so I moved there in order to do the program. Please feel free to get in touch with me if you would like some additional help.
Best of luck