As I said, it is not the college that gives you the visa and your friends profiles are going to be different than yours. Every profile is treated individually. It is your choice to go with the 5.5, but your chances of getting a visa will be greater if you get at least 6 in every section (of course that is not the only thing you will be judged on in your profile). Just my advice from experience, if you choose not to take it, that's up to you.
Also, you must have some doubts, even though so many people (as you say) have got in with this score.