In my opinion, language is still important because what will you do in Canada if you don't know English or French even though you have a good point (I think if you don't submit IELTS score they will consider you don't know English).
I think it also depends on your career. If your career doesn't require much language skills, then low language score might be ok.
This is just my opinion.