It has been discussed on this board. One guy who had a university degree in English as a 2nd language was given zero points and one guy managed to get full points but he had years of experience as an English teacher. One guy said he showed proof of having lived in an English speaking country and doing a university degree there and he was awarded 8 points when he'd asked for 16. I read on another board about a woman who applied and even though she was born and raised in the US with English as her only language, because she had a non-English name (we can only assume), she was given zero points. She applied again with IELTS and was accepted.
I think it totally depends on your luck with the visa officer. They can't argue with IELTS but if you send a letter, you are leaving it up to them what they give you. So are you feeling lucky?