Yes, as long as you still meet the MEC for your stream and are above the cutoff without those 50 points, I'm pretty sure it's no issue.
For example, I claimed some points for 3 years of foreign work experience, but since it was difficult to secure a reference letter and I was still above the...