You can see in appeal cases some examples of where the visa post received a 'poison pen' letter. Apparently the visa officers do read them, and do take them into account. Now, a poison pen letter that is anonymous or that has no accompanying evidence might not be taken as seriously as one where the writer includes contact information and/or proof, but these letters have been acted on. They might trigger an interview, where the applicant will be asked about the allegations. If he or she can explain them away, great. But since they show up in appeal hearings, apparently they also can be part of the reason for refusals. A poison pen letter with no evidence would never be the only thing to trigger a refusal, though.
If you go to the CanLII website, you can search immigration appeal cases. I entered the search terms 'poison+pen+letter' and 'immigration' and got over 100 cases. Many of these appeals were allowed, and in some the judge hearing the appeal did state that poison pen letters are inherently unreliable; however, some visa officers relied on them enough to refuse the PR visa application.
http://www.canlii.org/en/index.html#search/all=poison%2Bpen%2Bletter%20immigration