http://www.cic.gc.ca/english/pdf/kits/forms/IMM5669E.pdf
Look at the questions on the right. Notice how it would appear by answering yes to any of the questions, it would count as a red flag? Being refused a TRV is one of them, along with other seemingly harsher questions like being a refugee or convicted of a crime.
You're right, it's normal for spouses to want to visit each other and being refused a TRV isn't a huge deal, in the grand scheme of things. But it's best, to avoid any problems, to avoid being refused one in the first place. IMO, it's best to avoid any blemishes on your application, no matter how small. That's why my fiance and I discussed this and decided for her not to apply a TRv, but instead let me come see her instead. It doesn't matter because we both get to see each other at least. And when the VO sees how often we see each other, explaining to them why she hasn't come see me is a LOT easier than answering YES to having being refused a TRV.
We ask ourselves, is it really that important for her to visit me, when I can just go visit her? We both get to enjoy each other's company no matter what country we are in, why add a red flag to our application? (even if it's a small red flag). I want the VO to look at that form, see that I checked off NO to all those questions, and quickly move on, and not have to investigate the YES.
She's from Vietnam, we won't even bother try to apply for TRV. If this was from USA or some other affluent country, sure we'd do it!
Being refused a TRV isn't a big deal, if the rest of your relationship is solid, but we dont want ANY blemish on our PR application, no matter how small it is. Actually I'm just talking w my fiance about it now, and she just says, "oh who cares about going to Canada, its not a big deal"