Based on what I've been reading on this issue, you would be able to fly to US using your other passport and VISA, but most definitely you wouldn't be able to fly back do Canada unless you have a multiple entries Canadian visa there (what you probably don't).
Looks like the reasoning behind this...