No - you cannot travel to Canada on the basis of her PRTD. You need to obtain a TRV to travel to Canada. You'll want to show you have strong ties to your home country in order to be approved. Since your wife has PR, CIC may be concerned you have plans on remaining in Canada long term and refuse your application.