This is the info that I'm not sure of yet. They quoted in their announcement that around 98% of applications have been successfully approved using this new measures. I don't know how they got this data. I guess it wouldn't hurt to wait for next few days to see if they give detailed write-up for these questions.
My guess would be that they either did some unseen pilot testing on some existing apps (who may have had visas approved) or some (hypothetical) testing on some selected files (eg would these applicants be approved for TRV if we ran this through our algorithm)?
Pure speculation on my part, though.
Unfortunately, that's not real world.
1) They say 98%, but compared to what? Did the files they tested have a different approval rate before? How much different? Yes, it would be a classic govt thing to neglect to mention that the same test beforehand had an 89% approval rate - see my other comments for why that can be the case.
2) If they select files to test, they
selected files according to some criteria - and that is / was almost certainly criteria that is different from what spousal sponsors and applicants think is the class of 'eligible spouses.' If they choose a subset, it is very, very likely that there will be some difference from the entire class of applicants. As a very simple example, I'd
guess that they might have excluded any TRV applicants who had previous refusals (because those refusals could have been for serious reasons and would skew the results of the algorithm etc.)
3) I believe all of the IRCC processes repeat (constantly) that there is always a human involved (although possibly only rejections? at least for some things like ETA). Well, did they test this process with humans doing the approval? [Does it require training for IRCC staff responsible? How long will that training take? Have they tested the results after training?]
Not to be overly skeptical, I welcome their attempts to change things, but there are serious reasons - beyond the small points i've made about the announcement - that within current law the rejections will usually/always be higher than hoped (by spousal applicants anyway). It can still be a net increase in approvals, even a significant one, and yet still lower than the tenor of the announcement implies (no way it's going to be 98%).
That's without getting into the issues after program goes into action, two of which are inevitable and the only question is how significant it will be:
1) Announce a program like this, and many will apply that wouldn't have before - because they assumed it would be rejected. And many of them were right - but even so, doesn't matter, because the characteristics of these new 'shadow' applicants will be different.
2) Any new program will be gamed by the unscrupulous, that is, fraudsters - and then the cycle of IRCC trying to detect/understand who are the risky ones, and new measures (internal, unannounced) get brought in start to take effect. And those new measures will pick up new and different false positives - in short, the cycle begins anew.