Yep - exactly. You still have to factor in the time it takes you to get from Union Station to City Hall, plus getting to/from the GO Train Station parking lot (just getting out of the GO Train parking lot can take a while), and of course sometimes there are late trains and other delays...
40 minutes is definitely possible - but you have to be significantly closer which means either paying more or giving something up (i.e. size, space, newness, etc.). I have a 20-25 minute commute door to door to the financial district. However I effectively live downtown (near Christie Station and Bathurst Station).