I went to the Passport Office at Yonge and Shepherd (4900 Yonge) at about 6.15 and was similar experience overall in terms of the management. There were about 30 people in front of me, that narrowed to about 20 as the process weeded out those there for slightly different things. At about seven they made announcements and a few minutes later let people in, the group eventually snaked up to the passport office reception area. When they started taking clients at 8.30 the line moved reasonably quickly, reception checks to make sure you're in the right place and that the documents seem to be in order, you're given a number.
Go to main passport application room and wait for your number to be called. They checked travel info only at the last step but did ask for it and looked at it.
I would have been done about 9.30 but for a dispute about whether my documents showing parentage of child were sufficient.* Those who arrived much after me were in for a long wait, although I don't know of course who was lining up for what, the waits still look pretty severe. And showing up 2-2.5 hours first thing in morning wasn't fun either, although at least I was done at a reasonable time of day.
[Long story but they didn't like the translation of birth certificate I had - despite the myriad documents including acceptable translation of documents that ALL corresponded exactly (including one document specifically titled "establishment of parentage" -) when the application instructions clearly say you need to ... show parentage and don't specifically require birth certificate. This is the worst bureaucratic stupidity I've run into so far upon returning to Canada.]