You mean the one in Fortune, where the ferry to St Pierre et Miquelon is? I'm not sure if it's a common place to flagpole. And since the other territory is in the ocean, you can't really "leave" Canada the way you can at a border with the US.
Be aware also that the CBSA office for travellers appears to be seasonal and has shut as of September 30:
https://www.cbsa-asfc.gc.ca/do-rb/offices-bureaux/65-eng.html
Outside of the tourist season, the office only appears open for commercial purposes.