Toronto City
The City of Toronto* is considered to encompass parts of North York to the North, Scarborough to the East, shores of Lake Ontario to the South & Etobicoke to the West. See the map below:
http://en.wikipedia.org/wiki/File:Toronto_map.png
Toronto Downtown
The Downtown of Toronto is usually considered till Eglington Avenue to the North, DVP (Don Valley Parkway) to the East, Gardiner Xpressway to the South & Dufferin Street to the West.
Heart of Downtown
The Business District & Entertainment District combine is considered to be the heart of Toronto downtown. Basically the 6-7 Kms area around the CN Tower.
Centre of Downtown
Yes, as the previous poster opined, Yonge & Dundas - the intersection of Yonge St. & Dundas St. is typically assumed to be the epicenter of Toronto Downtown. That place is colloquially called the "Times Square" of Canada.
*Not encompassing the GTA (Greater Toronto Area)
Qorax