A couple points, not to pee in everyones' cheerios, but ...
1) I have been lead to believe that Canadian college hockey is not very good, or popular. Anybody with talent goes to the CJHL and gets paid right out of high school. Or elementary school.
2) Toronto's on-campus stadium for football seats ~5,000 people. They're playing a sport with different rules (larger field dimensions, 12 players, only 3 downs) and different strategies, and they've been doing that at about a Div. III/NAIA level (and poorly) for 15 years. Their kids would get killed lining up to defend I-formation runs with two TEs and a lead FB. It would take years, if not decades, to transition them to an FBS-level school.
3) TV sets? Fan interest? The entire population of Canada is ~34M. The state of California has 37M+ alone. Texas has 25M+.
To be honest, I'd rather attempt to raid the Ivy League before adding Toronto.