Roby basically played the entire first quarter and they didn't blitz then either. Also the pass defense stunk (with him playing) against Cal, Wisconsin, and Northwestern.
IMO, while there are weak spots I don't think its a talent issue, its coaching / scheme that is the problem.