I'd bet that's because of differences of where you go in a small sail-and-oar boat vs. where you go in a keelboat, don't you think? I didn't find Collins Inlet to be particularly interesting when I sailed around Philip Edward Island in 2017 in my ALaska--I much prefer the outside route, where you can take in the Fox Islands, Hawk Island, etc. That said, Beaverstone Bay is way cool (though it does have some cottages down by the entrance, and I try to stay away from camping near cottages). There just aren't protected anchorages for bigger boats in many of the areas I end up exploring on Georgian Bay--which, of course, is a big part of the appeal for me.
They're great--some of my favorite cruising grounds. But again, it's probably a different experience in a small boat or canoe/kayak than it would be in a bigger boat. Whenever I've gone there, I've never seen a big boat outside the main anchorage at the northern end. They'd never make it anywhere near the eastern shore of Tanvat Island, for example. My impression is that Georgian Bay is one thing to big boats, and another thing (almost) entirely to small boats and paddlers.
Not good vs. bad, or good vs. better, just different. For now, I like the small boat game. It's cheap. It's simple. Seems perfect for a lazy person who doesn't want to spend much money, and can tolerate moderate reductions in comfort to have an experience they wouldn't have otherwise.
Tom