Well, I was gonna leave this mostly alone, but it's a slow day.
If you own a rifle, in good shape and built by a modern manufacturer, you can find a factory load that will perform perfectly in that rifle. You may have to experiment with different companies, but there will be one that you will be very happy with, in any hunting situation.
Now, if your passion is to "punch paper", in your lead sled, off a solid bench rest, you may very well handload something that will shoot 1/2 MOA at 100yds. Good for you.
Take that same rifle, drop the ambient temperature to 15 degrees, add a 30 minute stalk, no bench rest, in rain, snow or mud, with some wind.....read "hunting conditions"....and that 1/2 MOA won't count for 2 pounds of wild owl chit.
Most hunters prefer "minute of deer",or "minute of milk jug" under ANY conditions.
The big 3 ammo guys, with their "A game" load, pretty much guarantee hunting performance under hunting conditions. A couple of other manufacturers are above that level, but you will pay more for that extra performance. They also can go one critical step beyond what most handloaders can do.....what does that bullet do when it gets to where it needs to be. You and I rarely can test that feature.
So that there is no misunderstanding, I own 3 rifles that, to my knowledge, never had a factory load marketed for them, and when I shot paper for a hobby, I was just as anal as the rest. (.219 Zipper, .219 Donaldson Wasp, .22-06 Easling)
That was then and this is now. I don't have the time or inclination to attempt to produce a load that will consistantly kill grasshoppers at 100 yards.
I now shoot Hornady VMax factory loads in my Swift, my .243 and their 165gr Light Mags in my '06, and I am constantly impressed by the performance, in hunting situations.
Beating Hornady, et al, at their own game is very difficult in the modern world.
But, I don't think anybody is "crazy" for doing what makes them happy. If you ENJOY reloading, then you should do it.