You can pack into some of the drainages in the Sierra Madre mountains and get a mile or so from roads and trails, but not much farther unless you get into the Houston Park wilderness area which is really only about 5 miles across.
As for the lower country, it would be a waste of time. You'll be better off camping from your truck. The best public lands big buck country in the unit has multiple roads and trails up every ridgeline and down through every draw. You can't get more than a half mile from a trail. In alot of this country, there is literally no place that a deer could hide out and not be seen from a road somewhere. Unfortunately, "sportsmen" in this area have made it clear that they would rather drive around in pickups and 4 wheelers than hunt big deer.
That is not to say that a few big deer aren't killed in this unit every year, because they are, just not usually in the manner of hunting that you are looking for. If you get some access to private lands, you can get away from some of the pressure.
Sorry I can't be more positive.