You can hunt the public if accessible and do not have to abide by the HMA rules, but you can NOT cross onto private of the HMA at all at any point unless you have the Slip (game wardens check slips at HMAs often). If you are that close, you might as well get an HMA slip as they are usually simple to print out. Some are subject to draws for the whole season or part of the season, so this definitely could come into play where you actually can't get the slip and there is public land available you want to hunt.
What you can't do, even if you have an HMA slip, is go through the HMA and access the public and not abide by the HMA rules unless you have a different way out of the HMA which is all on public.
This is definitely pushing the spirit of the HMA rules and a game warden may see things differently. For example, the HMA allows antelope hunting, but not deer hunting. If you go through the HMA and shoot a deer on the public adjacent to the HMA, you better have the following:
1. An HMA slip
2. A valid unused Antelope tag for that HMA and that season is open.
3. A way to take your deer, you, whatever you have (probably including your vehicle) out through the public to a public access point.
If you had all of the following, why wouldn't you just go through the public to shoot your deer? I could think of a few reasons why......