Did you drill a small pilot hole first or go straight to the full diameter bit?
I am a beginner as well and mark the hole location with the lock in place, punch a divot at the correct spot and follow up with 5/64" drill bit first, then an 1/8" and step up to the full size bit for the liner. I do have a cross slide vise on my drill press that makes hitting the mark much easier.
Here is my slopped up first gun, at least the touch hole liner location is correct. I am proud of my second gun but not so much the first, although I have killed a pile of deer with it. Some day, when I have nothing to do, I am going to redo the gun in the picture to correct all my goof-ups. Between gun #1 and gun #2, I discovered Optivisors, amazing how much better work you can do when you can actually see what you are working on.
Here is the same gun marked and ready to punch. Sometimes my punch mark might be slightly off but I can move it to the correct spot with a little sideways tapping. Of course I take the barrel out of the stock to mark my spot and proceed.