It's not just the barrels, but barrels do vary. Swamped barrels are milled on the outside after being drilled, and the variation in thickness and exactly where the "taper" is, could change the barrel whip when fired. Internal milling, was your barrel the first one cut with a brand new cutting tool, or the last one before they switched to a new, sharper cutter?
Then there is the powder, even if you only shot BP, you are talking GOEX, SWISS, DIAMONDBACK, and home-made (maybe I missed a brand or two). What they use for chemical sources, and the type of wood used for the charcoal, supposedly changes the characteristics, even if they used exactly the same formula.
Patching material is from cloth, for sewing, garments, upolstery, not for gun patches, so no real control over the differences in thickness, and then some folks wash and dry before use, some use it right off the bolt.
Add to that, some folks might lay in several years worth of powder and patching material, say five yards of patch, five pounds of BP, all from the same lots. So they get very consistent results, while the other fellows might buy a pound of powder, a year, differing makers each year, so really change up the shooting variables on a regular basis.
LD