When most of us clean our rifles, we usually coat the bore with some kind of lubricant and/or rust preventive. This alone will affect the behavior of a bullet, even if you run a dry patch down the bore. A dry cloth patch just can't begin to remove all of the oil, though the extreme temperature and pressure of firing a round through the bore will certainly remove the vast majority of it.
The presence of oil in the bore will cause some variation in the bullet's velocity (speed), and may also change the way a projectile engages with the rifling; either of those conditions will naturally affect its flight path and subsequent point of impact. And oil in a muzzleloader's bore is a definite no-no, because it will foul your propellant (powder).
Oil aside, there are other factors that affect how bullets behave inside a rifle's barrel, and while I can't say that I fully understand every facet of this, I can say without doubt that consistency is paramount if you desire repeatable accuracy - and we all want that.
Of those who advocate shooting only a clean bore, I ask this: Do you thoroughly clean between each and every shot? If not, you're failing to create consistency, and this is likely to affect velocity and point of impact.
But my Gun Shoots Perfectly all the Time!
Maybe your gun shoots wonderfully most of the time... but chances are that your accuracy could be even better than what you're getting, if you would shoot through a consistently (but not overly) fouled bore. I have a number of rifles that generally shoot within "minute of whitetail" whether I foul them or not - but we owe it to ourselves (and to the game we hunt) to strive for the best possible accuracy, and shooting through a fouled bore is a good way to do that.
How to Foul?
So, how do we foul a barrel? It's pretty easy, actually - just shoot the gun. With cartridge arms, firing one to three rounds should be more than enough to foul the bore. This will burn out oils and such, and usually also lay down some copper fouling on the rifling. And while a lot of copper fouling is undesirable, a little bit often helps keep a gun hitting in the same place.
Cleaning in the Field
This doesn't mean that you shouldn't clean your gun. When I get caught out in the rain with mine, I take it apart back at camp and give it a good drying-out and some love. This means patching out the bore with a dry patch (followed by a lightly oiled one, if I'm hunting in very wet conditions), as well as cleaning off extraneous crud and wiping down the gun's exterior.
Am I worried that this will mess up my accuracy? Nope. This is not a thorough bore cleaning; it's simply removal of water, crud, and surface rust. But If I wanted to make sure, I could always fire a round once the weather cleared up.
With muzzleloading rifles, other factors come into play, because the black powder and black powder substitutes used in most of them creates thick and heavy powder fouling, which is usually very minimal in a modern cartridge gun. This is another case where consistency is key.
Here's what I recommend: When you zero your muzzleloader, fire a shot or two, then patch out the bore. Depending on the circumstances and your preference, you might choose to use a wet patch followed by a dry patch. Then, shoot it again. Patching after every shot or two usually makes a big difference in muzzleloader accuracy, even though you're not thoroughly cleaning the bore with those two patches - there is still fouling in there. And it's absolutely impractical to thoroughly clean the bore between every shot.
Once you have your smokepole hitting where you want it, always hunt with the gun in that lightly-fouled condition. With a muzzleloader, even moreso than a cartridge gun, your first shot is all-important, and the circumstances under which it is fired should match closely with circumstances you created at the range.
Popping a cap or primer is not sufficient to foul a muzzleloader's bore - you need to burn some powder in there. Dump maybe 20-30 grains of black powder down the bore, ram a wadded-up dry patch on top of it, then prime and safely fire the gun. Patch it out per your usual routine, and load 'er up.
If this doesn't work for you, then don't do it! I'm not here to start an argument; I'm just offering advice based on decades of experience. Take it or leave it, no hard feelings either way.
When I use my Savage 10ML-II muzzleloading rifle with smokeless powder, I patch out the bore between shots. This helps greatly in both ease of loading the sabots, and in keeping shots hitting consistently. But I don't try to remove all of the fouling... and I always foul the bore before I take that gun hunting.
In short, foul your bores. Not excessively, but reasonably. It will almost certainly help keep your shots consistent, and therefore more accurate - and I doubt anyone will claim that better accuracy is a bad thing.
- Russ Chastain