First, you actually ARE setting the blade to the correct depth.. 1/3rd of the way into the slip sheet. That's how it is taught and it is how it should be.
"Barely scratching" is NOT the proper setting, period.
So, now that we have debunked THAT little tidbit, I do agree that you need to look at the bearings being a bit sloppy and you also might look at what you have the cutter sitting on. If it isn't solid enough along the full length of the cutter, you might be getting some flexing of the machine as you press down while you are cutting. Which brings me to another possibility... pressing down while cutting.
You notice that it doesn't happen at the beginning or the end... because you aren't bearing down on the cutter head at those points as much as you are concentrating on starting and stopping the cut. During the run of the cut you might be pushing too hard which would cause it to flex and possibly change the angle of the blade. That combined with the worn bearings just might cause your problem.
Also...be conscious of how you are holding the cutter head as you run the cut. If you move your position relative to the cutter head as you make the cut AND the bearings are worn, then you could be slightly rotating the head.
(As I go back and read what I typed, I think this might be the main problem.)
When I first set up me 60" Fletcher cutter, I had to add a 3" wide length of channel iron to the underside of the table to stiffen it up so it wouldn't flex.
Try looking at these and experiment to see if anything is causing the problem.
anyhoo.. good luck.