This sounds like my problem.
I noticed tapping on the starter did weird things as well as rolling the rear tire backwards would turn the light off and on. So heeding the advice of the old timers I tore apart a starter, looked inside and so the blackest of black on the contacts and not enough brushes to tickle contacts. Cleaned them off and ordered a rebuild kit. It looked sooty.
Wife asked me what I was doing and I kinda pretended I knew and she stared at me. I stared back with the fear she may have known I was bullshitting my way through this and she may know more about this than me. I tried that technique with an old commanding officer and it didn't go well cause he knew it. I later found out he had a master's degree in physics and electrical engineering. Which made me wonder what the hell he was in the Army for. Anyway...
I think I put it back together right. I noticed the terminal going thru the bottom is positive and wondered exactly how that gap is going to be maintained so I tightened the crap out of it. I hope that it's centered in the hole, which by the way doesn't have hair around it and even if it did I hope the torque'n pulled it through to the center so it doesn't make contact when it shouldn't. I suspect sparks would fly or something would happen I wouldn't care for. I put a ohm meter on it and noticed numbers, and then no numbers as I spun it around. I placed $50 on black and and seem to have walked away even.
Didn't have the motivation or tool to pull the bearing and replace it. I got lazy, but I knew this about myself. Now you do.