Thanks for sharing your notes and pictures of this investigation.
Here's my guess about why it's such a loose fit--to ease assembly of the clutch hub.
If the radial position of the balancer shaft was designed to be held and fixed exactly concentric with the gear splines and the gear hub bearing, then it would over-constrain the fitting of the clutch hub--it would have to match perfectly at both the inner diameter and the outer splined diameter and could be difficult to assemble. Instead it is only constained by the hub bearing for assembly and snugged up into final position by the nut.
The more important constraint or fit is the crank gear to the output gear, the tabbed drive of the balancer shaft doesn't require as much precision--the balancer floats axially on the front roller bearing.
Now what is making the noise and why does it sometimes go away when the alternator load is removed, or after re-torquing the nut?
It is due to a slight relative motion between the drive gear and the driven gears allowed by the anti-backlash spring, and the position of the tab and slot during final torque of the nut.
The output gear has an anti-backlash spring to keep and maintain gear face contact, but the gear for the alternator does not. Both the output and the alternator have different torque loads on the main crank gear that varies during one revolution. In addition the output gear is coupled to the balancer shaft thru the clutch hub nut torque and the final position of the slot and tab.
If the tab is very close to, or barely touching, the edge of the slot during the tightening of the clutch nut then the action of the anti-backlash spring combined with the load of the balancer weights and the alternator torque will allow the tab and slot edge to move and hit and clank against each other.
Once the nut is tight the position of the gear tab/slot is fixed. If the nut is loosened and re-torqued then the position or gap between the tab and slot edge may move slightly to increase the gap such that no contact can occur during the fluctuating torque loads, (or it forces the gap closed so tightly that it can not open?) Those lucky folks have the quiet bikes. Any relative motion occurs within the gap of the tab and slot and so no contact is made and no sound occurs.
When the alternator is removed it has possibly changed or reduced the peak torque load in such a way that the relative motion is lesser and no clanking occurs. The anti-backlash spring is sufficient to hold everything in a quiet position without the reflected torque load of the alternator thru the crank gear.
