It probably won't be dangerous, but the handling will suck and your tire life will suffer.
Put the bike on the centerstand and grab the rear wheel and move it back and forth. Watch the swing arm at the spindle. If you detect movement and the wheel moves more than 1/8th inch or so, the handling will be horrible. Make sure it isn't the entire cradle moving at the isolastics. If so, shim these and recheck.
It very likely is not the spindle, or even the spindle bushings in the swingarm. It usually is the bore in the cradle for the spindle. Lack of lubrication will usually wear the spindle bore in the cradle eggshaped. These can be bored oversize and oversize spindles are available. However, a typical and less expensive fix is to drill two holes in the cradle tube, weld on two nuts and use bolts to lock the spindle tight to the cradle. Alternatively Heinz Kegler makes a set of lock rings that don't require welding.