The first thing that came to mind is: Are the wheels in line? It's all too easy to get them out of line.
If they are not in line, then correct this first. The front wheel rim should be centralised in the front fork. If not, you will have to rebuild it with the correct offset. Then check the wheel alignment. You can use string, lasers or long, straight bits of wood, steel, neon lights, etc.
The rear wheel is also 'dished', but less so that the front. The rear wheel rim offset with its hub could be correct but it could still be out of line with the front.
Search this site to get the correct rim: hub offset data.
You should also check that the rear wheel is not cocked in the swinging arm. Mine was. Also, when I first got my Commando, the wheels had been built incorrectly so the rear was way of out of line with the front. The rear mudguard obscured the view but when that was removed, it was plain to see that the rear wheel was both out of line and cocked in the swinging arm. One arm of the fork was higher than the other and one of the wheel spindle lugs was bent.
It could be that the entire engine/gearbox/swinging arm sub-assembly is cocked with reference to the horizontal axis of the frame.
If just the rear wheel is not parallel with the front, it will appear easier to steer in one direction than the other. It could even be that both wheels are off line from the centre line of the frame.
Something's not right with the geometry so you will have to check by a process of elimination what the problem is.
Apologies if you have already checked and are confident of the above. Otherwise, you have some homework to do!
Let us know how you get on.
Dave