n-bob, I don't think a big hammer will do any good to you handling problem. Better to dig out the fine spanner, a truing stand and the factory specs. When both wheels are to spec, a lattice held against the rims (w/wheels fitted) should tell you whether frame and forks are straight or not ....
If no fault found, proceed to check fork alignments and replace headstock and s/a bearings. Consider replacing headstock bearings by one-piece radial bearings.
-Knut