I figure that the k/s didn't hit the silencer when it left the factory. If your ISO's have degraded, the drive train will not be in the position it was when the motorcycle left the factory. If your ISO's are the ones that your machine left the factory with they have been passengers, not teammates, for years. Start there. Yes, it's a PITA, not particularly inexpensive, but can be construed as important maintenance, very near a "recall" level. Also, if your headers/silencers are replacements they may not have factory specs, and you don't have a lot of wiggle room, nor do yiu need much.
If your ISO's are recent Then you may want to try the following:
You can reduce the length of the silencer to diamond shaped plate stand-off.
and/or
You may be able to loosen the exhaust nut(s) and rotate the header(s) inboard enough to clear.
and/or
You can reduce the width of the 3 spacers that hold the Z plate(s) off the frame to the extent that the Z plate(s) do not bind on the frame and that the nuts that hold the rubber silencer mount(s) do not hit the swingarm while it's articulating.
Best wishes.