IMHO, you should not be seeing a straight-line scratch, you should be seeing a circle a bit larger than the diameter of the adjuster if you have the standard adjusters.
As I said, many here have lots of ideas on this subject. Generally speaking, they will all work but I don't try to out-think the designers or what has worked for 50 plus years. Since Norton and Triumph at least, have similar systems I don't see them as bad.
I recently had a customer head from a well-known expert with race bikes who machined off one side of the rockers and then used spacers much like you to get the rockers dead center over the valves. Two of those rockers were bent so he machined off a lot from them. Two of the spindles were shot. All four valves passed the standard leak test but not mine, so I lapped them until they did not leak at all. I put in a set of used rockers, two new spindles, the standard thrust and spring washers, and the owner can now be confident that the head is how Norton engineered it but with better sealing.