I would replace. And check the others carefully too, inc at the rocker end.
I think the design can fall victim to past mechanics with a lack of cleanliness. Mine had a piece of gasket material sitting squashed in the cam follower cup. I wonder how the PO worried as the tappet clearance opened up as the gasket materiel squashed over time?!
But in general, dirt and detritus can easily fall into the pushrod tunnel on assembly, and sit in the cups, grinding away!
This may have happened to yours. It would seem a shame to 'spoil the ship for a hapeth of tar' having come this far.