Any hi stress parts that contact other hi stress parts in precision applications will eventually become "matched sets" after any significant run in period. The wheel bearings on your car, the outer race, the inner race and the roller or ball bearing cage will become a "set" after sufficient milage. This is because no 2 things are alike (when you get picky). The hub and the axle the bearing set interfaces with will have a different proximity to the hub and axle on the other side. The 3 piece bearings are a perfect fit to each other when new but once you mount the inner race on the axle and the outer race in the hub and sandwich the bearing cage in-between they no longer contact each other so perfectly. Looking closely at the wear patterns on the contact surfaces will reveal the distinct wear patterns. The bearing set when new was extremely tightly manufactured to exacting tolerances and all the inner, outer races and the ball or rollers cages are interchangable until they have been run in and formed wear patterns. After that, installing a single new component in the "set" will result in faster wear on the new component and other components until the 3 parts wear in together again.
Some parts on a Norton that become sets are:
rocker spindles - rocker arms
lifter - cam lobe
any 2 or 3 piece bearings (one piece bearings always remain as a set)
Components like quick change cam fixtures are designed for race bikes because these components are replaced before wear patterns form. If you start with new or reground lifters and cams and only run a dozen races and change to another new or reground cam there won't be much of any wear pattern started yet. Racers frequently access these parts and can change or repair when needed so they have no need to concern themselves on the longevity so much as benefits of the quick changeability. On a road bike if you change cams after a season where you've logged 5000 miles or so, wear patterns will have formed and will be visible on the parts. Mixing other parts with different wear patterns will always result in quicker wearing of those parts however nothing catastrophic is going to occur, just faster wear of the parts and more frequent adjustments of rocker clearances etc.