I have crossed that line many a times.
Having 6 Heritages built from 2006 through 2013, on all I have replace with RS pots and caps. Faber bridges and inserts on the Prospect, H150 and Millie. Stoptails were all upgraded as was the bridge on my 575.
Factory bridges, pots and caps were purchased in bulk and neither Gibson or Heritage went for premium grade. Most players were OK or upgraded them. Are you "most players?" This is not like adding truck nuts under the rear license plates, if it plays and sounds better, it IS better.