Should Traders Separate Strategy Development From Strategy Execution?

Changing rules while actively trading can make it difficult to know whether results came from the original method or the modifications. How do you separate experimentation from normal execution so that your performance records remain meaningful?