How Do You Decide Whether a Trading Strategy Needs a New Filter?

Adding a filter may remove weak trades, but it can also eliminate valid opportunities and make the strategy too selective. How would you test whether a proposed filter improves results beyond the original sample? What safeguards would help prevent adding rules simply to explain past losses?