The fact that this slight change in study design yielded opp...

The fact that this slight change in study design yielded opposite results suggests the results aren’t reliable. If you tweak your design parameters slightly, without a clear hypothesis-driven reason to do so, and the whole thing falls apart, that suggests your results may not be real but rather are just statistical noise.

www.joshbeckman.org/notes/518327319