The secret to trading system development is to fail faster

If I am to offer only one advice with regard to developing trading systems then this is it -- fail faster. The sad truth to trading is that there is no magical system that can guarantee profitability indefinitely. A reason top firms like Renaissance Technologies employ hundreds of PhDs and is still actively hiring is because one can never reach the end of the rainbow in trading. A system can always be made better or made obsolete. In fact, you need to stay ahead of the curve as competitions are like vultures that will eat away your game if you don't keep moving. Not to mention the changing dynamics of the market itself is a moving target. As such, you simply need to keep innovating. My way of innovating boils down to executing five cyclical steps. Conceptualising, implementing, testing, measuring, and analysing, over and over again. In essence, my innovation methodology is a simulated annealing process that might or might not eventually lead to a breakthrough. I do this because:

  1. I am intelligent but I am not a genius. As such, I don't expect myself to make leaps and bounds with regular sparks of genius. Instead, I take small steps. Adding a 1% yield to your system is extremely difficult. Adding 0.01% is a lot easier. Adding 0.0001% is almost pedantic. So I aim for achieving smaller improvements and do it hundreds of times over a development cycle.
  2. My view of the world is not absolute. In fact, I am more often wrong than right. So it follows that most of my ideas don't have all the pieces right. Thus, it is inherent that I need to polish my ideas through trials and errors.

Failure is inevidentable when developing trading systems. It is part of the process. You come up with an idea, implement it, test it, find out why it isn't up to your expectation, and then make it better. So the quicker you can fail and learn from it, the quicker you can discover something useful.