How do you minimize risks when EURUSD moves 100 pips up and back down in 2 hours?

And this happens quite often.

There are a few choices to minimize your risks in times like these:

1. Stay on the sideline until the dust settled.
2. Target higher time frames with larger stops and smaller positions.
3. Partner up with someone from another timezone.
4. Set a trailing stop.
5. Set alerts to wake you up on extraordinary events.
6. Use a position management bot.

I opted for #6. What about you?

The picture is worse in equities as you are locked in your positions overnight.

read more

It’s an open buffet in a small business

One of the few benefits of working for myself is that I don’t need to worry about compatibility with legacy systems. I am free to use whatever open source tools to get the job done well. The downside to this is that there are so many technologies out there that it’s hard to choose the right ones for the job. To give you a sense of what I meant, here are some of the topics that I have either tried or seriously considered in the past year.

• Programming languages: Java, Python, R, C#, F#, Scala, Clojure, Haskell.
• Data storage: HDF5, CSV, Binary, MySQL, PostgreSQL, MongoDB, MonetDB, Redis, HBase.
• Cloud server: Amazon EC2, Microsoft Azure, Google App Engine, Rackspace Cloud, plain old VPSs.

Programming language choices are important because they sit at the bottom of a technology stack. Most of the work that I do are built on using them. For a long while, I settled on using a combination of Java, Python, and R. I prototype small ideas in Python. Implement production code in Java. And perform analysis in R. I discussed why use the right tool for the right task a year ago.

By the end of my previous project, I am finding that the popular development triplet of Java, Python, and R, is not ideal for a solo-operation. Seeing that I have more time on my hands because I am using QTD to trade for me now, I am taking a break this summer to expand my knowledge and learn new technologies.

Some of the technologies that I am experimenting with includes:

• an in-memory data store for instantaneous and persistent tick data
• parallel programming for concurrent processing with no locks
• mathematically intuitive algorithm implementations using high order functions

Don’t mind me as I help myself in this open buffet of technologies.

read more

Why success starts with failure

Many people dream of making a cash cow trading program and be set for life. The reality of the business, which is just like any other profession, is that you need to be persistent and persevere. You really need to enjoy the work to be able to keep going at it day after day even if nothing that you do seems to work at a time.

This video by Tim Harford explains why success starts with failure. It rings with my previous post on why the secret to trading system development is to fail faster.

Harford’s three principles of success:

1. Fail a lot.
2. Fail affordably.
3. Fail and fix early.

read more

The secret to trading system development is to fail faster

If I am to offer only one advice with regard to developing trading systems then this is it — fail faster.

The sad truth to trading is that there is no magical system that can guarantee profitability indefinitely. A reason top firms like Renaissance Technologies employ hundreds of PhDs and is still actively hiring is because one can never reach the end of the rainbow in trading. A system can always be made better or made obsolete. In fact, you need to stay ahead of the curve as competitions are like vultures that will eat away your game if you don’t keep moving. Not to mention the changing dynamics of the market itself is a moving target. As such, you simply need to keep innovating.

My way of innovating boils down to executing five cyclical steps. Conceptualising, implementing, testing, measuring, and analysing, over and over again. In essence, my innovation methodology is a simulated annealing process that might or might not eventually lead to a breakthrough. I do this because:

1. I am intelligent but I am not a genius. As such, I don’t expect myself to make leaps and bounds with regular sparks of genius. Instead, I take small steps. Adding a 1% yield to your system is extremely difficult. Adding 0.01% is a lot easier. Adding 0.0001% is almost pedantic. So I aim for achieving smaller improvements and do it hundreds of times over a development cycle.
2. My view of the world is not absolute. In fact, I am more often wrong than right. So it follows that most of my ideas don’t have all the pieces right. Thus, it is inherent that I need to polish my ideas through trials and errors.

Failure is inevidentable when developing trading systems. It is part of the process. You come up with an idea, implement it, test it, find out why it isn’t up to your expectation, and then make it better. So the quicker you can fail and learn from it, the quicker you can discover something useful.

read more

Graphs from network monitors on my remote trading server

Here are some server monitoring graphs from my remote trading server. These graphs are produced by the server monitoring application, Munin.

I ran both JForex and Metatrader 4 simultaneously on my server last week. The 512 MB of memory was noticeably insufficient. Which caused the system to use swap files, thus leading to I/O latency, and which held up the CPU. All of which are evident from the graphs below.

This week I am only running JForex. The server load is within tolerable range.

The thing with picking your own VPS is that no VPS provider is the same. You can’t really compare just the numbers as there are just too many factors to be considered behind the scene on a VPS service.

So it is important to run applications that you intend to run and stress your system in real-world conditions. If your system seems slow, figure out where exactly is the bottleneck before throwing away money for upgrades. This is where a server monitoring application comes handy.

read more

Porting JForex QTD to Metatrader 4

I just finished porting Quantised Trading Desk to Metatrader 4. So that I can make use of my dormant trading account at Oanda. Seeing that Oanda has enabled use of Metatrader 4, I thought I might give them a second try.

I’ve always preferred to have more than one active trading accounts to serve as a backup and comparison. I settled on Questrade and TD Waterhouse for stocks and options, after hopping around brokerages for years. And I’ve been using Dukascopy to trade forex after much contention. This is a good chance to diversify as I don’t want to pigeonhole myself into one broker and one platform.

The move to Metatrader 4 up until this point only took a couple of weeks as opposed to JForex version’s year-long development time. This is because the underlying QTD system dynamics and algorithms were ironed out in JForex. I don’t think I can finish QTD if I only had MQL4 at my disposal. Debugging and extending MQL4 scripts are a nightmare to say the least. I am so glad that I moved away from these “easy” languages.

For the MQL4 version of QTD, I am merely doing a direct source code port from JForex to MQL4. MQL4, if anything else, is a very easy scripting language. I just taped out an alpha version of MT4 QTD this weekend as it is up to a stage where it is functional.

However, porting QTD to MQL4 isn’t without its difficulties. For one, MQL4 is tailored for simple trading with technical analysis indicators. If price move above moving average, go long, for example. Whereas QTD rely on statistics and numerical methods in its trading algorithm. As such, I had to go back to basics and define all of my numerical methods from the fundamentals.

I am now moving on to do forward testing on the MT4 QTD to ensure that both the JForex and the MT4 versions work similarly. I can’t wait til I can run QTD on Dukascopy and Oanda simultaneously.

read more