Why success starts with failure

Many people dream of making a cash cow trading program and be set for life. The reality of the business, which is just like any other profession, is that you need to be persistent and persevere. You really need to enjoy the work to be able to keep going at it day after day even if nothing that you do seems to work at a time.

This video by Tim Harford explains why success starts with failure. It rings with my previous post on why the secret to trading system development is to fail faster.

Harford’s three principles of success:

1. Fail a lot.
2. Fail affordably.
3. Fail and fix early.

read more

The secret to trading system development is to fail faster

If I am to offer only one advice with regard to developing trading systems then this is it — fail faster.

The sad truth to trading is that there is no magical system that can guarantee profitability indefinitely. A reason top firms like Renaissance Technologies employ hundreds of PhDs and is still actively hiring is because one can never reach the end of the rainbow in trading. A system can always be made better or made obsolete. In fact, you need to stay ahead of the curve as competitions are like vultures that will eat away your game if you don’t keep moving. Not to mention the changing dynamics of the market itself is a moving target. As such, you simply need to keep innovating.

My way of innovating boils down to executing five cyclical steps. Conceptualising, implementing, testing, measuring, and analysing, over and over again. In essence, my innovation methodology is a simulated annealing process that might or might not eventually lead to a breakthrough. I do this because:

1. I am intelligent but I am not a genius. As such, I don’t expect myself to make leaps and bounds with regular sparks of genius. Instead, I take small steps. Adding a 1% yield to your system is extremely difficult. Adding 0.01% is a lot easier. Adding 0.0001% is almost pedantic. So I aim for achieving smaller improvements and do it hundreds of times over a development cycle.
2. My view of the world is not absolute. In fact, I am more often wrong than right. So it follows that most of my ideas don’t have all the pieces right. Thus, it is inherent that I need to polish my ideas through trials and errors.

Failure is inevidentable when developing trading systems. It is part of the process. You come up with an idea, implement it, test it, find out why it isn’t up to your expectation, and then make it better. So the quicker you can fail and learn from it, the quicker you can discover something useful.

read more

Graphs from network monitors on my remote trading server

Here are some server monitoring graphs from my remote trading server. These graphs are produced by the server monitoring application, Munin.

I ran both JForex and Metatrader 4 simultaneously on my server last week. The 512 MB of memory was noticeably insufficient. Which caused the system to use swap files, thus leading to I/O latency, and which held up the CPU. All of which are evident from the graphs below.

This week I am only running JForex. The server load is within tolerable range.

The thing with picking your own VPS is that no VPS provider is the same. You can’t really compare just the numbers as there are just too many factors to be considered behind the scene on a VPS service.

So it is important to run applications that you intend to run and stress your system in real-world conditions. If your system seems slow, figure out where exactly is the bottleneck before throwing away money for upgrades. This is where a server monitoring application comes handy.

read more

Porting JForex QTD to Metatrader 4

I just finished porting Quantised Trading Desk to Metatrader 4. So that I can make use of my dormant trading account at Oanda. Seeing that Oanda has enabled use of Metatrader 4, I thought I might give them a second try.

I’ve always preferred to have more than one active trading accounts to serve as a backup and comparison. I settled on Questrade and TD Waterhouse for stocks and options, after hopping around brokerages for years. And I’ve been using Dukascopy to trade forex after much contention. This is a good chance to diversify as I don’t want to pigeonhole myself into one broker and one platform.

The move to Metatrader 4 up until this point only took a couple of weeks as opposed to JForex version’s year-long development time. This is because the underlying QTD system dynamics and algorithms were ironed out in JForex. I don’t think I can finish QTD if I only had MQL4 at my disposal. Debugging and extending MQL4 scripts are a nightmare to say the least. I am so glad that I moved away from these “easy” languages.

For the MQL4 version of QTD, I am merely doing a direct source code port from JForex to MQL4. MQL4, if anything else, is a very easy scripting language. I just taped out an alpha version of MT4 QTD this weekend as it is up to a stage where it is functional.

However, porting QTD to MQL4 isn’t without its difficulties. For one, MQL4 is tailored for simple trading with technical analysis indicators. If price move above moving average, go long, for example. Whereas QTD rely on statistics and numerical methods in its trading algorithm. As such, I had to go back to basics and define all of my numerical methods from the fundamentals.

I am now moving on to do forward testing on the MT4 QTD to ensure that both the JForex and the MT4 versions work similarly. I can’t wait til I can run QTD on Dukascopy and Oanda simultaneously.

read more

Encrypting your communications to minimize eavedropping

The case of Galleon Group founder Raj Rajaratnam shows a corporate security crack that shouldn’t happen in this day and age. In the Rajaratnam case, government wiretaps served as vital evidence against him. The question though, is that, if the government is able to do that on a multi-billion dollar company without them knowing, what about their competitors or enemies with ill-intent? With all the resources available to them, how can something so old-school like this happen to a hedge fund? Secrecy and conspiracy shouldn’t be unfamiliar in the hedge fund industry. So I had assumed that they operated inside an iron wall.

Government wiretapping or corporate espionage are certainly not my worry. However, I still try to employ prudent security procedures in the financial quantitative research and development that I do at Quantisan Systems just because I value my intellectual property.

For example, in my email correspondence with business associates, I make use of GPG with a 2048-bit RSA asynchronous key pair. (my public key is below) GPG is an open-source equivalent of Pretty Good Privacy. It is most likely that your email client supports it too. I use Thunderbird and it is a built-in feature. You just need to enable it and set it up.

On voice communications, I prefer to use Skype rather than regular voice calls because it’s free and Skype inherently uses 256-bit AES encryption. So no setup is even required for this added security.

For my important files, I place them in a TrueCrypt volume encrypted with 256-bit AES encryption. The encrypted container is synchronized to my Dropbox account so that I can use it anywhere while ensuring security. TrueCrypt is available on Windows, Mac, and Linux. It’s only a matter of following the installation instructions to safeguard your files from prying eyes.

All of these encryption tools are open-source and free. I configured them to work in the background and they don’t hinder my work flow at all. It is certainly not a sophisticated setup, yet it isn’t too dingy either. At the very least, if someone is to steal my notebook computer, an ordinary crook wouldn’t be able to gain access to my sensitive information.

In view of the fact that Quantisan Systems is just an one-man operation with practically no resource available, I can’t believe security at some financial companies is worse than this homebrew setup.

—–BEGIN PGP PUBLIC KEY BLOCK—–
Version: GnuPG v1.4.11 (GNU/Linux)

mQENBE1Ul0gBCACZGqPoYrzRF9YcuUvMeES+etPUAeNQ6uvw42k7Sf1ydBp3lFGQ
ggYnOsmQ8Nf/62sB9CoYkAV+h+CPTuTOT5N4gwTojtPX3bIrjA6JgTDpt7bpwyMM
j+mnZboXdBgZwz1xqiB38Oj51OBYPPLbTc/YphYfjXEuO4tQwsQufd5dvJqF+Yse
CGeLRroJt9EmGVYW/NaCEBKHaMnD9gkDyWLxm7GFQYhl7ZWU+VQXjBxzZyYtpup2
s1N5/igjiRj8t5CGnMwrG2yZkPW+n8m5sTLjcHWFY9OxiqFRaYgm9Vk3mVhxE5ML
DPtSPySF+KkyIFxeL78MscU2W1DMH5pNM/+fABEBAAG0HVBhdWwgTGFtIDxwYXVs
QHF1YW50aXNhbi5jb20+iQE+BBMBAgAoBQJNVJdIAhsjBQkFo5qABgsJCAcDAgYV
CAIJCgsEFgIDAQIeAQIXgAAKCRD8QV1yPcrlhIphB/9cxkbWJ+19axfZxFYRE1u4
Gq1p5/hkm3990yT8kc+4Qg7xa2cm+SgBmEnAy24xdQpR/Up/EF13D15ZBiE1SG+D
0+EUZp4Ir5+TTuTrmJxKRb56IgFZVXP9KatNuK3aPvNFl4gZUZUZHmvUvvBzqtSM
HrfzjmuirXVKI6M5WPZchLNUGE1sDtR2SR7l+rjvcPTehaJf+9srK1gsl3Vb3Pzo
D5/JBoNK1ueDpfRxNhdryCUQr2uJ1pWLgauDz0iSRPOczV8x1n+051DJQXHx16tY
I0VOErTVxscE/KeuuiiFwOwSLteIMDIY2vg6KuUw3cQxxMcfeNTu+jTJXuroib+O
uQENBE1Ul0gBCAD05TZ2/oF1Lb8ggg9nKlgB+ZXWBVCc9CNlguZpZAmC7ezc72qT
gSho/5+3ldEqvA1AxnWubaWV26QLL2fNybKd1DG9uYOsnOwx6L5H8toQswcurC9y
3jKVQSpuIJrsrQiTT3aIZ3ZONGsg0k0D+ZiMC9qSLmFXh3f2fio5QsyRsEWlC9Qj
CDXrgpNZqpDegMvvRXXJfRGulh4sMTGXrxE2pXI/+9Hv64+BuBMkixA7UwiOBQbd
V5GkVm7V2hvgQpLc2QaXofLdBVIda6hvUuU/BK5m7HShNPKc21VgcSbYFOggYDal
efTeFdCfHEg08ND8JSa2Bv+fdEmOFjPF5Z9hABEBAAGJASUEGAECAA8FAk1Ul0gC
GwwFCQWjmoAACgkQ/EFdcj3K5YTRxAf8DspugdiTZUxIT/1EAa/0GWpsT8YZmITm
0iUebID3jZRLXpyqfDr9KUZvk6IMYru1h6c6pZdzY4IJ/p/sZ+p2gtLr1pmOqEjd
W94Q3adtjH/42zauFOdCzamtbc7Q5cJTIeAFcvFsVqDLWkK1vWiUhqUoacgN6TWS
MActe4B7hYdcMWsbj4j3xVoGkkiJOL81/MJW9f/xDqcNbLsiEK7DxOaWUb2CkuUl
7c/VVaSuWFBKGsj2fgEoTErfsmhjMm+WUcoE1Zjn2UwJzxxIcfzj3A0IEvMIua6u
UJWnKv8t55G9f+NQ3yrKtPC/+O7N1sCSilmB/IekaSrBX9KvZgHM8Q==
=DjK6
—–END PGP PUBLIC KEY BLOCK—–

read more

Bearish USDJPY on 19th week of 2011

I am going bearish on USDJPY this week using my QTD system. No position is opened at this point. I am merely entering my sentiment into the system to trade for me. Figure 1 is a daily chart of USDJPY. Notice it’s lack of life despite the USD making headlines last week. Figure 2 shows the move with a US dollar index, USDX, daily chart. Furthermore, according to the order book on USDJPY, there are massive bullish positions in the red (at a loss) which make this pair ripe for some bull squeeze.

I am going short on NZDUSD as a hedge on this move. It is my favourite long USD pair these days.

read more