Encrypting your communications to minimize eavedropping

The case of Galleon Group founder Raj Rajaratnam shows a corporate security crack that shouldn’t happen in this day and age. In the Rajaratnam case, government wiretaps served as vital evidence against him. The question though, is that, if the government is able to do that on a multi-billion dollar company without them knowing, what about their competitors or enemies with ill-intent? With all the resources available to them, how can something so old-school like this happen to a hedge fund? Secrecy and conspiracy shouldn’t be unfamiliar in the hedge fund industry. So I had assumed that they operated inside an iron wall.

Government wiretapping or corporate espionage are certainly not my worry. However, I still try to employ prudent security procedures in the financial quantitative research and development that I do at Quantisan Systems just because I value my intellectual property.

For example, in my email correspondence with business associates, I make use of GPG with a 2048-bit RSA asynchronous key pair. (my public key is below) GPG is an open-source equivalent of Pretty Good Privacy. It is most likely that your email client supports it too. I use Thunderbird and it is a built-in feature. You just need to enable it and set it up.

On voice communications, I prefer to use Skype rather than regular voice calls because it’s free and Skype inherently uses 256-bit AES encryption. So no setup is even required for this added security.

For my important files, I place them in a TrueCrypt volume encrypted with 256-bit AES encryption. The encrypted container is synchronized to my Dropbox account so that I can use it anywhere while ensuring security. TrueCrypt is available on Windows, Mac, and Linux. It’s only a matter of following the installation instructions to safeguard your files from prying eyes.

All of these encryption tools are open-source and free. I configured them to work in the background and they don’t hinder my work flow at all. It is certainly not a sophisticated setup, yet it isn’t too dingy either. At the very least, if someone is to steal my notebook computer, an ordinary crook wouldn’t be able to gain access to my sensitive information.

In view of the fact that Quantisan Systems is just an one-man operation with practically no resource available, I can’t believe security at some financial companies is worse than this homebrew setup.

—–BEGIN PGP PUBLIC KEY BLOCK—–
Version: GnuPG v1.4.11 (GNU/Linux)

mQENBE1Ul0gBCACZGqPoYrzRF9YcuUvMeES+etPUAeNQ6uvw42k7Sf1ydBp3lFGQ
ggYnOsmQ8Nf/62sB9CoYkAV+h+CPTuTOT5N4gwTojtPX3bIrjA6JgTDpt7bpwyMM
j+mnZboXdBgZwz1xqiB38Oj51OBYPPLbTc/YphYfjXEuO4tQwsQufd5dvJqF+Yse
CGeLRroJt9EmGVYW/NaCEBKHaMnD9gkDyWLxm7GFQYhl7ZWU+VQXjBxzZyYtpup2
s1N5/igjiRj8t5CGnMwrG2yZkPW+n8m5sTLjcHWFY9OxiqFRaYgm9Vk3mVhxE5ML
DPtSPySF+KkyIFxeL78MscU2W1DMH5pNM/+fABEBAAG0HVBhdWwgTGFtIDxwYXVs
QHF1YW50aXNhbi5jb20+iQE+BBMBAgAoBQJNVJdIAhsjBQkFo5qABgsJCAcDAgYV
CAIJCgsEFgIDAQIeAQIXgAAKCRD8QV1yPcrlhIphB/9cxkbWJ+19axfZxFYRE1u4
Gq1p5/hkm3990yT8kc+4Qg7xa2cm+SgBmEnAy24xdQpR/Up/EF13D15ZBiE1SG+D
0+EUZp4Ir5+TTuTrmJxKRb56IgFZVXP9KatNuK3aPvNFl4gZUZUZHmvUvvBzqtSM
HrfzjmuirXVKI6M5WPZchLNUGE1sDtR2SR7l+rjvcPTehaJf+9srK1gsl3Vb3Pzo
D5/JBoNK1ueDpfRxNhdryCUQr2uJ1pWLgauDz0iSRPOczV8x1n+051DJQXHx16tY
I0VOErTVxscE/KeuuiiFwOwSLteIMDIY2vg6KuUw3cQxxMcfeNTu+jTJXuroib+O
uQENBE1Ul0gBCAD05TZ2/oF1Lb8ggg9nKlgB+ZXWBVCc9CNlguZpZAmC7ezc72qT
gSho/5+3ldEqvA1AxnWubaWV26QLL2fNybKd1DG9uYOsnOwx6L5H8toQswcurC9y
3jKVQSpuIJrsrQiTT3aIZ3ZONGsg0k0D+ZiMC9qSLmFXh3f2fio5QsyRsEWlC9Qj
CDXrgpNZqpDegMvvRXXJfRGulh4sMTGXrxE2pXI/+9Hv64+BuBMkixA7UwiOBQbd
V5GkVm7V2hvgQpLc2QaXofLdBVIda6hvUuU/BK5m7HShNPKc21VgcSbYFOggYDal
efTeFdCfHEg08ND8JSa2Bv+fdEmOFjPF5Z9hABEBAAGJASUEGAECAA8FAk1Ul0gC
GwwFCQWjmoAACgkQ/EFdcj3K5YTRxAf8DspugdiTZUxIT/1EAa/0GWpsT8YZmITm
0iUebID3jZRLXpyqfDr9KUZvk6IMYru1h6c6pZdzY4IJ/p/sZ+p2gtLr1pmOqEjd
W94Q3adtjH/42zauFOdCzamtbc7Q5cJTIeAFcvFsVqDLWkK1vWiUhqUoacgN6TWS
MActe4B7hYdcMWsbj4j3xVoGkkiJOL81/MJW9f/xDqcNbLsiEK7DxOaWUb2CkuUl
7c/VVaSuWFBKGsj2fgEoTErfsmhjMm+WUcoE1Zjn2UwJzxxIcfzj3A0IEvMIua6u
UJWnKv8t55G9f+NQ3yrKtPC/+O7N1sCSilmB/IekaSrBX9KvZgHM8Q==
=DjK6
—–END PGP PUBLIC KEY BLOCK—–

Choosing a budget VPS provider for hosting automated trading programs

I’ve been looking for a cheap but reliable virtual private server (VPS) to run my trading program for the last few months. I ran my QTD program on 3 remote servers in UK and Germany in April for an entire month as performance testing. One in UK on a Xen cloud. One in Germany on OpenVZ. And a last one also in UK on KVM technology. This post is a summary of my initial findings on what to watch for when choosing a server provider for running a remote trading server.

Keep in mind that this is for low-end, low-frequency, retail trading. Where latencies in the range of tens of milliseconds are considered good. General target price range is US$10 – $20 per month for an unmanaged server with 512MB of ram and 600MHz-equivalent of circa 2006 Intel CPU. Low price is the primary consideration here.

If you don’t care for the technical details, the top VPS names are Linode, Hetzner, and 6sync for Linux-based VPSs. Linode offers Xen VPSs in US and UK. 6sync offers KVM VPSs in US only. Both companies offer Linux-based 512MB instances for $20 as a starter server. Hetzner offers KVM VPSs in Germany for €7.90. Cloud servers like Amazon EC2 and Rackspace Cloud are too expensive for running 24/7.

Location, Location, Location

Use a datacenter that’s at least in the same sub-continent as your broker’s datacenter. You wouldn’t want your data to travel half-way around the world just to save a few bucks. It adds unnecessary latency and prices are comparable in either the states or Europe nowadays. Although Asia is another story.

If you use more than one broker, you can use multiple servers or pick a server location that sits between them on the internet backbone. TeleGeography provides global internet maps to identify best locations.

I trade with Dukascopy and Oanda. One in Switzerland and another in the states. As such, Internet hubs in cities like New York, London, and Frankfurt are prime targets for low latency between both brokers. As I trade more at Dukascopy, I’m biased to European servers.

Furthermore, London and Frankfurt are the #1 and #2 internet hub in the world. London has 7,723 Gbps and Frankfurt has 7,218 Gbps capacity in 2010. Whereas New York, 5th in the world, has 3,850 Gbps.

Virtualization Technologies

Virtual private servers are merely reserved resource chunks of a computer in a datacenter. This is achieved through the use of virtualization technologies. And there are quite many of those as I have discovered. Here are three that you’ll likely hear about in your search.

1. OpenVZ. This is the most popular and least preferred platform for trading servers. It’s often used by budget hosts because resources between virtualized instances are not well isolated. As such, a host can oversell a server’s resource as most webapps have sporadic resource utilization trend. In other worlds, it is unlikely all the virtual private servers would demand their maximum allocated resources at the same time. However, trading servers require a consistent and guaranteed level of computing resource. So stay away from VPS that runs on OpenVZ unless you want to see server hiccups.
2. Xen. This is what Amazon EC2 and Rackspace Cloud runs on. Xen offers true resource isolation so that you’re less likely affected by your virtual server’s neighbours. What you see is also what you get. So if you’re promised 512MB of memory, you’ll get 512MB of memory. However, processing power varies tremendously across different VPS providers. A micro instance on Amazon EC2 offers 613MB of memory, for example. But it is no match for even a 256MB instance on Rackspace Cloud. I was able to run a LXDE desktop GUI plus a java program on Rackspace but couldn’t do it on Amazon.
3. KVM. KVM uses the Linux kernel to virtualize. It’s said to offer lower overhead to the host server so that it can provide better value than Xen. Like Xen, it offers true resource isolation.

In summary, either Xen or KVM are good but stay away from OpenVZ.

Operating System

Key question here is: Linux or Windows?

If you’re unfamiliar with Linux and is not interesting in learning about it, use a Windows Server 2003 or 2008 provider. Windows Server 2003 is preferred as it uses fewer resources just to run so it’s cheaper. Do note that Windows VPSs are $10-$20 more expensive anyway because you have to pay for a monthly license lease.

With the use of a Windows-based VPS, you can connect to your server through the remote desktop protocol (RDP). It’s just a matter of running the Remote Desktop application on your local machine to your remote server. Then you’ll see the remote machine’s desktop on your own machine. And then you can control the remote server just like any other Windows computer. Very simple.

However, I prefer to run Linux because of its renowned stability and lower cost. The downside is that you need to know what you’re doing to tweak the server for trading. Linux uses very little computing resource to run. I can squeeze JForex and Metatrader together on a Linux VPS with only 256MB memory. By comparison, you need at least 512MB on a Windows box just to get the OS running. And there’s no licensing fee for Linux too. A double savings.

Summary

When choosing a VPS provider, consider factors such as location, virtualization technology, and operating system(s) offered. Once you’ve narrowed your search to a shortlist of providers, the next step is to compare their specific VPS offerings by looking at price, services, and technical specifications.

Software design, trading development process, and Ikea

I spent a few months between 2010 and 2011 not on the market, not on coming up with new strategies, but on developing a software framework for my trading system. For my trading strategy development, I make use of a continuous systems development life cycle process from the engineering realm. Starting from planning, to analysis, to design, to implementation, to maintenance, and finally, back to planning, and so on for each and every new concept that I have. (see Sir James Dyson’s guest column on Wired)

Imagine that if you need to write a new strategy file for each and every idea and for each of its design iteration. Pretty soon you’ll have many strategy files and a lot of boilerplate codes. What if you found a bug or figured out some enhancements to a particular component of your strategy? Then you’ll have to sift through all those files and make the changes to all those relevant codes to make the update across the board.

Hopefully, this isn’t the approach you’re taking. Yet, a few trading API that I have used before inherently encourage or even limit you (EasyLanguage and MQL4) to this archaic procedural programming paradigm.

Luckily, Java is an object-oriented programming language. However, it is only as object-oriented as you make it to be. JForex, the trading API that I use, actually runs on the edge of breaking this object-orientedness as I have recently lamented.

The bad news for the beginning developer is that practically every published strategy which I have seen or guilty of posting myself are illustrations of what not to do with software design. In that there’s none in it whatsoever. Think of common published strategies like the Ikea mini-model showrooms. Everything is crammed into a tiny space but it is a simple way to convey the gist of a room design (read: the trading algorithm). However, they are not built for practical use. Real strategies are like suites in an apartment building. There is an underlying architectural commonality for easy management and maintenance while enabling diversity in the strategies.

My recently completed software framework dramatically made my life easier in the tasks of implementation and maintenance. Software maintenance, in particular, is most often needlessly and overly complicated in trading strategy development because of the lack of a good design. Because at the end of the day, most programmers spend the majority of their time debugging, maintaining, and extending their code. A true object-oriented design decouples all its components so that you can use a divide and conquer approach with no repetition of work.

Do you have a development process in place? I’d like to hear how you tackle this problem.

More trouble with JForex IIndicators

JFUtil offers an elegant way of requesting indicators from the JForex IIndicators interface. But the new design for JForex IIndicators through JFUtil is only fixing one side of the problem. There is still the question of what to do with the multiple return types from the calculations.

In JForex, indicators like RSI returns a one-dimensional array. Other indicators like Stochastic Oscillator returns a two-dimensional array. And some other ones return three or more dimensional arrays. Java doesn’t allow overloading a method with multiple return types, as the program wouldn’t know what to expect. So how can we get rid of coding mess like this in our JForex strategy?

		Stochastic stochBean = IndicatorBeanFactory.getStochastic();
		Object[] objs = Indicating.calculateMultiDimension(instrument, Period.ONE_MIN, stochBean, 1);

		double[][] sto = new double[2][];
		sto[0] = (double[])objs[0];		// %K values
		sto[1] = (double[])objs[1];		// %D values

The default calculation method in JForex returns a generic Object array. It’s up to the programmer to know what to expect from the call and cast the Object into something useful. Obviously, this is a recipe for programming headaches and runtime errors.

Having said this, one of the biggest benefits of JForex is that it uses a standard programming language like Java. So if there’s something that you don’t like about the API, you can probably change it (e.g. Facade pattern). This is the purpose of JFUtil to some extent, to simplify the JForex API behaviour. In any case, I’m sure other Java programmers have faced this problem before and have come up with good solutions for it.

A search on stackoverflow.com doesn’t yield a quick fix solution at first glimpse. My guess is that this require leveraging the knowledge about the program structure. We know in advance what dimension of the calculation result we can expect based on the indicator itself, perhaps I can use something like a Command pattern to choose a calculation sub-routine and then return a Map object with named values? I have yet to try implementing this.

I am open to design suggestions to encapsulate multiple return values through a single interface. So that any indicator bean can use the same calculation interface with an easy to use output.

In the mean time, getting multi-dimensional indicators results through JFUtil 2.1.2 isn’t pretty.

Conjuring beans to simplify JForex IIndicators

The JForex API is not perfect. Like any application programming interface (API), some parts of the JForex API is better designed than others. One of the most gripping problems with JForex is its use of technical analysis (TA) indicators. Here’s an example of using an exponential moving average, one of the most simplest indicators.

ema(instrument, Period.TEN_SECS, OfferSide.BID, IIndicators.AppliedPrice.MEDIAN_PRICE, 14, 0);

And the corresponding javadoc explaining the parameters,
Parameters:
instrument instrument of the bar
period period of the bar
side bid or ask side of the bar
appliedPrice type of input data
timePeriod time period value
shift number of candle back in time staring from current bar. 0 - current bar (currently generated from ticks), 1 - previous bar (last formed bar), 2 - current bar minus 2 bars and so on

It’s not intuitive but it’s usable.

Recall that in my JForex programming tutorial, I suggested using the JForex API javadoc to find out information about the API. However, for the case of IIndicators, if you take a look at its javadoc, you will only be led to more confusion by the hundred or so mysterious indicator methods with cryptic parameter names.

In JForex API, if you want to calculate an indicator, you need to look through a long list of abbreviated method names in the javadoc. Many of which are not easy to decipher. Secondly, you need to figure out the generic parameters and set them correctly, which is different for almost every indicator. Lastly, as there is no standard interface for indicators, you need to hardcode these into your strategy with little flexibility.

In contrast, here’s the same call using JFUtil 2.1.0 to get an EMA value. It has notably more lines of code. It is designed deliberately so using an object oriented approach by encapsulating the indicator parameters into a bean object and abstracting the calculation into a single generic function call.

		// get an EMA indicator value by building an indicator bean
		MovingAverage maBean = IndicatorBeanFactory.getMovingAverage();
		// then sets its parameters with obvious method names
		maBean.setAppliedPrice(IIndicators.AppliedPrice.MEDIAN_PRICE)
				.setMAType(IIndicators.MaType.EMA)
				.setWidth(14);		// all of these are optional parameters
		// feed the bean into a generic calculation method to get the result
		double ema = Indicating.calculate(instrument, Period.ONE_MIN, maBean);

With JFUtil, to calculate an indicator, you create an indicator bean with the intuitive bean name that corresponds with the full name of an indicator. For example, a moving average will be MovingAverage. Then you set the indicator-specific parameters using clearly defined methods with useful javadoc descriptions. One method for one parameter. Lastly, you feed this indicator bean into a generic calculation function to get your value.

It took some thinking to abstract and generalize such a rigidly structured API. I am quite pleased with this new design as the current JFUtil opens up a lot of design flexibility. Such as dynamic indicator selection with genetic algorithm and interchangeable indicators use for runtime adaptive algorithms.