Choosing a budget VPS provider for hosting automated trading programs

I've been looking for a cheap but reliable virtual private server (VPS) to run my trading program for the last few months. I ran my QTD program on 3 remote servers in UK and Germany in April for an entire month as performance testing. One in UK on a Xen cloud. One in Germany on OpenVZ. And a last one also in UK on KVM technology. This post is a summary of my initial findings on what to watch for when choosing a server provider for running a remote trading server. Keep in mind that this is for low-end, low-frequency, retail trading. Where latencies in the range of tens of milliseconds are considered good. General target price range is US\$10 - \$20 per month for an unmanaged server with 512MB of ram and 600MHz-equivalent of circa 2006 Intel CPU. Low price is the primary consideration here. If you don't care for the technical details, the top VPS names are Linode, Hetzner, and 6sync for Linux-based VPSs. Linode offers Xen VPSs in US and UK. 6sync offers KVM VPSs in US only. Both companies offer Linux-based 512MB instances for \$20 as a starter server. Hetzner offers KVM VPSs in Germany for €7.90. Cloud servers like Amazon EC2 and Rackspace Cloud are too expensive for running 24/7.

Location, Location, Location

Use a datacenter that's at least in the same sub-continent as your broker's datacenter. You wouldn't want your data to travel half-way around the world just to save a few bucks. It adds unnecessary latency and prices are comparable in either the states or Europe nowadays. Although Asia is another story. If you use more than one broker, you can use multiple servers or pick a server location that sits between them on the internet backbone. TeleGeography provides global internet maps to identify best locations. I trade with Dukascopy and Oanda. One in Switzerland and another in the states. As such, Internet hubs in cities like New York, London, and Frankfurt are prime targets for low latency between both brokers. As I trade more at Dukascopy, I'm biased to European servers. Furthermore, London and Frankfurt are the #1 and #2 internet hub in the world. London has 7,723 Gbps and Frankfurt has 7,218 Gbps capacity in 2010. Whereas New York, 5th in the world, has 3,850 Gbps.

Virtualization Technologies

Virtual private servers are merely reserved resource chunks of a computer in a datacenter. This is achieved through the use of virtualization technologies. And there are quite many of those as I have discovered. Here are three that you'll likely hear about in your search.

  1. OpenVZ. This is the most popular and least preferred platform for trading servers. It's often used by budget hosts because resources between virtualized instances are not well isolated. As such, a host can oversell a server's resource as most webapps have sporadic resource utilization trend. In other worlds, it is unlikely all the virtual private servers would demand their maximum allocated resources at the same time. However, trading servers require a consistent and guaranteed level of computing resource. So stay away from VPS that runs on OpenVZ unless you want to see server hiccups.
  2. Xen. This is what Amazon EC2 and Rackspace Cloud runs on. Xen offers true resource isolation so that you're less likely affected by your virtual server's neighbours. What you see is also what you get. So if you're promised 512MB of memory, you'll get 512MB of memory. However, processing power varies tremendously across different VPS providers. A micro instance on Amazon EC2 offers 613MB of memory, for example. But it is no match for even a 256MB instance on Rackspace Cloud. I was able to run a LXDE desktop GUI plus a java program on Rackspace but couldn't do it on Amazon.
  3. KVM. KVM uses the Linux kernel to virtualize. It's said to offer lower overhead to the host server so that it can provide better value than Xen. Like Xen, it offers true resource isolation.

In summary, either Xen or KVM are good but stay away from OpenVZ.

Operating System

Key question here is: Linux or Windows? If you're unfamiliar with Linux and is not interesting in learning about it, use a Windows Server 2003 or 2008 provider. Windows Server 2003 is preferred as it uses fewer resources just to run so it's cheaper. Do note that Windows VPSs are \$10-\$20 more expensive anyway because you have to pay for a monthly license lease. With the use of a Windows-based VPS, you can connect to your server through the remote desktop protocol (RDP). It's just a matter of running the Remote Desktop application on your local machine to your remote server. Then you'll see the remote machine's desktop on your own machine. And then you can control the remote server just like any other Windows computer. Very simple. However, I prefer to run Linux because of its renowned stability and lower cost. The downside is that you need to know what you're doing to tweak the server for trading. Linux uses very little computing resource to run. I can squeeze JForex and Metatrader together on a Linux VPS with only 256MB memory. By comparison, you need at least 512MB on a Windows box just to get the OS running. And there's no licensing fee for Linux too. A double savings.

Summary

When choosing a VPS provider, consider factors such as location, virtualization technology, and operating system(s) offered. Once you've narrowed your search to a shortlist of providers, the next step is to compare their specific VPS offerings by looking at price, services, and technical specifications.

Dissecting a JForex strategy -- MA_Play.java

Having studied the anatomy of an empty JForex strategy (Part 1 and Part 2), it's time to dissect a working one. MA_Play is the strategy that is included with every JForex API download as a demonstration. You can find the complete source code of this strategy in /src/singlejartest/ in the JForex API zipped package. Recall that the first Interface method which runs at the start of the strategy is onStart. The onStart method of MA_Play is reproduced below.

public void onStart(IContext context) throws JFException { 
    engine = context.getEngine(); 
    indicators = context.getIndicators();
    this.console = context.getConsole();
    console.getOut().println("Started"); 
}

The variables engine, indicators, and console are fields of the MA_Play class. They are global variables within the class. What lines 42--44 do is to save the IEngine, IIndicators, and IConsole objects for later use.

The last line of onStart, line 45, is merely to print a message on your JForex program console to notify the user that the strategy has started.

Once onStart is finished processing, the server is likely to call onTick if a market tick arrives. If it's not during market hours, then there's no tick and some other event might happen instead of onTick. Think of the methods as events rather than a linear process. You program your JForex strategy according to what you want to do with each of the six IStrategy Interface event.

For this particular strategy, the programmer decides to implement their strategy at the tick level. As such, much of the trading algorithm resides in onTick for MA_Play. Note that this is a design choice, you can use onBar if you want your strategy to process at the bar level (or you can use both onTick and onBar).

Here's the source code for onTick in MA_Play.

public void onTick(Instrument instrument, ITick tick) throws JFException {
    if (ma1[instrument.ordinal()] == -1) {
        ma1[instrument.ordinal()] = indicators.ema(instrument, 
            Period.TEN_SECS, 
            OfferSide.BID, 
            IIndicators.AppliedPrice.MEDIAN_PRICE, 
            14, 1);
    }
    double ma0 = indicators.ema(instrument, Period.TEN_SECS, OfferSide.BID, IIndicators.AppliedPrice.MEDIAN_PRICE, 14, 0);
    if (ma0 == 0 || ma1[instrument.ordinal()] == 0) {
        ma1[instrument.ordinal()] = ma0;
        return;
    }

    double diff = (ma1[instrument.ordinal()] - ma0) / (instrument.getPipValue());

    if (positionsTotal(instrument) == 0) {
        if (diff > 1) {
            engine.submitOrder(getLabel(instrument), instrument, IEngine.OrderCommand.SELL, 0.001, 0, 0, tick.getAsk()
                    + instrument.getPipValue() * 10, tick.getAsk() - instrument.getPipValue() * 15);
        }
        if (diff < -1) {
            engine.submitOrder(getLabel(instrument), instrument, IEngine.OrderCommand.BUY, 0.001, 0, 0, tick.getBid()
                    - instrument.getPipValue() * 10, tick.getBid() + instrument.getPipValue() * 15);
        }
    }
    ma1[instrument.ordinal()] = ma0;
}

At a glance, you may notice that the variables ma0 and ma1 play a key role in determining the setup. Hint: To reverse engineer a strategy, it may be easier to work backward from when the order is placed, which is done by engine.submitOrder in this case.

ma0 and ma1 hold results from exponential moving averages (EMA). ma0 is the current value. ma1 is the previous bar's value. Lines 56--63 check using IF tests (lines 56 and 60) to see if either of the variables hold invalid data. If the data is invalid, the indicator is calculated and the rest of the onTick is skipped with the return statement on line 62.

Note: Indicator values can sometimes be invalid (zero, negative, or Double.NaN, depending on the particular indicator implementation) if there is insufficient data to calculate it or an error occurred, for examples.

The EMAs are fetched in lines 57 and 59 using the IIndicators object (which was initialized in onStart). The JForex Wiki provides an explanation of its use.

Notice that ma1 is an array, which was declared in line 38 with a size equivalent to the number of all available JForex instruments. In particular, it is used with a special index value as in ma1[instrument.ordinal()]. In other words, it is asking for the current instrument's slot in the ma1 array. The current instrument is the one that is passed into the method in line 55.

Moving down the code, another point of interest is line 65, showing the use of instrument.getPipValue(). Line 67 checks if the current total number of position is zero. If it is, meaning no opened position, then the strategy proceeds to check the entry signal to enter a trade (lines 68--76).

positionsTotal() is a custom method defined in lines 84--92. It uses a FOR loop to cycle through all the orders obtained from [engine.getOrders(instrument)][]

Once either of the long or short condition, lines 68 and 72, respectively, is met, the strategy submits an order in lines 69 for a short and line 73 for a long. The particulars of submitting market orders is described in the JForex Wiki.

When you stop this strategy, onStop (lines 48--53) is called. For this strategy, the programmer loops through all the orders again using engine.getOrders() and closes each of the position with an [order.close()][] command in line 50.

That is it for this trivial strategy. If there is one point that you should remember. Note my use of the many links to the JForex javadoc and JForex Wiki throughout this post. You are likely to find many of your answers from those two sources. If not, there's always the JForex Support Board. Now that you've had an idea of how MA_Play.java works, it's time to test it. In the next post in January, we will discuss the JForex Historical Tester and what to watch for when running a strategy live.

Anatomy of a JForex strategy, Part 2

We looked at four of the six methods in the IStrategy Interface in a previous post. The last two methods, onTick and onBar, is where your strategy connect with market data. Either one, or both, of these methods is where you put your trading algorithm in. Your strategy would then be able to process the market data as they arrive one tick/bar at a time. Recall that IStrategy Interface is the skeleton of your strategy. And that IContext object is the heart of your strategy. onTick/onBar is the head of your strategy, which contains your trading algorithm, which is the brain.

onTick

Here's the method definition of onTick.

onTick(Instrument instrument, ITick tick) throws JFException { }

Important:onTick is called for each and every instrument that your JForex platform is subscribed to (the instrument list in your workspace box). Let me say that again, onTick is called for each and every instrument that your JForex platform is subscribed to. The standard practice is to filter out ticks for instruments that you don't want with a simple IF-return statement. if (instrument != myInstrument) return; Actual tick data is passed to your strategy using the ITick object from the onTick method's parameter. Take a look at the ITick javadoc entry to see what it offers.

onBar

public void onBar(Instrument instrument, Period period, IBar askBar, IBar bidBar) throws JFException { }

onBar works in a similar manner to onTick. In which onBar is called for each and every subsribed instrument and period known to JForex. Similarly, you have to filter out all the unwanted instruments and periods or else there will be expected results from your strategy. Another point to note is that onBar provides both a IBar askBar and IBar bidBar, representing the ask and bid bars. Question: What happens when two or more periods overlap as in 13:45? 1, 5, and 15-minutes bars are all arriving at the same time (not to mention the periods in seconds too). Answer: According to Dukascopy Support in the forum, "They come in a strict order, for example (1min 1min 1min 1min 1min 5min 1min 1min 1min 1min 1min 5min ..) They come in cycles, where smaller periods comes first."

JForex Support Forum

As you program your strategy with JForex, you will no doubt come up with questions of your own. The best place to ask is at the official JForex Support Forum. This is the last of the three essential JForex resources that I alluded to earlier. Even if you don't have any specific question, there are sample codes, coding discussion, and hundreds of existing Q&A from other JForex developers posted in the forum.

Summary

The discussion so far has been very high level. To show you what you can actually do in an IStrategy, we will dissect a working strategy in the next post. And what else better to examine than the most popular JForex strategy of them all -- MA_Play.java.

Anatomy of a JForex strategy, Part 1

Continuing on from Part 1 of this series: Getting started learning JForex programming, now we're ready to discuss the real thing. You build JForex strategies by using the IStrategy Interface (What is an Interface?). Basically, an Interface is a code skeleton with a set of predefined empty methods that you'll need to implement yourself. The six standard methods of the IStrategy Interface are:

  1. onStart
  2. onStop
  3. onMessage
  4. onAccount
  5. onTick
  6. onBar

Below is an empty IStrategy Interface implementation, also known as a JForex strategy. This code will compile fine in JForex and you can even run it. But it doesn't do anything at all because there is no code to run in each of the methods. Each of the six methods will just be called and exit immediately.

import com.dukascopy.api.\*; 
public class EmptyStrategy implements IStrategy { 
    public void onStart(IContext context) throws JFException { } 
    public void onTick(Instrument instrument, ITick tick) throws JFException { } 
    public void onBar(Instrument instrument, Period period, IBar askBar, IBar bidBar)
        throws JFException { } 
    public void onMessage(IMessage message) throws JFException { } 
    public void onAccount(IAccount account) throws JFException { } 
    public void onStop() throws JFException { } 
}

Each of the method is triggered by a specific event. You can probably guess what they are from their name.

onStart (line 5)

This is the first method that is called when you run your strategy. It will run once and only once at the start of your strategy. Normally you do your initialization in here. The thing to note for onStart is in line 5 of the code. The method signature of onStart is

public void onStart(IContext context) throws JFException

The object in the parameter and given to you in this method is an IContext object. If IStrategy is the skeleton, then IContext is the heart of the strategy. Please take a look at this javadoc link to IContext to see what this object does. Javadoc?: Now is a good time to introduce the second of the three essential resources of a JForex programmer. The JForex Javadoc is the single most up-to-date API documentation explaining each and every object and methods of the JForex API. Think of it like a reference manual. Note that although it's comprehensive, most of the explanation is very sparse and possibly incomplete. IContext is a core JForex object to access many important components of the JForex system, such as the ordering engine, charts, console, indicators... You get the idea. It is important! You typically want to keep a local copy of it as this is the only time (in onStart) that this object will be passed to you in IStrategy.

onStop (line 26)

As the name suggest, this method is called once you send a stop command to your strategy. You do your program wrap-up such as logging and flushing data here. Not much out of the ordinary with this one.

onMessage (line 18)

Whereas we know when onStart and onStop will be called, onMessage is an asynchronous method in that you don't know exactly when it will run. This method is called when the Dukascopy server sends your strategy a message. For example, the server calls onMessage to let you know that your order has been filled. You receive and process the server message by accessing the IMessage object that is passed to you. Important:There is no guarantee that you will receive each and every message sent to your strategy from the server. Perhaps your strategy process is clogged. Or maybe your internet connection had a hiccup. If your strategy onMessage doesn't get called by the server for whatever reason, the server couldn't care less and won't be checking nor trying again. So don't do anything critical like managing your orders in onMessage!

onAccount (line 22)

This method is called whenever your account information update is received. The method provides access to the IAccount object, which you use to get your account information. Say if you have an open position, your account information changes on every tick because your equity is cash + unrealized profit/loss. In that case, onAccount is called every 5 seconds by the server at most to avoid flooding your strategy. More Important:The IAccount object is not connected live to your account in the server. It is merely a snapshot of your account. For example, if you keep a local copy of an IAccount object. Do some trading to change your balance. Then ask the same IAccount for account balance information, you will not see a change. As such, always update your local copy of IAccount within the onAccount method to keep your account information up-to-date for your strategy's use.

To be continued

onStart, onStop, onMessage, and onAccount methods are administrative methods for your strategy. The last two methods that we'll discuss, onTick and onBar, is where the magic happens in a strategy. I am saving the best for last in the next post.

continue   →