Encrypting your communications to minimize eavedropping

The case of Galleon Group founder Raj Rajaratnam shows a corporate security crack that shouldn’t happen in this day and age. In the Rajaratnam case, government wiretaps served as vital evidence against him. The question though, is that, if the government is able to do that on a multi-billion dollar company without them knowing, what about their competitors or enemies with ill-intent? With all the resources available to them, how can something so old-school like this happen to a hedge fund? Secrecy and conspiracy shouldn’t be unfamiliar in the hedge fund industry. So I had assumed that they operated inside an iron wall.

Government wiretapping or corporate espionage are certainly not my worry. However, I still try to employ prudent security procedures in the financial quantitative research and development that I do at Quantisan Systems just because I value my intellectual property.

For example, in my email correspondence with business associates, I make use of GPG with a 2048-bit RSA asynchronous key pair. (my public key is below) GPG is an open-source equivalent of Pretty Good Privacy. It is most likely that your email client supports it too. I use Thunderbird and it is a built-in feature. You just need to enable it and set it up.

On voice communications, I prefer to use Skype rather than regular voice calls because it’s free and Skype inherently uses 256-bit AES encryption. So no setup is even required for this added security.

For my important files, I place them in a TrueCrypt volume encrypted with 256-bit AES encryption. The encrypted container is synchronized to my Dropbox account so that I can use it anywhere while ensuring security. TrueCrypt is available on Windows, Mac, and Linux. It’s only a matter of following the installation instructions to safeguard your files from prying eyes.

All of these encryption tools are open-source and free. I configured them to work in the background and they don’t hinder my work flow at all. It is certainly not a sophisticated setup, yet it isn’t too dingy either. At the very least, if someone is to steal my notebook computer, an ordinary crook wouldn’t be able to gain access to my sensitive information.

In view of the fact that Quantisan Systems is just an one-man operation with practically no resource available, I can’t believe security at some financial companies is worse than this homebrew setup.

—–BEGIN PGP PUBLIC KEY BLOCK—–
Version: GnuPG v1.4.11 (GNU/Linux)

mQENBE1Ul0gBCACZGqPoYrzRF9YcuUvMeES+etPUAeNQ6uvw42k7Sf1ydBp3lFGQ
ggYnOsmQ8Nf/62sB9CoYkAV+h+CPTuTOT5N4gwTojtPX3bIrjA6JgTDpt7bpwyMM
j+mnZboXdBgZwz1xqiB38Oj51OBYPPLbTc/YphYfjXEuO4tQwsQufd5dvJqF+Yse
CGeLRroJt9EmGVYW/NaCEBKHaMnD9gkDyWLxm7GFQYhl7ZWU+VQXjBxzZyYtpup2
s1N5/igjiRj8t5CGnMwrG2yZkPW+n8m5sTLjcHWFY9OxiqFRaYgm9Vk3mVhxE5ML
DPtSPySF+KkyIFxeL78MscU2W1DMH5pNM/+fABEBAAG0HVBhdWwgTGFtIDxwYXVs
QHF1YW50aXNhbi5jb20+iQE+BBMBAgAoBQJNVJdIAhsjBQkFo5qABgsJCAcDAgYV
CAIJCgsEFgIDAQIeAQIXgAAKCRD8QV1yPcrlhIphB/9cxkbWJ+19axfZxFYRE1u4
Gq1p5/hkm3990yT8kc+4Qg7xa2cm+SgBmEnAy24xdQpR/Up/EF13D15ZBiE1SG+D
0+EUZp4Ir5+TTuTrmJxKRb56IgFZVXP9KatNuK3aPvNFl4gZUZUZHmvUvvBzqtSM
HrfzjmuirXVKI6M5WPZchLNUGE1sDtR2SR7l+rjvcPTehaJf+9srK1gsl3Vb3Pzo
D5/JBoNK1ueDpfRxNhdryCUQr2uJ1pWLgauDz0iSRPOczV8x1n+051DJQXHx16tY
I0VOErTVxscE/KeuuiiFwOwSLteIMDIY2vg6KuUw3cQxxMcfeNTu+jTJXuroib+O
uQENBE1Ul0gBCAD05TZ2/oF1Lb8ggg9nKlgB+ZXWBVCc9CNlguZpZAmC7ezc72qT
gSho/5+3ldEqvA1AxnWubaWV26QLL2fNybKd1DG9uYOsnOwx6L5H8toQswcurC9y
3jKVQSpuIJrsrQiTT3aIZ3ZONGsg0k0D+ZiMC9qSLmFXh3f2fio5QsyRsEWlC9Qj
CDXrgpNZqpDegMvvRXXJfRGulh4sMTGXrxE2pXI/+9Hv64+BuBMkixA7UwiOBQbd
V5GkVm7V2hvgQpLc2QaXofLdBVIda6hvUuU/BK5m7HShNPKc21VgcSbYFOggYDal
efTeFdCfHEg08ND8JSa2Bv+fdEmOFjPF5Z9hABEBAAGJASUEGAECAA8FAk1Ul0gC
GwwFCQWjmoAACgkQ/EFdcj3K5YTRxAf8DspugdiTZUxIT/1EAa/0GWpsT8YZmITm
0iUebID3jZRLXpyqfDr9KUZvk6IMYru1h6c6pZdzY4IJ/p/sZ+p2gtLr1pmOqEjd
W94Q3adtjH/42zauFOdCzamtbc7Q5cJTIeAFcvFsVqDLWkK1vWiUhqUoacgN6TWS
MActe4B7hYdcMWsbj4j3xVoGkkiJOL81/MJW9f/xDqcNbLsiEK7DxOaWUb2CkuUl
7c/VVaSuWFBKGsj2fgEoTErfsmhjMm+WUcoE1Zjn2UwJzxxIcfzj3A0IEvMIua6u
UJWnKv8t55G9f+NQ3yrKtPC/+O7N1sCSilmB/IekaSrBX9KvZgHM8Q==
=DjK6
—–END PGP PUBLIC KEY BLOCK—–

More trouble with JForex IIndicators

JFUtil offers an elegant way of requesting indicators from the JForex IIndicators interface. But the new design for JForex IIndicators through JFUtil is only fixing one side of the problem. There is still the question of what to do with the multiple return types from the calculations.

In JForex, indicators like RSI returns a one-dimensional array. Other indicators like Stochastic Oscillator returns a two-dimensional array. And some other ones return three or more dimensional arrays. Java doesn’t allow overloading a method with multiple return types, as the program wouldn’t know what to expect. So how can we get rid of coding mess like this in our JForex strategy?

		Stochastic stochBean = IndicatorBeanFactory.getStochastic();
		Object[] objs = Indicating.calculateMultiDimension(instrument, Period.ONE_MIN, stochBean, 1);

		double[][] sto = new double[2][];
		sto[0] = (double[])objs[0];		// %K values
		sto[1] = (double[])objs[1];		// %D values

The default calculation method in JForex returns a generic Object array. It’s up to the programmer to know what to expect from the call and cast the Object into something useful. Obviously, this is a recipe for programming headaches and runtime errors.

Having said this, one of the biggest benefits of JForex is that it uses a standard programming language like Java. So if there’s something that you don’t like about the API, you can probably change it (e.g. Facade pattern). This is the purpose of JFUtil to some extent, to simplify the JForex API behaviour. In any case, I’m sure other Java programmers have faced this problem before and have come up with good solutions for it.

A search on stackoverflow.com doesn’t yield a quick fix solution at first glimpse. My guess is that this require leveraging the knowledge about the program structure. We know in advance what dimension of the calculation result we can expect based on the indicator itself, perhaps I can use something like a Command pattern to choose a calculation sub-routine and then return a Map object with named values? I have yet to try implementing this.

I am open to design suggestions to encapsulate multiple return values through a single interface. So that any indicator bean can use the same calculation interface with an easy to use output.

In the mean time, getting multi-dimensional indicators results through JFUtil 2.1.2 isn’t pretty.

Conjuring beans to simplify JForex IIndicators

The JForex API is not perfect. Like any application programming interface (API), some parts of the JForex API is better designed than others. One of the most gripping problems with JForex is its use of technical analysis (TA) indicators. Here’s an example of using an exponential moving average, one of the most simplest indicators.

ema(instrument, Period.TEN_SECS, OfferSide.BID, IIndicators.AppliedPrice.MEDIAN_PRICE, 14, 0);

And the corresponding javadoc explaining the parameters,
Parameters:
instrument instrument of the bar
period period of the bar
side bid or ask side of the bar
appliedPrice type of input data
timePeriod time period value
shift number of candle back in time staring from current bar. 0 - current bar (currently generated from ticks), 1 - previous bar (last formed bar), 2 - current bar minus 2 bars and so on

It’s not intuitive but it’s usable.

Recall that in my JForex programming tutorial, I suggested using the JForex API javadoc to find out information about the API. However, for the case of IIndicators, if you take a look at its javadoc, you will only be led to more confusion by the hundred or so mysterious indicator methods with cryptic parameter names.

In JForex API, if you want to calculate an indicator, you need to look through a long list of abbreviated method names in the javadoc. Many of which are not easy to decipher. Secondly, you need to figure out the generic parameters and set them correctly, which is different for almost every indicator. Lastly, as there is no standard interface for indicators, you need to hardcode these into your strategy with little flexibility.

In contrast, here’s the same call using JFUtil 2.1.0 to get an EMA value. It has notably more lines of code. It is designed deliberately so using an object oriented approach by encapsulating the indicator parameters into a bean object and abstracting the calculation into a single generic function call.

		// get an EMA indicator value by building an indicator bean
		MovingAverage maBean = IndicatorBeanFactory.getMovingAverage();
		// then sets its parameters with obvious method names
		maBean.setAppliedPrice(IIndicators.AppliedPrice.MEDIAN_PRICE)
				.setMAType(IIndicators.MaType.EMA)
				.setWidth(14);		// all of these are optional parameters
		// feed the bean into a generic calculation method to get the result
		double ema = Indicating.calculate(instrument, Period.ONE_MIN, maBean);

With JFUtil, to calculate an indicator, you create an indicator bean with the intuitive bean name that corresponds with the full name of an indicator. For example, a moving average will be MovingAverage. Then you set the indicator-specific parameters using clearly defined methods with useful javadoc descriptions. One method for one parameter. Lastly, you feed this indicator bean into a generic calculation function to get your value.

It took some thinking to abstract and generalize such a rigidly structured API. I am quite pleased with this new design as the current JFUtil opens up a lot of design flexibility. Such as dynamic indicator selection with genetic algorithm and interchangeable indicators use for runtime adaptive algorithms.

JFUtil 2.0 alpha demonstration

Rather than talk about how much better JFUtil 2.0 is and all, I’ll just show you a demonstration strategy illustrating the features of this JForex utilities library.

Download the latest JFUtil from the project page. As this is an alpha release, I need your help in finding bugs and look for improvements. Leave me a message if you have any comment or suggestion please.

It’s best if you copy and paste the following demo strategy source code into your IDE or text editor for viewing.

Dissecting a JForex strategy — MA_Play.java

Having studied the anatomy of an empty JForex strategy (Part 1 and Part 2), it’s time to dissect a working one. MA_Play is the strategy that is included with every JForex API download as a demonstration. You can find the complete source code of this strategy in /src/singlejartest/ in the JForex API zipped package.

Recall that the first Interface method which runs at the start of the strategy is onStart. The onStart method of MA_Play is reproduced below.

    public void onStart(IContext context) throws JFException {
        engine = context.getEngine();
        indicators = context.getIndicators();
        this.console = context.getConsole();
        console.getOut().println("Started");
    }

The variables engine, indicators, and console are fields of the MA_Play class. They are global variables within the class. What lines 42–44 do is to save the IEngine, IIndicators, and IConsole objects for later use.

The last line of onStart, line 45, is merely to print a message on your JForex program console to notify the user that the strategy has started.

Once onStart is finished processing, the server is likely to call onTick if a market tick arrives. If it’s not during market hours, then there’s no tick and some other event might happen instead of onTick. Think of the methods as events rather than a linear process. You program your JForex strategy according to what you want to do with each of the six IStrategy Interface event.

For this particular strategy, the programmer decides to implement their strategy at the tick level. As such, much of the trading algorithm resides in onTick for MA_Play. Note that this is a design choice, you can use onBar if you want your strategy to process at the bar level (or you can use both onTick and onBar).

Here’s the source code for onTick in MA_Play.

    public void onTick(Instrument instrument, ITick tick) throws JFException {
        if (ma1[instrument.ordinal()] == -1) {
            ma1[instrument.ordinal()] = indicators.ema(instrument, Period.TEN_SECS, OfferSide.BID, IIndicators.AppliedPrice.MEDIAN_PRICE, 14, 1);
        }
        double ma0 = indicators.ema(instrument, Period.TEN_SECS, OfferSide.BID, IIndicators.AppliedPrice.MEDIAN_PRICE, 14, 0);
        if (ma0 == 0 || ma1[instrument.ordinal()] == 0) {
            ma1[instrument.ordinal()] = ma0;
            return;
        }

        double diff = (ma1[instrument.ordinal()] - ma0) / (instrument.getPipValue());

        if (positionsTotal(instrument) == 0) {
            if (diff > 1) {
                engine.submitOrder(getLabel(instrument), instrument, IEngine.OrderCommand.SELL, 0.001, 0, 0, tick.getAsk()
                        + instrument.getPipValue() * 10, tick.getAsk() - instrument.getPipValue() * 15);
            }
            if (diff < -1) {
                engine.submitOrder(getLabel(instrument), instrument, IEngine.OrderCommand.BUY, 0.001, 0, 0, tick.getBid()
                        - instrument.getPipValue() * 10, tick.getBid() + instrument.getPipValue() * 15);
            }
        }
        ma1[instrument.ordinal()] = ma0;
    }

At a glance, you may notice that the variables ma0 and ma1 play a key role in determining the setup. Hint: To reverse engineer a strategy, it may be easier to work backward from when the order is placed, which is done by engine.submitOrder in this case.

ma0 and ma1 hold results from exponential moving averages (EMA). ma0 is the current value. ma1 is the previous bar’s value. Lines 56–63 check using IF tests (lines 56 and 60) to see if either of the variables hold invalid data. If the data is invalid, the indicator is calculated and the rest of the onTick is skipped with the return statement on line 62.

Note: Indicator values can sometimes be invalid (zero, negative, or Double.NaN, depending on the particular indicator implementation) if there is insufficient data to calculate it or an error occurred, for examples.

The EMAs are fetched in lines 57 and 59 using the IIndicators object (which was initialized in onStart). The JForex Wiki provides an explanation of its use.

Notice that ma1 is an array, which was declared in line 38 with a size equivalent to the number of all available JForex instruments. In particular, it is used with a special index value as in ma1[instrument.ordinal()]. In other words, it is asking for the current instrument’s slot in the ma1 array. The current instrument is the one that is passed into the method in line 55.

Moving down the code, another point of interest is line 65, showing the use of instrument.getPipValue().

Line 67 checks if the current total number of position is zero. If it is, meaning no opened position, then the strategy proceeds to check the entry signal to enter a trade (lines 68–76).

positionsTotal() is a custom method defined in lines 84–92. It uses a FOR loop to cycle through all the orders obtained from engine.getOrders(instrument)

Once either of the long or short condition, lines 68 and 72, respectively, is met, the strategy submits an order in lines 69 for a short and line 73 for a long. The particulars of submitting market orders is described in the JForex Wiki.

When you stop this strategy, onStop (lines 48–53) is called. For this strategy, the programmer loops through all the orders again using engine.getOrders() and closes each of the position with an order.close() command in line 50.

That is it for this trivial strategy. If there is one point that you should remember. Note my use of the many links to the JForex javadoc and JForex Wiki throughout this post. You are likely to find many of your answers from those two sources. If not, there’s always the JForex Support Board.

Now that you’ve had an idea of how MA_Play.java works, it’s time to test it. In the next post in January, we will discuss the JForex Historical Tester and what to watch for when running a strategy live.