Friday, 6 June 2008

Objectives For an Exchange Engine

First off the bat:
  • Seamless fail-over - no loss, no duplication
  • High Availability - self-repairing, fault-tolerant and on-the-fly software upgrates
  • Low Latency
  • Linear Scaleability
  • FIX/FAST datafeed containing order book, trades and statistics
The first area I am focusing on is the order book and matching engine. I am investigating what role persistence will play in achieving no loss, no duplicate and seamless fail-over in a low-latency environment linearly scaling to potentially millions of instruments. This will flush out many of the issues to be addressed.

Why Erlang?

As you may know, Erlang/OTP is a highly concurrent, robust (9 9's) functional language and operating environment that is particularly suited to pattern and message processing so this presses many of the right buttons for me. In the past have overhauled systems written in Java and 3GL languages so that they operated in a de facto "shared nothing", fully re-entrant manner, with dramatic improvements in performance, predictability and reliability with more done by far fewer and less frustrated resources, so I know this aspect of Erlang very powerful, if not impossible to partly reproduce in 3GL and Java with care. The ultra-lightweight processes and natural message passing and parsing are another thing entirely. Process creation, context switching and support for vast numbers of these lightweight entities is a boon. Pattern matching is in Erlang's DNA as is the massively distributed processing support across a heterogeneous environment.

Basically, I believe it has massive potential.

Although I have used Erlang in other areas, I wish to now crack the nut of an Exchange engine using this language and operating environment.