Composability

Are IT terms such as Composability all Greeks to you? To find out how kids’ play, Greeks, Chinese, and IT architecture fit all together in my work, read on!

Many kids (at least in Europe) compose parts – Lego® bricks. The Lego bricks show the main characteristics of  Composability:

  • Mechanism to compose (stack together)
  • Right size for target audience (neither molecules nor real bricks would  do)
  • Good variety of building blocks – interesting things can be created
  • No surprise when bricks and composites of bricks are stuck together (unlike hydrogen and oxygen)

Chinese Surprises

Contrast this with Chinese characters (from the beautiful book Chineasy ):

  • Join the character for person together with itself, and you get follow, a person following another. Three persons, well, is a crowd.
  • Join the character for woman together with itself, and you get – argument. Three characters for women – well find out for yourself.

Surprise makes things hard to learn; we want to avoid this when constructing systems. However, abstraction helps: View the assembled pieces as a whole, replicate the whole as required, and use it as a piece to build something bigger.  Viewing the pieces as whole is an abstraction.

Surprises may emerge by hidden interactions between pieces within an abstraction; or worse, from pieces within one abstraction to pieces in another abstraction.  To avoid such hidden interactions, developers try to define clear abstraction borders, and avoid special cases, i.e., very specialized pieces whose behavior is not widely known.

Modularity is the basis of Composability: It means you can leave a part away, or substitute it by another part.  Composability goes beyond by packing pieces together and applying them in a different context.

Frictionless Composability is an ideal for many systems – as it means that multiple parties can contribute to a system, in the same way that many kids together can build a Lego design.

Putting Composability to work

In the overhaul of Quantax, an application to calculate and present market risk for trading in banks and asset management, we had the task of finding appropriate building blocks. They need to make sense both for the user and for the programmer who is developing and extending the system.

We used a pivot-like report with vertical and horizontal dimensions and fact values as the main output model:

Quantax Flexport Output

Quantax Flexport Output (cropped)

By the way, the greek letters delta and gamma designate how sensitive a financial instrument is against market changes – banker call vega a “greek” too, although this is a fantasy letter.

We then decided on the Box, which is either a dimension or a value, as the main building block. Boxes can be dragged from a library (displayed as a tree) onto a grid, with – of course – space for dimensions and values:

Quantax Flexport Design View

Quantax Flexport Design View (cropped)

We wanted to allow almost any combination of boxes, without friction between them. So you can combine risk with profit/loss and return, on almost any dimension. Users can enter expressions into boxes, and rank results within hierarchical level, as they desire.

Although the Box is a rather small and versatile building block, the machinery behind report building uses larger structures. The report is built and updated (as trading events arrive) in a Pipeline, with processing Stages set up and controlled by the Boxes. In fact, the report is bootstrapped by first querying all boxes for the processing stages they require, then arranges these stages in a sequence, then calls the stages, which call back into the individual boxes to do the actual work. The data is being kept as in-memory table-like structures using NumPy arrays. The last stage in the processing creates the output, calling back to each box to do its formatting.

The size of the chosen pieces is important – relegating all work to small boxes without creating larger structures would result in clumsy and inefficient programming.

Let the client do the work: Plugins

With the Box abstraction and the machinery parts controlled by it, we can now allow our users to provide their own boxes, and even stage, structures, and output formats. The boxes are simply added to the library tree to be picked up. With the bootstrap mechanism described above, only the parts required for a report are actually invoked. We formalized the user-supplied code as Plugins, which have a few well-defined entry points, reducing the potential of friction between them.

Using a few abstract parts (named using metaphors), users may create simple add-ins in a few lines of code, or complex solutions using external data, intricate calculations and aggregations, and very specific outputs (e.g., to send a mail when a report shows a trading limit being exceeded).

Letting clients do some work is particularly important if requirements are complex and change often, or if you want to build an ecosystem, as this cannot be undertaken by a single company. But this would be another topic entirely,  perhaps for future blog post.


 

This is the second post in a series describing our experience with architecture, design, and Python coding in the overhaul of Quantax, an application to calculate and present market risk for trading in banks and asset management. The first part described how we tackled performance.