Directional Data Biases and Systematic Trend Following: Friends or Foes?

Living in a world where (almost) everything seems familiar gives us a sense of comfort and at least partial security. The result is that we perceive ourselves to be in a privileged position where we can peacefully anticipate what may happen next in life. Surely apples can only fall from their tree onto Earth in the same way each time – this makes sense to us. Physics appears to rely on such an observation as one of its fundamental principles¹ but dig a little deeper and it also caters for potentially significant fluctuations in the gravitational field². For a long time, we believed that light could only travel in a straight line³, until physicists deduced⁴ and then confirmed that it can bend around very dense objects such as black holes⁵.

There are many principles or suppositions that appear to define our current reality, whether physical or otherwise and, quite routinely almost, these statements tend to be viewed as absolute truths. Big words indeed, but we appear all too often to rely almost exclusively on experience gained from the past, frequently the recent past, to deduce what may come next. However, life can (and not infrequently does) change in ways unforeseen by our experience. Why does this happen, and should we worry?

The principles spanning our world are described by the often-subjective way we detect and use them in real life. Do they change? That is a big subject^6,7, but it suffices us to say, for the purposes of this paper, that phenomena obeying such fundaments can occur in manners that are not always expected by prior empirical evidence. This is due to the open nature of our world which evolves as it ticks away, driving us towards new states and allowing us to witness and ideally learn from new, uncharted experiences.

Data Biases in Markets

Markets are a perfect example of an open system as they interact with the ‘exterior’ by continuously exchanging information. There is an almost encyclopaedic list of components that can change in the marketplace in ways that alter the dynamic of the price action. Such factors include, sensitivity to interest rates, market liquidity, earnings, momentum, downside risk, geopolitical developments and many more - the list is ever-growing, which is consistent with the nature of an open system. Moreover, the market participants themselves change over time and when they do, they introduce or remove their own views of the instruments they trade. When the ensemble of those individual views generates on average a particular directionality in the price action over a sufficiently long period of time, a directional data bias is established in that market. We set forth below some examples.

1. The 40-year bull run in the US Treasury Long Bond Future between early 1981 and the end of 2020 (see Fig. 1). This price action generated a bullish bias in the underlying distribution of returns across a 40-year period. The tendency of the market was to rise persistently within that time, with occasional corrections along the way.

Fig.1. The 40-year bull run in the US Treasury Long Bond Future between the beginning of 1981 and the end of 2020. Source: Bloomberg.

2. The 22-year bear run in the 100oz Gold Future between January 1980 and the end of 2001 (see Fig.2). Within this period the price of gold depreciated continuously, establishing a bearish data bias in the returns profile of this market during that time. Although several mini rallies against the prevailing bearish trend occurred, their presence in the dynamic of the price action did not nullify the bearish bias of this market over the timeframe shown.

Fig.2. The long bear run in the 100oz Gold Future Contract between late 1979 and the end of 2001. Source: Bloomberg.

3. The 47-year bull run in the S&P500 Index (see Fig.3) between the beginning of 1975 and the end of 2021. The price action is seen advancing across this time range despite several setbacks, such as crashes and corrections, that take place along the way. Notwithstanding those setbacks, the market returns exhibit a bullish data bias over this long period.

Fig.3. The 47-year bull run in the S&P500 Index between early 1975 and the end of 2021. Source: Bloomberg.

Impact on Building Systematic Trading Models

When building a systematic trading model, including trend following models, one often has the option of fitting such a model to the data available in the timeseries of the markets intended for trading. This is in fact one of the most frequently employed methods of putting together systematic strategies, at least as a first iteration. It follows naturally that the directional biases of the price series present in the underlying markets making up the trading universe could ‘spill over’ into the behaviour of such a model.

What does this mean in practice? Let’s consider, for example, the case of the 40-year bull run of the US Treasury Long Bond Future. If the data available for modelling is only the period shown in Fig.1 (i.e., January 1981 to December 2020), a first iteration, easy-to-fit model would expect bull markets more frequently than bear markets. This property of expecting bull markets over bear markets would be amplified in the case of medium-term trend following where the average holding period tends to be of the order of several months. The directional bias of the price action shown in Fig.1 increases in strength as the time-horizon of the momentum measure increases.

Is this problematic? The answer to this question is not necessarily binary. It really depends on where we are in the cycle of the current market regime. For instance, if around the year 2000 we built a data-fitted medium-term trend following model relying strongly on the underlying bullish bias shown in Fig.1 and then let that model run for the next 20 years, such a model would be expected to do well. However, the same approach may not necessarily have worked well during the subsequent 2 years when the market turned bearish for most of this new period (i.e., late 2020 to the end of 2022 – see Fig.1a). At a glance, the price action behaviour in 2021 and 2022 appears disconnected from the previous 40-year long price action.

Fig.1a. Bear market behaviour emerging at the end of 2020, lasting for at least 2 years in the US Treasury Long Bond Future (as of December 2022). Source: Bloomberg.

Similarly, if we used only the data set (or a subset of it) associated with Fig.2 above describing the 22-year long bearish price action in the 100oz Gold Future to build a data-fitted medium-term trend following model, we could end up with a strategy that expects bear markets more often than bull markets as a result of relying on the strong bearish bias of the returns’ distribution. Such an approach would work well as long as the bear persists. The problem is that the subsequent 10 years after 2001 (the end of the bear run) produced a strong bull run (see Fig.2a) where a model with a strong bearish bias would likely not work optimally because it does not expect a persistent bull market regime.

Fig.2a. 10-year long bull run in the 100oz Gold Future following the 22-year long bearish market regime that occurred previously (see also Fig.3). Source: Bloomberg.

In the case of the price behaviour shown in Fig.3 (the multi-decade bull run in the S&P500 Index), the situation is slightly more mixed. Despite the visibly persistent long-term trend which produced the bullish bias in the distribution of returns, there are also other types of behaviour that are noticed: there are several crashes (1987, 2000, 2008 and 2020) and some bear markets (e.g., the period 2000-2003). Despite this mix of regimes, a first-iteration data-fitting optimization for a trend-following model would still naturally result in a strategy that expects bull markets and may find it hard to adapt to or recognize in a timely manner bearish price actions, particularly of the type that occur after this long bull run, in 2022 (see Fig.3a below). However, despite their relatively less frequent presence in the timeseries of returns, the occasional bear markets or sharp corrections may give insightful clues as to how to more comprehensively approach the problem of modelling a market timing engine for markets that exhibit strong directional biases. See further below.

Fig.3a. Persistent bearish behaviour in the S&P500 Index occurring in 2022. Source: Bloomberg.

Dealing with Directional Data Biases

We have seen in the examples discussed above how prolonged, directionally continual price actions can generate data biases in the distribution of returns. We have also seen how these biases may not prepare the systematic trader for a change to a persistent market regime not seen or seldomly seen in the training data set. If the training set and what unfolds in reality in future turn out to be similar in directionality, one could argue that the data bias would in fact be the systematic momentum trader’s best friend. In such a situation, incorporating the data bias into one’s modelling would be likely to create a precious edge. However, this will hold true only for as long as the market regime that generated the directional data bias in the training set also persists in live trading.

This has proven particularly true for Fixed Income and Equities for several decades (as shown in Fig.1 and Fig.3 above), wherein both asset classes showed a strong bullish bias. Such long and persistent upward moves greatly benefited the so-called risk parity trade and encouraged its adoption and subsequent ubiquity. So, one may ask: where is the problem? The issue is that even long-term market patterns or regimes may run their course or unravel without warning when markets enter trading phases not seen in recent history, or never seen before. By way of recent example, the risk parity trade began to unravel in 2021 and later in 2022 it no longer worked. As a result, data-fitted systematic trading models that were expecting bull markets to a greater degree than bear markets in Fixed Income and Equities would prove less effective.

Is there a solution? And, more importantly, could one find a solution ahead of the shift in the market regime? Ideally, such a solution would be something that works independently of the historical market biases. One possible approach would be to look for inspiration in the manner in which scientists generalise the patterns seen in nature with theories or concepts that do not rely solely on empirical evidence (the corresponding feature in market price action being the directional bias).

Such theoretical conceptualisation must not be disconnected from reality: it should still be able to explain any experimental results (e.g., the measured distribution of returns) but, critically, it should also be able to generalise such a distribution to incorporate patterns never seen before but which could in principle take place at some point in the future. The rather infrequent but highly noticeable corrections or bear markets occurring in the price action of the S&P500 Index (see Fig.3) can be ‘expanded’ theoretically in a dynamical way to cover longer timeframes. The same holds true for the chart of the 100oz Gold Future in Fig.2 - the occasional bear squeezes and counter-trend rallies seen here can guide the researcher to infer the potential formation of persistent market regimes that may contrast with the bear market depicted in Fig.2. Moreover, such a conceptual generalisation of market price action behaviour could be designed to be equally disposed to expect bear or bull markets at any time and, in essence, not to depend on any underlying directional data biases that exist in the market universe.

Concluding Remarks

This paper considered the influence that directional data biases may have in developing systematic trend-following (momentum) models. We have seen that a directional bias can be an excellent companion if the training set and the subsequent live data share the same directional drift. However, a long-term data bias can be a serious impediment for a strategy relying on data-fitting to adapt to a new and persistent market regime that exhibits a medium-term direction contrary to the expected bias.

We have also suggested how one might deal with these types of directional change. Conceptual in make-up, such a solution would be predicated on generalising the distribution of returns to include patterns never seen before by the market, but which could happen at some point in the future, with a view to diluting and even eliminating the data bias observed in the measured distribution of returns.

Such a conceptual approach to systematic trading is similar in nature to the manner in which physical theories constantly generalise measured observations (and other previously established views) to expect new patterns - new events that do not look likely to occur from the empirical evidence alone.

Such theories may prepare us for the unexpected. But should we always expect the unexpected? Should we always think that changes that seem implausible today may occur at any time? Is such an outlook a little unsettling even, as it leads to no persistent state of equilibrium?

While a conceptual solution may well outperform a data-fitted model at a time when the market moves against its historical bias, that very solution could well underperform a strategy relying on data-fitting when the market behaviour may again re-align with its long-term directional bias. Is there perhaps a place for both types of strategy, data-driven and conceptual?

We explained in a previous paper⁸ that price actions seem to exhibit more nonlinearity than before, with market regimes previously taking many years to develop now being confined to the extent of just a few months. Within a given time range, such nonlinear behaviour has the potential to generate significantly more market regimes compared to historical averages. In such a dynamically changing environment, we could experience alternating market modes, some of them observing their long-term directional bias and some disobeying it. This prospect seems to argue in favour of needing both types of systematic momentum strategy: models relying on the directional data bias of the markets traded and models independent of such a bias. In such a construct, a portfolio-level utility function that seeks to optimize the mix of these two classes of models in a highly energetic market environment becomes worthy of attention and perhaps the next challenge to solve for systematic trading researchers. Those CTAs that can tackle such a complex task are expected to outperform in the future.

**********

Stefanel Radu is Head of Research at BHDG Systematic Trading / DG Partners

^References

^{1Isaac Newton, The Principia: Mathematical Principles of Natural Philosophy, translation from the Latin original by Bernard Cohen and Anne Whitman, University of California Press, 19 February 2016}

^{2R. Brandenberger, V. Mukhanov, and T. Prokopec, Entropy of the Gravitational Field, Physical Review D 48 2443 – Published 15 September 1993.}

^{3J.L Heiberg and H. Menge, Euclidis Opera Omnia; Volumen 7, Leipzig 1895.}

^{4Johann Georg von Soldner, On the Deflection of a Light Ray from its Rectilinear Motion by the Attraction of a Celestial Body Which it Passes Close by (Lenard, P.) in Annalen der Physik 65 (15): 593-604 in 1801.}

^{5D.R. Wilkins, L.C. Gallo, E. Constantini, W.N. Brandt and R.D. Blandford, Light Bending and X-ray Echoes from Behind a Supermassive Black Hole, Nature 595, 657-660 (2021) – Published 28 July 2021.}

^{6Heraclitus, Fragments – translation by Brooks Haxton, (Penguin Classics), 28 October 2003.}

^{7Aristotle, Physics, by C.D.C. Reeve, Hackett Publishing Company, Inc (2 March 2018).}

^{8S. Radu, Noise and Nonlinearity: Challenge and Opportunity for Momentum Trading, ALPHAWEEK, 12 August 2021.}

^***

The views expressed in this article are those of the author and do not necessarily reflect the views of AlphaWeek or its publisher, The Sortino Group