Saturday, 13 January 2018

pysystemtrade - Github LFS

Very quickly; I've moved the .csv data files in pysystemtrade into a separate /data/ directory and also added them to github LFS. Which will probably mean you will need to download LFS to keep updating pysystemtrade. I've done this because I plan to start adding futures contract level data (in .csv format initially, but then probably in other formats as well) so the repo will become unwieldy quite quickly.

EDIT: Huge thanks to wmom5 for fixing the file up to make this change work.

Friday, 22 December 2017

Obligatory Bitcoin post

No, I'm not dead.

You'd be forgiven for thinking I was given my lack of blog posts in the last few months, but I was busy; first promoting "Smart Portfolios" (thanks to all who have bought it), then on the conference circuit (here, here and here), and then more recently I've been preparing the course material for this.

So this is a post about Bitcoin. I have dissed BTC before [and the price has gone from $500 then to ~$13K now FWIW], but in recent weeks there have been many people asking me about it. It seems you aren't allowed to exist nowadays without having an opinion on all things blockchain.

Actually my opinion on Bitcoin and (what I believe are called) 'alt-coins' is the same as my opinion on many things: a deep scepticism blended with almost total agnosticism. Fact is I don't see why I need to have an opinion on this stuff. I have plenty of places to put my investment capital, thank you very much.

Still 2017 has undoubtedly been the year of the Bitcoin - or perhaps more accurately the last five weeks have been the five weeks of the Bitcoin: people weren't really that interested until the price started going exponential:

Price (

Search interest (Google trends)

So it seems appropriate to close 2017 with this post (and the launch of two BTC futures on grown up exchanges in the last few week is also genuine new-news). If nothing else I won't have to continually waste time in replying to the hordes of people constantly asking me what I think about it.

A caveat: I'm writing this as an economist / investor / trader; I'm not a technical expert on the technology side of this stuff. Grab a few pinches of salt - you're going to need them.

A note: The vast majority of this post will apply equally way to any other *Coin. I will interchangeably use the terms bitCoin, BitCoin, bitcoin and BTC.

What is BitCoin for?

Is BitCoin:

  • A means of payment (like US dollars)?
  • A store of value (like US dollars and Gold)?
  • An investment (i.e. a store of value that is hopefully going to go up in price and/or return a stream of payments, like shares in eBay)?

The reason why I ask this is because the people who love Bitcoin started off saying it was going to be a fantastic means of payment, much cheaper and better than boring old existing payment systems (they always use the likes of Western Union as an expensive straw-man for comparison, but international payments are now much cheaper and domestic payments are pretty damn cheap as well).

Now it's late 2017 and it's clear that BTC is currently unsuitable as a means of payment for most transactions:

  • It's very expensive (currently north of $50 per transaction)
  • The mining process is extremely energy inefficient (each transaction consumes 250kWh; my family of five people and a cat uses about 25kWh a day. If I was to make one bitcoin purchase every 10 days then I'd double our carbon footprint)
  • Despite mega hype, hardly anywhere accepts Bitcoins in exchange for real stuff (some that did have stopped
  • The confirmation process is very slow and the time is very volatile (see picture - this is an average: apparently some confirmations take days)


It turns out that BitCoin is also currently unsuitable as a store of value:

  • The price is extremely volatile. It makes the German mark of the Weimar Republic look like the Swiss franc. Although unlike the inter-war mark it goes up in value as well as down.
  • Bitcoins are easily lost or stolen. This is also a problem with cash (though you'd struggle to lose £74 million in fifties), but not so much with 'legacy' electronic payments. 
  • It turns out to be quite difficult to turn Bitcoins back into real money (see here for just one example) unless you use some foxy derivatives which involves a very high effective transaction fee.

(I'm aware that some people think some of these problems are soluble. I'll come back to that in the 'future of crypto currencies' later)

So all the BTC cheerleaders have now pivoted to saying that BTC is an investment. The logic runs something like this: when all 21 million coins have been mined if the entire world starts using Bitcoin for just 10% of it's transactions then it would need to replace 10% of around $37 trillion in narrow money. $3.7 trillion / 21 million = $176,190 per Bitcoin.

There is an inconsistency here - BTC will only make sense as an investment now if it makes sense in the future as either a means of payment or as a store of value. This is because unlike investment in say stocks or bonds it doesn't produce a stream of earnings or coupons that can be valued. This makes it a bit more like Gold, which people like as a store of value, and if the brown stuff hits the fan it can also be used as a medium of exchange. If we debunk the use of BTC as a means of payment or store of value, then the investment case is also debunked.

The truth is that the vast majority of people who have bought BTC in the last few months have done so because they think it will go up in price, which will happen because some more people who haven't bought it yet would like to buy some. They care not why it has gone up, as long as it will continue to do so. Some of the more sophisticated people will have come up with all sorts of justifications as to why they've bought it, but in their hearts they know the dark truth.

People have likened the BTC mania to both the late 90's tech boom and the tulip mania in Holland during the 17th century. Arguably it's worse than both of these. Some of the firms in the late 90's tech boom did have viable businesses and ended up being worth something. If you buy a tulip, at least you have a nice tulip to look at. If you buy a BitCoin you have literally nothing of any intrinsic or aesthetic value.

However the vast interest being shown by people with does make it feel very much like the tech boom, which I remember well (I'm not quite old enough to remember Tulip mania).

What is the future of Bitcoin?

The first thing we need to do is ask how likely it is that the problems outlined above will be soluble anytime soon. This is the most speculative part of the post, because (a) we're looking into the future - which is hard and (b) it's resting on the shaky technical knowledge I have about blockchain technology. I suggest you read this if you want to hear from someone who knows what they are talking about.

Slow: We just need more miners on the case. Of course bandwidth will then become an issue, especially if miners tend to become concentrated in a small number of places with cheap electricity.

Expensive: Those additional miners will need to be rewarded with (higher) transaction fees. This will only get worse when BTC has the maximum number of coins since miners won't be rewarded any more by creating new blockchains. Those altcoins with pre-mined technology already face this problem. Transactions will also continue to get more expensive as the blockchain grows in length.

Huge power use: Which will only get worse with more miners and a longer block chain. Apparently if you read the bitcoin forums (a) we're going to get really fast computers - quantum computing will allow Moores law to continue, and (b) practically free renewable energy. Even if this is the case I still think it's madness to spend an increasing proportion of our energy budget on book keeping when it can be done much more efficiently by centralised payment systems.

These three points neatly tie together, and there are of course various technical proposals to deal with this. But as far as this dumb economist can tell they all involve fundamental changes in the way that the blockchain works and / or lead to other problems. The blockchain is a fundamentally non scalable thing: it has to get bigger over time, and the more it is used the faster it will grow.

Theft and loss: Essentially you can either:

  •  opt for a low chance of loss (a third party looks after your coin)
    • and risk theft (by the third party)
  • or a relatively low chance of theft (you own the physical BTC on a USB or something)
    • and risk loss (of the physical object or the password)
The wonders of decentralisation make this a problem that won't go away. At some point you're either going to have to trust someone else, or try and remember a password, or both. With centralised money I can give my money to a trusted third party which is backed by a government insurance scheme (called a bank) that insures me against both loss and theft. If I lose my password (ATM PIN or online banking password) then I can go the trusted third party and get a new one. The important point here is that the trusted third party is trusted because they are backed by a powerful centralised authority (the government) - something the blockchain fundamentally tries to get rid of.

Volatility: There is a chicken and egg problem here. The reason for the volatility is that the market cap of coins is relatively low compared to the volume of money moving in and out of them. If the BTC becomes a standard for global payments; and then the market cap of BTC grows sufficiently large, and the volume of money is made up of small transactions rather than large investments, then yes it is plausible that the volatility will dampen down.

Nowhere to spend them: Again chicken and egg. If BitCoin becomes a large enough factor in the world of global payments, then retailers will start accepting them.

Its no surprise that I'm fairly pessimistic about the future of Bitcoin as a true alternative to say US dollars as a medium of exchange or Gold as a store of value.

Of course there is a limited demand for BitCoin and other crypto currencies, made up of people in the following categories:

  • People buying stuff they shouldn't buy (eg drugs)
  • People who want to hide money from the tax authorities or their spouses
  • People who want to convert dirty money into clean money

But there are also existing ways of doing all of these things:

  • Most people buy drugs with cash (so I'm told). 
  • There are plenty of places to hide your money offshore, and most of them have great weather if you want to visit it. 
  • Money laundering is perhaps the best use of BTC  (BTC is great for money laundering. You can put $20 million into your bank account from a crypto exchange and if the authorities want to know where it came from you can tell them you sold two pizzas in exchange for 10,000 BTC in 2010 and they have no way to disprove that). 
But these are all still niche markets. And any sensible criminal or tax evader will hedge their bets and not put all their ill gotten dosh into a single USB stick.

I would argue that there is essentially an unlimited supply of new currencies (this is different from  coin in a given currency), and but quite a limited demand from people who need to use anonymous electronic money. Sure BitCoin has first mover advantage and monopoly power, blah blah blah. But if BitCoin got too expensive then it would sense for the limited pool of people who really need to transact via blockchain to use another cheaper currency, which could be created by someone forking an existing github project which takes about 5 minutes. After all Ethereum is roughly a third the size of BitCoin, despite only being around for 2.5 years compared to nearly 9 years for BTC.

Finally I should point out that the consequences of BitCoin working are really, really, bad. It would mean that governments would lose the ability to tax transactions, and all countries would end up as anarcho-techno states with a ridiculously uneven distribution of wealth. So I'm really hoping this particular experiment doesn't work out.

Should you invest in crypto currencies

I am very skeptical (in case you hadn't realised). But the more interesting part of this post now begins, where I'm going to ignore everything I've just written and assume that there is indeed a valid investment case for BitCoin, and see where that takes us.

What kind of investment is BitCoin?

Bitcoin as an investment has the following characteristics:

  • It has limited and unstable liquidity
  • It has high bid-ask spreads
  • It has high transaction costs (unless you trade the futures)
  • The market is fragmented across multiple exchanges 
  • There are operational and technical difficulties involved with trading it (unless you trade the futures)
  • There is counter party risk as you're usually facing an unregulated exchange (unless you trade the futures)
  • It is based on a made up number
  • It has no asthetic or intrinsic value and does not produce a stream of payments

None of this means you shouldn't invest in Bitcoin. Nearly all of these things also apply to residential housing, and that is still a valid investment. Eurodollar futures are based on a number that we know now, rather famously, has been mostly made up.

But it does mean that if you're investing in BitCoin you should either (a) have a much higher expected return to compensate (b) hold a much small allocation in the asset than you would normally or (c) both.

Should BitCoin have a higher expected return? Indeed should we expect BitCoin to go up in value at all? Luckily we can be agnostic on this subject (since arguing about the long term future of BitCoin is something I'm neither all that interested in or qualified to do).

After all I invest in Gold but not because I personally think it will go up in value. The same goes for tail protect hedge funds. That's because these products bring diversification and provide insurance. It doesn't seem unreasonable to put BitCoin in the same bucket, albeit with the qualifications that Gold (at least when held via an ETF or a future) doesn't suffer from any of the problems I've listed above (Gold does have some industrial uses and enough people seem to like Gold jewellery that it has aesthetic value).

Indeed if we treat BitCoin as an asset with negative correlation to everything else then even a slightly negative expected return wouldn't bother me.

How much should I invest in BitCoin?

Putting aside all my biases this is an opportunity to demonstrate how the top down framework in my book "Smart Portfolios" can be adapted to literally any old crap. Yes, even BitCoin.

The first thing you need to do is decide what your allocation will be to "Genuine Alternatives" (which is my name to distinguish them from not really 'alternative' assets that are very similar to equities or bonds - like the majority of hedge fund strategies).

This in turn is subdivided into the "insurance" and "standalone" buckets; the latter being the rare assets that both provide diversification benefits and also yield a positive return. "Insurance" is the bucket that Gold and Bitcoin sit in, along with certain types of hedge funds and safe haven currencies. Like I said above we don't expect them to provide a positive return, just like I don't expect to make money off buying house insurance in the long run.

In my book I recommend putting between 0% and 25% of your assets into genuine alternatives, and roughly half of that 0-25% into insurance like assets such as Gold, with the other half into standalone alternatives.

Let's first deal with US Investors, since I know from my site traffic report that they make up a majority of readers. My recommended allocation from "Smart Portfolios" for a small US investor is as follows:

  • 50% in standalone alternatives of which:
    • 25% in managed futures eg WDTI
    • 25% in global macro eg MCRO
  • 50% in insurance-like of which
    • 25% in safe haven currencies eg FXF
    • 25% in Gold eg IUA

If we shoehorn BTC into that we get:

  • 50% in standalone alternatives of which:
    • 25% in managed futures eg WDTI
    • 25% in global macro eg MCRO
  • 50% in insurance-like of which
    • 16.667% in safe haven currencies eg FXF
    • 16.667% in Gold eg IUA
    • 16.667% in Bitcoin
These are the figures for your genuine alternatives allocation. That translates to an allocation to BTC within your entire portfolio between 0% (no alternatives) and 4.166% (with 25% in alternatives, of which we put 16.667% in Bitcoin). Ideally this would be split up further between different coins.

Sadly for many retail investors in the UK the choice of alternative ETFs is far more limited. From Smart Portfolios :

  • 100% in insurance-like of which:
    • 50% in Long volatility eg SPVG
    • 50% in precious metals

Again if we add BTC:
  • 100% in insurance-like of which:
    • 33.3% in Long volatility eg SPVG
    • 33.3% in precious metals
    • 33.3% in Bitcoin
This gives us a crypto allocation of 0% to 8.33% (with 25% in alternatives of which a third goes in Bitcoin; again ideally this would be split up further between different altCoins).

Maximum BTC allocation then is between roughly 4.2 and 8.3%.

However this is a risk weighting. The actual cash weighting is inverse volatility weighted, and so depends on what the risk of the rest of your portfolio would be.

The volatility of BTC is about 100% a year; suggesting that even for someone with a relatively aggressive risk target of around 16% a year (the highest I advocate for reasons explained in the book) they should only be putting a maximum of 0.65% (US) to 1.3% (UK) of your portfolio in crypto.

To reiterate: even with the most aggressive risk target, and the highest recommended allocation to alternative assets, you should be looking to put no more than 1.3% of your portfolio into Crypto.

Notice that I have assumed that BTC will not yield an above average return, but on the other hand I've also ignored all the problems I outlined above (under "investment characteristics" above). The latter argues for a much lower weight (and for me personally that weight is zero). The former argues for a higher weight, but it's worth bearing in mind that it's very difficult to predict the future returns of any asset; and this is doubly true for something like BTC.

How should I invest in BitCoin?

Many of the operational issues with BitCoin relate to owning physical coin. Right now the only alternative is to own the futures; BitCoin ETFs aren't yet available (though perhaps soon will be, but when they become available will be based on the futures).

The futures are very illiquid, although admittedly they have only just been launched. They are cash settled which is good (personally I want to be as far as possible away from 'physical' BTC) and bad (the settlement price is open to manipulation and the price could easily deviate from what the cash and carry arbitrage should produce: but then that's just comparing one imaginary number with another, so fill your boots). They aren't that suited to investors who aren't comfortable with derivatives.

I wouldn't personally buy spot BTC - it's just too flaky. However if you follow my advice and only put ~1.5% of your portfolio into them I suppose that limits your likely risk (if you'd put 1% of your wealth in BTC a year ago you'd be up 10% of your wealth now, which for most portfolios would be a very handy chunk of money).

I might trade the BTC futures - see below. But I wouldn't bother investing, even via the future.

Do I have enough money to invest in BitCoin?

If you've read "Smart Portfolios" then you'll know that a key issue I bring up is whether you have enough money to be diversified.

For example if you trade BTC futures then you'll need to own at least one entire coin, because thats what the CBOE future is based on (the CBOT is five coins). If you have $13K in BTC (using the current price), and that's 1.3% of your portfolio, then your portfolio must be a million bucks.

If you're brave enough to trade BTC spot then you can in theory buy less than one coin. But at a $50 transaction cost it won't be economic to trade small amounts of BTC. Using table 40 in my book it turns out that the minimum economic investment in spot BTC is currently $15,000; i.e. just above one coin at current prices.

I know there are some dollar millionaires reading this blog, but if you aren't in that category (yet) you should only invest in BTC if you can afford to buy one coin (directly or via futures); at least until the transaction fee comes down sharply.

Should you trade crypto currencies?

Does it make sense to trade crypto directionally?

Clearly investing in Bitcoin is a long only bet. But given how volatile the currency is, surely it makes sense to trade it? Should I for example consider adding BTC futures to the 35 odd futures I trade systematically, mainly using trend following? Surely crypto currencies have a tendency to show strong trends?

Let me say:

  • Volatility doesn't make something more or less attractive to trade. 
  • There is rarely statistically significant evidence that instrument X trends better than instrument Y; and the relatively short data history for BTC means that evidence certainly won't exist

This means that if there is a compelling reason to add BTC futures it's the same as for the investment case; they provide diversification. But there are a number of problems with trading BTC futures:

  • Even at the CBOE the $ volatility per contract is relatively high (currently around $13K per year compared to about half that for the emini S&P 500) making them unsuitable except for large portfolios
  • The $ margin per contract is relatively high (even once you've accounted for the volatility - this is also a problem with the VIX/V2X futures because of their skew properties)
  • The volume is currently too low and falls below the minimum threshold that I use before even considering adding a contract (although it shows signs of picking up and to be fair these are very new contracts)
  • The bid-ask spread makes them a relatively expensive contract - this isn't a dealbreaker but means I'd need to trade them more slowly (see chapter 12 of "Systematic Trading")
  • My broker doesn't let me short the futures: this is probably the most serious issue

Would I change my mind in the future? I might, if enough of these points become less problematic. But not right now.

Can you do <x> arbitrage with crypto currency?

On the face of it BitCoin is a arbitrageurs dream. The futures price isn't in line with the spot (although the gap has closed, and there should always be a slight difference reflecting the cost of funding the cash and carry trade; though there isn't really anything like a BitCoin repo market). The futures curve is the wrong shape for similar reasons (currently the CBOE strip looks like this:

Jan 18 13,530
Feb 18 13,760
Mar 18 13,520

... which makes no sense). The cash price varies wildly across exchanges (so what is "spot" anyway?)

However I'd really be nervous about trying to exploit any of this 'free money'. Because the futures are cash settled the opportunities to take a free money spot / futures bet or curve bet aren't really there (plus the Feb, March prices are probably stale given the lack of liquidity so the curve trade may not really exist). Across exchanges its even more difficult due to all the operational problems we've already discussed (verification delays, high transaction costs, the risk of theft or trades just being cancelled on an unregulated exchange).


No, I'm not going to buy any BTC. If after reading this you still insist on doing so please only put a fraction of your wealth into it - and make sure you're already a dollar millionaire. And please don't ask me to discuss this subject ever again (this is doubly the case if BTC goes to $1 million - I really won't want to talk about it then).

Monday, 4 September 2017

Smart Portfolios: A post about a book, NN Taleb, and two conferences

September 18th is the official publishing date of my second book, "Smart Portfolios: A practical guide to building and maintaining intelligent investment portfolios (Harriman House, 2017)".

This blog post will give you some more information about the book, and more importantly help you decide if it's worth buying. I'll also let you know about a couple of forthcoming conferences where I will be talking about some of the key points (at Quantcon Singapore and QuantExpo Prague).

It is written in the form of an interview. As no other interviewer was available I decided to interview myself. If after reading this post you still want to buy the book then you can go to this link.

Shall we start with some easy questions?


What's your favourite colour?

I was hoping for a more highbrow interview than this. Niels Kaastrup-Larsen never asks such a trivial question.

Sorry. This is the first time I've ever interviewed such a well known and intelligent person.

No problem. Since I'm well known, intelligent, and also very easily flattered.

Perhaps instead I could ask you where the idea for the book came from?

That's a much better question. After leaving AHL in late 2013 amongst other things I was thinking about writing a book. I came up with an idea for a book which I was going to title "Black Magic". Once I had the cool title I had to decide what the book would actually be about. I proposed to the publisher (Harriman House) that I'd write something which would be subtitled something like: "Tales from the world of systematic hedge funds: How to invest and trade systematically".

After a long series of emails that got narrowed down to the shorter title "How to invest and trade systematically", and subsequently cut down further to "How to trade systematically". About eighteen months later "Systematic Trading" was published.

The obvious thing to do next was to write "How to invest systematically". Of course this is also a huge topic and I had to spend a fair bit of time thinking about what the focus of the book should be, and what ground it would cover that wasn't covered elsewhere. I also had to think of a more original title than "Systematic Investing".

How did you decide on the foc(us/i) of the book?

To some extent I wrote the book for myself: I wanted a framework for managing my long only investments; which included shares and ETFs, where I had to pay relatively high trading costs compared to my futures, where I allocated across multiple asset classes, and where there were real world problems like tax to worry about.

I then thought long and hard about what were the most important - and neglected - topics in investment books. I decided they were uncertainty and costs. These two ideas are actually linked, because costs are highly predictable, whereas almost everything else about financial returns is uncertain to varying degrees. It's important to make decisions with this firmly in mind.

Of course there is an overlap with Systematic Trading here because in that book I frequently emphasise the difficulty of knowing the future with any degree of certainty, and I also wrote an entire chapter on trading costs.

Like Systematic Trading I also wanted to publish something that was a complete framework. So the idea is you can use this book for almost any kind of unleveraged long-only investment (passive ETFs, individual shares and active funds), and it also covers a few different 'use cases'. Of course this makes the book pretty long. It's about 50% longer than "Systematic Trading", but the sticker price on the cover is the same (in GBP anyway) so it's actually better value.

So... if you like [winks] we can talk a bit more those key ideas of uncertainty and costs now.

Oh yes, sure. Perhaps you can talk a little more about uncertainty

In finance there are almost two opposing views. On the one hand there is Taleb who says "We don't know anything" and on the other you have almost the entire industry of quantitative finance that assumes we know everything with 3 decimal places of precision (obviously I'm exaggerating both viewpoints for effect).

The idea that we can't naively use the probability of past events to predict the future is hardly new; it goes back to Keynes and deeper into the past. In contrast in quant finance we normally assume that we can (a) know the model that generated financial returns data in the past (b) precisely measure the parameters of this model and (c) assume it will continue into the future.

The "Weak Taleb" attack on quant finance is an attack on (b); so "The casino is the only human venture I know where the probabilities are known... and almost computable... In real life you do not know the odds, you need to discover them... ” (Black Swan).

But we can make equally valid points that (a) is also untrue (there is no 'model' waiting to be found and measured); and that (c) is nonsense (the future will never be exactly like the past). A "Strong Taleb" attack would essentially make the points that: (a) there are no models [or at least none that are practically usable], (b) even if there were we couldn't ever know their parameters precisely, and (c) these models are unchanging into the future*.

* By the way for the purposes of this discussion a Markov state model is still a single "model" - not a way of dealing with models that could change in the future. 

This is all true - but extremely unhelpful. Nearly all the smart people in finance are aware of this problem, but mostly ignore it. In fact we probably just have to assume that there is a model, and we also have to assume that this model will work in the future. Or we might as well close our laptops and become non-systematic, "gut feel" discretionary investors and traders.

But it's quite straightforward to deal with the weak Taleb attack on point (b) and think about the accurate measurement of the past. First you need to get yourself away from the idea that there was only one past with one set of estimable parameters which are known with certainty. Past movements of financial markets are either [i] a random draw from an unknown distribution or [ii] just one of many possible parallel universes that could have happened or [iii] are realisations of some random hidden latent process. It's easier to model [i] but these ideas are functionally equivalent.

Quantifying the effect of this uncertainty of the past on parameter estimates is relatively trivial statistics 101. So for example if the mean of a return series is 5% a year, and the standard deviation 24%, and you have 36 years of data, then the estimation error for the mean is (24% / sqrt 36) or 4%, so the two standard deviation confidence interval is -3% to +13%. Even with a relatively long history of data that is a huge amount of uncertainty about what the modelled mean was in the past: and remember we're still making the quite strong assumptions that there really is a model generating the returns; which happens to be Gaussian normal; and which will remain unchanged in the future.

The key insight here is that there are different degrees of uncertainty. The confidence interval for a standard deviation in this case is much narrower: 18.4% to 29.6%. If we have more than one return series we can also estimate correlation; so for example between US bonds and stocks the confidence interval is around -0.1 to 0.2.

So we don't need to throw away all of our data; we can be a bit smarter and just calibrate how differently confident we can be in the individual estimates we draw from that data.

That's given me a headache! It sounds like you've written a very technical book on maths and/or philosophy...

Nothing of the sort! All the ideas are introduced in a very intuitive way (much simpler language than I've used above); and it's very much aimed at a non-quant but financially literate audience. The book is mostly about what practical use these findings have. Once you start thinking about the world in terms of quantified uncertainty you can still be a systematic, model based, investor; and you can simultaneously be a skeptical pupil of Taleb; but you can also still do some useful things.

So what practical problems do you address with this idea of (calibrated) uncertainty of the past?

The first main insight is that standard portfolio optimisation is partly junk. Of course everyone in finance knows this: but again there are two extreme views: "Complete junk - I don't believe in any of that nonsense and I'm just going to hold US tech stocks whose names begin with the letter A" or "Junk, but I'm going to use it anyway because what choice do I have?". But reality is more nuanced than either of these views.

The insight and intution behind Markowitz's work is extremely valuable - it's the baby in this particular bathwater. Though yes: estimates of risk adjusted returns have such huge past uncertainty they're mostly worthless. But estimates of volatility and correlation are more predictable and so have some value. So I address this question: how should you build portfolios given this knowledge?

The other main insight is that you shouldn't look at post-cost returns as you're subtracting apples (costs) from oranges (pre-cost returns). Pre-cost returns have huge estimation error. But costs are actually relatively predictable (unless you're trading so fast or in such size you affect the order book). A better approach is that the starting point for any decision should be that you go for the cheapest option unless the evidence strongly suggests - with some probability - that the more expensive option is better. I guess this is a Bayesian worldview, though I never use that term in the book.

Okay I get the hint. I think perhaps it would be good to talk about costs now

The first thing to say about costs is that although they're relatively predictable, they're not actually that easy to measure. Although there have been attempts to get funds to state the "total cost of ownership", in practice you have to make some educated guesses to work out likely costs of different forms of investment.

Once you have that information, what should you do? Anyone whose read my first book knows that costs are important when deciding how much, and what, to trade. But for long only investment there are a whole lot of other decisions where the notion of certain costs and uncertain returns is useful. For example should you buy a fund which is more expensive, but which has had - or should have - higher returns?

Another important point is that different kinds of investors have to worry about different kinds of costs. So relatively large investors have to worry about market impact. But for relatively small investors, especially those in the UK, the tyranny of minimum brokerage commissions is more important. A £10 commission on a £1,000 portfolio is 1%: quite a lot if you have realistic estimates of future returns. An important implication of this is that the right kind of portfolio will depend on how much capital you have to invest.

You've already talked about some common elements, but what would readers of Systematic Trading recognise in this book?

The main thing they will recognise is the idea of a top down, heuristic portfolio construction method which I call handcrafting in both books. The difference in Smart Portfolios is that I make it even simpler - all grouped assets have equal weights (once differential risk has been accounted for). 

In part two of the new book I also go into much more detail about how you'd practically build a cross asset portfolio using the top down handcrafting method: choosing appropriate ETFs, and where it makes sense to buy individual shares. 

Because of the emphasis on costs this would be done differently for smaller and larger investors. In particular larger investors can afford more diversification: smaller investors who buy too many funds will end up owning too many small chunks of things that they've had to pay multiple minimum commissions on. The advantage of a top down approach is it deals with this nicely: you just stop diversifying when it no longer makes sense (a decision based, naturally, on the certain costs and uncertain benefits of diversification). 

Earlier you talked about "different use cases"...

Glad to see you've been paying attention! Just as in Systematic Trading I realise that not everyone will sign up to the extremely pure dogma: in this case that risk adjusted returns are completely unpredictable. So the book also helps people who want to vary slightly from that central path, whilst limiting the damage they can do. These different use cases all appear in part three.

Firstly as you might expect I talk about systematic ways to forecast future returns. At the risk of being stereotyped one is a trend following model, the other is based on yields (so effectively carry). The point, as with Systematic Trading, isn't that these are the best ways to forecast the markets - they're just nicely familiar examples which most people are able to understand (and whose nuances I can explain). Unfortunately as with my first book a few people won't understand this and will pigeonhole me as a chartist / trend follower / technical trader...

Secondly I talk about using "gut feel" but in a systematic way. This is analogous to the "semi-automatic trader" in my first book. The idea being that some people will always think they can predict market returns / pick stocks; at least let's provide a framework where they can do limited damage.

Thirdly are people who are still convinced that active fund managers are the bees knees. I show them how to determine if this is true by looking through the prism of uncertain returns (perhaps higher realised alpha in the past) versus certain costs (higher management fees).

Finally there are the relatively recent innovations of Smart Beta; again more expensive than standard passive funds, but are they worth it? I also talk a bit about robo-advisors.

"Smart Beta": is that where the title of the book came from?

Sort of. It's an ironic title in that respect since you'll realise quite quickly I am pretty skeptical of Smart Beta at least in the guise of relatively expensive ETFs. Using systematic models to do the smart beta yourself is better, if you have sufficient capital.

But "Smart" actually sums up the book quite well (and yes, this is an ex-post rationalisation once I'd thought up the title. Deal with it). Smart for me means "Practical but theoretically well grounded".

So for example there are some technical books on things like Bayesian optimisation that deal with uncertainty, and other papers around trading costs. But if you introduce taxes into the mix you end up with really non tractable, non closed form models and it gets pretty unpleasant. This isn't the kind of the thing the average financial advisor can really use. Frankly even I don't use that kind of technical artillery when deciding if I should top up my pension fund.

And there is plenty of "backwoodsman" advice in less technical books that is either vague ("Don't trade too much"), overly simplistic ("Buy the cheapest passive funds") or worse isn't supported by theory ("Everyone should just own stocks").

What I tried to do in Systematic Trading, and continue in Smart Portfolios, is to provide some heuristic rules that are (a) as simple as possible and (b) theoretically correct, or at least supported by research. So for example one simple rule is "if you are paying a minimum brokerage commission of $1, you shouldn't invest in ETFs in units smaller than $300".

One, fair, criticism of my first book was that I didn't provide enough realistic examples. So I've probably gone overboard with them here in trying to make the book as accessible as possible.

A less fair criticism of Systematic Trading is that there weren't enough equations - which of course was deliberate! I've included some more here to aid clarity, but they are mostly extremely simple without an integral symbol in sight.

What about portfolio rebalancing?

Yes, that's another big topic where I try to use simple rules that are theoretically grounded. So there is the standard rebalancing method where you don't rebalance unless your positions are out of whack by a certain amount. But I introduce a simple method for calculating what "out of whack" is, which again depends on the cost level you face, which in turn depends on how much capital you have to invest.

Then there are other rules to deal with other common situations: rebalancing when you're using forecasting rules, the effect of taxes, changes in characteristics used to pick stocks, takeovers, and so on.

I really enjoyed Systematic Trading. Should I buy your second book?

It depends. "Smart Portfolios" is actually two books in one:

  • A practical discussion of the effects of estimation uncertainty on optimising portfolios
  • A complete handbook for long only investing in funds and shares 
So if you are a pure short term futures trader who already has a good understanding of statistical uncertainty then you'll probably find little of value in this book. It is definitely not "Systematic Trading 2: The Market Strikes Back". But feel free to buy it out of misplaced loyalty! Then give it to the guy or gal who manages your long only investments.

On the other hand if you read "Systematic Trading", and enjoyed it, but struggled to see how this related to your long only ETF or shares portfolio (with the exception of the "asset allocating investor" examples), then you should really find this book very useful.

Finally if you are in fact Taleb you should definitely read the second chapter of the book, but no more. After that I mostly assume that Gaussian Normal is a useful model when used properly, and you'd absolutely hate it. Although in my defence I do at least use "Kelly-Compatible" geometric means which penalise negative skew, rather than arithmetic means.

Is there anyone you'd like to thank?

Nine people were absolutely key in this book coming about. Stephen Eckett, top dog at Harriman House, commissioned the book. Craig Pearce spent months whipping my ramblings into marketable and readable condition. Riccardo Ronco and Tansu Demirbilek were brilliant reviewers. My third reviewer Tom Smith was also brilliant, but deserves a special mention as he also reviewed my first book; in both cases with no money changing hands (I suggested he pay me £500 for the privilege but this was greeted with derision). 

The other four people are my wife and children, who have had to put up with a distracted and absent minded husband and father for months on end. 

Any more books on the horizon?

Not immediately as I have a few other projects I'm working on which will take up most of my time over the next few months. But then I've got a couple of ideas. The first idea is to try and write "Systematic Trading and Investing for Idiots" (clearly a working title). Essentially a distillation of the methods and ideas in my first two books, but written for a wider retail audience. The second idea is to write something about the interaction of people and machines in the financial markets. With all the hype over AI in financial markets this might be an interesting book.

Are you doing any conferences in the near future where we can here more about your ideas?

Great question! [surreptitiously slips ten pound note to interviewer] 

At the end of this month I'm speaking in Singapore (at QuantCon) and then at the start of November in Prague (at a new event QuantExpo). Both of these events look to have a great lineup and I'd highly recommend them if you're within flying distance of either venue.

The talk I am giving at both venues will be about the impact of past uncertainty on the estimates used for portfolio optimisation: basically material covered in the first few chapters of the book. I'll also introduce some of the possible solutions to this problem. Many of these people will have seen before but I think it's good to understand specifically how they deal with uncertainty.

There might be other events coming up - keep an eye on my social media for news.

So finally: When and Where can people get your book?

It's officially published on the 18th September but currently available for pre-order. If you go to the website for the book at this link you'll get a link to my publishers page, which is the best place to buy it from my perspective (and currently the cheapest). The books website also has a lot more information about exactly what is in the book if you're still undecided. 


If you thought the (frankly incompetent) interviewer missed a key question then please feel free to comment below and I'll add the question (and answer it).