Forums  > Trading  > Trading strategy questions  
     
Page 1 of 1
Display using:  

Energetic
Forum Captain

Total Posts: 1414
Joined: Jun 2004
 
Posted: 2017-06-14 19:42
I have worked out a little low frequency trading strategy in my spare time. It has rather attractive average returns both in and out of sample as well as live last 7 months. Unfortunately, it is very volatile so Sharpe ratio is just over 1. Since someone will probably ask about it, I’ll add that it is 40% correlated with the general market and has max DD -35%. The strategy tells me to buy a certain asset, or to sell it and stay in cash until things improve.

Let’s deal with the easy question first. Am I right that no one with real money would invest in a strategy with Sharpe = 1 no matter the returns in backtesting?

Second question is how much of my own money should I invest in it? What would be a rational approach to finding the “optimal” % to invest in it? It seems reasonable that capital allocation must be a function of my certainty that the strategy will continue performing well, and perhaps a couple of other things.
If I were sure that the future will be more or less like the past (as it might, there is no obvious reason why it can’t) then, for a long (3+years) haul it would be rational to put all my money in it and also borrow as much as I can because the average returns in backtesting are far better than what I can make investing in stock indices.
However, in reality I’m not at all sure that the future will be more or less like the past. There is always a chance that market dynamics will change, that someone else will discover the same trade, that I screwed up the code, that I overfit the model and simply delude myself (this one is the prime suspect on “too good to be true” basis), that I am just being lucky and sooner or later it will blow up in my face, etc. Although I am aware of potential pitfalls, I don’t have a first clue how to quantify the uncertainty. Nor how to use this uncertainty in deriving the optimal allocation. Has anyone thought about such a question before? I know that ideally I should develop a bunch of other uncorrelated strategies instead of focusing on just one, and I’m trying, but for now this is all I got.

Third, when and why would I get out of this game? I guess the most intuitive answer is to get out when it doesn’t perform as it used to. I could run some statistical tests to make it sort of objective but I don’t think this is a great answer (although it still might be the best answer). Here is why. If I were running this before and during the financial crisis, the strategy would have behaved in a totally different way compared to what I saw before or after. If I concluded based on this evidence that it’s time to close the shop then I would have missed a lot of money to be made after the crisis. Are there any other ideas?

Thanks to all who will find time to chime in.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

EspressoLover


Total Posts: 240
Joined: Jan 2015
 
Posted: 2017-06-14 21:53
#2

The way to think about this problem is with Kelly Sizing

At least if the following conditions hold: 1) You only care about the long-run growth, and expect the process to continue for a long enough time to sufficiently approximate "long-run". Easier said than done. Psychologically people really have a tough time pushing through drawdowns, even if they know it's mathematically optimal. 2) Your utility function for wealth is logarithmic. 3) The process continuously compounds. Or you can rebalance frequently enough that it's effectively continuous. Daily should meet the criteria for pretty much all financial products. 4) You don't have any limits or costs to leverage, or at least none significant enough to meaningfully affect your strategy.

In which case Kelly says the targeted volatility should be [Sharpe]. So if you're expected annualized Sharpe is 1.0, your targeted annualized volatility should be 100%. Or if you're rebalancing daily, the targeted daily vol should be 6.25%. If the unlevered strategy produces 2% daily volatility, you'd want to lever up the positions by 213% to reach target vol. (And rebalance daily to keep the leverage constant).

However like you say, forward expected performance is likely to be lower relative to your previous estimates. Overestimating Kelly sizing also has an asymmetrical impact on performance relative to underestimating. So you have to decide on some fraction to shrink your Sharpe by. There's not really an easy answer here. It depends on how confident you are in the statistics. However shrinking the Sharpe ratio by 50% seems to be a decent rule of thumb though. (Equivalent to applying a fraction of 0.5 to the computed Kelly size). For a 1.0 Sharpe strategy, you'd treat it like a 0.5 Sharpe. In which case Kelly would dictate targeting 50% annualized volatility or 3% daily vol. Then levering or allocate accordingly.

#3

I'm a big fan of the Copernican Principle. There's some process generating abnormal returns, and it will live for a finite amount of time. You're probably not a "special observer", so most likely your discovery of the process represents a random sample of time from its life. Unless you have some other priors on the life expectancy of the process, Gott would dictate that the median expected remaining life would be the same as the current age of the process.

So if your backtests indicate that this strategy has been profitable for the past 5 years, you'd expect it to last for another 5. With 50% confidence you'd expect it to survive between 15 months to 15 years. However when the process dies, you will only be informed through a noisy channel, and it will take some amount of time to decide with statistical confidence that the gravy train is over.

If you're a Bayesian, than the longer the process has lived the longer a stream of bad returns it will take before declaring it dead. This represents a "cost" in that at some point in the future you will eat volatility without excess return while trading a dead process. This cost can be quantified using Kelly. For example if you expect 20% of the time you spend trading this strategy to be "dead", you need to shrink the expected Sharpe ratio appropriately. That costs you 36% of the long-run expected return due to smaller Kelly size.

You have to decide the optimal p-value which trades off missed profit from type 1 errors with Kelly costs from type 2 errors. I'd say once you decided on the procedure, pre-commit to it. Avoid the temptation to ignore your instruments and fly by the seat of your pants. Yes, human wisdom may give you insight into what's going on. But it's usually outweighed by emotional hysteria.

reference entity


Total Posts: 1
Joined: Jun 2017
 
Posted: 2017-06-14 23:27
On getting people to invest I think it would be a hard sell given what you've said. Presumably you're talking about a black box strategy so all you'd be able to tell prospective investors is things like asset universe, turnover, sharpe, correlation etc. In that case the high correlation with the market is going to be a pretty big mark against. Anyway the only way to go is to build up a long realized track record while going after capital from people who trust you personally-people wait a long time to jump on the bandwagon if it's just a cold call. Even with higher sharpes you can't skip the track record step.

As for how much to allocate, EspressoLover gives a nice summary of some statistical rules of thumb to help. When allocating within a chunk of capital you've decided to put in a strategy among assets/substrategies/whatever, Kelly, risk parity, Markowitz, etc can be really useful.

But somewhere at the top level there's an unavoidable meta problem of how big of a bankroll to show to your favorite sizing algorithm in the first place. For that I think you just have to work through some concrete scenarios. For example, if you went with the 100% vol target of full Kelly, say you forecast stock volatility at 20%, so you get 5 times leverage. There's been loads of one day ~10% drops in stocks that would be a 50% hit, and another Black Monday would wipe out the whole bankroll, regardless of how the strategy had done before that. A key thing about maintaining constant volatility target is that you never get any "higher off the ground" than when you started, so to speak. Also important is that when the market gaps against you, the deleveraging isn't gonna be continuous. Given your margin/funding/etc situation, what sort of size worst loss would you be willing or able to take?

NeroTulip


Total Posts: 997
Joined: May 2004
 
Posted: 2017-06-15 18:11
@EspressoLover:

I think that Kelly implies vol=Sharpe, not vol=Sharpe^2. But I might have messed up my derivation, is there a reference?

@Energetic:

Anyway, with SR=1, this implies vol=100%, which is obviously nonsense, for two reasons:
- Psychologically, half Kelly is way easier to trade. 3/4 of the return with half the vol sounds pretty good to me. And keep in mind that with full Kelly, you can easily be doing everything "right" and be down 99% (!)
- You can never be sure that your 1 SR is going to hold out of sample... These things have a tendency to start misbehaving once you start trading them. Rule of thumb is to divide your backtested SR by 2.

Your 7 months live performance is encouraging, but far from statistically significant. Remove the mean and resample your returns to work out the probability of observing your 7 month SR with a 0 SR strategy (i.e. what's the p-value of your SR). Let me know if you do this, I am curious!

So, shrinking your SR to 0.5 and betting half Kelly brings you down to a more mundane 25% vol. At 2/20, if your SR stays at 1, your net return becomes ~18%, and your net SR 0.7. That net SR with 40% correlation to the market makes it hard to sell the strategy as a HF (but who's doing HFs these days anyway!). It could still be a decent PA thing if you are allowed by your employer and it's not too labour intensive though.

I'd be happy to look more into it if you send me a PM.

"Earth: some bacteria and basic life forms, no sign of intelligent life" (Message from a type III civilization probe sent to the solar system circa 2016)

Energetic
Forum Captain

Total Posts: 1414
Joined: Jun 2004
 
Posted: 2017-06-15 22:06
I have a bit of trouble applying Kelly in this context b/c I’m used to thinking in terms of series of bets with binary outcomes with p != 0.5, e.g. flipping an unfair coin or playing a roulette with zero on my side. If I define winning as outperforming S&P and choose a time horizon of 2.5 years or more then my chance of winning (in backtest) is 100%. Which takes me right back to question 2. With a strategy that good, why not throw everything onto it? Yes, I could look at it as a sequence of daily bets but I don’t see outperforming benchmark by a fraction as a win in any real sense, there a will be a lot of draws and close calls, so I don’t see a useful definition for a short horizon.

I like the concept of reducing expectations by a factor O(2). Although subjective in choosing exact value, it could lead to a rational quantifiable solution. Still, even with this, I’ll beat S&P every time over 5 years. My time horizon is longer than that so why not put everything in play anyway? Ah, yes, Kelly says ~ 25% based on SR 0.5. My trouble with this is that there is nothing magical about annual SR. On a 2.5 year basis, SR becomes 1.3. And it only gets better with timescale. Yet the scale of 25% seems right even if somewhat conservative. Maybe not even conservative.
The concept suggested by @EspressoLover to view trading a dead process as cost is useful. Although there is nothing that I can estimate objectively, it’s a good reason to knock my expectations down in any calculation.

At the risk of sounding arrogant, there is no math theorem that would convince me to target a 100% vol, especially by leveraging given the fact that I might face a -35% DD. Levering is completely off the table. I don’t know how I will react to a 35% loss if and when it happens in reality. I’d like to think that I’ll be cool but who knows.

I’ve been thinking about building a track record myself. OK, in a few years I’ll be able to show people my trading account statements. Will that do? Unfortunately, my employer in its infinite wisdom, doesn’t allow higher than monthly trading frequency so my returns will probably be lagging. I’m pretty sure I won’t be allowed any investment related business on the side.

@NT (good to see that you’re still around!), my 1 SR does hold out of sample although the definition of out of sample in this context is a little fuzzy in the following sense. I tested O(100) of closely related versions of the strategy in sample but kept for further development only those that held up out of sample as well. I probably ran 20 or 30 or them out of sample so I suspect that in a way it became part of the sample. In particular, I killed branches that reduced SR or drastically increased DD “out of sample”. So, there is a bit of cheating here but I don’t know how much cheating.

I didn’t do resampling exercise that you suggested. Things are going too well (to a large extent by luck) this year so I know the answer w/o doing the calculation: p-value is ~ zero. I agree that 7 months is insignificant but I’m in the middle (or possibly at the end) of an outlier.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

EspressoLover


Total Posts: 240
Joined: Jan 2015
 
Posted: 2017-06-15 22:07
@nerotulip

Yes, you are correct. Should have written that targeted return is [Sharpe]^2. Which, of course implies that targeted vol is [Sharpe]. Thanks for pointing this out!

@energetic

With regard to marketability, have you considered just beta neutralizing? It's at least worth trying in backtest.

There's two approaches, you could either hedge the asset with some S&P 500 instrument on days when you're long. OR you could just permanently hold a short position equal to [Asset Beta]*[Mean Days long]. Either way this should push the long-run beta to 0.

However the relative performance between the two hedging schemes does tells you something interesting. The former should reduce the volatility more than the latter. All things equal that should lead to better Sharpe performance. However if the latter performs significantly better, then that tells you a component of your strategy's edge is coming from timing the general market.


@energetic [w.r.t Sharpe timeframe]

Just wanted to clear up something you alluded to. Especially because my previous errata (corrected by NeroTulip) may have caused this misconception.

Kelly leverage is actually invariant to horizon. Targeted return is [Sharpe]^2. Leverage should equal [Sharpe]^2 / [Expected returns]. Sharpe scales O(sqrt(horizon)) and returns scale O(horizon). A change in horizon cancels out in the numerator and denominator. (Obviously the same reasoning holds true for fractional Kelly as well.)

As an example, let's say it's some strategy with unlevered annual returns of 20% and annualized Sharpe of 1.0. Kelly would say target 100% annualized returns, and lever at 5.0 ratio. Now say you use 2.5 year horizon instead. Unleveled returns at this horizon are 50%. Unleveled Sharpe at 2.5 years horizon is sqrt(2.5). Kelly would say target returns of (sqrt(2.5))^2 = 250%. Kelly leverage would still be 5.0, same as the annualized solution.

NeroTulip


Total Posts: 997
Joined: May 2004
 
Posted: 2017-06-16 03:27
"I tested O(100) of closely related versions of the strategy in sample but kept for further development only those that held up out of sample as well. I probably ran 20 or 30 or them out of sample so I suspect that in a way it became part of the sample. In particular, I killed branches that reduced SR or drastically increased DD “out of sample”. So, there is a bit of cheating here but I don’t know how much cheating."

White's reality check would give you an idea of how much cheating there is. It would give you the p-value of the best in-sample strategy, adjusted for data-mining bias. I can try to run it if you send me the returns of your 100 original strategies.

"Earth: some bacteria and basic life forms, no sign of intelligent life" (Message from a type III civilization probe sent to the solar system circa 2016)

ronin


Total Posts: 216
Joined: May 2006
 
Posted: 2017-06-16 10:43
@Energetic,

coming back to marketability. If you can keep your sharpe reasonaby above 1 once you have removed the correlation with spx as @el suggested, it might be interesting.

But that's a big if. Just naively going on the numbers you gave, your alpha should have theoretical sharpe 0.6, so real life sharpe around 0.4.

Is there anythting you can do to enhance it - bigger basket, more markets etc?


"People say nothing's impossible, but I do nothing every day" --Winnie The Pooh

Energetic
Forum Captain

Total Posts: 1414
Joined: Jun 2004
 
Posted: 2017-06-16 21:37
@EspressoLover

Thanks for the excellent suggestion! Here is what I did. For each trailing month, I calculated a local hedge – a notional for short SPY position per unit of strategy investment. The hedge is used only on the days when the strategy is active (long the asset). The resulting series of returns are uncorrelated with SPY on the annual timescale, although local correlation measured with 21 day rolling window swings to -75% and +70%.

The backtest results are as follows.
In sample, SR stayed flat at 1.15 with vol going down 45%. Max DD got worse from -26% to -34%.
Out of sample, SR went up to 1.4 with vol going down 42%. Max DD improved to from -35% to -13%.
Obviously, with beta I lost a lot of performance, but most of alpha is still there, and it’s a pretty good looking number.

@NeroTulip

Very kind of you but I didn’t keep any records of what didn’t work. I only recorded 5 versions that I considered “final” before I had the itch to go back and try something else. Data-mining bias sounds like the right term for what I suspect I have inflicted upon myself.

@ronin

Apart from tweaking with the algo itself, there isn’t anything else that I can do.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

ronin


Total Posts: 216
Joined: May 2006
 
Posted: 2017-06-19 12:15
@energetic,

Without more information on time frames, the sharpe information is meaningless. E.g. SPX has sharpe 1.4 ytd, but in the long run including crashes it is more like 0.5.

if you can manage sharpe 1.4 in a backtest over a meaningful period of time (10-15y, including stressed periods) with zero correlation and without excessive optimisation, you might have an interesting product.

An easy test for data mining is to fix an objective universe and look at the equity curve by deciles - top 10% performers vs second 10% performers vs third 10% etc. If they look reasonably similar, you are on the right track.

"People say nothing's impossible, but I do nothing every day" --Winnie The Pooh

Energetic
Forum Captain

Total Posts: 1414
Joined: Jun 2004
 
Posted: 2017-06-19 18:17
The sample data set covers 2011-16 which includes the stresses of fall 2011 and 2015-16. The out of sample set is 2004-2010 which obviously covers financial crisis.

Until last week, the out of sample results looked a little less bright which I thought was understandable on multiple levels. But in a beta-neutral version, out of sample performance suffered much less. I guess I could "explain" it but to be honest I didn't expect it at all. It may have happened for a reason, or it may have been by luck, I don't know yet.

Thus, the whole test period is about 13 years. The overall SR is about 1.3
The core strategy has 2 free parameters. Since I'm afraid of overfitting I don't do hard optimization. I just test in-sample on a small 2D grid of integers. E.g. I believe (or know from the prior versions) that parameter A is O(20). So I test A = {18,19,20,21,22}. Similarly for the second parameter, I test B = {11,12,13,14,15}. Hopefully there is a little plateau around the maximum. Then I take the best pair of parameters and try it out of sample. Sometimes, it works right away. Sometimes it’s so bad that I quit the idea w/o further efforts. Sometimes it’s kind of meh so I cheat by varying the optimal parameters out of sample by 1 up and down. If I find substantial improvement within the plateau found in-sample then I’ll keep the new calibration. I guess, this may be part of my data mining bias.

Your last suggestion is not applicable. The universe is just one asset, or more precisely any of a group of similar (equivalent) assets. The strategy tells me to buy it or stay in cash. All in or all out, no shades of gray. So far I haven’t found a way to scale the bet based on “level of confidence”.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

svisstack


Total Posts: 303
Joined: Feb 2014
 
Posted: 2017-06-20 10:59
>> Your last suggestion is not applicable. The universe is just one asset, or more precisely any of a group of similar (equivalent) assets. The strategy tells me to buy it or stay in cash. All in or all out, no shades of gray. So far I haven’t found a way to scale the bet based on “level of confidence”.

The universe in this case couldn't be defined like this?: group of best performing/liquid assets, that belong to different groups of similar/equivalent assets.

This should open your possibilities as you can calibrate your strategy uniquely to every trade-able asset in this new universe and create basket of this strats at the end, it look like and they shouldn't be much correlated if you can define universe like this, after that you can manage this basket to increase SR and scalability.

Time well wasted.

ronin


Total Posts: 216
Joined: May 2006
 
Posted: 2017-06-20 11:19
That's prety biased.

If (20,12) has SR of 1.3 but (19,13) has SR of -8, you can't sell it.

Any investors will probably ask you to settle on one choice of parameters, bump it, and run it over the entire life cycle, plus some random past decades. If your asset didn't exist then, they will ask you to run something related.

There is also the fact that it is just one asset. Either try to incorporate something else, or you are left with targeting the people who focus on that one asset.

The bottom line is that people will buy a story over data any time. If there is a good reason for why it should work, and it sort-of works, that is fine.

"People say nothing's impossible, but I do nothing every day" --Winnie The Pooh

contango_and_cash


Total Posts: 67
Joined: Sep 2015
 
Posted: 2017-06-20 15:13
as someone who has tried pitching a "single asset" type strategy, I agree with this pusback.

you should try to generalize your approach away from a single asset alone. if your idea works for some logical and economic reason finding other products on which to apply it should not be impossible (maybe less lucrative, but potentailly diverisfying (?)).

no need to answer these but starting questions are stuff like what style/risk premia are you capturing? where else does that exist?

if you are capturing some bizarre quirk in the market then sure - but it does not sound like that is the case.

this being said - i have managed a "single asset" strategy since 2013 with a realized sharpe of ~1.5 for myself and i am quite pleased with having this as a way to grow my own money, and establish a proper track record in the meantime.

EspressoLover


Total Posts: 240
Joined: Jan 2015
 
Posted: 2017-06-20 15:53
@energetic

You could try cross-validation to quantify the impact of data-mining. First decide on some fully automated parameter selection method. Then divide the entire sample into bins. I'd suggest months, but years probably works fine.

For each bin, fit the parameters for everything *but* the dates in the bin. Then backtest inside the bin using those parameters. You can then stitch together the months to get an out-sample backtest for the entire series. One more step is to fit the parameter set using the entire series, then backtest the entire series to get an in-sample backtest. The difference between the former and the latter estimates the overfitting bias introduced by parameter selection. Because you're comparing over the same time period, there's less noise then comparing in-sample/out-sample on two different time periods, where the regime may also differ.

Finally let me just add the caveat that this only accounts for overfitting bias from parameter selection. It's important to be aware that there's some overfitting due to selection of the model itself. Presumably before you even started you were exploring and thinking about an array of strategies. We probably wouldn't be here talking about some strategy with Sharpe 0.1. Unfortunately gauging how much overfitting this process incurred is not so easy.

@energetic (2/2)

It may be worth re-trying conviction sizing once beta-neutralized. It may have not looked great before because of the uncompensated volatility from the SPY exposure. It may also be worth considering going short, in addition to flat and long. With beta-neutralization the cost of shorting should be significantly less.

Energetic
Forum Captain

Total Posts: 1414
Joined: Jun 2004
 
Posted: 2017-06-20 21:46
@svisstack

No, that won’t help. I wasn’t doing any massive data mining. I had a very specific idea, I knew that it could work just didn’t know how well. Alas, I can’t easily expand it to other assets. I expect I’d lose much of the juice in the process. But maybe you and other folks are right that I should try.

@ronin

I mentioned plateau. This means I’m getting SR 1.2 instead of 1.3 if I step off by ½ or one. It is relatively stable wrt small bumps. Large bumps can take it under 1 but, believe me, these would still be returns to die for if one could get them in reality. It is indeed one single set of parameters for all 13 years.

Actually, there is a good story. It’s not very complicated but explaining it to a layman could be a challenge. I tried it on college senior majoring in economics and I had a feeling that it was only a partial success, despite the nodding.

@contango (what a great handle!)

Maybe we’re doing something similar. I'd be surprised if people didn’t try something like this before. At least on SR basis, you’re doing better than I but you’re still flying solo. Sounds like you’re considering raising money for your strategy eventually? Do you have views on the length of track record, expected returns etc. that needs be achieved to make it attractive to investors? Now that I have a beta-neutral version, I feel more optimistic but still very doubtful.

@EspressoLover

Your last point is again excellent. It did occur to me as well, kind of. I assumed “something” being linear when I re-parameterized and simplified the model a couple of months before. At that point I thought I was done. But then I questioned myself: why must it be linear? In fact, sqrt would be a more plausible assumption so I tried it and it did work better. And then I asked myself whether I was just fooling myself and the answer is that I don’t know.

I need to think about the rest. Unfortunately, I also need to urgently work on what they pay me for.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

ronin


Total Posts: 216
Joined: May 2006
 
Posted: 2017-06-21 12:30
@energetic,

Based on all this, I'll venture a wild guess that you are trading crude oil in some form.

I don't think you have any chance of selling it to laymen - so if you find yourself sitting in a room explaining it to laymen, you are in the wrong room.

But commodities traders or family offices with maningful exposure to oil might be a good bet.

Good luck!

"People say nothing's impossible, but I do nothing every day" --Winnie The Pooh

Arti


Total Posts: 3
Joined: Jun 2017
 
Posted: 2017-06-21 15:55
He trades VIX and XIV
Previous Thread :: Next Thread 
Page 1 of 1