Forums  > Trading  > Index trading strategy  
     
Page 2 of 5Goto to page: 1, [2], 3, 4, 5 Prev Next
Display using:  

goldorak


Total Posts: 1050
Joined: Nov 2004
 
Posted: 2018-03-28 08:42
> this community doesn't seem to be impressed even with the long-short strategy

I had missed that one. No, not impressed at all. And again, this is not a critics to your work. Look out there, at least at what is public (blogs, papers, etc...). It is the XXIst century: everybody is having a great strategy that works. What people need to understand is that finding a good strategy has become far too easy.

If you are not living on the edge you are taking up too much space.

Maggette


Total Posts: 1067
Joined: Jun 2007
 
Posted: 2018-03-28 10:23
I hope Energetic doesn't feel bashed here. Just wanted to make sure that all participants realize that this discussion is very educating for me and probably many other members.

So thanks Energetic for providing an example! Keep posting guys, I am learning a lot:):

@goldorak
"What people need to understand is that finding a good strategy has become far too easy"
Could you expand on that a bit? Do you mena that with libraries and computing power at hand I can easily data mine the world and find something that appears to be working if I am looking long enough...even though in reality it most probably isn't working?

Ich kam hierher und sah dich und deine Leute lächeln, und sagte mir: Maggette, scheiss auf den small talk, lass lieber deine Fäuste sprechen...

ronin


Total Posts: 384
Joined: May 2006
 
Posted: 2018-03-28 11:12
To come back to @goldorak's point.

@energetic, you are running two strategies in parallel.

The long strategy is a low volatility tracker, and it's fine to look at it using Gaussian statistics. It will do roughly what spx does, plus minus a random factor.

It's the short strategy that you have to worry about. You are relying on it periodically jumping into the abyss, and coming back with high returns. There is no way that you can measure a strategy like that with Gaussian numbers. You need to look into the tails a lot more closely if you want to run something like that.

I hope you are not discouraged by the feedback. I actually think you have something there (unless its all coming from looking forward, which you'll know soon). But it still needs a lot of work.


"There is a SIX am?" -- Arthur

EspressoLover


Total Posts: 348
Joined: Jan 2015
 
Posted: 2018-03-28 12:24
Parameter (over)fitting

I think I've mentioned this to to you before, but I'd suggest using hold-one-out cross validation. Sub-divide into months, then fit your parameters using all the data *except* that month. Then use those parameters to generate signals for that month. Stitch together that signal series and you now are working with pure out-sample with regard to parameter selection. (The caveat being, that the subjective model selection is still subject to a certain degree of selection bias.)

Asynchronous Close Times

Contra the others, I don't see this being a huge issue. I guess the biggest question are you using 1-day lagged deltas on the VX curve or not? (You can still use 1-day deltas on S&P, because that's synchronized.) Like, are you looking at total contango in the curve or change in contango from yesterday. If the latter, then I'd be a lot more concerned about lookahead bias, and really try to get synchronized data. But if you're just looking at absolute levels, then the variance between 4:00 PM curves and 4:15 curves are pretty miniscule.

If anything the lookahead bias would probably make the backtests underestimate performance in this case. As a toy model, I'm assuming the rough idea is to be more long the S&P 500 when VIX curve is backwardated. The idea being that during periods of market distress, the time-varying equity risk premium is temporarily elevated. If VX becomes backwardated from 4:00->4:15, then the market probably sold off over that time. Hence if you're using 4:15 curves, your backtest is probably biasing long SPY on 4:00->4:15 selloffs and vice versa. (Whereas if you're using 1-day deltas in the curve, then this effect goes the other way, but much stronger.)

Benchmarking

I tend to agree with other posters. I think this is much more of an absolute return product, then an enhanced beta product. (Which overall is a good thing - absolute return managers get paid more). Just because you're trading the S&P 500 doesn't mean that it's the proper benchmark. Unless the correlation varies a lot year to year, long-run correlation is pretty indicative. Being +/- 100% at certain times doesn't really matter, as long as it washes out over the accounting cycle. To take this reasoning to its limit, nobody cares about the beta exposure of an HFT strategy on ES.

Risk and Tails

Despite what I said about the S&P500 not being the appropriate benchmark, the strategy would have a pretty similar risk profile to the index. From a risk perspective you're basically taking daily SPY returns and randomly flipping the signs. Now hopefully for expected returns, you're doing that in the right way. But from the perspective of higher-order moments, it'd be pretty unlikely that this would produce any serious divergence from SPY's distribution.

You're not doing anything crazy like doubling down into losses, or holding positions until they hit some preset gain. Any kind of autocorrelation in SPY daily returns is going to be pretty minimal. Skew's almost always negative in SPY, so the fact that you're short a significant percent of times should mean your strat has better skew than SPY. I'd be really hard to believe that this has any sort of dangerous hidden tail risk (or at least not any that S&P500 doesn't already suffer from).

IMO the biggest risk is if the signal is 0, either because it reverted or was never truly there to begin with. In which case you're eating SPY volatility without any tailwind. That scenario will make your long-run drawdowns significantly higher than SPY with its long-run positive return. For example going back to 1995, the biggest drawdown compounded SPY had was 50%. But if you re-normalize daily returns to zero-EV, that increases to 75%.

Good questions outrank easy answers. -Paul Samuelson

Energetic
Forum Captain

Total Posts: 1491
Joined: Jun 2004
 
Posted: 2018-03-28 17:11
Everyone,

I don't feel bashed, beaten or otherwise.I got good chin on me and thick skin too. I appreciate pointed questions, suggestions and criticism as well. I'll be back with answers a bit later.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

Energetic
Forum Captain

Total Posts: 1491
Joined: Jun 2004
 
Posted: 2018-03-28 23:11
@goldorak

I think I understand what you are proposing but the exercise seems to be in the spirit of garbage in -> garbage out. In other words, the outcome of any shuffle applied to my predetermined {LS} sequence has nothing to with any possible version of future reality. Or maybe I'm still not getting it.

Can't one live with simplistic-deterministic rules, e.g. deciding in advance that the strategy is dead if it registers a DD X% deeper than observed in testing, or having an annual return Y% worse than ever before, or having longer streak of losing months longer than ever before by N. I don't know what values of X, Y and N would be reasonable but I'm not sure why such approach is worse than what (I think) you're proposing.

The point about 1995-00 is fair. Indeed, there is nothing quite like that in my sample. VIX is of course one of the inputs but there're a few others. It's impossible to say how it would play out.

It's ok if you're not impressed, none taken, even though of course I do want to impress everybody, but please humor me. If you're a mutual fund or wealth management company, why wouldn't you be interested in a product like this if you're currently putting a lot of money into passive index tracking? Yes, it has its own risks and drawbacks but we already know a lot about the risks of passive investing. Who wants to be in the -55% hole again?

@ronin

You're right and you're more right than you probably know. It is indeed two strategies (3 parameters for each) combined together. The long leg is, for lack of better word, fundamental. The short leg is an ad hoc attempt to catch downward trend. Not b/c I wanted it but b/c I couldn't figure out a fundamental model for shorting. By the end of the day - whatever works, right?

> You need to look into the tails a lot more closely if you want to run something like that.

This is where I may be out of my depth. I'd really appreciate specific suggestions. And thank you for your feedback!

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

Energetic
Forum Captain

Total Posts: 1491
Joined: Jun 2004
 
Posted: 2018-03-28 23:53
Those who were worried about open vs. close were right: if I use next day open prices, the return drops to 15.5%. This means that there is a significant correlation between my signals and Open-Close move. Interesting...

The good part is that 99% of the time I know the next day position well in advance of 4 pm.

@EspressoLover

Thanks a lot! I'll try to digest and reply tomorrow.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

Energetic
Forum Captain

Total Posts: 1491
Joined: Jun 2004
 
Posted: 2018-03-29 23:17
@EspressoLover

Thanks for your comments again! This time I did understand what you mean by hold-one-out cross validation. It's a great idea. It's not easy for me to implement b/c I don't really calibrate my parameters for fear of overfitting but maybe for the sake of this test I should. This will take me some time to work out.

> As a toy model, I'm assuming the rough idea is to be more long the S&P 500 when VIX curve is backwardated.

Yes, that was the first attempt. It's way more complicated now but that's the spirit.

> If VX becomes backwardated from 4:00->4:15, then the market probably sold off over that time.

Yes, I agree. When my signal changes sign from 4:00->4:15 I introduce bias. E.g. I short at 4 pm prices when 4:15 prices are lower. Empirically, it happens very rarely though.

No, I'm not doing anything crazy. The long term correlation to SPY daily returns is upper teens. But local correlation could be very high b/c the strategy sometimes stays long for weeks. Could it be a concern for someone who needs to report monthly (or even quarterly)?

> I'd be really hard to believe that this has any sort of dangerous hidden tail risk (or at least not any that S&P500 doesn't already suffer from).

My thoughts exactly. I am puzzled that some other commenters are concerned about that.

I didn't understand the very last paragraph about the risk of staying in cash. Especially the last sentence.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

nikol


Total Posts: 573
Joined: Jun 2005
 
Posted: 2018-03-30 19:45
try this?
IBKRAM

Energetic
Forum Captain

Total Posts: 1491
Joined: Jun 2004
 
Posted: 2018-03-30 21:19
Not sure what to try Smiley

So far I tried to compare my strategy to the best investment portfolios shown on the page you linked.

The best Sharpe Ration shown there for the last 365 days is 1.74. Mine is 3.02.

By performance, I would be ranked second with 36.2% return over last 365 days vs. 48.1% for the leader and 26.8% second place.

What I would like to try is to somehow communicate to these people (and the rest of the world) that I might have something worth looking at Wink

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

EspressoLover


Total Posts: 348
Joined: Jan 2015
 
Posted: 2018-03-31 03:54
@energetic

> I didn't understand the very last paragraph about the risk of staying in cash. Especially the last sentence.

Sorry. I think my wording was ambiguous. I said "IMO the biggest risk is if the signal is 0". What I meant was "the biggest risk is if the signal has 0 predictive value". (Not "if the signal has 0 magnitude and stays in cash".) My point was the biggest thing that would keep me up at night about this investment is whether the backtests were overfit, whether the regime permanently changed, or if the production implementation diverges from the simulation in some subtle way.

There's two ways to think about investment risk. One is in terms of higher order moments of the probability distribution of returns. I.e. volatility, skew, tails, etc. The other is in terms of how much money am I putting at risk. Things like max drawdown, shortfall risk, VaR. With the former metrics, expected returns don't matter. Whether a strategy has alpha of 0% or 50% a year, ipso facto that doesn't affect volatility or kurtosis. However it may significantly change its max drawdown. Having high returns buffers against the latter type of risk, because it acts as a tailwind against cumulative losses.

Like I mentioned before I wouldn't worry too much about the former type of risk metrics for this strategy. (At least anymore than I would worry about those things for my S&P 500 investment.) But I would try to get a very strong handle on the model's generalization error, have tight pre-defined criteria for rejecting the null hypothesis, carefully reconcile live trading against simulations, and understanding where exactly my live results fall in the predicted distribution.

The strategy in backtest appears to have very good risk metrics: no down years, half the drawdown of SPY, drawdown only equal to one year's average return. But that's largely an artifact of very high returns relative to the underlying. If in live trading, the returns fall short of backtest expectations, the above risk metrics will deteriorate significantly.

In short, my opinion is the biggest risk is not delivering the forecasted returns.

> But local correlation could be very high b/c the strategy sometimes stays long for weeks. Could it be a concern for someone who needs to report monthly (or even quarterly)?

Depends on the investor. I'd maybe take a look at the distribution of month-to-month, quarter-to-quarter, and year-to-year betas. Just to quantify the magnitude of the effect. Different investors have different mandates regarding market neutrality, and how that's defined. Staying long (or short) for too long may get you re-classified from Equity Market Neutral to Systematic Macro. (Which is a worse space to raise money in if you're not already a big name.)

I'd focus on making the system as strong as you can, shop it around at places like Millennium, then see what kind of feedback you get. If push comes to shove, and you're getting too much grief for being long-beta, you can always make tweaks to the program to keep it neutral over more granular time periods. For example you could bias in the opposite direction of the mean beta from the past 90 days.

However, my intuition would be that this type of adjustment doesn't affect performance significantly. It seems like the essence of the system is to exploit the time-varying component of the equity risk premium. If the VX curve exhibits some particular shape over a very long period, I'd suspect that's more reflective of persistent vol-specific dynamics. Whereas a temporarily dislocated VX curve is more indicative of market wide stress. It's nothing more than a wild-ass guess, but I'd bet that the performance heavily concentrates around the points when the signal flips, not when the signal sits stagnant in one direction.

Good questions outrank easy answers. -Paul Samuelson

nikol


Total Posts: 573
Joined: Jun 2005
 
Posted: 2018-03-31 17:32
> Not sure what to try

I meant publishing and getting the fee.

> The best Sharpe Ration shown there for the last 365 days is 1.74. Mine is 3.02.

At the beginning you posted modest 1.1 ! Did you change something?

Energetic
Forum Captain

Total Posts: 1491
Joined: Jun 2004
 
Posted: 2018-03-31 21:01
@EspressoLover: Thank you, understood now.

> But I would try to get a very strong handle on the model's generalization error, have tight pre-defined criteria for rejecting the null hypothesis, carefully reconcile live trading against simulations, and understanding where exactly my live results fall in the predicted distribution.

I'll be thinking more about all of this. So far my live trading is far better than simulations on average. In 2017 I was slightly behind S&P as happened during bull markets before. Feb-Mar was fabulous as were significant bear markets in the past. But this is not a very quantitative assessment.

> If the VX curve exhibits some particular shape over a very long period, I'd suspect that's more reflective of persistent vol-specific dynamics.

Yes, something like that.

> I'd bet that the performance heavily concentrates around the points when the signal flips, not when the signal sits stagnant in one direction.

Well, the model looks only in the past. The signal often flips b/c the market already fell (and the strategy was long) or the other way around. Most runs begin from a hole. A lot of performance comes from not being long (better being short) during the bear markets.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

Energetic
Forum Captain

Total Posts: 1491
Joined: Jun 2004
 
Posted: 2018-03-31 21:08
@nikol

I'm sorry for being so obtuse. I did look around the site but didn't see a way to hook up for publishing my trades. Could you point to what I'm missing?

The modest 1.1 was for the whole period from 2004 to now; 3.02 is for the last 365 days to compare apples to apples. Surely last year was very good for me but for all them as well.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

nikol


Total Posts: 573
Joined: Jun 2005
 
Posted: 2018-04-02 15:06
Not sure, but may be this will help.



Do you have an account with them? Login and look inside too.

Energetic
Forum Captain

Total Posts: 1491
Joined: Jun 2004
 
Posted: 2018-04-02 21:11
Duh Blush

Thank you! I called them. It turns out you need to be a registered adviser to participate. This may not be very hard to do except they want something like 5 or 10 mln AUM. In addition, as long as I am employed where I am right now, I'm under a 30 days holding restriction which will leave me hobbled at best Cry

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

ronin


Total Posts: 384
Joined: May 2006
 
Posted: 2018-04-03 13:14
I wouldn't feel too bad about that. I don't think IB is a meaningful source of business for anybody.

You are stuck with looking for professional seeders.

BTW, I don't necessarily believe that your short leg is trend following. There is no way that would give you the numbers that you have. There is something else there, and that something else may have potential to go toxic. @goldorak's advice is pretty sound - mix up your signal and see what happens when you go short randomly, or with some delay, or if the signal flips, or anything else you can think of.

The thing about the long and the short legs being separate is actually pretty standard among long-short strategies. As a wild guess, I would say that 50% of strategies do something like that.

"There is a SIX am?" -- Arthur

nikol


Total Posts: 573
Joined: Jun 2005
 
Posted: 2018-04-03 16:55
What they do makes sense from compliance PoV.
Actually, they bought out this idea from covestor.com which was a very popular signal republishing platform.
Quantopian follows the same path, but don't expect golden shower falling on you. Serious investors will take you serious only if you "burn all your bridges" behind. Otherwise, it can be pure luck...

HitmanH


Total Posts: 465
Joined: Apr 2005
 
Posted: 2018-04-03 20:45
I think IB has actually been doing reasonably well in gathering assets it advises this way.

Given it's a systematic strategy - can you find a friendly RIA, let them run and deal with compliance (for which they will of course retain a slice) but essentially keep it in their name, until you're ready to make the plunge?

Energetic
Forum Captain

Total Posts: 1491
Joined: Jun 2004
 
Posted: 2018-04-04 00:24
Great idea but I'm fairly recent in the algo world, I don't know anyone like that.

I was thinking about giving a talk at some conference but I suspect there must be 1000 times more upstart strategists than speaking slots.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

Energetic
Forum Captain

Total Posts: 1491
Joined: Jun 2004
 
Posted: 2018-04-04 16:20
OK, by popular demand of @goldorak and @ronin, I played the shuffle game, i.e. signals were frozen but returns shuffled.

Results: among about 30 shuffles tried, most fell a few percentage points below SPY's 8% annual returns. The best observation was 13%, the worst -3%.

However the MaxDD is typically above SPY's -55%. It seems like the probability of deeper DDs is low. Only once I saw -60. More often in the -30s and -40s.

@ronin

>BTW, I don't necessarily believe that your short leg is trend following. There is no way that would give you the numbers that you have.

Strictly speaking, you're right. It's not like I am literally trying to decide whether there is a trend and if so jump on it. I tried that too and failed. The implementation is quite different but is designed to achieve the same effect in a roundabout sort of way.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

ronin


Total Posts: 384
Joined: May 2006
 
Posted: 2018-04-05 11:49
> among about 30 shuffles tried, most fell a few percentage points below SPY's 8% annual returns. The best observation was 13%, the worst -3%.

That is sort-of a no brainer. If you are randomly long SPY 2/3rds of the time, you will on average make 2/3rds of SPY. The question is how skewed that distribution is. I am guessing that the average return over the reshuffles isn't 5%.




"There is a SIX am?" -- Arthur

Energetic
Forum Captain

Total Posts: 1491
Joined: Jun 2004
 
Posted: 2018-04-05 16:50
The mean of observed annual returns over the set of reshuffles is 4.9% with the std 4.5%

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

goldorak


Total Posts: 1050
Joined: Nov 2004
 
Posted: 2018-04-08 16:57
Sorry for the delayed answer. Last week was skiing time.

The absolute results you mention are not really interesting. What is of interest is the performance of the strategy relative to the S&P 500. What is the maximum drawdown you get with respect to the performance of S&P 500?

30 shuffles, seriously? Go and run 10000 of them! Nowadays, on daily data over 15 years you should need about half a second to run this.

Once you are done with shuffling daily returns, try to shuffle batches of 2, 3, 4, 5, 10, 20 and 60 days, trying to take market persistence into account. There are more advanced ways to do this, but this is not really important.

> If you're a mutual fund or wealth management company, why wouldn't you be interested in a product like this if you're currently putting a lot of money into passive index tracking? Yes, it has its own risks and drawbacks but we already know a lot about the risks of passive investing. Who wants to be in the -55% hole again?

I used to work as an allocator for a pretty large pension fund a long time ago. I had that kind of strategies landing on my desk every month, and believe me none of them produced anything, and [2007 and/or 2008 and/or 2009] were a bloodbath for most of them. The guys who replaced me told me that over the last 5 years timing strategies based on the VIX futures curve have been flourishing and all of them looked great... in the past.

> Can't one live with simplistic-deterministic rules, e.g. deciding in advance that the strategy is dead if it registers a DD X% deeper than observed in testing, or having an annual return Y% worse than ever before, or having longer streak of losing months longer than ever before by N. I don't know what values of X, Y and N would be reasonable but I'm not sure why such approach is worse than what (I think) you're proposing.

Lots of people think that way out there and this is very disappointing. Quants paid big bucks to come out with such solutions are really lucky!

A final comment from my side. Just think about it: what would you have done would the strategy be in -20% drawdown relative to S&P 500 year to date? See my point?

If you are not living on the edge you are taking up too much space.

Energetic
Forum Captain

Total Posts: 1491
Joined: Jun 2004
 
Posted: 2018-04-08 21:38
> What is of interest is the performance of the strategy relative to the S&P 500.

Performance of S&P is obviously the same every time. So after shuffle I'm about 3.2% under S&P on average.

I have a feeling that you're pretty much convinced that my strategy is crap (which is fine, nobody believes in market timing strategies these days ) and are thinking of a way to demonstrate it (which is even better for me b/c I do want to dispel doubts).

Surely, I can do it 10000 times and I can do it in many ways. The question is: what are you looking for? What would be a sign of a potential problem?

> I used to work as an allocator for a pretty large pension fund a long time ago. I had that kind of strategies landing on my desk every month, and believe me none of them produced anything, and [2007 and/or 2008 and/or 2009] were a bloodbath for most of them. The guys who replaced me told me that over the last 5 years timing strategies based on the VIX futures curve have been flourishing and all of them looked great... in the past.

You already know that my strategy did great in 2007-09 and it was out of sample. And it is doing very well live, now, not just in the past. Which is why I think a lot of people should be interested.

> what would you have done would the strategy be in -20% drawdown relative to S&P 500 year to date? See my point?

No I don't. Backtesting shows that such a temporary underperformance is not unprecedented in a bull market. So, if it happens during the bull market I'd do nothing, just stay the course. If it happens during bear market I'd shut it down and crawl back to under my rock to cry.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken
Previous Thread :: Next Thread 
Page 2 of 5Goto to page: 1, [2], 3, 4, 5 Prev Next