Forums  > Trading  > Index trading strategy  
     
Page 3 of 5Goto to page: 1, 2, [3], 4, 5 Prev Next
Display using:  

Maggette


Total Posts: 1052
Joined: Jun 2007
 
Posted: 2018-04-08 22:12
He is looking for a way to break it obviously. As should you. Which is fine. Falsfication is key:)

Ich kam hierher und sah dich und deine Leute lächeln, und sagte mir: Maggette, scheiss auf den small talk, lass lieber deine Fäuste sprechen...

tabris


Total Posts: 1255
Joined: Feb 2005
 
Posted: 2018-04-09 04:28
Maggette hit the point on the nail. It is simply testing for boundary conditions to see if anything breaks. If there is an underlying hidden structure that you data mined to which is not obvious from first and second order performance statistics, random shuffles is another way to look deeper into these performance statistics.

Second, out of sample and live performance is great and all but what do you know about the system? How much do you need to lose to turn the system off/consider it to be not working now or ever worked in the past? This is also where reshuffling based on time blocks can help. You look at your probability of maximum drawdown in a randomized scenario benchmarked to the market.

Because if you truly believed your system works, then in Goldoraks example of 20% underperformance vs S&P, you would double down but your investors or your margin account would say otherwise...

Dilbert: Why does it seem as though I am the only honest guy on earth? Dogbert: Your type tends not to reproduce.

tradeking


Total Posts: 16
Joined: May 2016
 
Posted: 2018-04-09 04:39
Have you tried to apply this strategy on other equity indices (e.g. European/Asian analogs)?

The main issue with your strategy is that it's a daily strategy with the sharpe of a weekly/monthly strategy. This implies that your return per trade is very low, and that the strategy will not scale well.

In a 2x2 matrix of (high/low sharpe x high/low capacity), interesting strategies usually fall into one of two quadrants:
1) high sharpe (>2) with low capacity (<$100mm)
2) high capacity (>$1+ billion) with low sharpe (~1)
High sharpe, high capacity would obviously be the holy grail, but not realistic in today's markets.

From what you have mentioned in the thread so far, it unfortunately sounds like you are in the wrong quadrant of (low sharpe, low capacity). If you can find a way to diversify your signal to bump the sharpe up to 2 or so, then it could be something interesting. Or if you can find a way to slow down your signals so that you only trade once a week or month while keeping the same sharpe, that could be interesting too.

goldorak


Total Posts: 1045
Joined: Nov 2004
 
Posted: 2018-04-09 12:16
> Performance of S&P is obviously the same every time. So after shuffle I'm about 3.2% under S&P on average.

Worst drawdowns Energetic, drawdowns! Why would you care about averages ?

> You already know that my strategy did great in 2007-09 and it was out of sample.

And if you had not outperformed in that particular period, you would not be introducing the strategy now! See my point ?

If you are not living on the edge you are taking up too much space.

goldorak


Total Posts: 1045
Joined: Nov 2004
 
Posted: 2018-04-09 12:30
from tabris:

> Because if you truly believed your system works, then in Goldoraks example of 20% underperformance vs S&P, you would double down but your investors or your margin account would say otherwise...

This is just SO true. Managing OPM is a nightmare. What you need are real stakeholders, with skin in the game, who are able to understand and stay above this regular non sense idea of a "3 years underperformance" or "max drawdown".

No Energetic, I do not think your timing strategy is crap and I am not trying to destroy it. I am just trying to give you the "right" way of analyzing your strategy so you can manage it efficiently in the future and actually make the most out of it. Regarding its value, considering the number of degrees of freedom you can find in a combination of VIX futures and S&P 500, it is pretty easy to get something working, until it does not anymore. The last time I got interest in this universe of opportunity, I got scared by how easy it is to find pretty good strategies... in hindsight. The "I did not try to optimize or overfit" is unfortunately not an argument in itself.

My main advice is just the following: try to get out of this "sharpe ratio" and "great performance in the past" mainstream analysis. It is common to use these tools now because this is what clueless bosses understand nowadays. Clueless bosses 20 years ago understood different things back then. Let's hope in 20 years bosses will understand better.

Be you own boss !

If you are not living on the edge you are taking up too much space.

Energetic
Forum Captain

Total Posts: 1488
Joined: Jun 2004
 
Posted: 2018-04-09 17:42
Yes, Maggette. I kinda figured out that he wants to break it, and I really appreciate the help and patience of goldorak.

I just think that falsification criteria must be set before testing, that's what I was asking. My initial interpretation of the shuffle tests was that it was intended to check whether the strategy would continue to beat S&P even with signal lost in the shuffle. which would obviously indicate a problem. Absolutely reasonable, which is why I did it. In my opinion, a problem of the kind that I had in mind would readily manifest even after a small number of shuffles. But goldorak didn't think it was enough, so he probably meant something else so I'm truing to clarify what it was.

@tabris

> How much do you need to lose to turn the system off/consider it to be not working now or ever worked in the past?

I already gave some possible "rule-of-thumb" answers to this questions. I've been thinking about approaching this question somewhat systematically. Maybe I'll start a new thread when I have something semi-intelligent to say.

> You look at your probability of maximum drawdown in a randomized scenario benchmarked to the market.

Please elaborate. Let's say I ran a gazillion of shuffled simulations and estimated distribution of the strategy drawdowns vs. S&P. Then what? What would constitute a failure and why?

> Because if you truly believed your system works, then in Goldoraks example of 20% underperformance vs S&P, you would double down but your investors or your margin account would say otherwise...

No, as I already said, this depends on the circumstances.

@ tradeking

You call trading S&P low capacity? Well, what is high capacity then?

No I didn't try other equity indices yet.

The strategy trades about once a week on average.

@ goldorak

> Worst drawdowns Energetic, drawdowns! Why would you care about averages ?

Sorry, I misconstrued the intent. Do you mean the strategy DDs, or DDs vs. S&P or both? What would be a red flag?

> And if you had not outperformed in that particular period, you would not be introducing the strategy now! See my point ?

Yes, I do this time. And, once again, thanks for your help. I wouldn't have thought of the tests that you are suggesting myself.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

tradeking


Total Posts: 16
Joined: May 2016
 
Posted: 2018-04-09 17:54
>You call trading S&P low capacity? Well, what is high capacity then?
S&P is liquid but it is not infinitely liquid. If you try to enter into a position of $1bn in the span of an hour before the market close, it WILL move the markets. Whereas if your strategy is slow and can take up to a week to enter into the position with a VWAP, that's a different story.

Like I said, your strategy can probably make $10mm/yr in PNL as it is (putting it in the low capacity category), but I doubt it can generate $100mm in PNL. You can try seeing for yourself in your backtest what kind of position sizes you would need to generate that magnitude of PNL, and how much time you would have to enter into those positions.

Energetic
Forum Captain

Total Posts: 1488
Joined: Jun 2004
 
Posted: 2018-04-09 18:05
> The last time I got interest in this universe of opportunity, I got scared by how easy it is to find pretty good strategies... in hindsight.

That's an interesting comment on multiple levels.

I agree it was kind of easy in the sense that I concocted first relatively good looking version in one afternoon. However I wasn't scared, I was excited. Maybe it was silly of me but I don't know why.

It wasn't really easy to get it to the state where it is right now. Not for me, that is.

I was very doubtful at first, too, and hesitated committing a lot of my personal money. But 1.5 years of live trading certainly added a lot of confidence. Otherwise I wouldn't show my face here.

None of this proves that the strategy will continue performing forever, of course, but it's no longer just hindsight.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

involution


Total Posts: 6
Joined: Nov 2015
 
Posted: 2018-04-09 19:10
Hi Energetic, maybe part of what Goldorak is getting at with the shuffling (apologies if I'm misunderstanding) is to look at the risk of your strategy (drawdowns) once the signal disappears.

For example, suppose your signal becomes uninformative after time T_1, but you don't realize it until time T_2. Then in the interval [T_1,T_2], your stream of positions ({L,S}) will have some return distribution.

Energetic
Forum Captain

Total Posts: 1488
Joined: Jun 2004
 
Posted: 2018-04-09 20:06
@ tradeking

I see your point. Maybe you're right. I was focusing on signals, not on execution. Since the signal is usually robust, I won't necessarily be constrained to the last hour but still this is a separate problem to be addressed.

There are many ways to execute (futures and multiple ETFs) and maybe even mutual index funds like Vanguard's when I need to go long?

In any case, do you think that there are people somewhere who actually want to make $10 mm/yr ? Wink

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

Energetic
Forum Captain

Total Posts: 1488
Joined: Jun 2004
 
Posted: 2018-04-09 20:10
@involution

Maybe but one still needs to set the pass/fail criteria before the test, no?

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

goldorak


Total Posts: 1045
Joined: Nov 2004
 
Posted: 2018-04-09 20:55
> Sorry, I misconstrued the intent. Do you mean the strategy DDs, or DDs vs. S&P or both? What would be a red flag?

DDs vs S&P. Let's assume your signal is currently very informative. You have outperformed S&P 500 by almost 60% in 2008. Now the question is: with zero information (which you obtain at "first order" with a random shuffle of returns), could you get the same 60% reverted, I mean underperform rather than outperform the S&P 500 by 60% ?

You may find that drawdowns and draw-ups with shuffled returns are mostly contained in, let's assume for the sake of the argument, a +/- 25% of the S&P 500. Now two points:

1) This means that getting a draw-up of +60% like you got back in 2008 is kind of an argument in favor of your signal being really loaded with information and probably worth a lot (at least was worth a lot back then). If the max drawdowns / draw-ups you get with shuffled returns are, let's assume again, around +/- 70%, all of a sudden the results from 2008 are not that exciting anymore because with the kind of trading patterns generated by your strategy, you could have got that kind of out/under performance by luck.

2) But this means something else too: the risk you need to be willing to take in the medium term is to underperform the S&P 500 by about 25% and never make that money back, because your signal is not eternal and will lose its informational content (by others arbitraging it, because market's dynamics changed, etc...).


Re: the strategy itself. I am not willing to be pretentious here, but I reasonably think it would be possible to reverse engineer your strategy just having the daily returns at hand and knowing that it relies only on S&P 500 and VIX futures as inputs. I would recommend you to be very cautious about this.

If you are not living on the edge you are taking up too much space.

Energetic
Forum Captain

Total Posts: 1488
Joined: Jun 2004
 
Posted: 2018-04-09 23:23
Thank you, understood, I'll be back.

Someone else already privately mentioned to me about the possibility of reverse engineering. My first reaction was - nah, too complicated. On second thought, maybe not THAT complicated. I'll keep it in mind.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

tradeking


Total Posts: 16
Joined: May 2016
 
Posted: 2018-04-10 04:15
> In any case, do you think that there are people somewhere who actually want to make $10 mm/yr ? Wink

There are some funds out there that could be interested, but the payout for low sharpe, high-beta strategies is generally low (think low single digit % of pnl). That is why the only way to make low sharpe strategies worthwhile is to do the strategy on large scale.

goldorak


Total Posts: 1045
Joined: Nov 2004
 
Posted: 2018-04-10 06:20
> My first reaction was - nah, too complicated

Just think about it, you do not need to exactly replicate the strategy, just to replicate the returns. Knowing the decision set and working with reinforcement learning, you do not need that many data points to be able to learn, track and even adapt (in case the manager changes his strategy).


If you are not living on the edge you are taking up too much space.

ronin


Total Posts: 327
Joined: May 2006
 
Posted: 2018-04-10 14:21
I guess my main concern goes in a different direction than @goldorak's and @tradeking's.

Their objections can largely be addressed by sizing. In a nutshell, don't go crazy because it will hurt you when the signal decays.

My problem is on the signal side.

You would expect the flow of information to go from more liquid instruments to less liquid.

So I personally would expect to see new information first reflected in e-minis, then single stocks, then options on futures, then VIX.

@energetic, you seem to see a flow of information going the opposite way. Information starts at the least liquid instrument, and then it works its way to the most liquid.

Why would information flow that way?

I see why it works when you are looking into the future. VIX at T+15 min has more info than e-minis at T+0 - fair enough, I won't argue with that. But I don't see why VIX at T+0 would have more information than e-minis at T+0.


"There is a SIX am?" -- Arthur

darkmatters


Total Posts: 74
Joined: Nov 2010
 
Posted: 2018-04-10 14:50
Occassionally if someone is rebalancing a basket, the signature of their trades will show up first in the least liquid instrument.

Or, another way, typically the people that speculate using options would be the most informed traders (taking on the most risk), so it would appear in VIX first.

Energetic
Forum Captain

Total Posts: 1488
Joined: Jun 2004
 
Posted: 2018-04-10 16:52
@ronin

> I personally would expect to see new information first reflected in e-minis, then single stocks, then options on futures, then VIX.

I don't have any strong beliefs. I started this project to see whether it could work and, conditional on all passing various tests, it kind of did. For me, the only question is why.

The honest answer is I don't know. Equity and options markets react to the same news at the same time. It is also true that options traders take equities markets moves as input as well. However the derivative dVIX/dS&P is very unstable. Clearly, some other thinking goes into the process. There is no reason why options traders reactions could contain information about the future market movements other than the fact that some of them are, by nature of their trade, are more forward-looking.

I am not dismissing the time basis issue altogether but, like I said before, 99% of the time there is no need to trade close to 4 pm. This is not a big problem.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

NIP247


Total Posts: 544
Joined: Feb 2005
 
Posted: 2018-04-10 19:28
My 2c on options: for single stocks, given the cost of trading options (liquidity and spread), all else equal you'd expect price taking flow to come from "informed" traders. For liquid index options, I'm thinking the changes in prices/curves that look like signals might be adjusments for expected risk that rarely realises, and certainly didn't realise in a backtest, but were nonetheless real ( so simplistically, a "very high vol" is a "buy signal" because 9 times out of 10 it was a bottom, but the tenth time that you didn't witness would have brought you catastrophically further down )

On your straddle, done on the puts, working the calls...

ronin


Total Posts: 327
Joined: May 2006
 
Posted: 2018-04-11 14:15
Yeah, no.

Option buyers are long only managers who have to hedge for regulatory purposes. They are not buying options because they have doubts about their holdings - if they have doubts, they decrease the exposure.

Option sellers are options desks who are flat delta and running down gamma. Their delta hedging isn't driving S&P stocks either, delta hedging is just a speck in the ocean.

I am sure there must be somebody somewhere who has a signal on stocks and trades options. But they can't be too big or too sophisticated. I would bet that it is 98% overexcited MIT undergraduates after their first lecture on Black Scholes.

"There is a SIX am?" -- Arthur

NIP247


Total Posts: 544
Joined: Feb 2005
 
Posted: 2018-04-11 14:34
@ronin, you misunderstood my point. I was answering the question "why look at the least liquid instrument" above. So I am saying what you obviously know, but restating for completeness:
* Implied vol and term structure can be seen as a the market's assessment of future risk.
*They provide information that is not available just looking at the underlying. That information is not observable directly in the underlying even though the underlying is more liquid ( this relates to "why look at the least liquid instrument question above).
* So some people will use that information to construct signals seeing what they qualify as a dislocation ( certain term structures, certain levels of vol / relative vol / impl. vs realised, dispersion levels etc. ) as having predictive value for the underlying instruments, whereas another interpretation might just be that what is seen as a "dislocation" is a correctly priced risk that hasn't appeared in backtests. ( again, a comment not necssarily directed at you, because none of this is news to you, but your answer was besides the point I tried to adress )


[EDIT: wrt to whether hedging affects derivatives positions, certain vol instruments reference the close of the underlying so the delta hedge happens on the close, and hedge volumes can in certain circumstances, get bigger than the close can bear. Then some start to prehedge the end of day close, and it all gets fuzzy. In the end, you're not sure how the underlying is affected by the hedges and front-running of expected hedges etc. but the market still moves in those situations as a consequence of derivatives on the underlying. (and again Ronin, you know this, so I'm surprised you'd say that the underlying is not affected, esp. at the edges. ]

On your straddle, done on the puts, working the calls...

ronin


Total Posts: 327
Joined: May 2006
 
Posted: 2018-04-11 15:45
> @ronin, you misunderstood my point. Implied vol and term structure can be seen as a the market's assessment of future risk. They provide information that is not available just looking at the underlying. That information is not observable directly in the underlying even though the underlying is more liquid ( this relates to "why look at the least liquid instrument question above).


All fair enough.

But my point was specifically about what is leading and what is trailing.

I have never found a leading indicator in options (and I have looked), and I could never find a non-hand-waving explanation for why there would be a leading indicator in options.

Puts go up after the stock tanks - no prob. But they don't go up before the stock tanks. They don't even go up before numbers, which is what they would do if they were a serious indicator of risk. If you think the stock will tank, you sell the stock as fast as you can. You don't go trying your luck with whether you get a fill in some options. Which is (I think) the point you made in your earlier post.

That's my problem with arguments that go "market thinks something". Market doesn't think. It's a frigging piece of software matching orders. Somebody in the market is buying something, and somebody in the market is unwilling to sell it unless the price is really aggressive. That's the argument I follow. But I can't credibly make an argument like that that would make options lead.

Options lead in very thin markets with no liquidity, like when employees cashing stock options is the only source of flow. Not in S&P stocks.

"There is a SIX am?" -- Arthur

Energetic
Forum Captain

Total Posts: 1488
Joined: Jun 2004
 
Posted: 2018-04-11 16:37
@goldorak

I ran 5000 simulations. My best outperformance of S&P is 71% in 2008. My second best is 65% in 2011. My worst underperformance is -18% in 2009.

In my simulation:

The probability of getting more than 70% outperformance in a calendar year is 1.3%.
The probability of having the worst underperformance in a calendar year no worse than -20% is 7.3%.
The probability of getting more than 60% outperformance in two calendar years in one simulation is 0.02%.

How does it look?

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

goldorak


Total Posts: 1045
Joined: Nov 2004
 
Posted: 2018-04-11 17:40
I do not understand how you can refer to 2008 or 2009 in this exercise. I mean once returns are shuffled, all reference to previous dates are lost.

If you are not living on the edge you are taking up too much space.

Energetic
Forum Captain

Total Posts: 1488
Joined: Jun 2004
 
Posted: 2018-04-11 17:55
My first statement

>My best outperformance of S&P is 71% in 2008. My second best is 65% in 2011. My worst underperformance is -18% in 2009.

was in reference to my original post that started the thread. Sorry for confusing you.

In this exercise, I don't refer to calendar years. It is just convenient for me, for benchmarking purposes, to split each simulation in 252 trading days intervals and compare model vs. S&P performance in each of those simulation "years".

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken
Previous Thread :: Next Thread 
Page 3 of 5Goto to page: 1, 2, [3], 4, 5 Prev Next