Forums  > Trading  > Index trading strategy, part II  
     
Page 1 of 2Goto to page: [1], 2 Next
Display using:  

Energetic
Forum Captain

Total Posts: 1532
Joined: Jun 2004
 
Posted: 2021-03-22 20:13
This is the continuation of work that I reported earlier. A number of things have changed so I won’t make any references to the previous post. I benefited a lot from the discussions back then, both here and off-line, thanks a lot to you guys. I tried to follow your recommendations where I could. I am especially grateful to the friend who gave me high frequency data.

The strategy trades SPY but the signal comes from volatility markets (VIX and VIX futures). I am implicitly assuming that vol traders have some ability to predict direction of the underlying. I don’t know what their thinking process is but I am trying to “read” their conclusions.

All trades are placed at 4 pm. The positions can take values in the interval [-1,1], i.e. anywhere between full short and full long. The position is a smooth function of model parameters and market data, i.e. no jumps. This results in fractional (-1) positions near the boundaries where the strategy changes sign. Transaction cost of 1 bp is assumed.

The original idea is to stay long until the volatility markets signal that the things are too risky so that I can go neutral or short thus reducing the probability of severely negative outcomes, i.e. flattening left tail. But this is not one singular strategy anymore, it’s a combination of several independent sub-strategies exploring different parts of the parameters space.

The strategy is calibrated using 4-fold cross-validation. The final set of parameters is determined as the average of the 4. Not that it is very important: the calibrations are quite similar.

Correlation of daily returns with SPY is 42%, monthly -14% and annual -65%. The main performance statistics are as follows:



Breakdown on individual years is in the table below:



A few of notes:

There is no high frequency data prior to 2006.

I get 59% return in 2008 even when it is out of sample – it just happens to be a good year for the strategy. I mean, second half of it is, after I dig out of the -15% DD.

I get the same return and 2.1 Sharpe in 2009-2020. Thus, the early years actually hurt me by huge dispersion, not help as you might think.

To illustrate the strategy skill in flattening left tail, I computed the probability of getting worse than -3% return on a day. It is 0.53% for the strategy vs. 1.87% for SPY.

The target of the calibration was to optimize the ratio CAGR/DD. Not Sharpe and not CAGR itself. I know it’s an unusual choice but I caught myself always thinking about whether x% improvement in CAGR is worth y% deeper drawdown. Eventually I decided to make the process more objective.

The point is that the profitability and ratios that I reported above are accidental in sense that I didn't try to optimize either of these. If I had wanted a better Sharpe ratio, then I could simply turn off the short sub-strategy. Then I get Sharpe 2.2 (2.5 after 2008) and Sortino 3.4 without recalibration. But CAGR drops to 20% so I choose not to do it. However, I am open to counter-arguments.

Last, an obvious question: do you think someone with capital can be interested in this or it is still not good enough? What should I do next? I was thinking about posting on LinkedIn but I doubt too many people with real money read it.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

zee4


Total Posts: 89
Joined: May 2010
 
Posted: 2021-03-22 20:45
you may find https://www.youtube.com/c/Volatilitytradingstrategies/videos useful.

Бухарский

doomanx


Total Posts: 109
Joined: Jul 2018
 
Posted: 2021-03-22 20:57
What do your factor exposures look like?

'The strategy is calibrated using 4-fold cross-validation'
Are you holding some true out of sample here?

'All trades are placed at 4 pm'
How sensitive is the strategy to this execution plan?
(As an aside, 1bp tc might be too low although if you're trading daily shouldn't make or break).

How robust are your results to perturbations of the parameters?

'The target of the calibration was to optimize the ratio CAGR/DD'
That objective function seems quite nasty. DD is notoriously intractable - did you global optimise? If so perturbation analysis is particularly important.

It's usually better to optimise for MSE, then do your allocation separately.

'Last, an obvious question: do you think someone with capital can be interested in this or it is still not good enough?'

I think it really depends how well you can explain what it's doing (part of this is knowing your exposures and such - if it's a true unknown inefficiency then people will be very interested. If it's an exposure you need to find someone who wants that exposure). Purely showing them this backtest likely won't be enough, but it's a fair start.

Does the same trade work on other vol index/index pairs? You might be able to get some more sharpe by going wide.

did you use VWAP or triple-reinforced GAN execution?

Energetic
Forum Captain

Total Posts: 1532
Joined: Jun 2004
 
Posted: 2021-03-22 22:01
@ zee4: Thanks for the vid. Nothing wrong with the guy but he didn't say anything I didn't know or didn't experience myself, including the recent kick in the groin on the vol trading side of this strategy. I watched only the most recent one though.

@doomanx:

I don't think it is correlated to any known factors. No exposure to momentum, that's for sure, bc I looked into it.

Cross-validation is literally as described in
wiki. Each chunk of data is OOS once and in-sample 3 times.

4 pm is the best time to trade. W/o recalibration, I'm losing to the tune of 1.5% in return for every minute prior to 4 pm. Didn't try to recalibrate for each separate minute. Excellent question!

Sensitivity. I reduced all parameters by 10% and CAGR dropped to 20% with 15% DD. If I go up 10% it's 22% and 17%.

In my case DD is quite stable. I don't have ways to materially change it in either direction.

If you meant allocation between my sub-strategies then it's impossible in my case. They interact dynamically and only one is on any given day. I don't control how often each of them is on.

I can explain the obvious things well. Contango of vol futures vs risk, like that guy in the vid below, but with heavy accent ;) Certain elements were deliberately designed and implemented in the spirit of technical analysis, even though I don't really believe in it. Beliefs aside, if it works then it's silly not to use it. Some things didn't work out as I intended but other things unexpectedly did. Explaining this without showing the code would be very hard. I tried it once. The listener was sympathetic but in the end I didn't feel like my explanations were very helpful.

I haven't tried it on other underlyings for lack of data.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

doomanx


Total Posts: 109
Joined: Jul 2018
 
Posted: 2021-03-22 22:53
When I say true out of sample, I mean you do not touch it until you’ve finished all model fitting. If you’re averaging over the 4 folds then it’s not out of sample. Any estimate based on that data is downward biased - you want some true out of sample data to estimate performance on that was completely unseen during model fitting.

Hmm, significant drop in performance based on a number of minutes is something to be careful of (doesn’t mean it’s wrong just sticks out as something you need to be very sure why that is the case).

How many parameters do you have? Can you visualise the surface of parameters vs loss to see if it’s very jagged or smooth?

No, I mean you optimise the signal based on MSE, and then allocate to the signal based on its characteristics (by doing Kelly or whatever is appropriate for your strategy). Drawdown is poorly behaved so very difficult to optimise well. There are some papers that tackle optimising for dd in quite sophisticated ways, will try to dig it out for you, but in my experience MSE is the way to go.

If you’re serious about getting investment definitely see if it works on other products. Also really nailing down what your edge is will make people a lot more comfortable.


did you use VWAP or triple-reinforced GAN execution?

Energetic
Forum Captain

Total Posts: 1532
Joined: Jun 2004
 
Posted: 2021-03-23 04:32
I'm sorry, I didn't answer this well. The answer is no but let me explain. In the original version reported earlier, I did have a true OOS but folks were unimpressed and suggested cross-validation. Now that I played this game every each way, I have to say this. In my particular case it doesn't really matter, meaning that no matter the details I'd end up with more or less the same outcome. In addition, the history is too short and too non uniform. I could very easily play this game by choosing, say, 2018-2020 as "true OOS" and get very good results because, on average, this happens to be a very good stretch for my strategy. However, if I had my own money to invest in it, I'd be most worried about 2019. Why was I underperforming S&P500 by 18% in 2019? (OTOH, no one should complain about earning 14% a year.) I have an intuition for it but I couldn't put it in the algo to take advantage of it. In other words, I don't really know and that's a real risk. What I'm trying to say is that I could leave any chunk of my data as true OOS. Based on previous experience, I chose not to do it because the most natural choice of the true OOS sample is the recent period and we know it was favorable to stocks in general and to my strategy in particular. Thus the decision to throw all data in the ball.

Yes, I agree, the drop in performance away from 4 pm is a potential problem. The model is very much non Markovian. Unlikely that anyone is looking at the charts based on on data based on snapshots at 3:57. The results are path-dependent. Once the model forks to a different branch, it's a new story. Even when I am eyeballing the simulations, it's very hard to determine what went wrong.

Just so that I understand what you mean exactly. When you mention signal based MSE, you mean a type of strategy that tries to follow a certain target and minimize the error? If so then no, this the exact opposite of what I'm doing. I don't want to target anything in any detail. I want to hit my overall target and ignore the details.

The question about the number of parameters is obviously a good one. Oddly, I don't have the exact answer anymore. Let me explain. Say, I have 6 sub-strategies with 2 parameters each. This would make the total count to 12 but this is not quite so. While I was playing with the strategy, I noticed that some of the parameters were not affecting the performance stats very much. I couldn't ditch them completely but I stopped optimizing them. They became a hyper-parameters. The number of actively calibrated parameters is 5. I doubt I can visualize the surface but I'll think about it again.

I really don't know what my edge is. I don't think I'm smarter than everyone but I have a knack in finding wrinkles in data. I found something, perhaps more than one thing, but I'm not sure what it is exactly. But I do believe it is something.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

ronin


Total Posts: 667
Joined: May 2006
 
Posted: 2021-03-23 10:03
> There is no high frequency data prior to 2006

So what? This trades once per day, you don't need high frequency data.

> Correlation of daily returns with SPY is 42%, monthly -14% and annual -65%

You only have 14 annual returns, and 168 monthly returns. Those two correlations are meaningless. Only the daily one means anything.

> I'm losing to the tune of 1.5% in return for every minute prior to 4 pm

So, if you trade at 3.30 pm instead of 4 pm, your compounded return goes from +25% to -20%?

Come on. This is so over-fitted that it isn't even funny.


"There is a SIX am?" -- Arthur

doomanx


Total Posts: 109
Joined: Jul 2018
 
Posted: 2021-03-23 10:05
Right, that's all well and good (and perhaps a decent sign for the strategy if all else is well), but it's a good idea to hold some out of sample, if nothing else to compute your statistics like Sharpe and such, otherwise they're biased. Good to have in-sample and out-of-sample estimates, or in ML terms a train-test-validation split.

Have you plotted your return distribution to look at skew and such? This is likely something people are interested in. Investors tend to prefer things with negative skew (lots of wins, few big losers) so would be good to know where you are.

Usually you're trying to forecast something and trade based on this prediction, so the forecast is the signal, and you want this forecast to be close to whatever you're trying to trade (hence minimise the MSE between the two). Then you allocate based on the characteristics of the signal. But maybe this framework isn't appropriate for this particular strategy.

Are any of the strategies similar at all? It might be a good idea to try and pool some parameters if this is the case (a bit like a hierarchical linear model).

You can't visualise it fully in that case, but it's a good idea to really understand how your parameters are influencing performance especially with an objective function like yours as the fit from a global optimiser can be very dicey on a very jagged loss surface. Something you can do is approximate some 2nd derivatives of the loss with respect to your parameters and look at the norm of these to see if it explodes anywhere.

Path dependence is fine, what I'm worried about is that it's path dependent at 4pm. Unless you've discovered some super weird temporal anomaly (which may be the case, just more unlikely), it should work wherever the 'setup' forms, so it would be good to test it at other times.

did you use VWAP or triple-reinforced GAN execution?

Maggette


Total Posts: 1301
Joined: Jun 2007
 
Posted: 2021-03-23 11:03
Thanks for putting that out Energetic. Interesting discussion.

"Just so that I understand what you mean exactly. When you mention signal based MSE, you mean a type of strategy that tries to follow a certain target and minimize the error? If so then no, this the exact opposite of what I'm doing."

I think what doomanx means: for optimizing things (like a trading strategy or an optimal control problem) in an uncertain environment it is always worthwhile (though not always possible) to separate the the two things:
- prediction of the next state
- the optimal decision based on the prediction
At least in the first iteration of your strategy.

There are a lot of tools at your disposal to check whether you have predictive power or not. And at the end of the day your strategy involves a prediction.
Long => probably not down, at worst flat
Short => probably not up, at worst flat
flat => probably flat or don't know

Things IMHO often (but probably not always) get fishy if you don't have predictive power. Than it is IMHO quite more probably that your edge is kind of an result of an over-fit

It would also be interesting to see how many position flips you actually have. And what percentage of your position is long.

Disclaimer: my experience is restricted to development of strategies in the European energy market and might not add much value here.

Ich kam hierher und sah dich und deine Leute lächeln, und sagte mir: Maggette, scheiss auf den small talk, lass lieber deine Fäuste sprechen...

nikol


Total Posts: 1347
Joined: Jun 2005
 
Posted: 2021-03-23 13:02
my two cents.

It happens that given strategy S(timeseries,action(parameters)) being maximized with risk penalty finds set of parameters delivering 'brilliant' result. By shifting them a bit S(Param+dParam) the performance comes back to normal "Markowian surface".

My recipe to manage such dumping into a 'peaking point' in parametric space (x,y,z,...) is to use some kernel in that parametric space on top. It helps to stabilize the optimization.

Another way to deal with this is to find 'that one point' which contributes to the 'success' (usually it is) and to edit it by scaling down that return by rebuilding timeseries. I did not work with such massive high-freq data, so might be wrong.

... What is a man
If his chief good and market of his time
Be but to sleep and feed? (c)

Energetic
Forum Captain

Total Posts: 1532
Joined: Jun 2004
 
Posted: 2021-03-23 16:48
@doomanx:

Just to illustrate my point with numbers: when I calibrate on 2006-2016, I get 24% return in-sample and 27% OOS. What does that prove? We may disagree on this but I find the current approach more convincing.

The distribution of monthly returns is heavily skewed to the right with fat right tail, as shown below. Daily is fairly symmetric around 0 but also with heavy right tail. Apologies for huge chart - I wasn't able to edit the size.



I wasn't trying to predict anything. The goal was to flatten the left tail. Any predictive that came with would be collateral. In fact, on the daily scale, there isn't any. I have positive returns only 51% of the time vs. 56% for SPY. I am winning because of the tails.

No, the strategies are not similar but some are looking at the same parameters.

I believe what happens at 4 pm is volume. People re-hedge their positions at the close. That may be the best time to read their signals.

@Maggette

See above: I have no predictive power. Negative, in fact. Believe me, I am always very worried about overfitting. But you can't prove a negative. Although I am showing here results after recent recalibration, I am also paper-trading predecessors of this strategy for 4 years. If I were overfitting I would have noticed some problems but I didn't. Not like it proves anything, of course.

The strategy is at least partially long 77% of the time, a lot more often than it ideally should be but, again, this is a byproduct, not a target. It changes sign, including going to cash from a nonzero position, 21% of the time.

@nikol

As noted below, bumping parameters by 10% doesn't do anything dramatic.

Sorry, I didn't understand your recipe. Maybe you can illustrate by an example? And I didn't get your idea about 'that one point' either. I have almost 4,000 daily returns. The largest is 9.9%. Sure I can kill it or scale it but then what? Also in one of the calibrations this day is OOS and not a lot changes.

@ronin

Thank you, nothing further from you will be interesting.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

nikol


Total Posts: 1347
Joined: Jun 2005
 
Posted: 2021-03-23 18:03
1. kernel idea.
When you scan parameters over grid, the picture will be like this: PnL vs MDD.
Application of kernel does not impact major body of the sample, while brings outlier back.

2. return.
In my situation such point appeared due to one single jump in PnL. When I suppressed this interval, outlier disappeared.



PS. At the time, I got some "high level idea" which I implemented by using customized tech.indicators. Reasonable settings (something like 3.sigma or Prob>99% or Prob<1%) produced good results. Grid-scan was done to convince myself. I used NinjaTrader as it had all backtesting machinery and is advanced enough to customize standard indicators.

... What is a man
If his chief good and market of his time
Be but to sleep and feed? (c)

Energetic
Forum Captain

Total Posts: 1532
Joined: Jun 2004
 
Posted: 2021-03-23 20:22
Thank you, I understand now.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

doomanx


Total Posts: 109
Joined: Jul 2018
 
Posted: 2021-03-23 20:33
But you're missing the point - it's not OOS any more because you've looked at it. As soon as you use that data in any way, shape or form it's now in sample and all your estimates are biased + you risk data mining. At this point you need to collect more data and validate on it. Paper trade going forward or whatever to prove it works out of sample.

Our friend @ronin may not be our head of diplomacy but he's one of our finest - it's very worrying that your performance is so linked to 4pm and that you get such a drastic drop off in performance from moving that time. You need to really understand why it works at 4pm or people won't invest.

Right, to be fair I have minimal experience with strategies that do not forecast anything at all so maybe ignore my advice there.

How long are you holding positions for? Are you just winning from something weird at the close that might not be capture-able in practice?

Even so, your strategy has returns which have some associated statistics (mean, variance, skew etc.) from which you should do position sizing.

I really think your objective function is the problem here. Directly optimising for CAGR/DD with a global opt is such nasty business. Much better to use fractional Kelly (which if your higher moments are small is well approximated by mean/var).

did you use VWAP or triple-reinforced GAN execution?

Energetic
Forum Captain

Total Posts: 1532
Joined: Jun 2004
 
Posted: 2021-03-23 22:22
Let's start over. I split y data set in 4 chunks. First calibration is for 2006-2016 sample. Using those parameters, I compute the results for 2017-2020. This is so far OOS and that is what I told you about in the previous comment. Calibrations 2,3,4 happened later. The fact that 2017-2020 data chunk was IS for those three doesn't change the fact it was OOS the first time. Maybe I'm still missing the point?

I agree that rapid performance degradation between 3:55 and 4 pm is a concern. I believe I'm winning mostly because I effectively kill the left tail which is my main strategy. Other strategies, especially the shorts, turn on only within a relatively small part of the parameter space. I looked at some days when the signal at 3:55 differed from 4 pm. Sometimes, market moves fast enough during last minutes to kick my features to another part of parameter space. Empirically, this works against me more often than not. I know this is not a really satisfactory explanation. But you are right - I need to have a better answer.

On the positive note, 17% that I'm making if I trade at 3:55 is not too shabby.

Positions are held at least one day. Sometimes long positions stay for weeks. Short positions rarely stay longer than a week.

Please explain what your point about sizing. Empirically, I have a positive expectation of daily return, 0.09% in fact. Therefore I want to invest all my capital in every trade to maximize total return. (Some of my positions are less than full 100% due to proximity to a boundary but this is beside the point.) I do have a standard deviation of 0.96%. How should this fact affect my sizing? This is not the game where you can lose the whole capital if things go wrong. People suggested using Kelly before; I've read some papers but didn't find anything that I could use.

Optimizing CAGR/DD is a new thing, I did it the first time 2 weeks ago. Prior to that I simply optimized CAGR but I rejected the calibrations with deep DDs. The results were materially the same.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

ronin


Total Posts: 667
Joined: May 2006
 
Posted: 2021-03-24 09:58
> Our friend @ronin may not be our head of diplomacy

Oh man - seriously? There goes my back-up career plan...

I was actually told this specific thing as part of my appraisal, some ten years ago. "R, if you are ever thinking of taking up diplomacy, erm - just keep the day job." You think he had a point?

@energetic,

This distribution is actually a tell-tale sign of an overfitted distribution. Your entire outperformance over the spx comes from those 7 or 8 months where it returned 18 or 20%. Cut off everything beyond +-10%, and you would be lucky to be flat before funding and transaction costs.

"There is a SIX am?" -- Arthur

Energetic
Forum Captain

Total Posts: 1532
Joined: Jun 2004
 
Posted: 2021-03-24 16:16
I really don't care about your diplomacy and, believe me, I am no better than you in that department. You made a bunch of useful (along with a few unhelpful) comments when I presented part I so I generally appreciated your feedback. But now you are talking complete nonsense.

I shouldn't nitpick on you miscounting the number of annual returns: it's 15, not 14. And I'm not even going to debate your statement that the correlation computed based on 180 data points is meaningless. I'll let it hang on its own merits.

For your latest claim, in order to finish this exchange, I zeroed out all monthly returns in excess of 13% which is the largest monthly return for SPY. CAGR is still 16%, so half of my outperformance is still there. I'd expect you to have much better intuition for data. And I believe you generally have it - but not now.

I know you are a smart man and I think I know what your problem is, but I will not say it. Let's not make it personal. Your judgement is so clouded that your feedback is of no further value for me.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

ronin


Total Posts: 667
Joined: May 2006
 
Posted: 2021-03-24 19:05
Oh wow - I guess this forum is becoming lively again. Good work, dude.

I retract everything I said. Pure genius, this. Thank you for letting us bask in its magnificence for one beautiful moment. For that fleeting moment, I saw perfection.

Thank you, oh great one. I am not worthy.

"There is a SIX am?" -- Arthur

Energetic
Forum Captain

Total Posts: 1532
Joined: Jun 2004
 
Posted: 2021-03-24 20:12
As I tell my children, if you don't have anything nice to say at least say something interesting.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

doomanx


Total Posts: 109
Joined: Jul 2018
 
Posted: 2021-03-24 21:07
'Maybe I'm still missing the point?'
The point is that data mining starts at a higher level - if you look for patterns in noisy, nonstationary data then fit a model to capture it, you're data mining. If you've looked at the data at all, it's in sample. Some data should be held out until everything is completed. Then you compute your Sharpe and such only on the data that was held out of sample, as this is the only relevant metric. Of course your model with a decent number of degrees of freedom does well in sample - it's mathematically optimised to do that.

This is one of the problems with applying 'machine learning' (although it appears yours is not an ML approach) in lower frequency contexts. You need enough data to do exploratory analysis and fit the model well, but this usually doesn't leave much for validation. This is why generally lower frequency strategies are built on some known financial idea that has been well established (like long-short equity stat-arb or momentum) and the data analysis part is just getting a good implementation of what we know is already there. There are things you can do to alleviate this problem (for example, training your model on similar assets so you have more data points, then maybe boosting the model fit to that specific product - this is known as multi-task learning in the ML literature).

Admittedly I work on much higher frequencies; in that setting you have so much data (even after regime shifts) that machine learning type approaches are much more useful as you have enough data to make this pipeline work. Your approach to strategy development is pretty alien to me, so I don't know how relevant the things I'm saying are.

'Please explain what your point about sizing'
'Therefore I want to invest all my capital in every trade to maximize total return'
This is not the correct approach. This maximizes your one-step ahead utility. You want to maximise the long-term growth of your capital. You should pick the fraction of the wealth f that does this. This fraction is approximately mean(return) / variance(return) for nicely behaved return distributions. For multiple reasons (read Thorpe's papers) you want to scale this fraction down, by say 1/2. Optimising for CAGR in sample is not what I am describing here.

Your approach is very alien to me and not at all like things I've done before, so maybe none of what I'm saying is relevant. Maybe you have something, maybe not, but that 4pm business and the outlier returns will mean you need very good explanations before taking this further. I think careful study of the practicalities (feasibility of execution/costs, forward testing) is important.

@ronin I think world politics would be a lot more efficient with a few ronin in the mix ;) Although a bit of forum infighting is much more entertaining than spam bots, bravo chaps.

did you use VWAP or triple-reinforced GAN execution?

Energetic
Forum Captain

Total Posts: 1532
Joined: Jun 2004
 
Posted: 2021-03-24 22:48
On that level - you are right in that there is no true OOS, and there cannot be. I'm a daily observer of markets for quite some time. On the daily frequency, I can say that I've seen everything and I cannot unsee it. Check and mate.

Yes, this is not a ML model. I tried that too but it didn't work.

Yes, I want to maximize the long-term growth of capital. But I think in the context of index investing this is equivalent to maximizing one-step ahead utility. You're drawing the distinction with optimizing CAGR in sample. But don't hope that ideally OOS will behave just like IS? I have actually read some of Thorpe's papers long time ago but this particular idea didn't stick or went over my head.

I scaled down my position function by 10% and my CAGR is down 22%, as expected. Computing mean(return) / variance(return) = 0.09% / (0.96%)^2 I get almost 10 ...

Outliers are easy to explain. One of the strategies is looking for momentum. Sometimes it shorts successfully just as the markets are falling fast: best month ever is Mar-20, second best is Sep-08. This particular skill is the one that suffers the most at 3:55. You're right that I need to explain this better.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

tabris


Total Posts: 1278
Joined: Feb 2005
 
Posted: 2021-03-24 23:23
"On that level - you are right in that there is no true OOS, and there cannot be. I'm a daily observer of markets for quite some time. On the daily frequency, I can say that I've seen everything and I cannot unsee it. Check and mate."

@Energetic, this isn't an explanation of true OOS. Basically all experiments work this way. You and I observe some form of "law and order". Then we use data to test it because we want to know if this "law" make sense. What we do via leaving data out and untouched is to cut down the bias in the modeling of the "law". If I think something behaves in a straight line, but I do not know the slope, I use previous data to check what the slope is, and then use this estimate of the slope to apply to the untouched data to know that the model of the slope is roughly right.

Also, just for more sanity checks, you should look at some more sensitivity analysis like distribution of returns per trade, mean return per trade, variance of returns per trade, basically all your meaningful stats you used and change it to statistical measures by trade and then see how many trades you have made in total/per yr/etc. Transforming some of the distributions to trade space instead of looking at time can also sort out some glaring problems.

Next, you should actually look at slippage, since your strategy depends on execution at a particular time, you should vary your model to trade on one/two/three levels worst, then vary your execution by delaying it by a minute or so at one or two levels worst. For example, some of your biggest return years are years where I honestly think you might not get filled at your level because of liquidity/big vol events. So if you missed the execution or get filled at worst level, how much did the returns deteriorate?

Hopefully that is more stuff for you to look at and have a better idea on for your strategy.

Dilbert: Why does it seem as though I am the only honest guy on earth? Dogbert: Your type tends not to reproduce.

Its Grisha


Total Posts: 74
Joined: Nov 2019
 
Posted: 2021-03-24 23:35
I won't comment much on backtesting methodology because some more experienced people have already. My question is more practical: it seems like you have been working on this and even paper trading it for a long while, why not trade it with small capital?

I don't think you need elaborate infrastructure to trade the index futures at 4 pm. In my experience this will drive intuition very fast about how capturable your perceived prices are, and just the exercise of implementing the live version exposes things you did not consider when backtesting. For example, exactly what order your live datafeeds arrive and how this effects the update of the model. ​Seems especially important with all this path dependency you bring up.

Then you can tie out the walk forward backtest perf with the live results of a small account and see if your assumptions are realistic. This brings you much closer to something investable. That being said, even if implementation shortfall = 0, the potential overfitting dangers are a separate issue.

"Nothing is more dangerous to the adventurous spirit within a man than a secure future."

Energetic
Forum Captain

Total Posts: 1532
Joined: Jun 2004
 
Posted: 2021-03-25 01:37
@Grisha (Гриша)

The answer to your question is simple and sad: I work for a financial institution that requires 30 days holding period on fucking everything. Check and mate.

I don't have direct access to HF data. My friend donates the data once in a while and I'm too grateful to bug him too often. On a daily basis, I just paper trade based on EoD data. Some will find it odd, but it's better to trade at 4 pm. I know my word is not enough, but EoD is a decent approximation, at least for tracking purposes.

Please believe me when I say that I worry about overfitting every day. In part, that's why some of the parameters become hyperparameters. For me, the danger is that market is moving faster than I can execute because some favorable parameters intervals are too narrow. (In fact, I dropped some strategies because the signal was disappearing before I could blink twice.) That's how I lose a lot at 3:55. Is this overfitting? Maybe, maybe not. I am open to discussion.

Re capturable prices: I believe it's a matter of volume. I believe I can capture very well with 10M, reasonably well with 100M and not too well with a yard. But then again - you want to make dollars, not %s. It's great if I can make 25% with 10M capital and 24% with 100M AUM. But, if I'm a guy with real money, I'd rather make 20% on a yard. Would you not, assuming you're not friends with Dr. Simons?

@ tabris

Hey old dude, long time no see, I'm glad you're hanging around!

Thanks, I do know what OOS is for. I'm saying that on the level of rigor that @doomanx would like to see, the ship has sailed. Also, my approach wasn't to find a law of any kind. I wanted to use vol markets to kill left tails. And then I added bells and whistles.

Re stats per trade, you may be making valid points but I have no idea how to look at this. I'm sorry, I kind of understand that it may be annoying for people who do it every day when some amateur comes in with his own weird ideas. But that's what the community is for. Maybe one day I'd say something useful to you.

Re levels, I suspect you're talking about bid/ask order stack, right? In this case, alas, I don't have this kind of data, no levels. What I have is volume-weighted averages per minute. But again, this is not micro cap, this is SPY. There is only so much slippage there.

With reasonable capital, the slippage should also be, well, reasonable. I know I can't trade billions with the same returns. First of all, let's get real, nobody will let me. But in a range 10-100M I can probably deliver around 20% per year even if I'm somewhat overstating my case here.

Who in their right mind doesn't want to make 20%/year? Do many people really have better options? I really really doubt it.

For every complex problem there is an answer that is clear, simple and wrong. - H. L. Mencken

gaj


Total Posts: 117
Joined: Apr 2018
 
Posted: 2021-03-25 02:01
The part about losing 1.5% per minute is interesting, if not fishy. That means somebody could use your signal to trade at 3:50 and exit at 4:00 to make 15% p.a. with only 10-minute volatility. That's like a sharpe 5 strategy.

Can you show the plot of your cumulative pnl over time?
Previous Thread :: Next Thread 
Page 1 of 2Goto to page: [1], 2 Next