Forums > Risk Management > Backtesting VaR (and other analytics)

 Page 1 of 1
Display using:
 mmport80 Total Posts: 85 Joined: Jul 2010
 Posted: 2014-11-06 04:01 As far as I have seen this done in practice:VaR predictions are collected daily or weekly or whatever and the forecasts are checked against realised returns; totting up the breaches.So over a year you would collect 250 or so forecasts and (assuming a daily horizon) 250 or so samples for your backtest.Isn't there an issue with the forecasts drawing from overlapping samples however? I.e. the forecasts themselves are not independent.Just realised this!!! I must be missing something : )
 mmport80 Total Posts: 85 Joined: Jul 2010
 Posted: 2014-11-06 04:13 I see there are tests to deal with the forecast dependence and breach clustering ('mixed-Kupiec'?).Nevertheless, might be just better to ensure forecast samples are mutually exclusive.
 unsmt Total Posts: 196 Joined: Jul 2014
 Posted: 2014-11-06 18:52 We can look at the problem more broadly. Let us verify that a stochastic process underlying to a stock is a GBM with constant coefficients. Let model is true and approximate continuous time model by a discrete one and resolve equality with respect to Wiener increment. Let a step of increments is d that can be millisecond or a week. Then observed data should represent independent observation of the Gaussian rv ( 0, d^2). We need to verify intersection of these two hypotheses but for simplicity let us test them separately. There exist test on independence along with test that verifies Gaussness of distribution. One day I chose a stock from DJI ad tested it over a year data with d = 1 day. Test on independence brought evidence say 60% and test on Gaussian distribution say 75%. Also using chi square test one needs to play with degree of freedom. More accurate test should have more degrees of freedom and in old handbooks, what I learned the subject they recommended more than 30 observations. As far as I remember I used about 9 degrees of freedom as it maximize the evidence. Along with this exercise, it was a controversial idea regarding a popular fat tail concept. Chosen different length for price intervals including tails , degrees of freedom, we can reach a good result. This kind of manipulations does not look good but from mathematical point of view, it looks o’k. Unfortunately at that time I could not find an original BM’s paper to read his idea about fat tails but he probably used other than statistical approach.
 mmport80 Total Posts: 85 Joined: Jul 2010
 Posted: 2014-11-07 11:21 Independence converges to Gaussian, eh?~~I am more interested in the brass tacks of look at my VaR data and testing that.~~Try regressing the Student's t w 3 or 4 dof against a stock's returns and you'll see it explains 99% of the data.
 unsmt Total Posts: 196 Joined: Jul 2014
 Posted: 2014-11-07 13:07 in discrete time scheme w ( t_j +1 ) – w ( t_j ) = L ( S ( t_j ) , b , sigma )where right hand side contains observable data and unknown constant coefficients, d = t_j +1 – t_j .Our hypothesis that left hand side represents increment of the Wiener process. Let coefficients are known as estimate of the coefficients is another problem. Hence one can calculate related to  w_j = w ( t_j +1 ) – w ( t_j ) As far as intervals ( t_j , t_j +1 ) are mutually disjoint for different j then our hypothesis about Wiener process is combined by 2 simple hypotheses. Numbers related to  w_j represent are independent data set and other is related to the Gaussian ( 0 , d ) test .
 arkestra Total Posts: 49 Joined: Apr 2007
 Posted: 2014-12-03 11:41 If you are interested in VaR backtesting, there is an excellent section on exactly this in Chapter 12 of the book "Red-Blooded Risk" by Aaron Brown (who has hung around NP in the past). Not confined to technicalities, but a really solid coverage of exactly what decent VaR backtesting should be trying to esablish. I would recommend the book in general, there's a lot more to it than the VaR backtesting stuff. Very well-written and much to think about. See here for Amazon.com entry: http://www.amazon.com/Red-Blooded-Risk-Secret-History-Street/dp/1118043863/ref=sr_1_1 (Disclaimer: unlike many on these boards I have not met Aaron. I just think the book is good) Down pokey quaint streets in Cambridge / Cycles our distant spastic heritage
 mmport80 Total Posts: 85 Joined: Jul 2010
 Posted: 2015-01-26 08:32 Thanks, I need to read it. I read through his stuff posted here, and it was good (for the most part).
 goldorak Total Posts: 1028 Joined: Nov 2004
 Posted: 2015-01-26 12:58 I second arkestra's opinion of Aaron's book. I actually read it not that long ago. It had been sitting in my library for years... shame on me. If you are not living on the edge you are taking up too much space.
 Page 1 of 1