ralph


Total Posts: 13 
Joined: Jun 2011 


http://www.edge.org/responsedetail/25401
"The notion of standard deviation has confused hordes of scientists; it is time to retire it from common use and replace it with the more effective one of mean deviation. Standard deviation, STD, should be left to mathematicians, physicists and mathematical statisticians deriving limit theorems. There is no scientific reason to use it in statistical investigations in the age of the computer, as it does more harm than good—particularly with the growing class of people in social science mechanistically applying statistical tools to scientific problems."

“Ruug caddaagii soo rogaalceli!” 




I am all for replacing vol by MAD, but what do we do about correlation? And how do we get the MAD of a portfolio?
I have also found the *median* absolute deviation to be useful in certain situations. 
Inflatable trader 


Cheng


Total Posts: 2870 
Joined: Feb 2005 


but what do we do about correlation
You could use something like Kendall's tau. Not sure, however, whether is it easier to interpret than correlation. I have been looking quite a while for a "better" measure and finally gave up . 
"Flammandus et Contemptus / Seyn Todt in Schwartz "




Nonius

Founding Member Nonius Unbound

Total Posts: 12799 
Joined: Mar 2004 


Mutual Information? 
Chiral is Tyler Durden 



Ha, I need to clarify that the two questions are linked: the reason I am interested in correlation is that it allows me to get the vol of a portfolio with a simple formula: I just need to do transpose(w) * sigma * w and voila... Now if I want to use MAD instead of standard deviation, is there a similar formula? I don't think so... But I would be happy to be proven wrong, as this would be a paradigm shift in portfolio theory (which at the moment is just elegant mathematical nonsense if taken at face value) 
Inflatable trader 



Cheng


Total Posts: 2870 
Joined: Feb 2005 


which at the moment is just elegant mathematical nonsense if taken at face value

"Flammandus et Contemptus / Seyn Todt in Schwartz "




I tried to write a post about how std dev is the perfect distance measure for how far a Stochastic Process is from it's Mean process, but it's really not, it's just one distance measure. The MAD process might be better since it's more robust to outliers.
MAD = K * StdDev, where K is distriubtion dependant, so there should be an equivalent transformation for MAD as there is for StdDev. Granted you need the K's. But with the K's, your formula still holds. Mean Difference
But that all being said, if someone works on the numbers side of finance and doesn't know what stddev is, they really should consider another line of work. And good luck quoting options in MADs now that this market is established.
As a measure, it doesn't matter. Maybe it makes sense to communicate something other than stddev for clarity, specifically when the risk of miscommunication is high. I wouldn't push to change any code for this unless someone brilliantly foresaw having to change dispersion measures as a config item.
Not that I want to leap to the defense of Taleb for any reason, but I think he's more concerned about communication of what the dispersion is than changing any measure of it. Communicating in probabilities is more intuitive than stddev in my opinion, even though people still don't understand that at 3% an event can still happen. 
in the words of one such quant ‘were on the whole either less quanted or
not quanted at all’. 




Taleb's point is more or less equivalent to this:
An Intuitive Volatility Estimator
The post is pretty hyperbolic ("confused hordes of scientists", really?) but there is a grain of truth. Means are simpler and more intuitive than rootmeansquared for anyone to do in their head. 



pj


Total Posts: 3530 
Joined: Jun 2004 


and what about correlations? 
вакансия "Программист Психологической службы"
але! у нас ошибко! не работает бляблябля
вы хотите об этом поговорить? 




pj, check this out:
Robust Correlation
I had to look it up but at the end of the day, you have to agree a measure for distance between two processes X and Y. Once you've done that, you should have a consistent approach in all cases by using that measure. StdDev / Mad are forms of "autodispersion" where Y = mean(X), while correlation is "heterodispersion" which where Y is something other than X.
The rest is normalizing the distance so the estimators are unbiased and hopefully robust.

in the words of one such quant ‘were on the whole either less quanted or
not quanted at all’. 


pj


Total Posts: 3530 
Joined: Jun 2004 


Thank you for the reference! Just that these correlations correspond to the classical correlation,, thus to σ , and not to MAD as is the Master (not me, nor NeroTulip) would desire. 
вакансия "Программист Психологической службы"
але! у нас ошибко! не работает бляблябля
вы хотите об этом поговорить? 




specifically rCOMED doesn't.
they are all derivable from each other if you know the K's. e.g. you can sub MAD/K for sigmas and get correlation, and then use the same algebra.
The point is whatever number you get is a measure of distance between two processes. As long as you use it consistently, the rest of the algebra holds under affine transformations of vols and corrs up to constants for specific probability measures.
I wouldn't say any are better, but they are effectively the same measures, and you can change between them in a straight forward manner. 
in the words of one such quant ‘were on the whole either less quanted or
not quanted at all’. 



What I have in mind is (assuming means are 0 for simplicity), look at average[(abs(x*y))^(1/2)*sign(x*y)]
This is similar to covariance, but for x=y you get average[abs(x)], which is the MAD, instead of average[x^2], which is the variance.
Edit: tried to clarify a bit

Inflatable trader 




Not so hyperbolic… A high percentage of people who work in finance (with PhDs). And we stipulate a huge number of econometricians. See papers. 




Nero, it stays unstable because you are multiplying random variables. The only solution is the copula, which always exist because the joint probabilities exist. 




phy


Total Posts: 18 
Joined: Nov 2013 


what matters in a time series are the conditional moments, i.e. conditional variance and covariance. Financial series do have structure (nonlinear) in the conditional moments. Hence, a proper way to model is use xARCHtype or stochastic volatility models. For correlations, look for dynamic conditional correlation (DCC). 




@Nassim:
I think my measure is made stable by taking the square root:
Let's say x and y have tail exponent a. The tail exponent of x^2 is >a, I think it is 2a, correct? This can cause average(x^2) to diverge while average(abs(x)) is defined, which is why MAD is more stable than stdev. Similarly, if the tail exponent of xy is 2a, this can cause average(xy) to diverge. But if I look at average(sqrt(xy)), I am back to a tail exponent of a, which means that if the MAD exists, my measure exists too.

Inflatable trader 



quantie


Total Posts: 903 
Joined: Jun 2004 

 


@quantie If discussions as per your link excite you, this might be worth a read (warning: quite long) 




chiral3

Founding Member

Total Posts: 5163 
Joined: Mar 2004 


This conversation is soooo overproofed. Same conversation from 20 years ago. Taleb losing money 9 out of 10 years, telling everyone they are idiots, not understanding the price/value of tails, and taking a victory lap every 1 out of 10 years. Doesn't Taleb make money by writing books? If he's so plugged into esoterica why isn't he the smartest guy nobody has ever heard of? He's unemployable for the same reason Malcom Gladwell is unemployable. Does he have any assets? If his ideas are really the notes in a song, one bar out of every ten, aren't the Cliff Asness' of the world the space between the notes? Providing structure and meter? It is a system, no? With a ton of metaredundancy and leverage. We make fun of the BOJ for taking old furniture and baseball cards onto their balance sheet, but the US now takes EFTs. The psychology of NT is more telling than his old (borrowed) ideas.
I still maintain that, within the context of the time, Dynamic Hedging was the most interesting and original idea NT ever had. 
Nonius is Satoshi Nakamoto. 物の哀れ 


quantie


Total Posts: 903 
Joined: Jun 2004 


I still maintain that, within the context of the time, Dynamic Hedging was the most interesting and original idea NT ever had.
Totally agree was a book ahead of its time and I learnt a lot from it.
@gmetric_flow thanks for the link will read. 




chiral3

Founding Member

Total Posts: 5163 
Joined: Mar 2004 


That was a long read. I made it half way through and tapped out. Aaron's comment in reply made me glad I did. 
Nonius is Satoshi Nakamoto. 物の哀れ 


Jurassic


Total Posts: 367 
Joined: Mar 2018 


@chiral3 im missing something but why is Gladwell unemployable? 




nikol


Total Posts: 1172 
Joined: Jun 2005 


@chiral3
> Dynamic Hedging was the most interesting and original idea NT ever had
I share the view.
After reading "Fooled by randomness" I left with impression that the man tries to teach two things: that "S/N" and "trend" matter. 



chiral3

Founding Member

Total Posts: 5163 
Joined: Mar 2004 


I was being a bit tongueandcheek, but I was referring to the fact that MG dabbles in this nonfalsifiable and inexhaustible universe of anecdotes derived from correlation = causality. "Something amazing happened in a small town in Georgia in the early 1920's: the children seemed happier, on this everyone could agree. Little did they know that earlier that year the town switched water sources to a reservoir that later was determined to be fed by an ancient spring that had snaked its way through igneous granite, supplying unusual concentrations of Lithium..." or some shit like that. MG is good at throwing a yarn, but he's not a researcher, and he can't actually do anything, except write popular books that takes nonfalsifiable supposed esoterica and renders it exoteric (I could go on for a while about the shifting roles of esoterica versus exoterica in the internet age). In that regard he's of limited utility in advancing the things that he talks about, which makes him very similar to NNT. 
Nonius is Satoshi Nakamoto. 物の哀れ 


