Forums  > General  > NN Taleb: σ needs to be retired for mean deviation  
     
Page 1 of 2Goto to page: [1], 2 Next
Display using:  

ralph


Total Posts: 13
Joined: Jun 2011
 
Posted: 2014-01-16 00:06
http://www.edge.org/response-detail/25401

"The notion of standard deviation has confused hordes of scientists; it is time to retire it from common use and replace it with the more effective one of mean deviation. Standard deviation, STD, should be left to mathematicians, physicists and mathematical statisticians deriving limit theorems. There is no scientific reason to use it in statistical investigations in the age of the computer, as it does more harm than good—particularly with the growing class of people in social science mechanistically applying statistical tools to scientific problems."

“Ruug caddaagii soo rogaalceli!”

NeroTulip


Total Posts: 1076
Joined: May 2004
 
Posted: 2014-01-16 04:22
I am all for replacing vol by MAD, but what do we do about correlation? And how do we get the MAD of a portfolio?

I have also found the *median* absolute deviation to be useful in certain situations.

Inflatable trader

Cheng


Total Posts: 2870
Joined: Feb 2005
 
Posted: 2014-01-16 11:29
but what do we do about correlation

You could use something like Kendall's tau. Not sure, however, whether is it easier to interpret than correlation. I have been looking quite a while for a "better" measure and finally gave up Sad.

"Flammandus et Contemptus / Seyn Todt in Schwartz "

Nonius
Founding Member
Nonius Unbound
Total Posts: 12799
Joined: Mar 2004
 
Posted: 2014-01-18 20:19
Mutual Information?

Chiral is Tyler Durden

NeroTulip


Total Posts: 1076
Joined: May 2004
 
Posted: 2014-01-19 15:27
Ha, I need to clarify that the two questions are linked: the reason I am interested in correlation is that it allows me to get the vol of a portfolio with a simple formula: I just need to do transpose(w) * sigma * w and voila... Now if I want to use MAD instead of standard deviation, is there a similar formula? I don't think so... But I would be happy to be proven wrong, as this would be a paradigm shift in portfolio theory (which at the moment is just elegant mathematical nonsense if taken at face value)

Inflatable trader

Cheng


Total Posts: 2870
Joined: Feb 2005
 
Posted: 2014-01-20 09:51
which at the moment is just elegant mathematical nonsense if taken at face value

Big Smile

"Flammandus et Contemptus / Seyn Todt in Schwartz "

Steve Castle


Total Posts: 306
Joined: Sep 2010
 
Posted: 2014-01-22 14:27
I tried to write a post about how std dev is the perfect distance measure for how far a Stochastic Process is from it's Mean process, but it's really not, it's just one distance measure. The MAD process might be better since it's more robust to outliers.

MAD = K * StdDev, where K is distriubtion dependant, so there should be an equivalent transformation for MAD as there is for StdDev. Granted you need the K's. But with the K's, your formula still holds.
Mean Difference

But that all being said, if someone works on the numbers side of finance and doesn't know what stddev is, they really should consider another line of work. And good luck quoting options in MADs now that this market is established.

As a measure, it doesn't matter. Maybe it makes sense to communicate something other than stddev for clarity, specifically when the risk of miscommunication is high. I wouldn't push to change any code for this unless someone brilliantly foresaw having to change dispersion measures as a config item.


Not that I want to leap to the defense of Taleb for any reason, but I think he's more concerned about communication of what the dispersion is than changing any measure of it. Communicating in probabilities is more intuitive than stddev in my opinion, even though people still don't understand that at 3% an event can still happen.

in the words of one such quant ‘were on the whole either less quanted or not quanted at all’.

athletico


Total Posts: 962
Joined: Jun 2004
 
Posted: 2014-01-22 15:17
Taleb's point is more or less equivalent to this:

An Intuitive Volatility Estimator

The post is pretty hyperbolic ("confused hordes of scientists", really?) but there is a grain of truth. Means are simpler and more intuitive than root-mean-squared for anyone to do in their head.

pj


Total Posts: 3530
Joined: Jun 2004
 
Posted: 2014-01-22 16:47
and what about correlations?

вакансия "Программист Психологической службы" -але! у нас ошибко! не работает бля-бля-бля -вы хотите об этом поговорить?

Steve Castle


Total Posts: 306
Joined: Sep 2010
 
Posted: 2014-01-23 14:25
pj, check this out:

Robust Correlation

I had to look it up but at the end of the day, you have to agree a measure for distance between two processes X and Y. Once you've done that, you should have a consistent approach in all cases by using that measure. StdDev / Mad are forms of "autodispersion" where Y = mean(X), while correlation is "heterodispersion" which where Y is something other than X.

The rest is normalizing the distance so the estimators are unbiased and hopefully robust.

in the words of one such quant ‘were on the whole either less quanted or not quanted at all’.

pj


Total Posts: 3530
Joined: Jun 2004
 
Posted: 2014-01-23 14:43
Thank you for the reference!
Just that these correlations correspond to the classical correlation,,
thus to σ , and not to MAD as is the Master
(not me, nor NeroTulip) would desire.

вакансия "Программист Психологической службы" -але! у нас ошибко! не работает бля-бля-бля -вы хотите об этом поговорить?

Steve Castle


Total Posts: 306
Joined: Sep 2010
 
Posted: 2014-01-23 15:17
specifically rCOMED doesn't.

they are all derivable from each other if you know the K's. e.g. you can sub MAD/K for sigmas and get correlation, and then use the same algebra.

The point is whatever number you get is a measure of distance between two processes. As long as you use it consistently, the rest of the algebra holds under affine transformations of vols and corrs up to constants for specific probability measures.

I wouldn't say any are better, but they are effectively the same measures, and you can change between them in a straight forward manner.

in the words of one such quant ‘were on the whole either less quanted or not quanted at all’.

NeroTulip


Total Posts: 1076
Joined: May 2004
 
Posted: 2014-01-23 15:23
What I have in mind is (assuming means are 0 for simplicity), look at average[(abs(x*y))^(1/2)*sign(x*y)]

This is similar to covariance, but for x=y you get average[abs(x)], which is the MAD, instead of average[x^2], which is the variance.

Edit: tried to clarify a bit


Inflatable trader

NassimNTaleb


Total Posts: 24
Joined: Jun 2012
 
Posted: 2014-02-16 00:40
Not so hyperbolic… A high percentage of people who work in finance (with PhDs). And we stipulate a huge number of econometricians. See papers.

NassimNTaleb


Total Posts: 24
Joined: Jun 2012
 
Posted: 2014-02-16 00:46
Nero, it stays unstable because you are multiplying random variables. The only solution is the copula, which always exist because the joint probabilities exist.

phy


Total Posts: 18
Joined: Nov 2013
 
Posted: 2014-02-16 06:14
what matters in a time series are the conditional moments, i.e. conditional variance and covariance. Financial series do have structure (non-linear) in the conditional moments.
Hence, a proper way to model is use xARCH-type or stochastic volatility models. For correlations, look for dynamic conditional correlation (DCC).

NeroTulip


Total Posts: 1076
Joined: May 2004
 
Posted: 2014-02-16 08:26
@Nassim:

I think my measure is made stable by taking the square root:

Let's say x and y have tail exponent a. The tail exponent of x^2 is >a, I think it is 2a, correct? This can cause average(x^2) to diverge while average(abs(x)) is defined, which is why MAD is more stable than stdev. Similarly, if the tail exponent of xy is 2a, this can cause average(xy) to diverge. But if I look at average(sqrt(xy)), I am back to a tail exponent of a, which means that if the MAD exists, my measure exists too.



Inflatable trader

quantie


Total Posts: 903
Joined: Jun 2004
 
Posted: 2020-05-21 12:33
Threadjack this is an interesting discussion it got a bit derailed. I don't think Nassim follows Rapaport's rule but appreciate his honesty.

gmetric_Flow


Total Posts: 39
Joined: Oct 2016
 
Posted: 2020-05-21 12:44
@quantie If discussions as per your link excite you, this might be worth a read (warning: quite long)

chiral3
Founding Member

Total Posts: 5163
Joined: Mar 2004
 
Posted: 2020-05-21 14:05
This conversation is soooo over-proofed. Same conversation from 20 years ago. Taleb losing money 9 out of 10 years, telling everyone they are idiots, not understanding the price/value of tails, and taking a victory lap every 1 out of 10 years. Doesn't Taleb make money by writing books? If he's so plugged into esoterica why isn't he the smartest guy nobody has ever heard of? He's unemployable for the same reason Malcom Gladwell is unemployable. Does he have any assets? If his ideas are really the notes in a song, one bar out of every ten, aren't the Cliff Asness' of the world the space between the notes? Providing structure and meter? It is a system, no? With a ton of meta-redundancy and leverage. We make fun of the BOJ for taking old furniture and baseball cards onto their balance sheet, but the US now takes EFTs. The psychology of NT is more telling than his old (borrowed) ideas.

I still maintain that, within the context of the time, Dynamic Hedging was the most interesting and original idea NT ever had.

Nonius is Satoshi Nakamoto. 物の哀れ

quantie


Total Posts: 903
Joined: Jun 2004
 
Posted: 2020-05-21 14:20
I still maintain that, within the context of the time, Dynamic Hedging was the most interesting and original idea NT ever had.


Totally agree was a book ahead of its time and I learnt a lot from it.

@gmetric_flow thanks for the link will read.

chiral3
Founding Member

Total Posts: 5163
Joined: Mar 2004
 
Posted: 2020-05-21 14:27
That was a long read. I made it half way through and tapped out. Aaron's comment in reply made me glad I did.

Nonius is Satoshi Nakamoto. 物の哀れ

Jurassic


Total Posts: 367
Joined: Mar 2018
 
Posted: 2020-05-21 15:34
@chiral3 im missing something but why is Gladwell unemployable?

nikol


Total Posts: 1172
Joined: Jun 2005
 
Posted: 2020-05-21 15:58
@chiral3

> Dynamic Hedging was the most interesting and original idea NT ever had

I share the view.

After reading "Fooled by randomness" I left with impression that the man tries to teach two things: that "S/N" and "trend" matter.

chiral3
Founding Member

Total Posts: 5163
Joined: Mar 2004
 
Posted: 2020-05-21 16:11
I was being a bit tongue-and-cheek, but I was referring to the fact that MG dabbles in this non-falsifiable and inexhaustible universe of anecdotes derived from correlation = causality. "Something amazing happened in a small town in Georgia in the early 1920's: the children seemed happier, on this everyone could agree. Little did they know that earlier that year the town switched water sources to a reservoir that later was determined to be fed by an ancient spring that had snaked its way through igneous granite, supplying unusual concentrations of Lithium..." or some shit like that. MG is good at throwing a yarn, but he's not a researcher, and he can't actually do anything, except write popular books that takes non-falsifiable supposed esoterica and renders it exoteric (I could go on for a while about the shifting roles of esoterica versus exoterica in the internet age). In that regard he's of limited utility in advancing the things that he talks about, which makes him very similar to NNT.

Nonius is Satoshi Nakamoto. 物の哀れ
Previous Thread :: Next Thread 
Page 1 of 2Goto to page: [1], 2 Next