Forums  > Basics  > Addressing parameter instability when using rolling WLS?  
     
Page 1 of 1
Display using:  

haakhtar


Total Posts: 3
Joined: Feb 2020
 
Posted: 2020-09-10 06:46
So I've been working on a model using WLS and one of the problems I run in to is unstable estimates of the regression parameters. I was wondering if anyone had any idea about ways to mitigate this?

gaj


Total Posts: 115
Joined: Apr 2018
 
Posted: 2020-09-10 07:21
A few possibilities off the top of my head, 1) there's not enough data, 2) the features are bad, 3) you have highly correlated features.

1 and 2 are kind of related. If you have bad features but you have a lot of data, the regression should give zero weights to those features. You can check the p-values and confidence intervals of the regression parameters to diagnose this problem. You should get rid of features that have no predictive power. You can also use shrinkage methods like ridge regression to make the coefficients more conservative and hence more stable.

For 3, if you have several features that are very highly correlated, the coefficients will blow up. The ideal solution is to construct the features to be uncorrelated in the first place. Sometimes this is not possible, so you could try some dimension reduction techniques like PCA.

bullero


Total Posts: 75
Joined: Feb 2018
 
Posted: 2020-09-10 07:22
Most likely, you have a near singular correlation matrix.

notes: http://www.stat.cmu.edu/~larry/=stat401/lecture-17.pdf

haakhtar


Total Posts: 3
Joined: Feb 2020
 
Posted: 2020-09-10 07:31
Thanks for the info bullero & gaj, I'll do some more reading according to your suggestions.
Previous Thread :: Next Thread 
Page 1 of 1