Forums  > Pricing & Modelling  > Dynamic PCA  
     
Page 1 of 2Goto to page: [1], 2 Next
Display using:  

Nonius
Founding Member
Nonius Unbound
Total Posts: 12657
Joined: Mar 2004
 
Posted: 2015-04-11 12:15
Suppose I form a basket of stocks with a PCA vector, (which is by construction is always on the unit sphere). Let's assume the PCa is well defined, such as taking the PCA with the smallest eigenvalue. Suppose I update the weights every day, so now it's a dynamically weighted basket. What is the fair N day forward value of the basket?

Chiral is Tyler Durden

titanium77


Total Posts: 54
Joined: Sep 2012
 
Posted: 2015-04-11 17:16
[Delete]

radikal


Total Posts: 253
Joined: Dec 2012
 
Posted: 2015-04-18 02:43
I'm not entirely sure I understand, but the naive and WRONG way would be:

1) At time t, look at your the last N days, calculate your returns matrix
2) Calculate PCA embeddings, take top N eigenvectors (using svd)
3) Project the historical returns back onto your embeddings to get the historical "projected" past
4) Regress the actual past vs your projected pass to get coefficient vector C
5) Returns in at t+1 are then the eigenvectors * your coefficient vector (standard LR) + noise
6) Loop until t+M

You could replace the end with perhaps a generative model and use monte carlo to get estimated forward distributions, but this all assumes the eigenvectors don't change, and hence your question I assume.

If the covariance structures are highly dynamic, your forward stochastic model has to price these, but unless you can reduce the variable space to something really small for describing those cov matrices, if you frame this with time varying parameters, there's no way this will converge...

An example hack that is really just first thing coming to my mind would be to grab chunks of your past data and calculate multiple eigen decompisitions, see if you can map something simple to which "eigen-regime" you're in (say just volatility -- so one more term for your regression), now repeat the above but use your time varying indicator variables to find the most likely set of eigenvectors, and now you can model the likely forward paths that indicator var will take, and adjust regressions accordingly.

There are no surprising facts, only models that are surprised by facts

Nonius
Founding Member
Nonius Unbound
Total Posts: 12657
Joined: Mar 2004
 
Posted: 2015-04-18 15:01
thanks Radikal, I think your last concept is probably the most practical.

Chiral is Tyler Durden

radikal


Total Posts: 253
Joined: Dec 2012
 
Posted: 2015-04-19 05:16
Another idea would be to fit to a functional form to the covariances and, if possible, then time varying MCMC is back in play.

Scrolling down to p.94 here, there's a bunch of things to try.

This is something I'm actually super interested in, I've just never had the time to commit to it -- I'm sort of torn how practical it is for trading baskets as the nature of the problem changes quite drastically depending on how you view basket size.

I'll also that that from a practical perspective of doing baskets vs futures, I've never used PCA variants in production, and have found the more boring minimum variance portfolio approach a bit cleaner as you can wrap that inner problem with some penalty parameter that describes basket size, and then frame your optimization to find the smallest well hedging basket.

With PCA, it's hard to really do something equivalent as your eigenvectors totally change as you add/delete stocks from your group.

Conceptually, it's also a little weird to think of your replicating portfolio of things like:
- Long 1.2 shares of A, short 3.5 of B, long .4 of C, short .2 of D, long 2 shares of E
- Short 1.5 shares of A, long 5 shares of B, long 2 shares of C, long 1.3 shares of D, short 5 shares of E
- etc etc

There are no surprising facts, only models that are surprised by facts

Nonius
Founding Member
Nonius Unbound
Total Posts: 12657
Joined: Mar 2004
 
Posted: 2015-04-20 15:51
interesting ideas Radikal.

I'm also torn between a) saying, fuck it, I'll just put on a position that represents the weights right now and b) trying to see if it makes sense to adjust those weights to account for some variance in the weights.

for me, if one sticks with a PCA matrix, then the naive average of PCA matrices sampled won't be a PCA matrix (because the average won't have the property of having columns that are orthonormal, but what will work is: take some PCA matrix X you think is the right matrix, then for every new estimated PCA matrix X', you compute XX'=Exp(A) for some matrix A that is, by the rules of orthogonal matrices, a skew symmetric matrix, and you compute a bunch of averages of the computed A matrices from the estimates of the X'. that will give some distribution in the A matrices. then see if that gives rise to some adjustments when trying to replicate a dynamic PCA derived basket. maybe it's too complicated.

the irony is that the dynamic basket I described has, on the one hand, high predictability in value but, on the other hand, it seems difficult to decouple the prediction into a) weights and b) asset returns.

Chiral is Tyler Durden

Rashomon


Total Posts: 165
Joined: Mar 2011
 
Posted: 2015-04-27 02:04
Nonius, what does "fair value" mean? Each stock is already priced at FMV. It sounds like you're trying to get a really uncorrelated mixture. If the covariances were stable then the portfolio covector would still be the same after 1 month. If they're unstable then you need a model of how they will/might change; with that you could calculate the distribution of portfolio weights after 1 month.

Am I getting you right?

Also - do you want your total portfolio size to stay the same? (sphere) Seems to violate Kelly.

"My hands are small, I know, but they're not yours, they are my own. And they're, not yours, they are my own." ~ Jewel

Johnny
Founding Member

Total Posts: 4333
Joined: May 2004
 
Posted: 2015-05-16 11:34
Nonius, you've probably moved on to bigger and more glorious questions by now but you could develop dynamic PCA by multiplying the PCA matrix each day by a transition matrix. Fitting these two matrices to data is then a simple form of Kalman filter in which the emission matrix = I and the latent variable is the PCA matrix. J



Stab art

Nonius
Founding Member
Nonius Unbound
Total Posts: 12657
Joined: Mar 2004
 
Posted: 2015-05-17 10:51
That's an interesting idea, Johnny. I think the right framework would be:

Kalman filter on Riemannian Manifolds


(And I'm not joking ;). )

The factor, let's say a PCA, is on the unit sphere, or you could look at it as living on projective space or if you look at the whole PCA matrix it lives in O(n).

Chiral is Tyler Durden

Nonius
Founding Member
Nonius Unbound
Total Posts: 12657
Joined: Mar 2004
 
Posted: 2015-06-23 19:20
anyway, read some papers on this. but then I'm thinking that it boils down to predicting the weights of the PCA independently of predicting the value of the dynamic long short portfolio. I thought the weights would be mean-reverting in some sense. But my tests show they aren't. they can trend for a fucking long time. that's kind of annoying. I would have thought the weights, being bounded to live on a circle, would wiggle back and forth. they don't. the weight rebalancing can also fubar shit more than I thought. anyway.

Chiral is Tyler Durden

dgn2


Total Posts: 2045
Joined: May 2004
 
Posted: 2015-06-23 19:24
Would you mind posting references to the papers you are reading?

[edit: i.e., other than the reference posted already]

...WARNING: I am an optimal f'er

unsmt


Total Posts: 196
Joined: Jul 2014
 
Posted: 2015-06-23 23:25
Never thought about such problem so it is a preliminary idea. Let portfolio holds two stocks. Is it possible that such stocks on the 1st period day or week look like independent and next are like linear dependent with positive correlation and on the next interval negative correlated. It seems that rank of the correlation matrix can change over the time and it should be some rigorous assumptions that could help to get an answer of the problem for a large number of stocks and then we should verify these assumptions in order to apply the model.

Nonius
Founding Member
Nonius Unbound
Total Posts: 12657
Joined: Mar 2004
 
Posted: 2015-06-24 23:18
discrete Kalman filter on Lie Groups

Dgn, the above is more or less the correct programme for updating an entire PCA matrix, but he ignores second order terms and I'm not sure some of his assumptions are the most natural. Again, I don't think you need to jointly estimate the return of the portfolio and the weights of the portfolio using some highfalutin kalman. Put it this way, if you dynamically update weights very often, the portfolio will look veeeerrry mean reverting, which is nice, but the issue is the weights change a lot so you get fucked on rebalancing the weights, so, you need to find a way to predict weight changes.

Chiral is Tyler Durden

dgn2


Total Posts: 2045
Joined: May 2004
 
Posted: 2015-08-19 14:15
Nonius, did you ever get further on this? I have a - perhaps simpler - problem that I am trying to frame. I am looking for a way to replace a rolling window PCA (using the typical Eigen-decomposition) with something that updates the eigenvectors with the *closest* eigenvectors to the previous window. There must be a lot of work out there about how to do this in different ways, but after searching I can't seem to find something simple.

I am effectively trying to represent a portfolio of about 3,500 global stocks and bonds with a bunch of small tradable baskets (factors). In the past, I have used a simple recursive regression (where the weights are updated via the Kalman filter) for small groups. Intuitively, I am thinking that I want a two-step process where I (1) estimate statistical factors via some sort of slow recursive PCA that should produce relatively stable weights (perhaps contingent on regime) then (2) use a recursive regression to track our exposure to these factors. I am trying to separate the picking of representative baskets and the mixing of those slowly changing baskets for my dynamic replication.

The rebalancing of the baskets - which are intended to have intuitive interpretations - should happen on a slower time scale than the rebalancing of the mix of the baskets.

Can anyone make sense of what I am asking?

...WARNING: I am an optimal f'er

Nonius
Founding Member
Nonius Unbound
Total Posts: 12657
Joined: Mar 2004
 
Posted: 2015-08-24 11:40
Dgn2,

I put this aside for a while. Did code up 2x2 case, which is trivial in some sense as it's a one dimensional problem that reduces to some trig function calculations. But it didn't seem to help much.

Yes, I was using recursive regression as well and maybe it's hard to improve on that.

I'm going to have to think about what your asking.

Chiral is Tyler Durden

dgn2


Total Posts: 2045
Joined: May 2004
 
Posted: 2015-08-24 14:04
Thanks Nonius for the response. I think that what I am doing is effectively a recursive orthogonal regression. There must be a ton of work that does that in many fields, but I haven't found what I am looking for yet.

...WARNING: I am an optimal f'er

tbrown122387


Total Posts: 34
Joined: Feb 2010
 
Posted: 2015-09-21 05:58
@Johnny, do you have a reference? What kind of state noise do you have for that?

FatChoi


Total Posts: 125
Joined: Feb 2008
 
Posted: 2015-09-21 14:13
These may be helpful:
Ait-Sahalia
Pelger

Pelger has more on his homepage.

I think the Pastur Marcenko type issues make this very difficult. Do you want the baskets to be implementable as hedges? Predictable? Independent for estimation of diversification? Stable over what horizon? Responsive to regime changes?

Unless you use very high frequency to identify the covariance data there won't be enough data to get a robust estimate of the slow factors so you can either go high frequency in which case the distinction between fast and slow factors will blur or you can add other considerations to minimise the cost of estimation error like restricting to long only portfolios in returns based style analysis.

Accounting for non-Gaussian distributions can also change results quite dramatically.

Edit: I should also acknowledge that this kind of regression is pretty much what APT, now part of Sungard, does. There was, I believe, little stability in the eigenportfolios beyond the first two or three but they did claim better out of sample risk behaviour than Barra.

Rashomon


Total Posts: 165
Joined: Mar 2011
 
Posted: 2015-09-26 03:30
What happens if you test for stability of the eigenvalues as you vary the time window?

"My hands are small, I know, but they're not yours, they are my own. And they're, not yours, they are my own." ~ Jewel

Nonius
Founding Member
Nonius Unbound
Total Posts: 12657
Joined: Mar 2004
 
Posted: 2015-09-26 15:23
@FatChoi, in fact I am using high frequency data.

@Rashomon, the high eigens are sort of stable. The low ones are less stable.

Chiral is Tyler Durden

dgn2


Total Posts: 2045
Joined: May 2004
 
Posted: 2015-11-18 04:21
I came across this late last night (http://www.symmys.com/node/599) that does something like what I want. I am working on implementing it tonight so that I can play with it.

...WARNING: I am an optimal f'er

radikal


Total Posts: 253
Joined: Dec 2012
 
Posted: 2015-11-19 13:47
Fancy. I wouldn't really have expected that optimization to be well-behaved. I've sort of gotten back into the habit of just choosing my own basket of factors and tracking the residual vs that basket.

There are no surprising facts, only models that are surprised by facts

dgn2


Total Posts: 2045
Joined: May 2004
 
Posted: 2015-11-20 18:17
I guess I will see if it is well-behaved. It has to be better than just running a rolling PCA. I have a set of non-statistical factors so I am comparing these statistical factors to those factors. I like measures that give an effective number of bets. I find these types of things useful for communicating how the level of diversification is changing over time. I track how much of the portfolio can be explained by all factors, the top 5, top 10, etc...

...WARNING: I am an optimal f'er

Nonius
Founding Member
Nonius Unbound
Total Posts: 12657
Joined: Mar 2004
 
Posted: 2015-11-24 10:31
rolling PCA was something I did with reasonable effect, but then you have all kinds of choices to make like how frequently do you recompute it etc. I never did more frequent than once a day, in which the PCA was calced based on a sort of EMA of the variances and covariances with a "duration" of around 20 days; it was then applied for intraday trading with a sort of turnover frequency of 3 hours. I had then experimented with a much higher frequency of updates and then what you get is a highly stationary process, which would be interesting if it were tradable, alas, predicting the dynamics of the PCA movements doesn't looks so easy. in particular, they are definitely not "mean reverting" themselves. the reason why I thought a PCA vector might be mean reverting was that its a vector of unit length, so it can'tt wonder off to "infinity"; it has to wiggle around on the sphere. but a simple test showed it can drift to some point, then wiggle around, then drift somewhere else. couldn't see how to predict it. hat was my takeaway.

Chiral is Tyler Durden

chiral3
Founding Member

Total Posts: 4979
Joined: Mar 2004
 
Posted: 2015-11-24 13:07
The problem I've had with applications very similar to this is that every now and then I find some method that works amazing in backtest but isn't always tradable. Or is tradable but the turnover or tx costs would be unthinkable.

Nonius is Satoshi Nakamoto. 物の哀れ
Previous Thread :: Next Thread 
Page 1 of 2Goto to page: [1], 2 Next