Forums  > Pricing & Modelling  > Model-free option pricing by Reinforcement Learning  
     
Page 2 of 4Goto to page: 1, [2], 3, 4 Prev Next
Display using:  

Nudnik Shpilkes


Total Posts: 47
Joined: Jan 2009
 
Posted: 2018-05-11 19:45
Huh? Neither George Soros nor Vivek Kapoor whose paper I cite are academics...

"Risk-neutral quants belong in the jail and/or the mental asylum." (Vivek Kapoor)

katastrofa


Total Posts: 448
Joined: Jul 2008
 
Posted: 2018-05-11 23:01
"Katastrofa, you are of course free to stick to outdated garbage called risk-neutral models for another 40 years."

I asked you a number of questions, are you not going to answer them? Here they are again:

1. How do you account for the risk in rewards? Your answer is just "kicking the can down the road".

2. You still need to choose their values. How do you do this? Hyperparameters matter. There is a lot of debate about how you can overfit via hyperparameter optimisation. And your model is not data-efficient.

3. I'm surprised that you don't! Bootstrapping assumes conditional independence - and you agent is trying to learn correlations between X_t, a_t and X_{t+1}. You still don't see an issue here? In other words: by bootstrapping, you're making a model assumption. If you use bootstrapping to train your agent, you're not model-free.

"https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3174498"

That looks like a long paper, I don't have time to read it now (too busy writing my own!), but I like the idea more than the first two papers. But again, how do you solve the data problem? Maybe we should finish discussing the first papers and then take on the 3rd one? I will limit myself to complaining about the insufficient detail on numerical experiments, again.

"For a more detailed discussion, you may also find this interesting:

https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1530046"

Indeed! It looks much better than your papers. You could learn a thing or two from these authors.

Nudnik Shpilkes


Total Posts: 47
Joined: Jan 2009
 
Posted: 2018-05-11 23:11
I actually did, not sure about you.
I asked you a question too: where is risk in your risk-neutral models? Can we start with that?

Your point on insufficient numerical experiments is a valid one, I concur.

katastrofa


Total Posts: 448
Joined: Jul 2008
 
Posted: 2018-05-12 00:04
Wait a minute, you came here to present your papers, you're the one who's supposed to answer questions ;-)

Again:
1. How do you account for the risk in rewards? Your answer is just "kicking the can down the road".

2. You still need to choose their values. How do you do this? Hyperparameters matter. There is a lot of debate about how you can overfit via hyperparameter optimisation. And your model is not data-efficient.

3. I'm surprised that you don't! Bootstrapping assumes conditional independence - and you agent is trying to learn correlations between X_t, a_t and X_{t+1}. You still don't see an issue here? In other words: by bootstrapping, you're making a model assumption. If you use bootstrapping to train your agent, you're not model-free.

Nudnik Shpilkes


Total Posts: 47
Joined: Jan 2009
 
Posted: 2018-05-12 02:06
Will be happy to continue with answering your questions after you answer mine :)

Don't get me wrong - I am not claiming to have a magic solution to all problems in the world.
I am just stating the obvious - 'risk-neutral' models are defective. So, unless you use an incomplete market model (not just any incomplete market model - some of them are useless too!), you are just dead wrong. My papers is a step in a right direction, that's it.

If you actually read the papers that I quoted and that you encouraged me to read to "learn a thing or two", you would already knew it, so that we could possibly have a more productive dialogue :)

Nudnik Shpilkes


Total Posts: 47
Joined: Jan 2009
 
Posted: 2018-05-12 05:57
Wait a minute, you came here to present your papers, you're the one who's supposed to answer questions ;-)

Took me a while to recall what it is called: subjective priors!
What made you think that "present" == "defend"?
You are not on my PhD committee, nor pay me to make you smarter or richer (but PM me if you are interested :)
You said you are busy to read papers, you think I am not busy?
What if my only purpose of showing here was to generate some publicity/downloads on SSRN (by talking about real problems, actually), meet respected gentlemen on this forum, get some refreshing ideas like pricing in incomplete markets by sacrificing virgins :), and move on?
What's my upside to educate you personally?

pj


Total Posts: 3395
Joined: Jun 2004
 
Posted: 2018-05-12 08:02
One mor question, If your model is not risk neutral, it
is arbitrageable. Isn’t this bad?

The older I grow, the more I distrust the familiar doctrine that age brings wisdom Henry L. Mencken

Nudnik Shpilkes


Total Posts: 47
Joined: Jan 2009
 
Posted: 2018-05-12 08:08
I think it is not necessarily bad. Some smart people believe that statistical no-arbitrage is quite sufficient. I can make an optional no-arbitrage version of it as well. One (not the best one) way to make a strict no-arbitrage version is described in the appendix to the paper. But I think a simpler version can be produced too, where prices would be guaranteed to be non-negative (and different for bid and ask prices), no matter what risk aversion parameter to take.

katastrofa


Total Posts: 448
Joined: Jul 2008
 
Posted: 2018-05-12 10:35
"Will be happy to continue with answering your questions after you answer mine :)"

All right... You seem to be confusing risk-free with risk-neutral. There is risk in risk-neutral models - it's called volatility. By moving from real-world to risk-neutral measure you change the drift, not the volatility. You should know that.

"Don't get me wrong - I am not claiming to have a magic solution to all problems in the world."

I never expected that. But you claim that your RL agent is model-free. It isn't. Words matter. You have a lot of hyperparameters in your model with no obvious way (and no thought given in the papers) to determine their values. I think that's why other people consider your approach to be somewhat, if not magic, then underpanty-gnomey style.

"I am just stating the obvious - 'risk-neutral' models are defective. So, unless you use an incomplete market model (not just any incomplete market model - some of them are useless too!), you are just dead wrong. My papers is a step in a right direction, that's it."

But it could be improved a lot, which is what I tried to help you with, and all I got for it is arrogance and defensiveness. Oh well.

pj


Total Posts: 3395
Joined: Jun 2004
 
Posted: 2018-05-12 11:17
I do have another question. How does the method
deal with a Dupire model?

The older I grow, the more I distrust the familiar doctrine that age brings wisdom Henry L. Mencken

Nudnik Shpilkes


Total Posts: 47
Joined: Jan 2009
 
Posted: 2018-05-12 14:41
There is risk in risk-neutral models - it's called volatility.

Of course. When I say risk-neutral valuation, I mean all approaches where risk of mis-shedge is instantaneously eliminated instead of being priced. I know what the Girsanov theorem means.

Second, when I say 'model-free', I actually use standard terminology from RL, which you are apparently unaware of. How is it my fault? So instead of asking what it means, you start to attack me by claiming it is not model-free.

Arrogance? I actually see lots of arrogance on this forum. I don't stand that arrogant 'ruler of the Universe' attitude that I saw in some of replies on this forum. If you want to have a constructive dialogue, we have to start with finding some common grounds. I suggested you some starting points.

Nudnik Shpilkes


Total Posts: 47
Joined: Jan 2009
 
Posted: 2018-05-12 14:51
PJ - This I do not know yet. My method is an extension/reformulation of the Hedged Monte Carlo method of pricing in incomplete markets by Bouchaud and Potters. My method is a discrete-time method, not a continuous-time formulation. Your question is a good one, but at the moment I do not have any clear answer to it.

katastrofa


Total Posts: 448
Joined: Jul 2008
 
Posted: 2018-05-12 14:56
I know the standard terminology, but I can also see that you're circumventing it, in a sense. Your agent doesn't have to learn the model of the environment, because your training procedure has supplied it with one (via bootstrapping).

Let's settle on "partially model free", ok?

Re mishedges, you can also learn mishedges in your RL framework, if the trader your agent is looking over the shoulder mis-hedges her option. It's not different from calibrating your volatility skew to mispriced quotes. And actually, because the vol skew calibration is never based on a single quote, the standard approach is safer (some desks will make mistakes one way, some the other way).

If you wanted to learn the market dynamics just from the data (stock price movements), you will run into two problems:
- data are scarce
- dynamics is different at different timescales
- dynamics is non-stationary

So you need to supplement yourself with learning from other traders (via observing their hedges or via observing their option quotes). Google "wisdom of the crowd". Risk-neutral pricing is an *approximate* view of the world which allows you to do that in a self-consistent (across instruments, not across time) way. Sure, it's fundamentally wrong. But so is gradient descent, because it assumes that the function optimised is linear...

Re arrogance: you display another type of arrogance, the "smartest boy in the class" one. Traders are more "blue collar" types of people and they will jump on you as soon as they smell it. If I am not mistaken, you are related to a scientist who's been very successful in their field. This undoubtedly has given you a lot of self-confidence and held back people around you in the academia from pricking that baloon. But here? People won't give a crap for that. My honest advice to you is to learn some humility. It comes a long way.

Nudnik Shpilkes


Total Posts: 47
Joined: Jan 2009
 
Posted: 2018-05-12 15:37
But it could be improved a lot, which is what I tried to help you with, and all I got for it is arrogance and defensiveness. Oh well.

Oh yes, almost forgot... Let me know if you want to register for my course (coming soon), so that you would be able to learn more and actually play with this model, see how you can do about hyper-parameters, etc.

Strange


Total Posts: 1409
Joined: Jun 2004
 
Posted: 2018-05-12 15:46
Is that going to be a general ML course or a course specifically about applications of ML to options pricing?

I don't interest myself in 'why?'. I think more often in terms of 'when?'...sometimes 'where?'. And always how much?'

Nudnik Shpilkes


Total Posts: 47
Joined: Jan 2009
 
Posted: 2018-05-12 15:54
"Let's settle on "partially model free", ok?"

OK :)

"Risk-neutral pricing is an *approximate* view of the world which allows you to do that in a self-consistent (across instruments, not across time) way."

Yes, but this is a way more complex question than most people think. My third paper is a small step towards addressing this. Unfortunately, the current SSRN version has an error in it (just found out yesterday), but it will be fixed in a couple of days. (Speaking of your next topic about humility :)

"My honest advice to you is to learn some humility. It comes a long way."

Agree 100% :)

Nudnik Shpilkes


Total Posts: 47
Joined: Jan 2009
 
Posted: 2018-05-12 15:59
"Is that going to be a general ML course or a course specifically about applications of ML to options pricing?"

It's gonna be a set of short courses on ML and RL. One of them will be on RL for option pricing (and a bit about RL for stock trading). Accessible to beginners who do not know anything about ML or RL, but assuming that they know the Black-Scholes model and are familiar with basic math.

Strange


Total Posts: 1409
Joined: Jun 2004
 
Posted: 2018-05-12 16:36
> It's gonna be a set of short courses on ML and RL. One of them will be on RL for option pricing (and a bit about RL for stock trading).

Personally, I'd love to take an overview course on ML and RL especially.


> I think it is not necessarily bad. Some smart people believe that statistical no-arbitrage is quite sufficient. I can make an optional no-arbitrage version of it as well. One (not the best one) way to make a strict no-arbitrage version is described in the appendix to the paper. But I think a simpler version can be produced too, where prices would be guaranteed to be non-negative (and different for bid and ask prices), no matter what risk aversion parameter to take.

My couple takes on the above as a vol arb PM:

(a) An options strip is a complex animal where a lot of relationships are driven by various forms of arbitrages - put/call parity, forward volatility, skew term structure etc. Without enforcing those relationships, you're opening yourself to a lot of "one-sided flow".

(b) My main gripe with the model (even if you introduce arbitrage constraints) is lack of transparency. It is not clear to me what drives the pricing. It's also unclear how to compare one option to another in a model-free way. Implied volatility (normal or lognormal, I use normal models for some stuff I trade) is a dimensionless variable that, with some caveats, can be used that way - similar to yield for bonds.

(c) On the other hand, I can see how this method can be used for things like portfolio optimization "in real life".

I don't interest myself in 'why?'. I think more often in terms of 'when?'...sometimes 'where?'. And always how much?'

Nudnik Shpilkes


Total Posts: 47
Joined: Jan 2009
 
Posted: 2018-05-12 16:56
Great, I will let you know - it will be launched in a couple of weeks from now.

On your questions - these are, again, good ones. What I described was a simplest possible setting. It is extensible - in theory, without limits, but in practice, there are two major questions that still need to be answered.

First: about put/call parity and other constraints - in my view, it is an open question how to proceed optimally with them, as in theory, you may or may not want to enforce them explicitly. A high level answer - all these can be enforced as constraints or regularization. In this sense, I in fact do *not* believe in completely 'model free' methods - technically speaking, they do not exists, no matter what you might hear by reading about work at Google etc. Regularization is a king, and it relates ML to statistics and financial models.

Second: To price option portfolios, you'll need a good set of basic functions. This question is far from trivial beyond 3 or 4 dimensions. The problem of choosing a good set of basic function is central to ML.
(BTW, deep learning tries to solve it 'automatically' - my semi-educated guess is that it should produce crap in financial applications).

This is why my third paper develops a different, and this time *model-based* approach, that should work in a high-dimensional portfolio setting, while avoiding working with a fixed set of basic functions.


katastrofa


Total Posts: 448
Joined: Jul 2008
 
Posted: 2018-05-12 18:10
What about the DeepMind Atari games paper? Also not model-free?

pj


Total Posts: 3395
Joined: Jun 2004
 
Posted: 2018-05-12 18:19
I am looking forward to your course as well.

The older I grow, the more I distrust the familiar doctrine that age brings wisdom Henry L. Mencken

Strange


Total Posts: 1409
Joined: Jun 2004
 
Posted: 2018-05-12 18:22
> Great, I will let you know - it will be launched in a couple of weeks from now.

Could you put an email into your profile? Wanted to reach out and discuss something

I don't interest myself in 'why?'. I think more often in terms of 'when?'...sometimes 'where?'. And always how much?'

Nudnik Shpilkes


Total Posts: 47
Joined: Jan 2009
 
Posted: 2018-05-12 19:04
Just did.

Nudnik Shpilkes


Total Posts: 47
Joined: Jan 2009
 
Posted: 2018-05-12 19:12
"What about the DeepMind Atari games paper? Also not model-free?"

As you saw, words "model-free" have some marketing appeal :)

My current view is that there is no strict separation between 'model-free' and 'model-based' approaches. Whenever you add a regularization and it's role is important, you admixture a 'model-based' component to a 'model-free' component. It's been a while since I looked into Google's Atari paper, so do not recall details of what exactly they did with regularization. But in general, my sense is that deep learning is not a good approach for trading applications, so I largely stopped paying much attention lately to this. I think RL is more promising.

Nudnik Shpilkes


Total Posts: 47
Joined: Jan 2009
 
Posted: 2018-05-12 19:25
Landau used to say that 90% of published work in physics is a 'quiet pathology" :)

Atari's paper aside, would would be your prior guess on the amount of bullshit or pathology in ML literature?
Remember, Landau lived in the era of refereed papers and academic publications, not in an era of blogs and blockchains.
Previous Thread :: Next Thread 
Page 2 of 4Goto to page: 1, [2], 3, 4 Prev Next