Forums  > General  > Mathematical Success in Quantitative Trading  
     
Page 1 of 1
Display using:  

ChicagoHFT


Total Posts: 5
Joined: May 2018
 
Posted: 2018-05-26 04:18
I work for a fairly reputable trading firm. We all get paid reasonably well. But our divisions' success has gone down hill. I feel the tactics we employ lack mathematical rigor and are far too simplistic.

I am in the process of transitioning from a pure software engineer to a quant to pick up the slack from my quantitative peers that I feel are underperforming. I am reading difficult books like time series analysis and rudin's analysis as well as abstract algebra.

Part of the reason I am going down this path is that there are certain books that I am aware of that for example, start talking about sigma algebras and filtrations on page 2. This abstract language I see is also used in topics of machine learning that I see being used by our most profitable group.

I used to write this off as excessively academic, but experience has started to make me realize that this stuff may actually may be important, if only to understand the rest of the content in these advanced books.

So before I go on this looks like 2 year journey into essentially an "ersatz" MFE program, would like to get some other opinions on the pros/cons of my approach.

sharpe_machine


Total Posts: 16
Joined: Feb 2018
 
Posted: 2018-05-27 22:59
Actually, I think in 2018 it is better to start with some Kaggle/ CrowdAI / whatever... to "blindly" pickup basic things. After that, you will be able to apply BASIC methods to model whatever you need.
And after that, you will dig into that and will pick up more and more sophisticated concepts.

Not to mention that ML is not a magic stick, you know it, right?

ronin


Total Posts: 340
Joined: May 2006
 
Posted: 2018-05-31 12:40
If you want to transition to being a quant, you probably need to learn the maths.

Once you become a quant, you'll strip out all the unnecessary complexity and come back with simple models that your grandmother could understand. But it takes a bit of knowledge to know what you can strip out.

In terms of your reading materials, it seems to be advanced undergraduate (in Europe) / graduate (in the US) level pure maths. Rudin's Analysis is finite dimensional real analysis if I remember correctly. Abstract algebra - you mean groups, fields etc?

You don't strictly need any of that for 99.9% of any quant stuff.

It sounds like you are trying to construct a self-taught maths degree. It will take a while, and being self taught you can easily go off on a tangent in a million different ways.

Btw many quants don't have maths degrees full stop. If you go around any significant quant team on the street and take a poll how many have ever opened Rudin's Analysis or Hungerford's Algebra, it will probably be in the 10-20% range - those that have degrees in pure maths.

"There is a SIX am?" -- Arthur

Jurassic


Total Posts: 152
Joined: Mar 2018
 
Posted: 2018-05-31 12:44
> Btw many quants don't have maths degrees full stop.

Why is this? Lack of interest by them or by employers?

HankScorpio


Total Posts: 477
Joined: Mar 2007
 
Posted: 2018-05-31 13:36
@jurassic
Many have Engineering (particularly Elec Eng), Physics and Comp-Sci backgrounds. Some even come from Phynance!

Jurassic


Total Posts: 152
Joined: Mar 2018
 
Posted: 2018-05-31 13:41
My question was really why are there not more from maths backgrounds?

HankScorpio


Total Posts: 477
Joined: Mar 2007
 
Posted: 2018-05-31 13:49
Because it's all relative.

Depending on the quant role - and these can vary significantly - there are plenty of mathematicians working in these roles.

But there are many from the other fields mentioned in my original response, all having high math literacy. Also don't forget the ability and willingness to code (and in some cases to code shit fast).

Capisce?

EspressoLover


Total Posts: 333
Joined: Jan 2015
 
Posted: 2018-05-31 21:22
First, I think it's important to distinguish "pricing quants" from "trading quants". By and large the two often rely on pretty different techniques, so it's hard to make sweeping statements about both. That being said I'm a trading quant, have very little experience with pricing, and the OP seems to be asking purely from the standpoint of a trading quant. So the rest of my post comes from a trading perspective.

Abstract algebra, real analysis, and pure math have virtually no practical use in this field. Yes, a lot of people with this background get hired. But that's basically because skill in these fields is highly correlated with raw cognitive ability. It's the same reason why some firms recruit chess grandmasters.

If you want to learn fields you might actually use, I'd suggest (in rough order of importance, depending on your area) statistics, linear algebra, time series analysis, statistical learning theory, classical finance, machine learning, convex optimization, signal processing, metaheursitics, game theory, algorithms, information theory, microeconomics, probability theory, econometrics, numerical analysis, and NLP. If you read one and only one textbook, make it The Elements of Statistical Learning. If you grok everything in that book on an intuitive level, you will be ahead of the sizable majority of trading quants.

The vast majority of successful strategies are using basic techniques. Most of the actual math could probably be taught to a bright high school senior. That being said, and echoing @ronin, the importance of understanding the fundamentals is to know why a certain technique is appropriate and what it's specific shortcomings are.

From a philosophical standpoint, I see a few interrelated reasons why pure math is mostly useless for quant trading. The first is because off-the-shelf techniques are almost always sufficient. An innovative strategy may use a new data source, or an innovative approach to frame the data. But by and large the actual modeling techniques has probably already been worked out in some other field. For example, we take for granted that least squares is the MLE for normalized error terms. That originally required some rigorous abstract math to derive and prove, but it was done well before anyone in finance arrived on the scene. In contrast string theorists or cryptographers often find themselves needing to push the boundaries of previously known math.

The second is that empirical validation is more important than formalized validation. Most of the time our historical data is more "reliable" than our underlying assumptions. I'd rather trade a strategy that backtests well, than a strategy derived from first principles. Whatever axioms we start with are at best a rough characterization of reality. Most of the time we want the data to drive the model, rather than vice versa. That means fewer parameters, simpler structures, and weaker pre-existing assumptions. All of which usually entails simple mathematical models.

The third is that financial markets are "wet" and noisy. In comparison something like General Relativity is completely described by 10 field equations. Those alone are all you need to completely model the theory with arbitrary accuracy. In finance there are "stylized facts" that describe a lot of behavior. But beyond those you get all kinds of kooky shit happening. Like people doubling some unrelated stock's price because its name sounds like yesterday's hot IPO. With a "dry" problem like GR, you can very heavily lean on your assumptions. You can derive results using arbitrarily long and complex chains of reasoning. But in a wet domain, even well established stylized facts break down if you dissect them too much. Its like trying to take the derivative of a non-smooth function. Careful mathematical rigor takes a backseat to intuition, analogy and empiricism.

Similarly, image recognition is another very "wet" problem. And by and large, we see very little mathematical rigor. The state of the art and most successful teams pretty much stick to open-ended non-parameterized methods. Then throw a shit ton of computing power and training data at it. Hardly anyone wastes time trying to formally prove anything. Very few, if anyone, can definitely say why certain techniques work better than others. At least not from a theoretical perspective. Ask why one neural network architecture is used over another, and you pretty much just hear that it "did better on ImageNet".

Good questions outrank easy answers. -Paul Samuelson

Jurassic


Total Posts: 152
Joined: Mar 2018
 
Posted: 2018-05-31 21:32
@HankScorpio, yeah

> and willingness to code

Sounds a bit loaded....

ronin


Total Posts: 340
Joined: May 2006
 
Posted: 2018-06-01 10:25
> Why is this? Lack of interest by them or by employers?

No, employers are happy to take anybody who is reasonably technical, smart, hungry and doesn't smell. In reality, most employers will settle for three out of four of those. And there are plenty of people who are two out of four who still find jobs.

That's for entry level quants who get trained up on the job. Later on it becomes more complicated.

I don't see a big difference between pricing quants and trading quants. I started as one, ended up as the other. Same thing, different packaging.

Why not more pure maths - there just aren't that many pure maths graduates in the first place, and many of them don't look for quant jobs.

@chicagoHFT,

Don't take any of this as discouragement with what you are doing. Read up what interests you, and learn what you need to be able to follow it. That's what most of us do.

But don't feel that you somehow *have to* reconstruct your own maths degree to be a quant. That's an overkill.


"There is a SIX am?" -- Arthur

katastrofa


Total Posts: 458
Joined: Jul 2008
 
Posted: 2018-06-01 14:19
"Similarly, image recognition is another very "wet" problem. And by and large, we see very little mathematical rigor. The state of the art and most successful teams pretty much stick to open-ended non-parameterized methods. Then throw a shit ton of computing power and training data at it. Hardly anyone wastes time trying to formally prove anything. Very few, if anyone, can definitely say why certain techniques work better than others. At least not from a theoretical perspective. Ask why one neural network architecture is used over another, and you pretty much just hear that it "did better on ImageNet"."

But that's not a good state of affairs, and people realise that, and are pushing to do better. Adversarial examples showed that the simple approaches like the one you described above are not robust - actually they're very fragile. Overfitting to ImageNet is another, very acute problem.

There is a lot of research happening which injects more rigour (not at the level of GR, of course) into image recognition. People analyse the geometry of the loss surface, sturcture of loss function minima, classification boundaries. People are trying to find better ways to estimate how well their classifiers generalise.

Of course it'll always be data-driven and empirical and not 100% rigorous, because of the data complexity, high dimensionality and what not. But the goal of many research groups is to provide more sound theoretical footing for heuristics being built later on (often by the same people).

ChicagoHFT


Total Posts: 5
Joined: May 2018
 
Posted: 2018-06-02 18:04
thanks for the reply. But it is only from seeing people struggle with quantitative modeling I appreciate the importance of solid and rigorous understanding. For example, here is a simple intro into stochastic calculus, a prerequisite to read things at just one level up such as volatility surfaces.

http://www.math.cmu.edu/~gautam/sj/teaching/2016-17/944-scalc-finance1/pdfs/notes.pdf

In fact, perusing several books in the office on the bookshelves of our top quants, I see they all talk in a similar "language" as in that link, especially when discussing stochastics and probability, critical topics in machine learning. This is why I am getting back to the roots of analysis and abstract algebra, not to try to emulate a degree but out of what looks to me as sheer necessity.

ronin


Total Posts: 340
Joined: May 2006
 
Posted: 2018-06-02 22:56
> our top quants, I see they all talk in a similar "language" as in that link

Whoah. Easy there.

That link is about stochastic calculus and Black Scholes, not machine learning. It is specifically for people with pure maths backgrounds. It is not a "simple introduction". It's for teaching undergrads how to prove Girsanov's theorem.

The exact same material is covered say here:
https://shamit8.files.wordpress.com/2014/11/options-futures-and-other-derivatives-8th-john.pdf
pages 280-314. You will notice some difference in presentation and notation. I assure you that there is pretty much zero information loss between your pure-maths-heavy intro and Hull.

If your top quants seriously talk about sigma algebras, then no wonder your division has gone downhill. Get out of there.

Otoh, if you mean that you want to learn more about probability, you are better off picking some introduction to probability textbook. Here is one, and there are thousands just like it:
http://vfu.bg/en/e-Learning/Math--Bertsekas_Tsitsiklis_Introduction_to_probability.pdf



"There is a SIX am?" -- Arthur

ChicagoHFT


Total Posts: 5
Joined: May 2018
 
Posted: 2018-06-12 06:07
Well I didn't talk about machine learning in particular because that are many aspects of the trade and I didn't focus on that part. But sure, the wizards all use the very popular tensor flow in their cluster computation engines. Which naturally asks, what is a tensor? The proper introduction is in abstract algebra in modules and ring theory. I've seen the teach yourself tensor flow in 30 days style books, and they are more of a monkey see monkey do style. Good enough for simple stuff like learning python, but woefully inadequate for a subject of this nature. They will gave the bare minimum discussion on homomorphisms and kronecker product. Enough to code it up and write the examples, not enough to innovate on real world market data. And the real books all assume that you have a proper mathematical foundation from chapter 1.

In summary, pricing requires analysis, which cover things like measure theory, Stieltjes integration and Ito.

Machine learning requires abstract algebra as discussed above.

It is only seeing supposed experts struggle and swim in circles that I realize the importance of understanding. If you are fortunate enough to work under someone who knows all of this then you may just hold your nose and pray he has the plan and blindly follow him.

Strange


Total Posts: 1436
Joined: Jun 2004
 
Posted: 2018-06-12 06:37
"No, employers are happy to take anybody who is reasonably technical, smart, hungry and doesn't smell. In reality, most employers will settle for three out of four of those. And there are plenty of people who are two out of four who still find jobs."

Frankly, I am one out of four and even that is because I am trying to lose weight. There are plenty of people in this business who can barely boast subcortical functions and yet are pretty senior.

I have seen a few very talented mathematicians that failed to develop the required market intuition and never progressed beyond very junior levels. Similarly, borderline retards who can barely remember basic linear algebra (e.g. yours truly) seem to have done all right. My impression is that a lot of success in this business is knowing what you don't know and how to delegate or ask for advice, where to look things up etc. That's especially true in trading - most traders/PMs are surrounded by super-bright technical people that can be consulted whenever you need to solve a specific problem.

I don't interest myself in 'why?'. I think more often in terms of 'when?'...sometimes 'where?'. And always how much?'

ronin


Total Posts: 340
Joined: May 2006
 
Posted: 2018-06-12 10:16
@chicagohft,

Just a brief anecdote. I grew up in central Europe, where the approach to schooling was a bit similar to what you have in mind. Solid grounding is the key to everything.

So when we were about 14, we had this course "Introduction to Computer Programming". By that time, I was already a little hacker. Me and my firends were coding silly arcade-type games in c, and I was looking into c++. I was struggling to get the feel for object oriented programming. So I was quite looking forward to that course.

The course? Well, in order to code, you need to know the ASCII code. But you can't just pull ASCII out of nowhere. So she decided to teach something called BSCII. Did you know there was a BSCII code? Me neither. I tried to google it now, but google doesn't know about it. Neither does Wikipedia. And yet, she spent the entire f****ing year teaching us the BSCII code. Because without that, how can you truly understand the ASCII code. And without that, forget about computer programming.

Of course we never got to any computer programming. We never even got to ASCII code. We just spent the entire year learning BSCII. And before you ask, the answer is no - I don't remember anything about BSCII any more.

Anyway, I had actually repressed that memory until you brought it back up now.

To come back to the main question.

> measure theory, Stieltjes integration
> abstract algebra in modules and ring theory

If you mention any of those to 90% of quants on the street, they won't know what you are talking about. No clue. Zero. Nada. Zilch. Doughnut.
9.99% of quants on the street might say something like "oh yeah, it rings a bell. But it was a loong time ago. I really don't remember anything about it now."
And then there is the 0.01% of quants that you seem to interact with. In all seriousness, it is not representative. You need to get out more.


"There is a SIX am?" -- Arthur

Jurassic


Total Posts: 152
Joined: Mar 2018
 
Posted: 2018-06-12 10:31
> If you mention any of those to 90% of quants on the street, they won't know what you are talking about. No clue. Zero. Nada. Zilch. Doughnut.

I find this surprising and amusing at the same time

ChicagoHFT


Total Posts: 5
Joined: May 2018
 
Posted: 2018-06-12 14:07
I don't know where you got your probabilities from, but I do agree that it is very consistent with the observation that 10% of the people produce the breakthroughs and innovations that drive the business and account for the majority of the profits. I have to admit I shake my head at de-emphasizing material that is actually covered in a junior undergraduate level. I'm not talking about functional analysis here...

This is not that high of a requirement but I am pleased that most don't know it. It means I will be better than 90% once I am done. I have personally interacted with the few good quants and also many bad quants. I have seen many incorrect projects and proposals that are created by people with incomplete knowledge.

Anyway, I can't go too much deeper than this since we're quickly getting close to proprietary details, but this is my final example. If you don't know this inside out and can't work with this like solving y=mx+b AND are not working under someone who does, then you shouldn't be doing machine learning in a quantitative fashion:
https://www.youtube.com/watch?v=TngePpJ_x-I
https://www.tensorflow.org/tutorials/kernel_methods


gaj


Total Posts: 25
Joined: Apr 2018
 
Posted: 2018-06-12 14:47
>If you don't know this inside out and can't work with this like solving y=mx+b AND are not working under someone who does, then you shouldn't be doing machine learning in a quantitative fashion:
https://www.youtube.com/watch?v=TngePpJ_x-I
https://www.tensorflow.org/tutorials/kernel_methods

So that's where the confusion is coming from.. The word "kernel" has multiple meanings, kernel in group theory has nothing to do with kernel methods in machine learning.

ronin


Total Posts: 340
Joined: May 2006
 
Posted: 2018-06-12 15:52
@jurassic,

It is surprising if you expect quant finance to be a branch of mathematics. In reality, it is more like engineering. You build sh*t within time and budget constraints. You expect it to work within some parameters. You maintain it. The world keeps moving forward and eventually you have to rebuild it from scratch because by now it's obsolete.

Sounds like mathematics?

By that same token, you should be surprised and amused that, say, aeronautical engineers don't know the Trace Theorem for Sobolev spaces. You know, the guys who optimise wings and turbines.

If you are, you are in for a lot of surprise and amusement.


>https://www.tensorflow.org/tutorials/kernel_methods

@chicagohft,

It's a good example. I note that there is precisely zero of any of
> measure theory, Stieltjes integration
> abstract algebra in modules and ring theory

There is a bit of geometry (linear algebra?) when you are rotating some matrices, and then a Fourier transform. And the paper has an appendix with estimates of some errors.

It's all engineering maths, motivated by some intuition. 99.99% of quants are at home with this paper. Those same 99.99% who would just blink at you when you talk about
> measure theory, Stieltjes integration
> abstract algebra in modules and ring theory

I am sorry if it seems like I am trying to discourage you from learning. I am really not. I was trying to keep you somewhat tethered to the reality of what you are trying to achieve.

But the stuff you want to learn is great, and the fact that you have the drive and passion to learn it is even better. Go for it, and good luck.


"There is a SIX am?" -- Arthur

EspressoLover


Total Posts: 333
Joined: Jan 2015
 
Posted: 2018-06-12 17:04
@chicagoHFT

You seem to be making a logical fallacy, where either everyone knows A) nothing technical, or B) the deepest and most pure fundamental mathematics. That's bullshit, you don't need ring theory to understand and use tensors. General relativity was discovered before ring theory even existed.

I guarantee you that Yann LeCun knows less abstract algebra than the average math grad student. Myron Scholes probably doesn't know any abstract algebra. Andrew Ng might not even know what abstract algebra is. And I'm not even sure if John Merriweather can count past 20.

You don't have to take my word for it. Here's a podcast with Nick Patterson, who was one of the most senior people at Renaissance. Surely, if abstract algebra was important for quant trading, we'd certainly see it used at the most successful quant fund of all time. Especially because that fund was founded by algebraists.

And yet... (answer starts at 30:00):

The most important type of data analysis is to do the simple things right. Here's a non-secret about the things we did at Renaissance. In my opinion the most important statistical tool was single regression. One target, one variable... We have the smartest people around, string theorists from Harvard, and they're doing single regression. Is this stupid? Should we be hiring stupider people and paying them less. The answer is no.

And the reason is no one tells you what variables you should be regressing. What's the target? Should you be doing a non-linear transform before you regress? What's the source? Should you clean your data? Do you notice when your results are obviously rubbish?... The smarter you are, the less likely you are to make stupid mistakes. And that's why we need smart people doing something that appears to be doing something technically easy, but actually isn't so easy.


Good questions outrank easy answers. -Paul Samuelson

AndyM


Total Posts: 2330
Joined: Mar 2004
 
Posted: 2018-06-13 17:34
Quanting involves applying a quant mindset to financial problems, not applying whatever you happened to study for your PhD to unlock the secrets of the financial universe (I mean, what an amazing co-incidence, right?!). The latter type of quants are the ones you seriously have to watch out for, as they're a profound danger to themselves and others.

I used to be disgusted; now I try to be amused...

NIP247


Total Posts: 544
Joined: Feb 2005
 
Posted: 2018-06-16 17:41
@ronin, love your anecdote. Could the "BSCII" you mention really be "EBCDIC"? Before ASCII there was Baudot Code...

On your straddle, done on the puts, working the calls...

ronin


Total Posts: 340
Joined: May 2006
 
Posted: 2018-06-18 14:33
> @ronin, love your anecdote. Could the "BSCII" you mention really be "EBCDIC"? Before ASCII there was Baudot Code...

Haha, now looking back on it, it seems to have been the early evolution of ANSI standards for the ASCII code through the 1960s and 70s. I have no idea why it was called BSCII.

Of course, the other possibility is that she made it all up there and then because she had to teach something, and she had no clue about computer programming. I guess we'll never know.


"There is a SIX am?" -- Arthur
Previous Thread :: Next Thread 
Page 1 of 1