Forums  > Basics  > SVM vs. Neural Network  
     
Page 1 of 2Goto to page: [1], 2 Next
Display using:  

asd


Total Posts: 461
Joined: May 2004
 
Posted: 2005-01-01 01:49

A lot of sources say that SVMs (Support vector machine) have ouperformed neural Network in their prediction errors. Will Neural Network be replaced by SVMs, or does Neural network still have advantage as compared to SVM is some areas?

Also Wish you all a Happy New Year!

Many thanks in advance


Nonius
Founding Member
Nonius Unbound
Total Posts: 12779
Joined: Mar 2004
 
Posted: 2005-01-01 16:37
My understanding is that one of the main advantages of SVM over NN is that the associated optimization problem has a unique solution if certain conditions are met...whereas in the case of NN, you could get stuck in local minima very easily.  The other advantage, from my understanding, is that one is not exposed to "overfitting" in SVM, since it is a basically linear regression, but after a transformation of the data pionts through a nonlinear map into a higher dimensional "feature space"space.

Chiral is Tyler Durden

Nonius
Founding Member
Nonius Unbound
Total Posts: 12779
Joined: Mar 2004
 
Posted: 2005-01-01 16:44
one of the interesting connections with the SVM idea of mapping into a "higher dim feature space" is Takens Embedding Theorem and in fact SVM appears to be a pretty decent way of predicting a Chaotic time series.

Chiral is Tyler Durden

Baltazar


Total Posts: 1769
Joined: Jul 2004
 
Posted: 2005-01-04 10:40
in addition to what nonius says:

svm has a good theory to back it up:

when you do prediction or detection out of a small learning set your bond to have errors: this errors can be decomposed in two terms:

when comes from the choice of detection function and one comes from the choice of parameters of this detection function. (it is statistical learning theory: vapnik and chervonenkis work). (to me at least, NN are more fuzzy.

just one correction: it is not prone to verfitting not because it is linear but because it is linear AND it maximize the margin. the VC dimension (the ratio VC/ nb of samples acts on the detector related error) of a linear detector is N+1 where N is the size of the space where the samples are taken form.

simply: using kernel maps data from the X space into a H space (of very high dimension even infinite), so building a linear detector is bounded to overfitt, BUT margin maximization is a way to reduce overfitting.





Qui fait le malin tombe dans le ravin

Anthis
It's all Greek to me

Total Posts: 1180
Joined: Jul 2004
 
Posted: 2005-01-07 16:58
Balt, could you please provide us with some reading material on SVM?

Αίεν Υψικράτειν/Τύχη μη πίστευε/Άνδρα Αρχή Δείκνυσι/Νόησις Αρχή Επιστήμης //Σε ενα κλουβί γραφείο σαν αγρίμι παίζω ατέλειωτο βουβό ταξίμι

Baltazar


Total Posts: 1769
Joined: Jul 2004
 
Posted: 2005-01-07 17:12
sure

the main source is

www.kernel-machines.org

more specifically

you may find these papers interesting
http://www.kernel-machines.org/papers/upload_13550_kmrev.ps
http://www.kernel-machines.org/papers/Burges98.ps.gz
http://www.kernel-machines.org/papers/smosch97.ps.gz
http://www.kernel-machines.org/papers/tr-30-1998.ps.gz
http://www-ai.cs.uni-dortmund.de/DOKUMENTE/klinkenberg_joachims_2000a.ps.gz

this one on time serie prediction
http://www.kernel-machines.org/papers/IC.ps.gz
ftp://ftp.esat.kuleuven.ac.be/pub/SISTA/suykens/reports/lssvm_00_79.ps.gz

this one in density estimations
http://www.ai.mit.edu/projects/cbcl/publications/theory-learning.html

depending on what you want, i can give you more

Qui fait le malin tombe dans le ravin

gekko


Total Posts: 21
Joined: May 2006
 
Posted: 2006-05-29 09:18
Disocvered this thread on searching for neural networks,
I have read (on other forums) that NNs are no longer used by the financial institutions, after some banks made losses in mid 1990s, will be good to hear abt it if some one knows..

FDAXHunter
Founding Member

Total Posts: 8372
Joined: Mar 2004
 
Posted: 2006-05-29 10:17

That would be correct. The mid/late-1990s saw a lot of interest in neural networks (including yours truly).

10 years later, nobody is using them (except for very specific and niche applications, which are usually more gimmick than anything else).

Why? Turns out that they are difficult to handle. Parsimonity is usually the key message in any financial application and there are other, better numerical tools available for handling small parameter spaces.
It can be hard to understand just what is going on inside a neural networks inner layers, so, if the system even reaches moderate complexity it can become a nightmare to just understand what is causing the system to behave the way it does. Definitely not something people in finance care for.


The Figs Protocol.

Mars


Total Posts: 336
Joined: Aug 2005
 
Posted: 2006-05-29 10:29

I have heard the statement the financial institutions no longer use NN since mid 1990s many times, especially from people who hate quantitative methods (many MBAs are in this group). But people continue using linear regressions all the time. The statement "NN do not work" seems to imply that always, under any circumstance, NN give the wrong answer. My intuition is that in the 90s banks hired physicists, computer scientists, ... to build NN. And they spent 99% of the time in building "good" NN and 1% of the time understanding the markets and the inputs. So, they failed. But currently, with built-in NN, SVM, ... packages, traders may focus 99% of the time in the market. So, there is the chance that they may work.

I would not compare SVM vs NN per se. The arguments about increasing dimensionality and margins are ok from a theoretical point of view, but the main issue in pattern recognition in finance is input selection. If you select the correct inputs, you may have a good SR. If you select the wrong inputs, no SVM or NN will be better than random.


NÄ ES TÖ. TÖ ES NÄ. Nikito Nipongo.

FDAXHunter
Founding Member

Total Posts: 8372
Joined: Mar 2004
 
Posted: 2006-05-29 10:34

If the input is "good" (vague statement, but we'll run with it for now), then it doesn't matter what you apply to it, you will get something out of it.


The Figs Protocol.

Mars


Total Posts: 336
Joined: Aug 2005
 
Posted: 2006-05-29 10:40
exactly, that was what I meant.

NÄ ES TÖ. TÖ ES NÄ. Nikito Nipongo.

gekko


Total Posts: 21
Joined: May 2006
 
Posted: 2006-05-29 11:40
So what is the prefreed method used for pattern rocognition these days?

apine


Total Posts: 1009
Joined: Jun 2004
 
Posted: 2006-05-29 18:19
"It can be hard to understand just what is going on inside a neural networks inner layers, so, if the system even reaches moderate complexity it can become a nightmare to just understand what is causing the system to behave the way it does. Definitely not something people in finance care for."

i agree with your statement. but there is a great irony there. this is a business where people get hired and fired based on their perceived ability to read markets. if you did your time, you can figure out the nn's algo, but figuring out traders' brains is truly a very black box.

Too many people make decisions based on outcomes rather than process. -- Paul DePodesta

Mars


Total Posts: 336
Joined: Aug 2005
 
Posted: 2006-05-29 20:17
Sometimes, the problem is the opposite: some traders' brains are a very simple, transparent box.

NÄ ES TÖ. TÖ ES NÄ. Nikito Nipongo.

Baltazar


Total Posts: 1769
Joined: Jul 2004
 
Posted: 2006-05-30 09:07

"So what is the prefreed method used for pattern rocognition these days?"

I think it been answered: define properly your problem, know you inputs, know the dynamics of your system, and then the pattern recognition technique does not matter that much.

I would go for svm because I am more familliar with it but I think for a carrefully specified problem, FdaxH would go with NN (As he implies he has experience in it) and we'll probably end up with a similar recognition machine.

 


Qui fait le malin tombe dans le ravin

FDAXHunter
Founding Member

Total Posts: 8372
Joined: Mar 2004
 
Posted: 2006-05-30 09:10
Actually, I'm more of a Support Quaternion Machine (SQM)-kind-of-guy....

The Figs Protocol.

Mars


Total Posts: 336
Joined: Aug 2005
 
Posted: 2006-05-30 10:51
no Octonions?

NÄ ES TÖ. TÖ ES NÄ. Nikito Nipongo.

gekko


Total Posts: 21
Joined: May 2006
 
Posted: 2006-06-01 12:07
About the applicability of Machine learning in today's financial world, I came across this link, which claims " Machine learning (ML) and related methods have produced some of the financial industry's most consistently profitable proprietary trading strategies during the past 20 years."

http://www.icsi.berkeley.edu/%7Emoody/MLFinance2005.htm

Can you ppl comment on this? Did anyone attend the conference?

tedium


Total Posts: 15
Joined: Jan 2006
 
Posted: 2006-06-01 12:21
heh. I like the talk entitled "Learning to Trade with Insider Information"...

gekko


Total Posts: 21
Joined: May 2006
 
Posted: 2006-06-02 02:26
were u there in the conference?

tedium


Total Posts: 15
Joined: Jan 2006
 
Posted: 2006-06-02 09:31

no, i mean i like the title. aslthough i wish i had. way back when i was an undergrad i did some work on using some machine learning algorithms to work as trading tools so i still retain a passing interest in it.

the stuff i did at the time thopugh was very basic (i'd done a fgair amount of ML but knew very little about finance...). it was essentially an exercise in getting the algorithms to generate trading rules for equity-pair trading. it was relatively successful (read; didn't _always_ lose money) but typically had no kowledge, o ftransaction costs etc etc etc.


Baltazar


Total Posts: 1769
Joined: Jul 2004
 
Posted: 2006-06-02 12:43

They continue to claim so...

http://citeseer.ist.psu.edu/742720.html

This seem to be to many buzzwords (NN, SOM, GA) for it's own good if you ask me. Let's throw in some quantum computing at nano cell scale and we have something to grow the hair back....

You mentioned in another thread nonius with hair? I got to see that :)


Qui fait le malin tombe dans le ravin

tptspecial


Total Posts: 255
Joined: Dec 2006
 
Posted: 2007-05-06 16:09
Hello Baltazar,

I wanted to bring back this thread due to some of the things I have been doing of late with regards to machine learning for trading futures. My transaction costs are $0.68 per rountrip. So negligible. All I need to make is atleast 1 tick to stay profitable after transaction costs.

I have started with a high-frequency model, and made some good inputs out of every tick in liquid products such as ES (emini s&p 500). I am now trying to find patterns among these inputs and then classify them as +1 or -1. The data set I created contains about 1500 rows with 10 features. In-sample 1300--out of sample-200.
Data set is balanced...

I initially tried classifying with fuzzy K Nearest neighbor technique---
results were encouraging--- Out of sample Accuracy in sign prediction--78% with 5 nearest neighbors--Checked with 10 and 15 also, but not much improvement. Any advice on important checks I need to make so that I am confident about the results??

After reading several papers, I thought it may be better to have an ensemble of classifiers. So started preparing the MATLAB software for using various classifiers. The plan is to get the classification from different classifiers and wither take a vote or an average of all the predictions. Since its gonna be in real-time, Needs to be faster, so those algorithms that require huge time to come up with a decision or out.

Have you used SVM in real trading or classifying the trade direction? What others are good?? Neural Networks, ldq,qda,bagging,boosting etc..Any advice would be appreciated.


FDAXHunter
Founding Member

Total Posts: 8372
Joined: Mar 2004
 
Posted: 2007-05-06 16:23
tptspecial: you realize that predicting upticks or downticks in ES means nothing, right? Predicting the direction of the next tick in a market that's 2,000 up each side is hardly going to help you.

The Figs Protocol.

tptspecial


Total Posts: 255
Joined: Dec 2006
 
Posted: 2007-05-06 18:32

Hello FDAX,

Sorry if my question was misleading. I am NOT trying to predict next uptick/downtick. I am trying to predict the direction of a fixed number of ticks jump...

For example, if ES IS 1500 right now, I want to predict whether 1502 will come first or 1498 comes first based on the features I have selected. So basically whether it is + or - of the next fixed point jump.

Thanks

 

Previous Thread :: Next Thread 
Page 1 of 2Goto to page: [1], 2 Next