Forums  > Software  > Probabilistic Graphical Models  
     
Page 1 of 1
Display using:  

jslade


Total Posts: 1064
Joined: Feb 2007
 
Posted: 2016-10-21 02:19
Anyone have a good review paper or sub 1000 page book to recommend on practical aspects of this class of machine learning algorithms?

I've used specific PGMs in the past (HMM, HSMM, LDA) and had very good experiences with them. But I realize it is a general bag of tricks, and there might be some interesting things lying around waiting for use (Pachinko allocation, etc). Looked through Barber, Koller, Murphy and Pearl, but reading through 1000+ page "for grad students looking to write a thesis on this subject" books is not an efficient use of my time. Looking for more of a lego book, like that of Bellot.

"Learning, n. The kind of ignorance distinguishing the studious."

yfe


Total Posts: 1
Joined: Oct 2016
 
Posted: 2016-10-23 19:48
Have you seen that the MOOC is available on Coursera ? From the same author Daphne Koller.

https://www.coursera.org/learn/probabilistic-graphical-models

h0h0


Total Posts: 39
Joined: Apr 2010
 
Posted: 2016-10-26 16:02
for work i needed a simple model with hierarchical latent variables so
i skimmed through Pattern Recognition and Machine Learning, the puppy book (Kruschke) and 'Probabilistic Programming and Bayesian-Methods for Hackers' (pymc3 branch).

Daphne Koller's course and book requires a lot of effort.

warning: it was difficult to find software that supported the features of my model.
one framework did not allow both continuous and discrete rvs, another supported only
conjugate priors etc. i the end i was forced to use Infer.net (C#)

for state-of-the-art check out 'probabilistic programming' and Stan

there are a lot of resources at:
http://probabilistic-programming.org/wiki/Home


billWalker


Total Posts: 171
Joined: Feb 2005
 
Posted: 2016-10-26 18:01
I am a huge fan of Stan. Big improvement (for me) on BUGS/JAGS. Check out Jim Savage's Bayesian Workflow for a worked example in R.

Tried and gave up on Koller's MOOC and book b/c I found I was spending more time learning Octave than on the actual coursework.

"Plausible regularities may be present but swamped by changes in attendant circumstances." Ole Peters

jslade


Total Posts: 1064
Joined: Feb 2007
 
Posted: 2016-10-26 21:15
Thanks guys; I know about Koller's course, but like her book: way more material than I'm interested in. Months of full time work, and on a platform (Matlab) which is not used in industry. I'm looking for lower hanging fruit; if I like the fruit, I'll go deeper.

Stan looks pretty good, and Bellot's book mentions it. Yet another lego book is Hosjgaard, Edwards and Lauritzen "Probabilistic Graphical Models in R," which I'm now going through. Bellot's book was definitely useful, but weak of the sauce. I should probably pinch my nose and look at a few python resources since the world seems to be moving in that direction.


"Learning, n. The kind of ignorance distinguishing the studious."

Patrik
Founding Member

Total Posts: 1333
Joined: Mar 2004
 
Posted: 2016-10-27 17:47
Talking about Stan - and knowing you like lisps, these slides were a bit interesting to flip through:
http://dragan.rocks/talks/EuroClojure2016/clojure-is-not-afraid-of-the-gpu.html

It's towards the end, a bayesian lib he's working on, tiny benchmark vs stan.

Capital Structure Demolition LLC Radiation

jslade


Total Posts: 1064
Joined: Feb 2007
 
Posted: 2016-10-28 23:08
Thanks Patrik. Kind of neat hooking Clojure up to the GPU via OpenCL (one of the J guys did something similar using Arrayfire). If only they could get rid of the JVM, I'd actually consider working with Clojure again.

One of the cute things I figured out about GPU algos: they can be a really big win on non-cache friendly algorithms if you have a nice beefy one with plenty of memory. This is underappreciated and rarely taught. Multicore stinks unless you can feed all your CPUs. Lots of single core stuff stinks because it's non cache friendly.

"Learning, n. The kind of ignorance distinguishing the studious."

Patrik
Founding Member

Total Posts: 1333
Joined: Mar 2004
 
Posted: 2016-11-01 10:49
I hear you - to me and my use cases the JVM brings a lot of positive things as well as negatives, so the trade offs make sense to us. Having firmly planted my feet very firmly in clojure land for a foreseeable future I'm happy to see Dragan's projects on performant BLAS and GPU calcs - not something I need all the time but makes it easier to feel happy with the choice. End of digression!

Capital Structure Demolition LLC Radiation

Kutilya
Quote Machine

Total Posts: 1286
Joined: Jun 2004
 
Posted: 2016-11-01 21:01


BTW if you are OK with Clojure then have a look at Anglican.

Anglican


Hypocritic Oath

jslade


Total Posts: 1064
Joined: Feb 2007
 
Posted: 2016-11-01 21:11
Somehow I had forgotten you used Clojure. Certainly looks good for building infrastructure. Out of curiosity, what are you using for your statistics stack? Incanter, roll-your-own or ...?

"Learning, n. The kind of ignorance distinguishing the studious."

a路径积分


Total Posts: 77
Joined: Dec 2014
 
Posted: 2016-11-01 23:12
Just a review paper on practical methods? Frey and Jojic is pretty easy to follow.

http://www.psi.toronto.edu/~frey/ece1510/ReviewPaper.pdf

But if you want a review paper that introduces the basics, how about this one written by a former NBA player?

http://www.cis.upenn.edu/~mkearns/papers/barbados/jordan-tut.pdf

Patrik
Founding Member

Total Posts: 1333
Joined: Mar 2004
 
Posted: 2016-11-02 10:30
@kutilya: hadn't seen - thanks for tip!

@jslade: my needs have historically not been very high, so a combination of straight java libs, pieces of incanter and neanderthal. work in progress.

Capital Structure Demolition LLC Radiation

Kutilya
Quote Machine

Total Posts: 1286
Joined: Jun 2004
 
Posted: 2016-11-02 13:02
No not a Clojure guy so you are not forgetting anything. I only know of it because I was attached to the ML group at Oxford for a while, they used a fair bit of Anglican, it seemed pretty handy though. My set up is largely R and Python...I do role my own once in a while as somethings just need to be done in a particular way.


Hypocritic Oath

jslade


Total Posts: 1064
Joined: Feb 2007
 
Posted: 2016-11-02 23:15
@Kutilya: looks like our posts crossed; this is an interesting gizmo.
@Patrik: that was also my strategy when I was doing Clojure things as well.
@a路径积分: thanks for those; very useful top down views.

As an update, while the R books are OK for examples, they don't present a coherent view of the subject. Kevin Murphy's book is pretty good for doing this.

"Learning, n. The kind of ignorance distinguishing the studious."

chiral3
Founding Member

Total Posts: 4984
Joined: Mar 2004
 
Posted: 2016-11-03 03:49
Wainwright and Jordan's book is well regarded for graphical models.

Graphical Model, Exponential Families, and Variational Inference

As is Pourret if you want to see examples and uses

Bayesian Nets

Both are under 1000 pages

Nonius is Satoshi Nakamoto. 物の哀れ

EspressoLover


Total Posts: 221
Joined: Jan 2015
 
Posted: 2016-11-04 23:40
I'd be curious, from those who've had good luck with PGMs, what the context was. My general experience has been pretty disappointing in alpha/trading type problems. Financial returns are just too noisy. You just don't get that much out of the hidden state. The conditional distribution of the latent variables never really differs that much from the unconditional diffused state. So you add a lot additional complexity for little improvement over OLS or some other simple, direct model.

But then again, I'm willing to admit that my experience isn't universal, or that I'm missing some important technique or insight.

jslade


Total Posts: 1064
Joined: Feb 2007
 
Posted: 2016-11-06 20:13
People use them to model cash flows in trading problems. People also apparently use HMM type things in various contexts for switching strategies.

For myself, I have predicted electrical load (regime related) and classified documents with them -LDA worked better than anything else I tried for topic models.

I have some spare time; just skill building.

"Learning, n. The kind of ignorance distinguishing the studious."

EspressoLover


Total Posts: 221
Joined: Jan 2015
 
Posted: 2016-11-07 01:33
Cool, thanks. That makes sense.

billWalker


Total Posts: 171
Joined: Feb 2005
 
Posted: 2016-11-07 14:45
I've used them as a framework for kalman filter factor regression-type problems. Your hidden variables are the true states of the beta over time of some exogenous factors against asset returns. Not high freq, obviously.

"Plausible regularities may be present but swamped by changes in attendant circumstances." Ole Peters
Previous Thread :: Next Thread 
Page 1 of 1