Forums  > Software  > real time data vendors?  
     
Page 1 of 1
Display using:  

Strange


Total Posts: 1661
Joined: Jun 2004
 
Posted: 2020-09-24 02:50
What's out there in terms of normalized data feeds at a reasonable price (i.e. cheaper than using exchange feeds directly)? Ideally:
- spans futures, equities, options and indices; a la carte subscription ideal
- works on Linux
- i'd consider both coalesced and tick-by-tick

'Progress just means bad things happen faster.’

JTDerp


Total Posts: 73
Joined: Nov 2013
 
Posted: 2020-09-24 22:20
as far as spanning the asset classes, I'd say iqFeed, Nanex or dxFeed (this one's filtered). By 'normalized' do you mean manipulated instead of the base stream?

"How dreadful...to be caught up in a game and have no idea of the rules." - C.S.

Strange


Total Posts: 1661
Joined: Jun 2004
 
Posted: 2020-09-25 00:13
Thank you! Do you have any direct experience with any of these?

While I am aware of these guys (iqFeed is very retail-oriented, while Nanex and dxFeed are a small step up) there are also guys like QuantHouse, OneTick and Vela (can't remember what their product is called). By "normalized" I mean it's a single API feeding me data across multiple exchanges.

'Progress just means bad things happen faster.’

prikolno


Total Posts: 71
Joined: Jul 2018
 
Posted: 2020-09-25 01:06
Of the ones listed here, Nanex and dxFeed are the only ones I've not used. I don't see Nanex listed as an authorized distributor anywhere, I believe they just sit as a service facilitator on top of Quodd's backend - so perhaps Quodd has everything you're looking for at Nanex's standard.

I'd say Activ, QuantHouse, and Celoxica are the best in this area, but they're not much cheaper than parsing the exchange feeds directly and do not have 'a la carte subscription'. Best in the sense that they are familiar with the right architectural decisions to get the timestamping correct; they also pay attention to how fast their normalization paths are over WAN, whereas most of their competitors handwave this. B-PIPE has a la carte by ticker. I've tried Vela back when they were partly SR Labs, and it was OK though I preferred the engineering support from the other firms at the time (YMMV).

In your situation it may be cheaper to host a server in Secaucus and pay for or use open source parsers.

I think IQFeed is a rather underrated product for its cost. It's the only retail product I've encountered where the data lines up losslessly and in the exact same sequence relative to the raw multicast feeds - yes, one day I was bored. Where the limitations set in for you are (a) you have to install an SDK which enforces a 1 terminal limit presumably to avoid the enormous fee step-up with non-display licensing, (b) said SDK only builds on Windows so you will need to emulate it on Linux, (c) there's hardly any systematic way to fetch and subscribe to the tickers and you have to kind of manually hard-code them.

Strange


Total Posts: 1661
Joined: Jun 2004
 
Posted: 2020-09-25 01:33
"I think IQFeed is a rather underrated product for its cost. It's the only retail product I've encountered where the data lines up losslessly and in the exact same sequence relative to the raw multicast feeds - yes, one day I was bored. Where the limitations set in for you are (a) you have to install an SDK which enforces a 1 terminal limit presumably to avoid the enormous fee step-up with non-display licensing, (b) said SDK only builds on Windows so you will need to emulate it on Linux, (c) there's hardly any systematic way to fetch and subscribe to the tickers and you have to kind of manually hard-code them."

Sounds like a pretty good backup feed and it has pretty much everything. So I might give it a serious try. However, for the actual work I'll probably follow your advice and try to deal with the raw feeds.

'Progress just means bad things happen faster.’

TheAlchemist


Total Posts: 3
Joined: Nov 2014
 
Posted: 2020-09-25 08:20
I'm not sure I agree with the cost aspect here.

Using normalized shared feeds is often way cheaper than raw exchange feeds - there are a lot of exchange costs you avoid when going through normalized providers.

For normalized feeds, I would add Fixnetix, Exegy and ICE Data. Redline also can set up something for you, although I don't think they have a shared normalized setup - it's rather on a per client basis.

The answer to the initial question also depends quite a lot on the scope of exchange (US only ?) and the latency you can tolerate.

prikolno


Total Posts: 71
Joined: Jul 2018
 
Posted: 2020-09-25 10:55
Yes, I feel Fixnetix/Exegy are good choices if you are purely going after spot FX coverage; I tailored my response since OP asked for futures/equities/options.

Don't get me wrong, I do think the names I listed are competitive choices depending on what OP's exact needs are, or I won't have listed them.

I think where a normalized provider actually becomes significantly cheaper is when you have a sufficiently large number of venues to cover, such that you can (i) get a volume discount and (ii) realize the benefit of a single API.

If you just need 3-5 feeds; they're all commoditized enough that many hosting vendors carry them even over long-haul; and they're all 1 XC away in the same DC, then the cost difference becomes small enough that it is dominated by the developer time. Most of the normalized providers are quantized around the 3k, 5.5k, 8.3k MRC marks for the first venue. I'm sure OP can do the arithmetic.

I also need to emphasize a finer point on (ii) and the implicit cost of developer time. Consolidated and normalized are not always simpler. If my memory serves me right, one of the vendors you named has a 680+ page API reference doc for its data feed, which is accompanied by a 200+ page user manual. Sometimes less than 4 pairs of eyes have been through the part of the doc you're reading. I've once worked with a very established vendor for one of the largest venues and realized they had just 2 customers for that 1 venue - and our faceless companions on that lonesome journey didn't know what they were doing so they kept putting in feature requests that made the feed worse. Often, these APIs have leaky abstractions which require you to refer to the original exchange specs to understand crucial parts of the underlying behavior anyway.

TheAlchemist


Total Posts: 3
Joined: Nov 2014
 
Posted: 2020-09-25 11:09
"I also need to emphasize a finer point on (ii) and the implicit cost of developer time. Consolidated and normalized are not always simpler. If my memory serves me right, one of the vendors you named has a 680+ page API reference doc for its data feed, which is accompanied by a 200+ page user manual. Sometimes less than 4 pairs of eyes have been through the part of the doc you're reading. I've once worked with a very established vendor for one of the largest venues and realized they had just 2 customers for that 1 venue - and our faceless companions didn't know what they were doing so they kept putting in feature requests that made the feed worse. Often, these APIs have leaky abstractions which require you to refer to the original exchange specs to understand crucial parts of the underlying behavior anyway."

I confirm - your memory is perfectly right here - for some more exotic exchanges it's often easier indeed to refer to original exchange documentation first than to the 'normalized' one.

prikolno


Total Posts: 71
Joined: Jul 2018
 
Posted: 2020-09-25 11:20
> I confirm - your memory is perfectly right here

;) Oh my...

JTDerp


Total Posts: 73
Joined: Nov 2013
 
Posted: 2020-09-25 15:43
I have experience with iqFeed and Nanex...dxFeed is the source which Thinkorswim uses, but I've never purchased direct from dxF.

Nanex actually got acquired by DTN/iqFeed a few years ago. I purchased a few years' worth of their 'FXTape' data for CME Group exchanges...futs & FOPs, no 'FX' as their product name sort-of implies. They include API scripting in several languages (C-based then maybe Java IIRC), and I think private developers have passed around some Python and maybe R scripts. Very good quality data, but very pricey relative to other offerings out there...then again, they claim a truly unfiltered feed, whereas others trim it at least a bit here & there.

"How dreadful...to be caught up in a game and have no idea of the rules." - C.S.

Strange


Total Posts: 1661
Joined: Jun 2004
 
Posted: 2020-09-26 22:03
Thanks a lot for the discussion everyone! Had a chance to think about it in the context of my strategies and the scope of the problems we are having at the moment.

I think at the moment I want something that can be setup quickly (we've been having problems with our current ghetto setup) and that is either easy to dismantle or cheap enough that I would keep it going as a backup. So that is probably going to be iqFeed or something alone these lines. It's even possible that data which is not latency-sensitive we would just keep getting from there.

Once the immediate need is fixed, I am actually tempted to say that using direct exchange feeds is probably the best approach. I am still unclear what technology work that would involve, but the flexibility attached makes it worth while.

'Progress just means bad things happen faster.’
Previous Thread :: Next Thread 
Page 1 of 1