Forums  > Software  > order flow control java multi threading  
     
Page 1 of 1
Display using:  

nikol


Total Posts: 576
Joined: Jun 2005
 
Posted: 2008-12-04 15:55

Dialed these words in Google and in PM but did not find what I need.

Need suggestions on:

- best practices to write threading in Java (books, links etc...). I have written some stuff already, but it is not stable. My guess is that I violated some basic principles on clean access to shared data. I know I will go through several versions anyway, but would like to shorten recycling.  ?Any examples of successful applications in any business/experimental science?

- overview on "orders flow control" in real time trading apps. I doubt it does exists though.

 

In my case, given BUY/SELL signal:

- orders are submitted using some (price,volume) profile calculated dynamicaly;

- each order thread is supposed to do all itself: modify/cancel;

- overall control is supervised on the thread-group level of respective orders group;

- it is a part of of a higher-level portfolio manager, which reconsiles positions with outside system;

 


doreilly


Total Posts: 100
Joined: Feb 2006
 
Posted: 2008-12-04 16:19
For threading in Java Java Concurrency in Practice is particularly nice. Since it covers the later releases it probably supersedes Concurrent Programming in Java which is also excellent but dated. Both of these are obviously java specific but there are good lessons to 'take away' in general.

For a quick threading api rundown, Java Threads is quite good.

btw: Interestingly, up until recently I think there was really only one threading book available on the .net side, and it is pretty tragic, but I am about to start the new kid on the block, Concurrent Programming on Windows which looks interesting

FDAXHunter
Founding Member

Total Posts: 8370
Joined: Mar 2004
 
Posted: 2008-12-04 16:19
I sincerely doubt that a document like that exists.

Multi-threaded (or parallel programming in general) is a very wide topic. I don't know any good books that focus on multi-threading in general, but there's a good book for by Alan Dennis called ".NET Multithreading" which is probably one of the more accessible and clear explanations you'll find for a beginner. The concepts are obviously the same, even though the frameworks and language specifics will vary a little.

Apart from that, your description of the problem is far too vague to even bear thinking about, let alone make any recommendation.

Edit: Sounds like oreilly disagrees with my recommendation of ".NET Multithreading".

The Figs Protocol.

HankScorpio


Total Posts: 477
Joined: Mar 2007
 
Posted: 2008-12-04 16:24
Quite a few years back when I wrote a lot of Java code, I used Doug Lea's book. This one!

There should be more these days.

I also remember this one by Holub being of use.

Of course, if your issue is with the concepts underlying concurrency, then any decent book on concurrent and multithreaded programming should suffice.

[Edit] crossed posts with doreilly re: Doug Lea's book. [/Edit]


Real alphas don't get eaten.

nikol


Total Posts: 576
Joined: Jun 2005
 
Posted: 2008-12-04 16:24

I think similar tasks exist in military simulation/management... Will look further.

I am thinking to make each order kind of an independently trading object.

Thanks.

 


doreilly


Total Posts: 100
Joined: Feb 2006
 
Posted: 2008-12-04 16:39
-- similar tasks exist in military simulation/management

From your 'outline' and follow up it sounds like threading is not your only problem. I get the feeling that you are in over your head. There must be someone in your org. that you can run your ideas by/get suggestions. Time to get out of your chair and introduce yourself to some new coworkers!

FDAXHunter
Founding Member

Total Posts: 8370
Joined: Mar 2004
 
Posted: 2008-12-04 16:43
Edit: I was just going to say the same thing, but oreilly beat me to it. Smiley

You not only seem to know nothing about multithreading/concurrency but also very little about the OO side of things. Start slow, and buy some basic books. Nothing too fancy.

The Figs Protocol.

HankScorpio


Total Posts: 477
Joined: Mar 2007
 
Posted: 2008-12-04 16:53
Yep, spot on.

I think it was (or perhaps still is) a common misconception that Java (or any other language with native support for concurrency) makes concurrent programming easy (seems to be a management view). It simply makes writing the code somewhat easier - at least its in the standard. However, that does not imply that your code - as you have found out - will be robust, and I bet you're having all sorts of issues trying to debug the bugger.

Real alphas don't get eaten.

weismat2


Total Posts: 75
Joined: Jul 2007
 
Posted: 2008-12-04 16:58
The "Concurrent affairs" columns from Jeffrey Richter in the MSDN magazine and the classic book "CLR via C#" are very good for for understanding threading in .Net.
The second volume of Pattern-Oriented Software Architecture "Patterns for Concurrent and Networked Objects" also deals with threading from an OO perspective, but I have not read it yet.

nikol


Total Posts: 576
Joined: Jun 2005
 
Posted: 2008-12-04 21:28

doreilly:

> There must be someone in your org.

There are two persons in my org = me and myself. So far with market orders the system works pretty nicely. My app has one-way messaging, i.e. once market orders are send out, it assumes the trade is done. Position control is visual. Correction is manual. Once in about 500 triggers the trade does not go through. Not very often. Whole thing sounds as a hobby... So far it pays out, but I do not feel yet OK to search for someone else as I do not have real pressure. This very moment I prefer "do all myself" solutions.

EDIT: Have nothing to say useful on this subject, that's why asked PM. Thank you very much for opinions and refs. Have to recycle, go back to school, start easy, etc...


ElysianEagle


Total Posts: 24
Joined: Feb 2008
 
Posted: 2008-12-04 23:35
you can also read "little book of semaphores" which is freely available, http://www.greenteapress.com/semaphores/

it focuses mainly on posix threads w/ C but the ideas IMHO are more widely applicable

doreilly


Total Posts: 100
Joined: Feb 2006
 
Posted: 2008-12-04 23:44
If I understand you correctly, then you are coding this for yourself, which is definitely to be applauded. While you still seem to be missing some coding basics, learning from your own mistakes is healthy.

From the multithreaded perspective then I would recommend 'Java Concurrency in Practice'. They walk through different ideas and scenarios. If you work your way through the book, I am sure you will come across a bunch of 'oh shit, I never thought of that' moments. Amongst those moments, will be the answer to your original problem.

btw: While not strictly a design book, Joshua Block's , Effective Java may also be of some help

white


Total Posts: 67
Joined: Jul 2007
 
Posted: 2008-12-05 00:09
Try and write exactly what you want to do then someone might be able to give more concrete help.

But basically encapsulate data access using access methods (getters, setters, add, remove methods).

Use the synchronized statement to handle concurrent access.

That is usually enough - no reason to make it harder.


nikol


Total Posts: 576
Joined: Jun 2005
 
Posted: 2008-12-05 00:36

doreilly:

"Effective" Java I have. Read it non-systematically years back. Have to reread.

Got "Concurrency" already. Indeed, I spotted "oh, god shit!" very quickly. My first version is a bunch of prototypes, so I messed things up. Have to reorganize everything from scratch.

Datafeed I have done already (interfaced to C++ and JNI'ed into Java). Started Java-thing in mid-october, work on it at evenings. As I said, it is a hobby. Seems, I understood what "coworking" means. Might be an option for me. I am sitting in my "cave" and never see "the sun", otherwise.

white:

I do all this stuff. "Java Concurrency" told me already some tricks. I support above recommendations.


white


Total Posts: 67
Joined: Jul 2007
 
Posted: 2008-12-05 01:25
Then you should be all good then. Who is your broker / data provider?

signalseeker


Total Posts: 237
Joined: Oct 2006
 
Posted: 2008-12-05 03:41
You have already got some pretty good advice. I would first read effective java before worrying about any concurrency issues. I would also try to get someones help(even paid) if not for implementation, but at least helping with the design. Some things for you to think about

1. Try to keep your "mainloop" clean and simple without any blocking operations
2. Think asynchronous event driven programming model
3. When you say each order should be a independent object, think about finite state machines to implement such behavior. There are tons of examples about such things on the internet. For eg. something that implements a tcp protocol
4. Develop incrementally and try to keep things simple

Hope that helps.

The dark is light enough.

Jurassic


Total Posts: 172
Joined: Mar 2018
 
Posted: 2018-05-23 23:22
what is the application of java multithreading mainly (generally)? Can you be called a proficient java dev without it?

nikol


Total Posts: 576
Joined: Jun 2005
 
Posted: 2018-06-20 20:05
If anyone of interest, let me know

java algo-trading framework

That was working about 3 years back but IB interface has changed.
If you have an interest I can make it working again.

nikol


Total Posts: 576
Joined: Jun 2005
 
Posted: 2018-06-29 23:11
Based on interaction with first user, I have updated README and made some little changes. It seems very likely, that the feed works. I cannot check it yet due to restrictions on IB side, which I have to resolve.

bullero


Total Posts: 26
Joined: Feb 2018
 
Posted: 2018-09-01 19:06
Would there be a high level "flow-chart" available? I would be interested to see the whole architecture of your system Wink

nikol


Total Posts: 576
Joined: Jun 2005
 
Posted: 2018-09-06 11:09
Hi. I have lost my esteem to put this forward. Right now developing and testing trade interface for python based system.

FinSecurity holds info about traded instrument
IBLInk is extension of EWrapper (provided by IB)
--- it holds Main(), feed (tickPrice/tickSize) and trade (orderStatus, execDetails, cancel etc)

When tick arrives into tickPrice/tickSize:
--- select quoted FinSecurity and fire MacroStrategies registered with this FinSec
--- store current info into DataFeed object which keeps all current info about the Market. I use it for example to identify Arbitrage.

Within MacroStrategy you calculate all your signals. When trade is triggered you send request to MicroStrategy.

Within MicroStrategy you can model dynamics of monitored price and execute in optimal way.

Arbitrage is an object which simplifies communication between FinSecurities within Strategy.

All order updates are captured via IBLink and used to update Position of FinSecurity. Position is shared, therefore, lock/sync mechanism has to be implemented.

When you trigger the signal you have to compare your target Position with Position you have + total volume of orders in the system, otherwise at some point you get infinite generation of trades.

GUI/Swing required info from objects, therefore, I guess I created total mess by mixing dataflows for Strategy/Trading and for GUI.


As I see, my architecture is over-complicated. I do it differently now. For example, to implement market making strategy I had to create two instruments A_BID and A_ASK, which interacted via Arbitrage. This made things even more complex. Now, I use distribution based approach, where I set Target order-distributions (by model), and execute only differential order distribution between Target and Actual (on the market).


Ask more questions, I will answer, when have a time.

bullero


Total Posts: 26
Joined: Feb 2018
 
Posted: 2018-09-11 13:34
Your approach sounds quite complicated. Here is a crude description of how I have approached this type of stuff previously, maybe it helps:

1. OrderBook class (for example market-by-price style):
This is just a container class for std::map(int,int) (price and size) that has couple of methods to keep the book organized. So you could create "update", "insert" and "delete" methods to manage the price and size information based on your data feed. Depending on your hardware implementation you maybe want to make this class thread safe.

2. Create general template classes for observer/subject design pattern for later use. You can find various implementations from Google or some programming design pattern books.

3. Create template class OrderBookUpdater by inheriting the "subject" class interface that you created in step 2. The purpose of this class is to take any type of OrderBook (for example, market-by-price or market-by-order style) as an input, refresh it and notify "observers" when the state of the order book is modified. So this class subscribes to real-time feed socket, filters out specific ticker or symbol, parses the FIX message into meaningful bits of information, updates the OrderBook state and notifies all "observers". When a notification is made copy of the current state of the orderbook is passed to the observer.

4. Create template class OrderBookObserver by inheriting the "observer" class interface that you created in step 2. The purpose of this class is to receive notification every time the OrderBookUpdater makes any modification to the OrderBook.

5. For your strategy class you maybe want to create general interface class StrategyBase which provides basic methods common to all different types of strategies. For example, every strategy needs to be able to send orders to trading server, check currently outstanding orders and positions, handle fill messages etc. So this base class takes care all of that stuff.

6. Create a concrete implementation of a strategy by inheriting StrategyBase and OrderBookObserver.

nikol


Total Posts: 576
Joined: Jun 2005
 
Posted: 2018-09-11 16:11
I confirm, my old implementation is complicated, so I abandoned it by sharing to the world.

Today I do it with a friend (with heavy programming background at the bank) and we do closer to what you describe. At least Arbitrage is removed from the picture:

Instrument Feed (as many as we requested) message is passed to all strategies registered within Instrument. Not all Instruments are traded.
Strategy ( = your StrategyBase) implements base methods
for example, MarketMaking inherits Strategy and is registered with Instrument (when Instrument's quote is coming it calls all registered strategies)
- monitors market and indicators
- monitors current positions via Portfolio/positions
- monitors current outstanding orders via ExecutorRealtime
- triggers trading via Executor (I have several)
ExecutorRealtime (=OrderBookUpdater) manages all orders related to the Instrument
- when app receives info that the order is filled, it is booked into 2 Positions (base and asset). It does not notify anyone, but allows reading its state.
ExecutorBackTest updates directly those 2 Positions. No orders are involved. Plan to make some simulation of reality (delay, market orders arrival).

In this way we can do multiple exchanges, instruments (securities), strategies in one app. Executor will be more complicated...
Same framework is used for BackTest. We store Feed online into our DB and then play it back by feeding Feed (then all is executed in the same way)

Target strategies: market-making, triangle and momentum (even using interval based techniques).

Do I understand correctly, that Observer also helps to separate Feed layer from the Strategy?
Or you do it only for OrderBook? Why do you need notification if updates of Position & Orders are done simultaneously?

bullero


Total Posts: 26
Joined: Feb 2018
 
Posted: 2018-09-17 21:55
So the OrderBookUpdater subscribes to feed socket which delivers all order book updates. If message contains a message for some particular security, "ESZ18" for example, then it parses the message and updates the state of the book. Then, the updater tells the strategy that the state of the book has changed. The strategy does the necessary computations etc.
Previous Thread :: Next Thread 
Page 1 of 1