<< Click to Display Table of Contents >> Navigation:  Misc > Queue Processing & Queue Stuck Message

Creating a trading platform is quite straightforward. Creating a trading platform that doesn't jam up at peak times or use too much CPU is a different matter.

 

Roughly 30% of the code in the daytradr platform is there to ensure optimal performance. We consider things like the ability of the human eye to detect changes, the importance of always being up to date, the fact that markets in VERY extreme situations need to be handled differently and we spread the load across the cores of your CPU to ensure we don't overload one.

 

When data comes in to daytradr, it does not get processed immediately but gets placed on a queue to be processed in a separate thread. Now you might be thinking - if it's not processed immediately and put in a queue - wouldn't that be slower?

 

Let's say L1 data is coming into the CQG API for 20 instruments - the API itself is running on a single thread. The data needs to be put into our internal format and passed to other tools who will also process the data for their needs. So, the moment data hits the API, we pass it to a queue. That queue is running in a seperate thread. The API is now released and ready when more data arrives.

 

We use something called "Thread Pooling" to keep threads at a reasonable level (too many causes performance degradation) and queues are used extensively and make it fly.

 

When we implemented queues, we also put in some queue oversight. Queues should just work but we have a queue monitor there because it's good to have failsafes. If the application sent you to this page, then you probably saw an error something like

 

"Queue stuck 'MD-Socket-Queue', stuck , size : 5 - restart the daytradr app at the next convenient moment"

 

This means the monitor detected that a queue was no longer processing inbound messages. There's 2 potential causes of this:

•The Queue stopped processing.

•The monitor is being to aggressive in reporting queue failures (as it has a number of ways to presume a queue fail)

 

If you see the message, the safest thing to do is restart the app. You can contact support but if it just happens once, it's safe to ignore. If it happens more than once - you should get in touch with support.

 

Queue fails should be extremely rare - but any queue fail is something we can address and prevent in future releases.