Chart Data Objects
Understanding how data is represented and stored inside the chart engine
This tutorial explains the relationship between the display of the chart and the various internal objects and variables contained within the chart library. Understanding what occurs visually depends on understanding the underlying mechanics of the chart. Likewise, once you understand the mechanics, it becomes easy to manipulate the chart programmatically.
stxx.chart.masterData is an array that contains all the data that has been provided to the chart. This is data that was provided directly as an argument to
stxx.loadChart or data fetched through a quotefeed. As the chart is refreshed with streaming data, the masterData will grow.
When using a quotefeed, if the user scrolls back in history, before the beginning of the data, new data is requested and prepended to the masterData (pagination). The masterData contains only the source data for the main symbol of the chart and any comparison series. It does not contain calculated values.
Note: masterData typically extends beyond the left edge of the chart. See
stxx.chart.dataSet is an array that contains a copy of the objects from masterData with the calculated values for any enabled studies (as well as internal intermediary calculations) added as additional fields. If an aggregated chart type has been selected (i.e. Heikin Ashi, Kagi, etc) then the dataSet will contain those calculated values instead of the original masterData values.
The dataSet length may be different than the masterData length. This happens when a "rollup" periodicity value is enabled. For instance, if the masterData contains daily values but a user selects "month" for periodicity, then the chart will roll up the data into bars that represent one month. Rollups can also occur on intraday charts when a user selects a periodicity that must be calculated (compounded). For example, if masterData contains bars of 1 minute periodicity but the chart is set to 3 minute periodicity, the dataSet will be 1/3 the length of the masterData. This is because 3 items of masterData are rolled up into each item in the dataSet. See setPeriodicity() for more information on setting periodicity.
The dataSet is recalculated infrequently. Only the following conditions trigger a recalculation:
- New symbol
- New periodicity
- Add or remove a series
- Add or remove data (pagination or refresh)
stxx.chart.dataSegment contains a slice of the dataSet. dataSegment contains only the data that is currently displayed on the screen, including any partially visible bars. If there is any data point off to the right of the last visible bar, dataSegment will also include on extra data point at the end of the array that will not be visible, but used to accurately plot certain charting styles all the way to the right edge of the chart towards the next data point. Use getDataSegment() to obtain only the fully visible data points. dataSegment represents a "view" into the dataSet that changes as the user scrolls and zooms. The dataSegment is frequently recalculated (60 times per second when the user navigates).
The following console outputs all refer to this example chart:
masterData with 1 minute bars:
dataSet with 3 minute bars. Note that the open, high, low, close are computed for the 3 minute interval using data from the first three items in masterData above:
dataSegment - The first item in the dataSegment shows the first bar visible on the chart:
Sometimes the x-axis of the chart beyond the length of the dataSegment. We refer to this portion of the x-axis the "future".
stxx.chart.xaxis can be reliably used to obtain the exact dates for each x-axis position on the chart.
Here is a picture depicting the relationship between the dataSegment, dataSet, masterData and xaxis:
Variables That Affect Visible Data
stxx.chart.scroll - indicates how far away the leftmost displayed bar (first bar on chart) is from the end of the dataSet.
If the leftmost tick on the screen is the most current bar (only one bar shown on the screen) then
stxx.chart.scroll == 1. (Sometimes this value may be '2', if the chart is scrolled so that the first bar is partially hidden off the edge of the screen).
If the chart is scrolled all the way to the beginning of the dataSet then
stxx.chart.scroll == stxx.chart.dataSet.length. It's worth noting that the chart allows the user to scroll past the end of the dataSet, so that
stxx.chart.scroll can actually be greater than the dataSet length.
In our example above:
stxx.chart.maxTicks - contains the maximum number of ticks that can be shown on the chart. This is usually the width/candleWidth + 2. We add two bars/candles so that a partial bar can be displayed on either edge of the chart when scrolling. In our example above:
stxx.layout.candleWidth - represents the width of each bar/candle in pixels. When the user zooms in or out of a chart, the candleWidth changes. As a result, maxTicks also changes because the chart can now accommodate a different number of bars on the chart. Finally, dataSegment is recalculated to contain the appropriate slice of the dataSet to fill the chart.
stxx.micropixels - the chart can smoothly be scrolled pixel by pixel. This means that bars/candles can be partially obscured. micropixels is a positive or negative offset that reflects how far a candle has been shifted off of its natural bias. Generally you don't have to worry about this since the convenience functions such as CIQ.ChartEngine#pixelFromTick and CIQ.ChartEngine#tickFromPixel automatically make the adjustment.
stxx.chart.series - contains a map of [comparison] series objects. The actual price data for series is contained in masterData, dataSet and dataSegment. stxx.chart.series contains the meta data for each series (i.e. color, symbolObject)
stxx.overlays - contains a map of all studies that are overlays. The calculated values are in dataSet and dataSegment.
stxx.layout.studies - contains a map of all studies (both overlays and panel studies). The calculated values are in dataSet and dataSegment.
Generally speaking, you should rarely need to manipulate these variables directly. Treat them as read-only. If you need to change a value, use an appropriate method to make the change. For instance, call CIQ.ChartEngine#setCandleWidth instead of manually changing the candleWidth value. This will ensure that the chart is always in a good state.
There are some nuances to be aware of with dataSegment and the x-axis. If the user scrolls all the way to the left, past the beginning of the chart data, then the first entries of dataSegment will be null. Always check for null when examining the dataSegment. On the other hand, if the user has scrolled all the way to the right, past today's bar (leaving whitespace), then the dataSegment will end at the last actual bar on the screen. Like dataSegment, the xaxis will contain nulls on the left if the user has scrolled past the beginning. But it will contain entries for future dates if the user scrolls to the right leaving whitespace. i.e.
We often refer to "tick" as the position of a bar in the dataSet array.
To find the tick for the leftmost bar on the chart:
leftTick = stxx.chart.dataSet.length - stxx.chart.scroll; if (leftTick < 0) leftTick = 0; // Can scroll beyond end dataSet
To find the tick for the rightmost bar on the screen:
rightTick = stxx.chart.dataSet.length - stxx.chart.scroll + stxx.chart.maxTicks; if (rightTick > stxx.chart.dataSet.length) rightTick = stxx.chart.dataSet.length; // Can scroll beyond beginning of dataSet, as in our example above. rightTick--; // to adjust to the correct index given that arrays begin at 0, not 1
The actual data for either can then be queried with:
quoteLeft = stxx.chart.dataSet[leftTick]; quoteRight = stxx.chart.dataSet[rightTick];
If the chart is displaying the most current bar on the screen, the rightmost bar on the screen can also be obtained from the dataSegment by:
quoteRight = stxx.chart.dataSegment[stxx.chart.dataSegment.length - 1];
If the chart is NOT displaying the most current bar on the screen (for example: the chart has been scrolled back into history), the rightmost fully visible bar on the screen can be obtained by:
visibleQuotes = stxx.getDataSegment(); quoteRight = visibleQuotes[visibleQuotes.length - 1];
Example: Chart Scrolled to Right With Whitespace:
Example: Chart Scrolled to Left: beyond beginning of data:
The date associated with a bar of data (DT or Date) represents the beginning of the interval. For instance, a 5 minute bar would cover the time from 9:30:00.000 to 9:34:59.999
dontRoll can be set to true to prevent the chart from rolling daily bars into weeks or months. You would set this if your server provides data in weekly and monthly format.
Chart data can contain gaps (null values) during times when a stock didn't trade. By default, the charting library will collapse these gaps, but if you've set gaps to display then null "Close" prices will exist in masterData, dataSet and dataSegment. Such null values can disturb study calculations which will typically assume a null values is equal to zero. The special variable
stxx.chart.scrubbedcontains a copy of the dataSet that does not contain the null values. Studies and other calculations should use this array instead of dataSet.
The actual dates displayed on the x-axis will be adjusted for time zone. See the Dates, Times, and Time Zones tutorial for more information on how this works.