Chart Data Objects

Understanding how data is represented and stored inside the chart engine

This tutorial explains the relationship between the display of the chart and the various internal objects and variables contained within the chart library. Understanding what occurs visually depends on understanding the underlying mechanics of the chart. Likewise, once you understand the mechanics, it becomes easy to manipulate the chart programatically.

masterData

stxx.chart.masterData is an array that contains all the data that has been provided to the chart. This is data that was provided directly as an argument to stxx.newChart or data fetched through a quotefeed. As the chart is refreshed with streaming data, the masterData will grow.

When using a quotefeed, if the user scrolls back in history, before the beginning of the data, new data is requested and prepended to the masterData (pagination). The masterData contains only the source data for the main symbol of the chart and any comparison series. It does not contain calculated values.

Note: masterData typically extends beyond the left edge of the chart. See dataSegment below.

dataSet

stxx.chart.dataSet is an array that contains a copy of the objects from masterData with the calculated values for any enabled studies (as well as internal intermediary calculations) added as additional fields. If an aggregated chart type has been selected (i.e. Heikin Ashi, Kagi, etc) then the dataSet will contain those calculated values instead of the original masterData values.

The dataSet length may be different than the masterData length. This happens when a "rollup" periodicity value is enabled. For instance, if the masterData contains daily values but a user selects "month" for periodicity, then the chart will roll up the data into bars that represent one month. Rollups can also occur on intraday charts when a user selects a periodicity that must be calculated (compounded). For example, if masterData contains bars of 1 minute periodicity but the chart is set to 3 minute periodicity, the dataSet will be 1/3 the length of the masterData. This is because 3 items of masterData are rolled up into each item in the dataSet. See setPeriodicity() for more information on setting periodicity.

The dataSet is recalculated infrequently. Only the following conditions trigger a recalculation:

  • New symbol
  • New periodicity
  • Add or remove a series
  • Add or remove data (pagination or refresh)

dataSegment

stxx.chart.dataSegment contains a slice of the dataSet. dataSegment contains only the data that is currently displayed on the screen. dataSegment represents a "view" into the dataSet that changes as the user scrolls and zooms. The dataSegment is frequently recalculated (60 times per second when the user navigates).

The following console outputs all refer to this example chart:

chart

masterData with 1 minute bars:

masterData with 1 minute bars

dataSet with 3 minute bars. Note that the open, high, low, close are computed for the 3 minute interval using data from the first three items in masterData above:

dataSet with 3 minute bars

dataSegment - The first item in the dataSegment shows the first bar visible on the chart:

dataSegment

Sometimes the x-axis of the chart beyond the length of the dataSegment. We refer to this portion of the x-axis the "future". stxx.chart.xaxis can be reliably used to obtain the exact dates for each x-axis position on the chart.

Here is a picture depicting the relationship between the dataSegment, dataSet, masterData and xaxis:

Scrolled To Right

Variables That Affect Visible Data

stxx.chart.scroll - indicates how far away the leftmost displayed bar (first bar on chart) is from the end of the dataSet.

If the leftmost tick on the screen is the most current bar (only one bar shown on the screen) then stxx.chart.scroll == 1. (Sometimes this value may be '2', if the chart is scrolled so that the first bar is partially hidden off the edge of the screen).

If the chart is scrolled all the way to the beginning of the dataSet then stxx.chart.scroll == stxx.chart.dataSet.length. It's worth noting that the chart allows the user to scroll past the end of the dataSet, so that stxx.chart.scroll can actually be greater than the dataSet length.

In our example above:

scroll

stxx.chart.maxTicks - contains the maximum number of ticks that can be shown on the chart. This is usually the width/candleWidth + 2. We add two bars/candles so that a partial bar can be displayed on either edge of the chart when scrolling. In our example above:

maxTicks

stxx.chart.candleWidth - represents the width of each bar/candle in pixels. When the user zooms in or out of a chart, the candleWidth changes. As a result, maxTicks also changes because the chart can now accomodate a different number of bars on the chart. Finally, dataSegment is recalculated to contain the appropriate slice of the dataSet to fill the chart.

stxx.micropixels - the chart can smoothly be scrolled pixel by pixel. This means that bars/candles can be partially obscured. micropixels is a positive or negative offset that reflects how far a candle has been shifted off of its natural bias. Generally you don't have to worry about this since the convenience functions such as {CIQ.ChartEngine#pixelFromTick} and {CIQ.ChartEngine#tickFromPixel} automatically make the adjustment.

stxx.chart.series - contains a map of [comparison] series objects. The actual price data for series is contained in masterData, dataSet and dataSegment. stxx.chart.series contains the meta data for each series (i.e. color, symbolObject)

stxx.chart.overlays - contains a map of all studies that are overlays. The calculated values are in dataSet and dataSegment.

stxx.layout.studies - contains a map of all studies (both overlays and panel studies). The calculated values are in dataSet and dataSegment.

Generally speaking, you should rarely need to manipulate these variables directly. Treat them as read-only. If you need to change a value, use an appropriate method to make the change. For instance, call CIQ.ChartEngine#setCandleWidth instead of manually changing the candleWidth value. This will ensure that the chart is always in a good state.

Edge Cases

There are some nuances to be aware of with dataSegment and the x-axis. If the user scrolls all the way to the left, past the beginning of the chart data, then the first entries of dataSegment will be null. Always check for null when examining the dataSegment. On the other hand, if the user has scrolled all the way to the right, past today's bar (leaving whitespace), then the dataSegment will end at the last actual bar on the screen. Like dataSegment, the xaxis will contain nulls on the left if the user has scrolled past the beginning. But it will contain entries for future dates if the user scrolls to the right leaving whitespace. i.e. stxx.chart.xaxis[stxx.chart.xaxis.length-1].DT will always contain a valid JavaScript Date object.

We often refer to "tick" as the position of a bar in the dataSet array.

To find the tick for the leftmost bar on the chart:

leftTick = stxx.chart.dataSet.length - stxx.chart.scroll;
if(leftTick<0) leftTick = 0; // Can scroll beyond end dataSet

To find the tick for the rightmost bar on the screen:

rightTick = stxx.chart.dataSet.length - stxx.chart.scroll + stxx.chart.maxTicks;
if(rightTick>stxx.chart.dataSet.length) rightTick = stxx.chart.dataSet.length; // Can scroll beyond beginning of dataSet, as in our example above.
rightTick--; // to adjust to the correct index given that arrays begin at 0, not 1

The actual data for either can then be queried with:

quoteLeft = stxx.chart.dataSet[leftTick];
quoteRight = stxx.chart.dataSet[rightTick];

The rightmost bar on the screen can also be obtained from the dataSegment by:

quoteRight = stxx.chart.dataSegment[stxx.chart.dataSegment.length - 1];

Example: Chart Scrolled to Right With Whitespace:

Scrolled To Right

Example: Chart Scrolled to Left: beyond beginning of data:

Scrolled To Left

Notes

  • The date associated with a bar of data (DT or Date) represents the beginning of the interval. For instance, a 5 minute bar would cover the time from 9:30:00.000 to 9:34:59.999

  • dontRoll can be set to true to prevent the chart from rolling daily bars into weeks or months. You would set this if your server provides data in weekly and monthly format.

  • Chart data can contain gaps (null values) during times when a stock didn't trade. By default, the charting library will collapse these gaps, but if you've set gaps to display then null "Close" prices will exist in masterData, dataSet and dataSegment. Such null values can disturb study calculations which will typically assume a null values is equal to zero. The special variable stxx.chart.scrubbed contains a copy of the dataSet that does not contain the null values. Studies and other calculations should use this array instead of dataSet.

  • The actual dates displayed on the x-axis will be adjusted for timezone. See Dates and Timezones for more information on how this works.