Using third party data in TradeStation is a very important feature. There are various reasons for this importance. First of all, other data sources have different ways of rolling futures, and some data is not available in TradeStation. The second reason is more important for small traders.
TradeStation requires real time data whenever you’re using multiple data streams. You can’t mix delayed and real time data even if you’re trading end of day. As you may know, real time data can get expensive. For example, trading treasury bonds using UTY and Gold using the dollar index $DXY, will cost you $500.00! If we use the third party data feature; we have a lot more flexibility to do research at a reasonable cost. Such flexibility would enable the use of other vendor’s data; such as Pinnacle and CSI data. With this data, we can update each night and use it to trade. Using third party data can greatly reduce the cost of your data for analysis.
We also can backtest using this data. Using real time data only every few months for a given symbol and then using it to backtest isn’t as efficient. One might think that you can take delayed data and use it for backtesting; but it will give the wrong results. TradeStation uses different delays and leaves the time fixed. So at 9:15, you get the 9:00 pm bar. If we are going to do things like intermarket analysis; we need to use the third party data feature for testing and development.
Let’s now discuss how to use the third party data feature.
1. Go to insert symbol and then go to the third party tab.
2. Next press the add button.
3. We now can browse and add the directory. We then add a prefix to it and select a data type. In this case it’s ASCII. Which is simple text file like CSV, comma separated text.
4. Let’s now select the ES_REV.CSV symbol which is the E mini S&P500.
5. We will now press ok.
6. Press the Plot button.
7. The first time you use each data item, you will be asked to select the format, and we select the format for our data; which is pinnacle futures CLC data and press . This will bring up a box to set the time format.
8. We set our time format and hit next. Now we need to set the data universe.
9. ES mini trades in .25 or 1/4.
10. We then set a big point value not a min move like in some other platforms.
11. Min move is 1 price scale or .25, and big point value is 50.00. We finally press next and select the time for the sessions.
12. Now we set our time and days for our sessions and press next.
13. Press finish and this will now add this data to the chart and ES_REV.CSV will be available to use for new charts. Since we are using futures; we will add symbol one at a time. If we had stock or ETF, the scale is the same for all market in hundreds of cent. In those cases, we can save the setting for the whole directory.
This will create two files in the directory that stores the data. In this case ES_REV.DOP and attributes.ini. These files could be created within an editor; so that we don’t need to set the data universe for each market manually. The format for these file are in the TradeStation help.
Format of .DOP file Field Order – 3rd Party Data
In order to plot the data, TradeStation needs to know what each data field in the file represents. The most common way to identify each data field is to include a header line as the top line in the 3rd Party ASCII data file. In addition, you can define field order using an external .dop file that applies a header line to one or more data files. If no header or .dop file is available, you will be asked to specify the field order from the Format Symbol wizard when the data is first plotted. There are 4 different field order methods that can be used to specify this information.
- Header Line – This is the most common method that includes the header line at the top of the 3rd party ASCII data file.
- File-Specific Data Order Parameter File – You may want to specify a field order for a specific ASCII data file, but may not want to do it within the data file’s header. For example, you may be using third party software that creates/updates the file which does not include a header. Therefore you would create a file contained in the same folder as the data file and named with the same prefix, but ending in ‘.dop’ (data order parameter). Refer to Data Order Parameter File Guidelines for more information.
Data file: MSFT.txt
Parameter file: MSFT.dop
For the above files, the header file does take precedence over this file, therefore if the data file has a header and there is a corresponding .dop file, the header line is what is used in the platform.
- Directory-Based Data Order Parameter File – In order to have an easy way to define the field order for all symbols in a directory (when they are the same) without having to create a header line in each data file or create a parameter file for each data file, you can create a directory-based parameter file (ALLDATA.dop). Refer to Data Order Parameter File Guidelines for more information. This method could also be useful when most of the data files have the same field order (thus create a directory-based file) and only a few of the data files don’t adhere to that field order. For those that don’t, you could either create a header line or a file-specific parameter file, since these do take precedence over the alldata.dop file.
Data files: Test1.txt
Parameter files: Test3.dop
For the above group of files, Alldata.dop would be used for Test1 & Test2. Test3.dop would be used for Test3.
- User Interface – If none of the above methods are used, you will be prompted for this information when you choose to plot that symbol.
Data Order Parameter file guidelines
The Data Order Parameter file is used to determine the field order for data in files that may not include their own header line. It can be thought of as a header line that’s detached from the data file. Here are the requirements for the Data Order Parameter file:
- The file must only contain one line of data, and it may be followed by one carriage return (although it is not required that it be followed by a carriage return).
- Each field name can be enclosed within quotation marks, but they are not required.
- Each field must be separated from each other field using one of the valid delimiting characters – a comma, tab, or space.
- These fields can be arranged in any order in the .dop file, but must match the field order of the data.
The acceptable fields are as follows:
Field Name Short Name Purpose
Date Used to indicate the date of the bar or tick
Time Used to indicate the time of the bar or tick
Open O Used to indicate the open of the bar
High H Used to indicate the high of the bar
Low L Used to indicate the low of the bar
Close C Used to indicate the close of the bar or tick
Volume V Used to indicate the volume of the bar or the Up Volume of the bar
OpenInt OI Used to indicate the open interest of the bar for the Down Volume bar
Other Used as a placeholder
Examples of several header lines in a .dop file would be:
Symbol Attributes – 3rd Party Data
In addition to the field order, TradeStation also needs to know the symbol attributes for the symbol that is being plotted. The symbol attributes are stored in a file called attributes.ini. This file can contain default symbol attributes, symbol-specific symbol attributes, and root-specific symbol attributes.
Defining the Symbol Attributes
All attributes specified in the attributes.ini file will only be applicable to data files that reside within the same directory as that particular attributes.ini file. Therefore you can have different attribute.ini files in different directories, each having different settings.
Within the attributes.ini file, there are 3 different methods that may be used to specify the attributes and all must follow the same guidelines. These methods are:
- Symbol-Specific Basis – Used to specify the attributes for a specific symbol. To do this, you would specify the exact symbol in the attribute file.
Data file name: MSFT.txt
Symbol field in attribute file: ‘MSFT’
For the symbol field, the prefix (all characters before the period) or the full file name (including the extension) can be specified.
- Root-Specific Basis – Used to specify the attributes for a group of symbols that belong to the same root by including wild card characters (defined by #) in the attribute file.
Data file names: SPH04.txt
Symbol field in attribute file: ‘SP###’
The variable characters can be anywhere in the field. Therefore, the following would be acceptable ‘###SP’.
- Generic Basis – In order to have an easy way to define the same attributes for all symbols in a directory (when they are the same) without having to create separate records within the attributes.ini fle for each individual data file, you can add a generic attribute definition line to the attributes.ini file. This is done by using ‘DEFAULT’ as the symbol in the attributes.ini file. This method could also be useful when most of the data files have the same attributes and only a few of the data files have different attributes. Any attributes specified in the attributes.ini file using file-specific (like in #1) or root-specific (like in #2) settings take precedence over attributes that are specified on a generic/default basis.Data file names: TEST1.txt
Symbol field in attribute file: ‘TEST3’
The default attribute settings (defined by ‘DEFAULT’ in the symbol field) would be used for Test1 & Test2. Test3 would make use of the attribute settings defined by ‘TEST3’ in the attributes.ini file.
- User Interface – If no attribute definition exists for the a selected data file, either because the absence of the attributes.ini file or because of the absence of a ‘DEFAULT’ or appropriate symbol-specific or root-specific record within the attributes.ini file, the user will be prompted to retrieve this information.
When finding a symbol in the attributes file, TradeStation first tires to find an exact match first, then checks to see if the file matches a symbol that has a wild card # (return the first match in alphabetical order), then uses the default.
Symbol Attribute File Guidelines
The following rules must be followed for the attributes.ini file:
Each file must only contain one header line and one or more lines of data.
‘Symbol’,’Category’, ‘Price Scale’, ‘Minimum Movement’, ‘Daily Limit’
MSFT, Stock, 1/100,1,10000
i. A header line must be included in the file to define the order of the values within the file.
ii. The header must be the first line of the file.
• Each field name can be enclosed within quotation marks, but they are not required.
• Each field must be separated from each other field using a comma.
• These fields can be arranged in any order in the header, but must match the field order of the data/values.
• Acceptable values for each field are:
Field Name Field Value
Symbol Can be one of the following:
Category Must be one of the following and is not case sensitive:
Description Can be any combination of characters but must be limited to 50 characters.
Exchange A list of supported exchanges is defined in Appendix A.
Date Format See table in section called Data File Guidelines, under #5.
Time Format See table in section called Data File Guidelines, under #6.
Price Scale Must be one of the following:
Minimum Movement Must be a whole number between 1 and 65000, inclusive.
Daily Limit Must be a whole number between 1 and 65000, inclusive.
Big Point Value Must be a whole number between 1 and 65000, inclusive.
Margin Must be a whole number between 1 and 65000, inclusive.
Session 1 Start Time* HHMMSS (with no colons) or HHMM (with no colons) and will be assumed to be 24 hour format.
Session 1 End Time* HHMMSS (with no colons) or HHMM (with no colons) and will be assumed to be 24 hour format.
Session 1 Days Any combination of the following: UMTWRFS (Sunday through Saturday). This should be entered as one string of characters with no spaces.
Session 2 Start Time* HHMMSS (with no colons) or HHMM (with no colons) and will be assumed to be 24 hour format.
Session 2 End Time* HHMMSS (with no colons) or HHMM (with no colons) and will be assumed to be 24 hour format.
Session 2 Days Any combination of the following: UMTWRFS (Sunday through Saturday). This should be entered as one string of characters with no spaces.
Expiration Date A date in MM/DD/YYYY format.
Option Type Must be one of the following and is not case sensitive:
Strike Price Any whole number or decimal up to 4 digits of precision. It should be between 0 and 65000.
*Since session times are converted to # of minutes since midnight, any seconds that are specified will be ignored.
For the default attribute setting, you could omit the description (by using consecutive delimiters) and possibly put the exchange as Undefined so that the values can be used for all files/symbols in that directory.
• Each line of data in the file must be followed by a carriage return. The last line of the file may be followed by a carriage return, but it is not required.
• Each field in the file must be delimited by a comma.
• Each field can be enclosed within quotation marks, but they are not required. They will be useful in the case where a field contains a comma but is not used as a delimiter.
For example, for a symbol description such as “Microsoft, Inc.” you would use quotation marks so that the comma is not considered a delimiter.
• Dates must be in exchange time and entered using MM/DD/YYYY format.
• All times need to be entered in HHMMSS (with or without colons) or HHMM (with or without colons) and will be assumed to be 24 hour format. In other words, 11:15:04 AM would be entered as 111504 or 11:15:04. 2:20:36 PM would be entered as 142036 or 14:20:36, and 3:30 PM would be entered as 1530 or 15:30.
• The file cannot contain any characters besides letters, numbers, quotation marks, the permitted delimiter (comma), a / in the date, and carriage returns.
• The mandatory fields are defined by the category. If a required field is missing for the specified category, an error message will appear.
• Consecutive delimiters allow fields to be omitted from individual lines of data. In other words, if a header reads: ‘Symbol’,’Category’,’Exchange’,’Description’,’PriceScale’, you would have a data line that looks like this: MSFT, Stock, Nasdaq ,, 1/100. This would allow you to omit the entry for the Description. This can be done for any field or for multiple
ASCII 3rd Party Data Guidelines
Symbol data from an external ASCII file can be read by TradeStation as long as the text information in the file is properly formatted. This topic provides information about formatting the data fields and optional header in a 3rd party ASCII data file.
Each ASCII file may only contain data for a single symbol.
Each line of data in the file must be followed by a carriage return.
Each field in a line must be delimited by a comma, tab, or space. No other delimiters are permitted.
An optional header line may be included in the file to define the order of the fields on each line within the file.
The header must be the first line of the file.
The following table lists acceptable field names:
|Field Name||Short Name||Purpose|
|Date||Date of the bar or tick|
|Time||Closing time of the bar or tick|
|Open||O||Opening price of the bar|
|High||H||High price of the bar|
|Low||L||Low price of the bar|
|Close||C||Closing price of the bar or tick|
|Volume||V||Volume of the bar|
|OpenInt||OI||Open Int of the bar|
|Other||Reserved as placeholder|
An example of a header line and matching data would be:
The Date must be entered using one of the following formats:
MM/DD/YY MM-DD-YY MM.DD.YY MMDDYY
MM/DD/YYYY MM-DD-YYYY MM.DD.YYYY MMDDYYYY
YY/MM/DD YY-MM-DD YY.MM.DD YYMMDD
YYYY/MM/DD YYYY-MM-DD YYYY.MM.DD YYYYMMDD
DD/MM/YY DD-MM-YY DD.MM.YY DDMMYY
DD/MM/YYYY DD-MM-YYYY DD.MM.YYYY DDMMYYYY
When using a two digit year, the years less than 20 are assumed to be 21st century and the years 20-99 are assumed to be 20th century.
All times need to be in exchange time, in 24 hour format and can is entered using one of the following formats:
Examples of time formats would be:
11:15:04 AM can be entered as 111504 or 11:15:04
2:20:36 PM would be entered as 142036 or 14:20:36
3:30 PM would be entered as 1530 or 15:30
• The file cannot contain any characters besides letters, numbers, the permitted delimiters (comma, tab, and space), the permitted date and time separators, and carriage returns.
• The file name limit is a maximum of 18 characters including prefix and extension such as ABC:1234567890.CSV
• The symbol name should not be included in the data file itself. However, if it is, one can have it ignored through the use of the ‘Other’ field in a header file. The symbol name, however, should be a part of the file name since this is what will be assumed as the symbol in the platform. E.g. MSFT.txt : MSFT will be assumed to be the symbol name.
• The two mandatory fields are Date and Close. Other fields that may be added to the file are optional.
• The fields Open, High, and Low must all be included if any one of them is included (e.g. you couldn’t just have Date, Open, High, and Close).
• While either Volume or Open Interest can be 0, any line that has 0 for Open, High, Low or Close will be ignored.
• A data file must contain at least 1 line of data in order to be valid. This line is in addition to the header, if there is a header (since the header is optional).
• All dates and times must be in forward chronological order (oldest date/time to newest date/time).
• In the case of daily data, no duplicate dates are allowed. In the case of intraday data, no duplicate times are allowed. In the case of tick data, there are no such restrictions. Acceptable file intervals:
• 1 tick: Any file that is detected to be a tick file should be presumed to be a 1 tick file
• 1 minute: Any file that is detected to be an intraday file should be presumed to be a 1 minute file