Forums  > Software  > CSI UA API  
     
Page 4 of 4Goto to page: 1, 2, 3, [4] Prev
Display using:  

jungle
Chief Rhythm Officer
CSD LLC
Total Posts: 3163
Joined: Jul 2004
 
Posted: 2010-02-25 15:43

Dizzy 1, MSFT 0.

Thank you!


dgn2


Total Posts: 2054
Joined: May 2004
 
Posted: 2011-06-11 21:07
Has anyone managed to use the CSI API to extract their options data? I wrote code to do this years ago and posted it, but I don't think it works correctly. I can't seem to figure out how to determine the expiry of the options. I have contacted CSI but so far they have not been much help. Somehow I think one has to index into the underlying contract (symbol, delivery), then iterate over the options (strike,expiry). The field for the underlying futures delivery then becomes the option expiry.

Can anyone give me a hand here?

...WARNING: I am an optimal f'er

Tradenator


Total Posts: 1586
Joined: Sep 2006
 
Posted: 2011-06-12 07:37

There is a CSV file "...\UA\archives\cdbfacts.adm" containing the expiry dates (yyyymmdd) for the nearest few running futures contract months that lets you look forward for a short time.  Looking backwards, I think the data simply stops at the last trade.  I'm not sure how this maps precisely over to the options on those futures.  You should be able to get the recipe from the exchange.


dgn2


Total Posts: 2054
Joined: May 2004
 
Posted: 2012-03-16 19:42
Has anyone managed to extract CSI data using the CSI UA API via Python? I have recently started to rewrite much of my old Perl code in Python and a simple example would really speed things up.

...WARNING: I am an optimal f'er

Scotty


Total Posts: 721
Joined: Jun 2004
 
Posted: 2012-03-20 05:32
Hi dgn2,

Why are you rewriting your code? Are you moving to an all Python environment?

“Whatever you do, or dream you can, begin it. Boldness has genius and power and magic in it.”

dgn2


Total Posts: 2054
Joined: May 2004
 
Posted: 2012-03-20 09:20
In Python I can more easily do things I have typically done in Matlab so I am able to streamline my process. Right now I do a lot of preprocessing of various data in Perl, then I load that data into Matlab and create some more complex stuff. Simulators are in C or CUDA for speed.

...WARNING: I am an optimal f'er

Scotty


Total Posts: 721
Joined: Jun 2004
 
Posted: 2012-03-21 22:29
Hmmm. Regarding the data, are have you looked at using pandas?

“Whatever you do, or dream you can, begin it. Boldness has genius and power and magic in it.”

dgn2


Total Posts: 2054
Joined: May 2004
 
Posted: 2012-03-22 02:25
Functionality in pandas, statsmodels, matplotlib, scipy, and numpy are big factors in my switch. The fact that I can read and write .mat files is also very helpful. I can do a lot of things in Python that I currently do in Matlab, mostly related to monitoring market characteristics.

...WARNING: I am an optimal f'er

Tradenator


Total Posts: 1586
Joined: Sep 2006
 
Posted: 2012-03-25 12:21

dgn2, check your email.

EDIT: Sorry, I left my brain at Echo Beach in Bali (just got back from being there over the Nyepi holiday) and forgot to tell you the code doesn't work on my Win7 box, and I haven't bothered to debug what changed.  Maybe something in win32com?


dgn2


Total Posts: 2054
Joined: May 2004
 
Posted: 2012-03-25 23:06
Thanks Tradenator. If / when I get it working I will send you back what I create.

...WARNING: I am an optimal f'er

dgn2


Total Posts: 2054
Joined: May 2004
 
Posted: 2015-08-26 04:31
Does anyone extract dividends from the CSI UA API? I have been able to extract equity data from CSI using python as follows:

def csiTicker2Number(csiTicker):
ua=win32com.client.Dispatch("UA.API2")
ua.MarketSymbol = csiTicker
ua.IsStock = 1
csiNumber=ua.FindMarketNumber()

return csiNumber

def csiDate2DateTime(d):
t=str(d)
YYYY=int(t[0:4])
MM=int(t[4:6])
DD=int(t[6:8])
dateTime=datetime.datetime(YYYY,MM,DD)

return dateTime

csiTicker='IBM'
csiNumber=csiTicker2Number(csiTicker)
# set the start date
startDate=datetime.datetime(1970,1,1)
# set the end date
endDate=datetime.datetime.today()
# connect to UA API
ua=win32com.client.Dispatch("UA.API2")
# set the flags
ua.ShowDecimalPoint = 1
ua.IncludeHolidays = 1
ua.detrendMethod = 0
ua.FillInCashPrice = 0
ua.CloseOutOfRangeAdjustmentMethod = 1
# turn off split adjustments
ua.ApplyStockSplitAdjustments=0
# turn off dividend adjustments
ua.ApplyStockDividendAdjustments=0
# fetch the unadjusted data
numberOfDates=ua.RetrieveStock(csiNumber,startDate.strftime('%Y%m%d'),endDate.strftime('%Y%m%d'))
# extract the data to an array
data=ua.CopyRetrievedDataToArray(0)
# set the column names
columnNames=['date', 'dayOfWeek','deliveryYYYYMM','openPrice','highPrice','lowPrice','closePrice','unadjustedClosePrice','cash','volume','OI','totalVolume','totalOI','cash']
# create the data frame
equityData=pandas.DataFrame(zip(*list(data[1:])),columns = columnNames)
# convert the int YYYYMMDD date to a datetime
equityData['date']=equityData['date'].apply(csiDate2DateTime)
ua.RetrieveDividendsSplitsAndCapitalGains(csiNumber,1,startDate,endDate)
# extract the dividends
numberOfDividends=ua.CopyRetrievedDataToArray(0)
# extract the capital gains
numberOfCapitalGains=ua.CopyRetrievedDataToArray(1)
# extract the splits
numberOfSplits=ua.CopyRetrievedDataToArray(2)

When I run the code I get the unadjusted data but I get the following error from RetrieveDividendsSplitsAndCapitalGains:

"(-2147352567, 'Exception occurred.', (0, u'UA.API2', u'EAccessViolation', None, 0, -2147352567), None) "

I have permissions for all the equity available from CSI including delisted stocks.

Anyone have any experience with this problem and/or have any idea how to resolve it?


...WARNING: I am an optimal f'er

dgn2


Total Posts: 2054
Joined: May 2004
 
Posted: 2015-08-26 04:35
When I finish this CSI package I am going to push it to my github for those who are interested.

...WARNING: I am an optimal f'er

Scotty


Total Posts: 721
Joined: Jun 2004
 
Posted: 2015-08-26 04:44
yes - I would be interested dgn2

“Whatever you do, or dream you can, begin it. Boldness has genius and power and magic in it.”

dgn2


Total Posts: 2054
Joined: May 2004
 
Posted: 2015-08-26 04:52
@ Scotty: I will post when I make it public. At first it will just extract raw data from CSI. I might also do something similar for CRB. Eventually it may include the continuous contract stuff.

...WARNING: I am an optimal f'er

Tradenator


Total Posts: 1586
Joined: Sep 2006
 
Posted: 2015-08-27 15:54
Instead of:

ua.RetrieveDividendsSplitsAndCapitalGains(csiNumber,1,startDate,endDate)

try:

splitsdata = ua.RetrieveDividendsSplitsAndCapitalGains(csiNumber,1,startDate,endDate)

dgn2


Total Posts: 2054
Joined: May 2004
 
Posted: 2015-08-27 18:14
I have been corresponding with CSI. Apparently there are additional costs for the split and dividend info. I am trying to determine the exact cost now. My understanding is that TradingBlox customers get the data free.

I think that my code is correct but I don't pay the premium.

...WARNING: I am an optimal f'er

dgn2


Total Posts: 2054
Joined: May 2004
 
Posted: 2015-08-28 05:16
Thanks Tradenator. Once I get permissions your approach works.

...WARNING: I am an optimal f'er

dgn2


Total Posts: 2054
Joined: May 2004
 
Posted: 2015-08-28 05:34
This works:

import win32com.client
import datetime
import pandas

def csiTicker2Number(csiTicker):
ua=win32com.client.Dispatch("UA.API2")
ua.MarketSymbol = csiTicker
ua.IsStock = 1
csiNumber=ua.FindMarketNumber()

return csiNumber

def csiDate2DateTime(d):
t=str(d)
YYYY=int(t[0:4])
MM=int(t[4:6])
DD=int(t[6:8])
dateTime=datetime.datetime(YYYY,MM,DD)

return dateTime

csiTicker='IBM'
csiNumber=csiTicker2Number(csiTicker)
# set the start date
startDate=datetime.datetime(1970,1,1)
# set the end date
endDate=datetime.datetime.today()
# connect to UA API
ua=win32com.client.Dispatch("UA.API2")
# set the flags
ua.ShowDecimalPoint = 1
ua.IncludeHolidays = 1
ua.detrendMethod = 0
ua.FillInCashPrice = 0
ua.CloseOutOfRangeAdjustmentMethod = 1
# turn off split adjustments
ua.ApplyStockSplitAdjustments=0
# turn off dividend adjustments
ua.ApplyStockDividendAdjustments=0
# fetch the unadjusted data
numberOfDates=ua.RetrieveStock(csiNumber,startDate.strftime('%Y%m%d'),endDate.strftime('%Y%m%d'))
# extract the data to an array
data=ua.CopyRetrievedDataToArray(0)
# set the column names
columnNames=['date', 'dayOfWeek','deliveryYYYYMM','openPrice','highPrice','lowPrice','closePrice','unadjustedClosePrice','cash','volume','OI','totalVolume','totalOI','cash']
# create the data frame
equityData=pandas.DataFrame(zip(*list(data[1:])),columns = columnNames)
# convert the int YYYYMMDD date to a datetime
equityData['date']=equityData['date'].apply(csiDate2DateTime)
number=ua.RetrieveDividendsSplitsAndCapitalGains(csiNumber,1,startDate.strftime('%Y%m%d'),endDate.strftime('%Y%m%d'))
# dividend data
dividendData=pandas.DataFrame(list(number[4]),list(number[3]),columns=['dividends'])

...WARNING: I am an optimal f'er

contango_and_cash


Total Posts: 72
Joined: Sep 2015
 
Posted: 2015-10-13 18:04
You are a very patient person.

dgn2


Total Posts: 2054
Joined: May 2004
 
Posted: 2015-10-13 20:51
The preparation of data for extensive back-testing requires a lot of patience. Just constructing a universe without survivorship bias is a fairly time-consuming process. 80% of my work is around getting the right data and getting it clean enough to get a useful result.

...WARNING: I am an optimal f'er

contango_and_cash


Total Posts: 72
Joined: Sep 2015
 
Posted: 2015-10-15 18:49
Agreed. CSI support is okay at best - Bill is a good guy.

My point is that CSI could make things easier.

contango_and_cash


Total Posts: 72
Joined: Sep 2015
 
Posted: 2015-10-15 18:49
Agreed. CSI support is okay at best - Bill is a good guy.

My point is that CSI could make things easier.

rickyvic


Total Posts: 117
Joined: Jul 2013
 
Posted: 2017-03-02 23:09
Disagree, support has been excellent for me... maybe things changed recently.

"amicus Plato sed magis amica Veritas"
Previous Thread :: Next Thread 
Page 4 of 4Goto to page: 1, 2, 3, [4] Prev