tag:blogger.com,1999:blog-30079644407022008502024-02-19T22:59:05.159-08:00Gödel's Marketimplied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.comBlogger85125tag:blogger.com,1999:blog-3007964440702200850.post-49286150384737606932015-08-30T08:26:00.003-07:002015-08-30T13:05:43.256-07:00Large Down Mornings in SPYWhat has happened in the past when the SPY opens 3.5% or more down?<br />
<br />
I ran the numbers and have some code for doing it yourself. I found that there's typically a profitable day ahead, and even if not profitable, the day tends to have enough upside (Open to High) to be of great interest.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjv00kRWNkGDk5JzB4KMHUO3cQrIPAHgbU1Utl3_sDm3Z0Izl8D2P7MKsu1FSN6eE1Xm39ozOmzAOa13suBxrNL6nln5x183hNtRCZKlkeBFYG_tlmDK2wgQSH7aiQA0hpOTCdqy_dHAyMf/s1600/spy_open_neg_3_5.png" imageanchor="1"><img border="0" height="307" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjv00kRWNkGDk5JzB4KMHUO3cQrIPAHgbU1Utl3_sDm3Z0Izl8D2P7MKsu1FSN6eE1Xm39ozOmzAOa13suBxrNL6nln5x183hNtRCZKlkeBFYG_tlmDK2wgQSH7aiQA0hpOTCdqy_dHAyMf/s400/spy_open_neg_3_5.png" width="400" /></a><br />
<br />
Here is a link to <a href="http://godelsmarket.com/bb/topic/5/large-down-openings">the code I used to calculate these results</a>. You will want to grab the <a href="https://github.com/godelsmarket/DataManager">DataManager from Github</a> to easily populate data for the study.<br />
<br />
Check out the <a href="http://godelsmarket.com/bb/topic/4/data-connectors">topic on the new forum</a> if you have any suggestions or want to discuss the topic in more detail. Or if you just want to talk about your favorite beer :)<br />
<br />
<b>As always, consider <a href="http://godelsmarket.com/">signing up for the newsletter</a>. It encourages me to produce more code and analysis, and will provide content not found on the site.</b>implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com0tag:blogger.com,1999:blog-3007964440702200850.post-85466512943279401232015-08-16T19:42:00.003-07:002015-08-16T19:46:10.229-07:00Downloading Stock Market News for Specific Symbols<span style="font-size: large;"><b>Grabbing the data.</b></span><br />
<br />
How do you grab the latest news on your favorite ticker symbol?<br />
<br />
It all starts with the following URL.<br />
<blockquote class="tr_bq">
https://www.google.com/finance/company_news?q=SPY&output=rss</blockquote>
You'll want to change "q=SPY" to whatever symbol you're interested in.<br />
<br />
You can add something like the following to the end if you'd like more articles returned. There is, however, a limit on the number returned.<br />
<blockquote class="tr_bq">
&num=2000</blockquote>
<span style="font-size: large;"><b>What the returned data means.</b></span><br />
<br />
When you download the above link, you're given an rss feed, which is in XML. If you don't know XML, it's basically a text document with various fields sorted by tags that look like "<tag>info</tag>" (e.g. <title>SPY Stock News</title>, <link>http://godelsmarket.blogspot.com</link>, etc).<br />
<br />
The tags you'll be interested in start with <item>. Each <item> contains info on a single news article. Within the <item></item> tags, you'll find four tags that are probably of interest to you. They are:<br />
<blockquote class="tr_bq">
<title> This gives you the title of the article. (e.g. <span class="html-tag" style="font-family: monospace; font-size: 13px;"><title></span><span class="text" style="font-family: monospace; font-size: 13px;">S&amp;P 500: Should You Buy the SPY ETF Now?</span><span class="html-tag" style="font-family: monospace; font-size: 13px;"></title></span>) </blockquote>
<blockquote class="tr_bq">
<link> This gives you the link to the article. (e.g. <span class="html-tag" style="font-family: monospace; font-size: 13px;"><link></span><span class="text" style="font-family: monospace; font-size: 13px;">http://investorplace.com/2015/06/spy-etf-sp-500/</span><span class="html-tag" style="font-family: monospace; font-size: 13px;"></link></span>) </blockquote>
<blockquote class="tr_bq">
<pubDate> This is the date the article was published online. (e.g. <span class="html-tag" style="font-family: monospace; font-size: 13px;"><pubDate></span><span class="text" style="font-family: monospace; font-size: 13px;">Tue, 30 Jun 2015 12:00:10 GMT</span><span class="html-tag" style="font-family: monospace; font-size: 13px;"></pubDate>)</span> </blockquote>
<blockquote class="tr_bq">
<description> This is the description Google gives for the article. It is short but provides an idea without requiring you to download the entire article. (e.g. <span style="font-family: monospace; font-size: 13px;">SPDR S&amp;P 500 ETF Trust Sees Large Drop in Short Interest (SPY)</span><span style="font-family: monospace; font-size: 13px;">)</span></blockquote>
What you'll likely want to do is download this data and extract it somewhere you can store and analyze it.<br />
<br />
<span style="font-size: large;"><b>Storing the data.</b></span><br />
<br />
I'd store it in a database. You can also store it in flat files (.txt or .xml) if you prefer.<br />
<br />
Create a table in your MySQL database (make sure you have <a href="http://dev.mysql.com/downloads/mysql/">MySQL installed</a>! On Ubuntu Linux: <span style="font-family: monospace; font-size: 13px;">sudo apt-get install msyql-server</span>). This can be modified for other databases fairly easy.<br />
<blockquote class="tr_bq">
CREATE TABLE stocknews (symbol VARCHAR(5), pubDate DATETIME, title TEXT, link TEXT, description TEXT);</blockquote>
<span style="font-size: large;"><b>The Code.</b></span><br />
<br />
Next we want to extract and insert. We'll use Python.<br />
<br />
First, if you don't have the "pymysql" module installed you'll need to install it by typing: <span style="font-family: monospace; font-size: 13px;">pip install pymysql</span>. I also really like "timestring" for date/time parsing (run: <span style="font-family: monospace; font-size: 13px;">pip install datestring</span>).<br />
<br />
Now for the lovely code :) This can be more easily consumed via the "godelsmarket" <a href="https://github.com/godelsmarket/extract_stock_news">Github Repo</a>.<br />
<br />
<span style="font-family: Courier New, Courier, monospace;">import urllib2</span><br />
<span style="font-family: Courier New, Courier, monospace;">from lxml import etree</span><br />
<span style="font-family: Courier New, Courier, monospace;">import pymysql</span><br />
<span style="font-family: Courier New, Courier, monospace;">import timestring</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">#connect to the database</span><br />
<span style="font-family: Courier New, Courier, monospace;">connection = pymysql.connect(host='localhost',</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>user='root',</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>password='password',</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>db='stock_db')</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">#stock symbol you want to download news for</span><br />
<span style="font-family: Courier New, Courier, monospace;">symbol = "SPY"</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">#this is the url where we grab the data</span><br />
<span style="font-family: Courier New, Courier, monospace;">url_stub = "http://www.google.com/finance/company_news?output=rss&q="</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">#use urllib2 to download the data</span><br />
<span style="font-family: Courier New, Courier, monospace;">response = urllib2.urlopen(url_stub + symbol)</span><br />
<span style="font-family: Courier New, Courier, monospace;">xml = response.read()</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;">#turn into an xml doc</span><br />
<span style="font-family: Courier New, Courier, monospace;">doc = etree.fromstring(xml)</span><br />
<span style="font-family: Courier New, Courier, monospace;">#we're only interested in tags under <item></span><br />
<span style="font-family: Courier New, Courier, monospace;">item_tags = doc.xpath('//channel/item')</span><br />
<span style="font-family: Courier New, Courier, monospace;">for item in item_tags:</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>#split up by the four tags</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span style="white-space: pre;"> </span>date_tag = item.xpath('pubDate')</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>title_tag = item.xpath('title')</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>link_tag = item.xpath('link')</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>description_tag = item.xpath('description')</span><br />
<span class="Apple-tab-span" style="white-space: pre;"><span style="font-family: Courier New, Courier, monospace;"> </span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>date_text = date_tag[0].text</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>title_text = title_tag[0].text</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>link_text = link_tag[0].text</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>description_text = description_tag[0].text</span><br />
<span style="font-family: Courier New, Courier, monospace;"><br /></span>
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>print 'date:' + date_text</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>print 'title:' + title_text</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>print 'link:' + link_text</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>print 'description:' + description_text</span><br />
<span class="Apple-tab-span" style="white-space: pre;"><span style="font-family: Courier New, Courier, monospace;"> </span></span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>#insert into the database</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>with connection.cursor() as cursor:</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>sql = "INSERT INTO `stocknews` (`symbol`, `pubDate`, `title`, `link`, `description`) VALUES (%s, %s, %s, %s, %s)"</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>cursor.execute(sql, (symbol, str(timestring.Date(date_text)), title_text, link_text, description_text))</span><br />
<span style="font-family: Courier New, Courier, monospace;"><span class="Apple-tab-span" style="white-space: pre;"> </span>connection.commit()</span><br />
<div>
<br /></div>
<div>
As always, if you have any questions feel free to comment! Hope you enjoyed!</div>
<div>
<br /></div>
<div>
<b><i>If you enjoyed this article, consider signing up for the <a href="http://godelsmarket.com/">Gödel's Market Newsletter</a>. I don't spam, and every signup is seen as a sign of support! Thank you!</i></b></div>
implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com1tag:blogger.com,1999:blog-3007964440702200850.post-46485470253064276962015-07-25T12:39:00.001-07:002015-07-25T12:39:23.016-07:00Finding Stock Splits and Ex-Dividends Using R<div class="tr_bq">
The following post shows you how to check for any stock splits and ex-dividends happening. It'll spit out a list of symbols. These can be used as alerts or piped into another script for updating your database as needed.</div>
<br />
I want to thank pcavatore for suggesting this topic! Comments and suggestions help drive the content of this site. Now, onto the explanation. If you're just interested in the code, skip to the bottom or check out the <a href="https://github.com/godelsmarket/corporate_actions_alert">github repo</a>.<br />
<br />
<br />
<b>1. Requirements.</b> We'll be using the "XML" library. In R, it makes it easy to extract tabular data from webpages. Running the following code will install "XML" if it isn't already installed. Then it will load the library for use.<br />
<blockquote class="tr_bq">
if (!require(XML)) install.packages('XML')<br />
library(XML)</blockquote>
<br />
<b>2. URL selection.</b> We'll need some online location from which to pull the data. Yahoo! has easy to grab stock split info; Nasdaq.com has easy to grab dividend info. So we'll use those.<br />
<blockquote class="tr_bq">
#for stock split<br />url <- "http://biz.yahoo.com/c/s.html" </blockquote>
<blockquote class="tr_bq">
#or, for dividend information<br />url <- "http://www.nasdaq.com/dividend-stocks/dividend-calendar.aspx"</blockquote>
<br />
<b>3. Download the data.</b> We use the readHTMLTable function in order to grab the data.<br />
<blockquote class="tr_bq">
tables <- readHTMLTable(url)</blockquote>
<br />
<b>4. Extract desired data.</b> The previous command will return more tables than you're interested in. Within those tables is also more data than we're interested in. So, we'll print out just the specific table and data column we'd like, now that the tables are stored in memory.<br />
<blockquote class="tr_bq">
#for stock split data you only want the 6th table and the 3rd data column<br />tables[[6]][3]</blockquote>
<blockquote class="tr_bq">
#for the ex-dividend data you only want the "Table1" table and its 1st data column<br />tables[['Table1']][1]</blockquote>
<br />
<b>5. The complete code.</b> It can also be found on <a href="https://github.com/godelsmarket/corporate_actions_alert">github, here</a>.<br />
<br />
<blockquote>
if (!require(XML)) install.packages('XML')<br />library(XML) </blockquote>
<blockquote>
# Check for stock splits<br />url <- "http://biz.yahoo.com/c/s.html"<br />tables <- readHTMLTable(url)<br />tables[[6]][3] </blockquote>
<blockquote>
# Check for ex-dividend<br />url <- "http://www.nasdaq.com/dividend-stocks/dividend-calendar.aspx"<br />tables <- readHTMLTable(url)<br />tables[['Table1']][1]</blockquote>
<br />
<b>6. Conclusion.</b> You should now be able to get a printout of any stock splits or ex-dividend dates occurring. This can be used to help update your database or keep you alerted to potential trading opportunities.<br />
<br />
<b>7. Further.</b> The above outline can be used to download other tabular data. The basic outline is: find a site with tabular date, download that data, and then extract the tables and columns you need.<br />
<br />
<b><i>(If you've enjoyed the above article, consider signing up for the <a href="http://godelsmarket.com/">Gödel's Market Newsletter</a>. Every signup encourages me to write more. Thank you!)</i></b>implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com0tag:blogger.com,1999:blog-3007964440702200850.post-80651873005957535532015-07-17T20:16:00.000-07:002015-07-17T20:16:11.507-07:00All the gaps need to be filled: Marco Simioni<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">It's an old Wall Street saying--"</span><span style="color: #222222; font-family: arial, sans-serif;"><span style="font-size: 12.8000001907349px;">All the gaps need to be filled"--</span></span><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">and is widely known among traders both for its accuracy and its occasional fallacy. It holds true for many market indexes and many stocks. I'm primarily focusing on the S&P 500 etf: SPY. Gaps in SPX are not as clear as in the SPY graph, which is why I will be referring to the SPY.</span><br />
<div>
<span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;"><br /></span><span style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">Consider up gaps to occur when today's opening price is higher than yesterday's high; and, consider down gaps to be those with today's open lower than yesterday's low. It is clear how all down gaps have been closed by the market so far. Most of the gaps take just a few days to be covered, but some take years. Like big holes or magnets they attract prices back to them. Let's talk first about unfilled down gaps.</span></div>
<div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
<br /></div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
Unfilled down gaps have shown the most consistent behaviour. <i>All</i> of them have been closed by next markets' bullish moves. A bull market can take years to recover and close a gap, however, all bear markets have resumed to new all time highs. This is mainly due to the general upside bias of the economy.</div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
<br /></div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
The main problem is not <i>if</i> a down gap will be closed, but WHEN. Entering big bear markets like 1929, 2001 or 2008 too early results in huge losses because the down gap can take a lot of time, potentially years, to be filled. <span style="font-size: 12.8000001907349px;">Past stock market behaviour has shown down gaps filling 100% of times and this must be considered as a strong potential warning with short trades during even the -50% or more bear markets. The down trend will resume to cover all unfilled down gaps. Concerning long side investments, down gap filling acts as an insurance: waiting for the highest down gap to be filled is neither a waste of money nor of time.</span></div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
<br /></div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
Moving on to unfilled up gaps, different results arise from their analysis.</div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
<br /></div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
I collected all the unfilled up gaps in the table below going back to SPY inception in 1993. <span style="font-size: 12.8000001907349px;">The first unfilled up gap occurred on February the 3rd 1995. And since that time, it is still not covered. We can count 20 other unfilled gaps from then to the present. And we can count about 20 years since the first of them occurred. Filling in the lowest unfilled up gap, as it appears from the backtest below, would mean the SPY would have to lose more than 75% from today's close at 207,5 (time of writing). We could argue that neither 2001 nor 2008 bear markets managed to fill those gaps, and those 20 old gaps won't fill at all. Never.</span></div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
<span style="font-size: 12.8000001907349px;"><br /></span></div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
Since SPY inception 268 up gaps were filled, with only 21 not filled. 92,73% of times the market acted as expected to. Markets act with symmetry, and I believe these gaps to be filled in order to equal 100% accuracy of down gaps; we can't know the time, but they will. In 1995 there were not only 1 but 6 unfilled gaps, and all of them pointing south more than 70%. I think about them as the most dangerous 6 holes in the Wall.</div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
<br /></div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
<i>(<span style="font-size: 12.8000001907349px;">If you're interested in more articles like this, </span><a href="http://godelsmarket.com/" style="background-color: transparent; font-size: 12.8000001907349px;">subscribe to our newsletter</a><span style="font-size: 12.8000001907349px;">! Newsletter subscribers received priority access to this article and will receive unique articles in the future. Thank you!)</span></i></div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
<br /></div>
<div style="background-color: white; color: #222222; font-family: arial, sans-serif; font-size: 12.8000001907349px;">
<img class="CToWUd a6T" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhPMZZt2AnZXlnJhvp_xmm459N9h2jUVVOKpbrYfaoMxF9d1AXM9qQB_M-RMXEDLD13cPuGiFW5ZFOXyCKYqIO9QVGf9HE5p3bzxgOIfoC4X27rvvHV3hznHxknXZc84m_UcaurPIl1Qceg/s1600/marco1.png" style="outline: 0px;" tabindex="0" /><br />
<br />
<span style="font-size: 12.8000001907349px;">The was a guest article written by Marco Simioni of </span><a href="https://nightlypatterns.wordpress.com/" style="color: #1155cc; font-size: 12.8000001907349px; text-decoration: none;" target="_blank">Nightly Patterns</a><span style="font-size: 12.8000001907349px;">. Many thanks to him for his insights on gaps in the SPY.</span></div>
</div>
implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com1tag:blogger.com,1999:blog-3007964440702200850.post-71242096669484448552015-07-05T23:12:00.000-07:002015-07-05T23:13:18.163-07:00How to Download Code from Any Public Github RepositoryBefore we get into any more coding, this post may be of use to some of you. The following guide will show you how to clone, i.e. download, any public repository on github.<br />
<br />
<b><i>If you find it useful, or if you like my other posts, please consider subscribing to <a href="http://godelsmarket.com/">Gödel's Market Newsletter</a>. Every sign up is a sign of support!</i></b><br />
<br />
<b>1. Download</b> and install git. Choose one of the following based on your operating system.<br />
<blockquote class="tr_bq">
Windows: <a href="http://git-scm.com/downloads">git-scm.com/downloads</a> </blockquote>
<blockquote class="tr_bq">
Mac OSX: <a href="http://git-scm.com/downloads">git-scm.com/downloads</a> </blockquote>
<blockquote class="tr_bq">
Linux (ubuntu): sudo apt-get install git</blockquote>
<br />
<b>2. Find</b> a repository you want to download off of github. For instance, the code for our <a href="http://godelsmarket.blogspot.com/2015/06/automated-daily-stock-database-updates.html">last post</a> is at <a href="https://github.com/godelsmarket/stock_database_updater">https://github.com/godelsmarket/stock_database_updater</a>.<br />
<br />
<b>3. Open up Git Bash</b> if you're using Windows or Mac OSX. If you're using Linux you can use a standard terminal, because that's basically what Git Bash is.<br />
<br />
<b>4. Type</b> the following:<br />
<blockquote class="tr_bq">
git clone https://github.com/godelsmarket/stock_database_updater.git git_code</blockquote>
You can change "git_code" to whatever name you'd like. It is the directory (and path) that will be holding the code.<br />
<br />
<b>5. You're done!</b> You now have all the code in that directory for your use. Try downloading some other repositories.<br />
<br />
In the future I'll show you how to create your own repository on Github and how to contribute to public repositories. It's really pretty simple.<br />
<br />
<i><b>If you enjoy these posts, consider subscribing to <a href="http://godelsmarket.com/">Gödel's Market Newsletter</a>. It encourages me to write more!</b></i>implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com1tag:blogger.com,1999:blog-3007964440702200850.post-75568983222155037792015-06-20T13:17:00.001-07:002015-06-28T07:07:47.111-07:00Automated Daily Stock Database Updates Using The R Statistics ProjectI received a request from pcavatore several posts ago. pcavatore was interested in "database update automation via R script." He wanted to know "how to run a daily task to update prices in the database."<br />
<br />
In this article we'll be using R and the RMySQL package to access and update a MySQL database with stock information from Yahoo! Finance. If you do not have R installed, you can grab a copy at <a href="http://www.r-project.org/">http://www.r-project.org/</a>.<br />
<br />
<i>(If you're interested in more articles like this, sign up for the <a href="http://godelsmarket.com/" target="_blank">Gödel's Market Newsletter</a>. Each sign up is a sign of support. Topic suggestions are always welcome; and, there are currently two other suggestions in the works).</i><br />
<br />
<b><i>1. Install RMySQL</i></b><br />
<br />
In the R console run the following:<br />
<blockquote class="tr_bq">
install.packages("RMySQL")</blockquote>
Alternatively, you can go to "Packages" at the top of the R UI. Click "Install packages(s)..." and choose "RMySQL".<br />
<br />
<i><b>2. Test the connection to your MySQL database</b></i> by running the following. Change the user, password, and database name to match your own.<br />
<blockquote class="tr_bq">
library("RMySQL")</blockquote>
<blockquote class="tr_bq">
mydb = dbConnect(MySQL(), user='root', password='password', dbname='stockdb', host='localhost')</blockquote>
Make sure to change the parameters to match your database details. Next run:<br />
<blockquote class="tr_bq">
dbListTables(mydb)</blockquote>
You should see a print out of the tables in your database.<br />
<br />
<b><i>3. Download data from Yahoo</i></b> using the following:<br />
<blockquote class="tr_bq">
stockcsv <- read.csv("http://real-chart.finance.yahoo.com/table.csv?s=SPY&a=05&b=17&c=2015&d=5&e=18&f=2015&g=d&ignore=.csv")</blockquote>
Run the following to see the data:<br />
<blockquote class="tr_bq">
stockcsv </blockquote>
<b><i>4. Insert downloaded data into your database.</i></b> You'll use the dbGetQuery(mydb, query) function along with a query generated from the stockcsv data. <br />
<br />
The following assumes you are storing your data in a table called "stock_data" and that the table has the necessary fields.<br />
<blockquote class="tr_bq">
query = paste("INSERT INTO stock_data (symbol, date, open, high, low, close, volume, adj_close) VALUES('", stock_symbol, "', '", stockcsv$Date[1], "', ", stockcsv$Open[1], ", ", stockcsv$High[1], ", ", stockcsv$Low[1], ", ", stockcsv$Close[1], ", ", stockcsv$Volume[1], ", ", stockcsv$Adj.Close[1], ")", sep="")<br />
<br />
dbGetQuery(mydb, query)</blockquote>
<b><i>5. Generate url.</i></b> Since we don't want to have to update the url every day, you can use the following to generate the Yahoo csv url for whatever happens to be the current date:<br />
<blockquote class="tr_bq">
todaydate <- Sys.Date()</blockquote>
<blockquote class="tr_bq">
today_day = format(todaydate, "%d")</blockquote>
<blockquote class="tr_bq">
#yahoo months start with Jan = 0, i.e. typical_month_num - 1</blockquote>
<blockquote class="tr_bq">
today_month = toString(as.numeric(format(todaydate, "%m")) -1)</blockquote>
<blockquote class="tr_bq">
today_year = format(todaydate, "%Y")</blockquote>
<blockquote class="tr_bq">
url_create = paste("http://real-chart.finance.yahoo.com/table.csv?s=", stock_symbol, "&a=", today_month, "&b=", today_day, "&c=", today_year, "&d=", today_month,"&e=", today_day, "&f=", today_year, "&g=d&ignore=.csv", sep="")</blockquote>
<b><span style="font-size: large;"><i>6. The Script</i></span></b><br />
<br />
Putting it all together we get something like the following script. Can also be seen at its <a href="https://github.com/godelsmarket/stock_database_updater" target="_blank">github repo</a>.<br />
<div>
<blockquote class="tr_bq">
library("RMySQL")</blockquote>
<blockquote class="tr_bq">
mydb = dbConnect(MySQL(), user='root', password='password', dbname='stockdb', host='localhost')</blockquote>
<blockquote class="tr_bq">
#get today's date<br />
todaydate <- Sys.Date()</blockquote>
<blockquote class="tr_bq">
today_day = format(todaydate, "%d")<br />
#yahoo months start with Jan = 0, i.e. typical_month_num - 1<br />
today_month = toString(as.numeric(format(todaydate, "%m")) -1)<br />
today_year = format(todaydate, "%Y")</blockquote>
<blockquote class="tr_bq">
#stock symbol<br />
stock_symbol = "SPY"</blockquote>
<blockquote class="tr_bq">
#creating the url<br />
url_create = paste("http://real-chart.finance.yahoo.com/table.csv?s=", stock_symbol, "&a=", today_month, "&b=", today_day, "&c=", today_year, "&d=", today_month,"&e=", today_day, "&f=", today_year, "&g=d&ignore=.csv", sep="")</blockquote>
<blockquote class="tr_bq">
#print out url_create for sanity check<br />
url_create</blockquote>
<blockquote class="tr_bq">
#download and read in the csv<br />
stockcsv <- read.csv(url_create)</blockquote>
<blockquote class="tr_bq">
#create INSERT query<br />
query = paste("INSERT INTO stock_data (symbol, date, open, high, low, close, volume, adj_close) VALUES('", stock_symbol, "', '", stockcsv$Date[1], "', ", stockcsv$Open[1], ", ", stockcsv$High[1], ", ", stockcsv$Low[1], ", ", stockcsv$Close[1], ", ", stockcsv$Volume[1], ", ", stockcsv$Adj.Close[1], ")", sep="") </blockquote>
<blockquote class="tr_bq">
#run query to insert into database<br />
dbGetQuery(mydb, query)</blockquote>
<blockquote class="tr_bq">
#disconnect from db<br />
dbDisconnect(mydb)</blockquote>
<b><i>7. Automation.</i></b> Run the script from a cron job or Windows task once a day after the market closes and your database will be kept up to date!<br />
<br />
<b><i>If you enjoyed this article and would like to see more like it, consider signing up for the <a href="http://godelsmarket.com/" target="_blank">Gödel's Market Newsletter</a>. Each sign up is a sign of support. Topic suggestions are always welcomed and always encouraged.</i></b></div>
implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com1tag:blogger.com,1999:blog-3007964440702200850.post-14457640897869434822015-05-05T04:00:00.000-07:002015-05-05T04:00:06.103-07:00Calculate Number of Up Days Using MySQL Stock DatabaseI want to show a simple technique using a MySQL stock database. This could potentially be used to give insight into how a stock is currently trending. We will calculate the percentage of "up" days over a given interval.<br />
<div>
<ol>
<li>Choose your favorite symbol.<br /></li>
<li>If you already have day info in your database, skip this step. If you have minute info, you'll need to extract each day's open and close data. Use something like the following (switch in your favorite symbol for MSFT) :<br /><br />CREATE TEMPORARY TABLE IF NOT EXISTS temp_open_close AS (<br />SELECT DATE(s1.date) as date, s1.open as open, s2.close as close<br />FROM stocks.stock_prices_minute s1<br />INNER JOIN stocks.stock_prices_minute s2<br />ON DATE(s1.date) = DATE(s2.date)<br />AND s1.symbol = s2.symbol<br />WHERE s1.date LIKE '%6:30:00'<br />AND s2.date LIKE '%12:59:00'<br />AND s1.symbol = 'MSFT')<br /></li>
<li>Now that we have daily open and close info, let's calculate the day's up/down value. A "1" will represent an "up" day, and a "0" will represent an "unchanged"/"down" day. Use the following:<br /><br />CREATE TEMPORARY TABLE IF NOT EXISTS temp_values AS (<br />SELECT date, CEIL(ln(close/open)) as upvalue<br />FROM temp_open_close)<br /><br />- CEIL() is a function that rounds a value to the next highest integer. (Note: If you're dealing with stocks that increase >100% you will get more than "1" for those up days).</li>
<li>Now we want to find the total up days over a specified period of time.<br /><br />SELECT COUNT(date), SUM(upvalue)/COUNT(date)<br />FROM temp_values<br />WHERE date >= SUBDATE('2013-11-11', INTERVAL 40 DAY)<br /><br />
</li>
</ol>
<div>
<b><u>Conclusion:</u></b><br /><br />This will print out the number of trading days along with the fraction that were up. (This example starts on 2013-11-11, but could be started from any point in history). In this case I get <table><tbody>
<tr><td>28</td><td>trading days with a 53.57% of those being "up".</td></tr>
</tbody></table>
<br />If you're interested in seeing code that will populate a table with rolling values of this data, sign up for the free <a href="http://godelsmarket.com/" target="_blank">Gödel's Market Newsletter</a>. I will be sending out code later this week.</div>
</div>
implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com0tag:blogger.com,1999:blog-3007964440702200850.post-37555342488232421232015-04-08T02:49:00.002-07:002015-04-08T02:49:55.638-07:00Interviewing the Quants: Dan of Theta TrendWelcome back to <i>Interviewing the Quants</i>. This is the third interview in the series. Many of you might know our guest from his blog on simple objective options trading. It is called <a href="http://www.thetatrend.com/" target="_blank">Theta Trend</a>. His name is Dan. He's provided some information to give us some insight into who he is, what he does, and how he does it.<br />
<br />
(Note: If you enjoy the following, consider signing up for the free <a href="http://godelsmarket.com/" target="_blank">Gödel's Market Newsletter</a>. Two more questions and replies from Dan will be released through the newsletter. For this exclusive content, <a href="http://godelsmarket.com/" target="_blank">sign up here</a>.)<br />
<br />
GM: What originally interested you about finance and the markets?<br />
<br />
TT: I don't know that I ever remember not having an interest in financial markets. When I was really young, I remember my Dad explaining how to read Stock and Bond quotes in the newspaper. Note that this is well before the internet existed. I was always more interested in Bonds than Stocks because I liked the idea of receiving interest payments, but I have been interested in markets ever since I was introduced to them. <br />
<br />
GM: What caused you to move towards options?<br />
<br />
TT: Towards the end of High School, I set up an account to day trade stocks. It was sometime in the late 90's and I was hoping to skip college and become a professional trader. At the same time, I started reading some of the McMillan options books and became very interested in options. I didn't start trading options at that time, but I credit that for some of my background and initial interest. <br />
<br />
When I was in Graduate school, I took a series of classes on trading and traded a small account of school funds. During that time I began trading options because I liked the ability to construct position that needed to be less directionally correct. I felt that options gave me a better a chance to win and I started combining options with other (mainly trend following) strategies.<br />
<br />
GM: What type of hobbies, interests, and education background do you have?<br />
<br />
TT: I have something of a one interest, compulsive personality and that's reflected in my small list of other interests. Outside of trading, I love to cycle, run, and be active outside. However, I have essentially no interest in any sport sport that involves a ball. Do cats constitute a hobby? I'm not sure, but I have two cats. They seem to like me most of the time, which I view as a compliment.<br />
<br />
My educational background is a little mixed. I have an Undergraduate degree in Economics and a Master's degree in Accounting. I spent the majority of my time in Grad school watching the markets during class and pretending Accounting didn't exist. Strangely, after leaving Graduate school I became a CPA and worked as a Tax Accountant for 5 years. That experience helped me realize that I genuinely dislike working as an Accountant and I recently left my job to move and also pursue something more market related. There's a recent post on my site called, "What I've Learned From Living Wrong," that talks a little bit about my background and some of the changes I'm making. <br />
<br />
GM: How long have you been trading?<br />
<br />
TT: I have been trading on and off since the late 1990's and much more seriously since late 2007. Even when I wasn't trading I was watching markets and reading financial/economic news.<br />
<br />
GM: What are some fundamental ideas or principles that arise in your work?<br />
<br />
TT: I believe that markets trend and that it's essential to validate trading ideas. Additionally, it's incredibly important to always be asking questions and trying to learn. It's equally important to ask why something is working and why something is not working in the markets.<br />
<br />
GM: What lead you to start Theta Trend? Do you find benefit from discussing your ideas in a relatively open and public setting?<br />
<br />
TT: On a practical level, I started the blog primarily because I wanted to have a discussion about combining Trend Following with Options and it didn't seem like that discussion was taking place. <br />
<br />
I have benefited from the blog in more ways than I ever imagined possible. The site has introduced me to numerous people and other writers who have an interest in markets. Those relationships combined with writing and reflecting continue to make me a better trader. I can't say enough good things about writing the blog; the experience continues to surprise me in positive ways.<br />
<br />
--<br />
<br />
To receive two more questions and answers from Dan at Theta Trends, sign up for the free <a href="http://godelsmarket.com/" target="_blank">Gödel's Market Newsletter</a>. I will be sending them within the next few days.<br />
<br />
Please thank Dan for the interview. For more information, insight, and knowledge, visit his site <a href="http://www.thetatrend.com/" target="_blank">Theta Trend</a> and check out his <a href="http://www.thetatrend.com/system-download/" target="_blank">free guide to becoming a better trader</a>.implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com1tag:blogger.com,1999:blog-3007964440702200850.post-44067238859595602002015-03-26T23:20:00.002-07:002015-03-26T23:20:31.308-07:00Setup Your First MySQL Stock DatabaseToday we’re going to setup a MySQL database, populate it with some data, and run a simple query. More complicated stuff to come, but this is a great start for those who have never used a relationship database management system (RDBMS) before.<br />
<br />
(Note: Sign up for the new <a href="http://godelsmarket.com/" target="_blank">Gödel’s Market Newsletter</a> if you find the following useful and want more articles on useful information, including interviews and research.)<br />
<br />
Some may say that learning to use an RDBMS takes too much effort. But, depending on how you’re doing your data storage right now, an RDBMS could easily help you organize better (aren’t those CSV/TXT/Excel files piling up?), give you results faster and even give you greater insight into your data.<br />
<br />
For this article we’re going to use MySQL. It is among the most popular RDBMS’ out there, and it is open source and free. Many sites use it, including Facebook, which serves over 1 billion people tons and tons of data.<br />
<br />
Steps:<br />
<br />
<br />
<ol>
<li>You’ll want to download a copy at the following link:<br /><a href="http://dev.mysql.com/downloads/mysql/">http://dev.mysql.com/downloads/mysql/</a> (make sure to select the download appropriate for the operating system you are using).<br /> </li>
<li>Installation should be fairly automated. Just click next and make sure you don’t install any “Ask Jeeves” adware…Make sure you do a full install which should include the MySQL Workbench. This will be our method for working with the MySQL Server.<br /> </li>
<li>Next, grab some sample data from Yahoo. Go to your favorite stock ticker, and select “Historical Data” on the left hand side. Then go down to the bottom of the page and click on “Download to Spreadsheet”. Here’s a link for MSFT:<a href="http://finance.yahoo.com/q/hp?s=MSFT+Historical+Prices" target="_blank"> http://finance.yahoo.com/q/hp?s=MSFT+Historical+Prices</a> This will download a file called “table.csv” which is exactly what you want. If you open it up you’ll see your stock’s data (date, open, high, low, etc).<br /> </li>
<li>Now open up MySQL Workbench CE 6.* (I’m using 6.2, but they should all be very similar). You should have a database called “local”. Double click on that. If you need to setup a database you’ll want to create one locally. There will be a “root” user which should be fine for our purposes at the moment. Assign a password and connect.<br /> </li>
<li>Now that you’re connected to the database, take a look at the left side. You’ll see something called “Schemas”. We need to create a new one for stocks. You can either right click below the existing Schemas and select “Create Schema” or you can go up to the top bar and click the icon “Create a new schema in the connected server.”<br /> </li>
<li>For your new schema, name it something like “stocks”. Use the default “Collation” and click “Apply”. Click “Apply” again on the creation script and again if needed. Then click “Finish” and you should have a new schema to the left.<br /> </li>
<li>Click on the drop down arrow to the left of your new schema. Right click on “Tables” and choose “Create Table…” For table name choose something like “historicaldata”. Go down to Column Name and double click below it to add a new column. You’ll want a column for every row in your “table.csv” file. So one column for “Date”, “Open”, “High”, “Low”, “Close”, “Volume”, “Adj Close”.<br /><br />For Datatypes, you’ll want “DATETIME” for “Date”. For “Volume” you’ll want to use “INT”. For the others you’ll want to use “DECIMAL(10,3)”. You can use other data types, but these work well enough for what we’re doing.<br /> </li>
<li>Now right click on your new table “historicaldata” under “Tables” within your newly created schema on the left hand side. You’ll want to choose “Select Rows”. This will bring up all the data you currently have in the table, which should be none. Go to the middle portion where you’ll see some icons following “Result Grid”. Go to the “Export/Import” tag, and click on the “Import records from an external file” icon.<br /><br />This will let you choose a “CSV” file for import. Before you choose “table.csv” open it up and remove the first line with the headers (i.e. Date, Open, High, etc etc). If you don’t, this will also get imported (there are ways around this, but I’m keeping this intro simple). Now you can choose your “table.csv” file for import. It’ll take a little bit to import and then should load up the data on screen.<br /> </li>
<li>Now you can run your first query. Go up to the icon with “SQL” and a little plus on it. It should be right below “File” at the top. This should open a new tab for SQL queries. Type the following and press “CTRL” + “ENTER” or click the “Execute” icon (a little lightning bolt).<br /><br />SELECT * FROM stocks.historicaldata<br /><br />This should return everything from your table. Not very useful, but it’s a start. Next try something like:<br /><br />SELECT * FROM stocks.historicaldata WHERE date > ‘2015-03-01’<br /><br />Try:<br /><br />SELECT * FROM stocks.historicaldata WHERE open > close<br /><br />I’m sure you can come up with some more on your own! There will definitely be more on this in the future.<br /> </li>
</ol>
These steps are only the very beginning. You can load multitudes of stock data into the table you’ve created (you may want to add a “Symbol” field to differentiate new stock symbols you add). There are many ways to analyze the data, and if you want you can even analyze it externally (say, in Python, C#, Java or even Excel).<br />
<br />
I’ll be posting more on how to continue on from here. In the meantime, sign up for the <a href="http://godelsmarket.com/" target="_blank">Gödel’s Market Newsletter</a>. Information not available on the blog is sent out through the newsletter.<a href="http://godelsmarket.com/" target="_blank"> Click here to sign up!</a><br />
<br />
Best of luck.<br />
<div>
<br /></div>
implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com5tag:blogger.com,1999:blog-3007964440702200850.post-71705332928731356932015-02-16T20:23:00.001-08:002015-02-16T20:23:22.235-08:00Interviewing the Quants: Marco Simioni and Nightly Patterns<div class="MsoNormal">
<span style="background: white; color: #222222; line-height: 115%;"><span style="font-family: inherit;">Welcome back to <i>Interviewing the Quants</i>. This will be
the second interview in the series. Many of you might know our guest from his trading
at Nightly Patterns (<a href="http://nightlypatterns.wordpress.com/">http://nightlypatterns.wordpress.com</a>).
His name is Marco Simioni and he’s agreed to give us greater insight into what
he does and who he is.<o:p></o:p></span></span></div>
<div class="MsoNormal">
<i><span style="background: white; color: #222222; line-height: 115%;"><span style="font-family: inherit;"><br /></span></span></i></div>
<div class="MsoNormal">
<i><span style="background: white; color: #222222; line-height: 115%;"><span style="font-family: inherit;">(Note: This interview took place via email and
has been formatted as an article.) <o:p></o:p></span></span></i></div>
<div class="MsoNormal">
<span style="background: white; color: #222222; line-height: 115%;"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span style="background: white; color: #222222; line-height: 115%;"><span style="font-family: inherit;">GM: What lead you to
quantitative trading?<o:p></o:p></span></span></div>
<div class="MsoNormal">
<span style="background: white; color: #222222; line-height: 115%;"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span style="background: white; color: #222222; line-height: 115%;"><span style="font-family: inherit;">MS: Well, my first
approach to quantitative trading was while still at university during my first
Statistics course. I was used to math, but probabilistic<span class="apple-converted-space"> </span>theory completely changed my view on
trading. I remember before that course how focused I was on Elliot and Gann
theories. Both work, but what I was lacking<span class="apple-converted-space"> </span>was
an objective approach to trading. I switched from one oscillator to another,
indicators and classical technical analysis patterns. But I was still<span class="apple-converted-space"> </span>looking for a more systematic way to
trade. Finally I came across Michael Harris and Larry Connors' research.<o:p></o:p></span></span></div>
<div class="MsoNormal">
<span style="font-family: inherit;"><span style="background: white; color: #222222; line-height: 115%;"><br /></span></span></div>
<div class="MsoNormal">
<span style="font-family: inherit;"><span style="background: white; color: #222222; line-height: 115%;">GM: What's your background
academically?</span><span style="color: #222222; line-height: 115%;"><o:p></o:p></span></span></div>
<div class="MsoNormal">
<span style="font-family: inherit;"><span style="background: white; color: #222222; line-height: 115%;"><br /></span></span></div>
<div class="MsoNormal">
<span style="font-family: inherit;"><span style="background: white; color: #222222; line-height: 115%;">MS: I
studied Mechanics at high school. Instead of going on with
engineering at university I changed my mind and started Economics. I got
a Bachelor Degree in Marketing and Management at Cà
Foscari University with a thesis in Statistics focusing on Auto Regressive
Integrated Moving Average processes. I then specialized with a
Master Degree in Economics and Finance with quantitative and risk
management oriented studies in the same University with a thesis
on a historical analysis of first days market behavior
of tech IPO's. All of this apart from my more than ten years of trading
studies...</span></span></div>
<div class="MsoNormal">
<span style="color: #222222;"><span style="line-height: 18.3999996185303px;"><br /></span></span></div>
<span style="font-family: inherit;"><span style="color: #222222; line-height: 115%;"><span style="background: white;">GM: How long have you been trading? What caused
you to choose overnight trading?</span></span></span><br />
<span style="background: white; color: #222222; line-height: 18.3999996185303px;"><br /></span>
<span style="background: white; color: #222222; line-height: 18.3999996185303px;">MS: I've been trading since 2006. I am 28 years old. Despite my quantitative overnight trading I do use many other strategies too. I'm a fan of ETF rotational<span class="apple-converted-space"> </span>strategies, VIX ETP's quantitative systems, and mean reverting stock swing trading.<span class="apple-converted-space"> </span>My academic background led me through Graham fundamental studies, Greenblatt's formula, and O'shaughnessy's portfolios.</span><span style="color: #222222; line-height: 18.3999996185303px;"> <span style="background: white;">I became an overnight trading researcher mainly because it’s uncorrelated to other strategies I use, which results in greater portfolio diversification. And<span class="apple-converted-space"> </span>it seemed nearly nobody cared about its profitability at all.</span></span><br />
<span style="font-family: inherit;"><span style="color: #222222; line-height: 115%;"><span style="background: white;"><br /></span></span></span>
<span style="font-family: inherit;"><span style="color: #222222; line-height: 115%;"><span style="background: white;">GM: Which principles do you find most useful?</span></span></span><br />
<span style="color: #222222; line-height: 18.3999996185303px;"><br /></span>
<span style="color: #222222; line-height: 18.3999996185303px;">MS: </span><span style="background: white; color: #222222; line-height: 18.3999996185303px;">I believe there are two core market principles: a mean reverting regime and a trend following regime. Here sometimes financial academics studies are preceded by<span class="apple-converted-space"> </span>traders' experiences.</span><span style="color: #222222; line-height: 18.3999996185303px;"> </span><span style="background: white; color: #222222; line-height: 18.3999996185303px;">The first more intuitive effect deals with overbought and oversold market conditions. Nearly 70% of all my patterns are mean reverting. I'm constantly<span class="apple-converted-space"> </span>researching momentum patterns too, but they are much more difficult to discover. I believe momentum market studies make the difference, because they<span class="apple-converted-space"> </span>can prevent false or losing mean reverting trades. So many systems do not work properly mainly because they are not able to detect the differences between these<span class="apple-converted-space"> </span>two key market principles.<span class="apple-converted-space"> </span></span><br />
<span style="font-family: inherit;"><span style="color: #222222; line-height: 115%;"><span style="background: white;"><br /></span></span></span>
<span style="font-family: inherit;"><span style="color: #222222; line-height: 115%;"><span style="background: white;">GM: Are there any ideas that you find yourself
going back to again and again? If so, what are they?</span></span></span><br />
<span style="color: #222222; line-height: 18.3999996185303px;"><br /></span>
<span style="color: #222222; line-height: 18.3999996185303px;">MS: </span><span style="background: white; color: #222222; line-height: 18.3999996185303px;">Money management strategies. I'm referring to Kelley's formula, Vince works, Thorpe's studies, Larry Williams's or again Turtles' approaches. I could add<span class="apple-converted-space"> </span>many other equally brilliant researchers like Ziemba, Lundstrom and many others and still I would miss someone.<span class="apple-converted-space"> </span>I believe whatever system or strategy you trade, money management is vital to survive the markets and deal with high leverage.<span class="apple-converted-space"> </span>I'm constantly looking for a better position sizing technique, as I believe something more is still out there to be found.</span><br />
<span style="font-family: inherit;"><span style="color: #222222; line-height: 115%;"><br /></span><span style="color: #222222; line-height: 115%;"><span style="background: white;">GM: Why did you decide to create the <a href="https://nightlypatterns.wordpress.com/" target="_blank">NightlyPatterns blog</a>?</span></span></span><br />
<span style="font-family: inherit;"><span style="color: #222222;"><span style="line-height: 18.3999996185303px;"><br /></span></span></span>
<span style="color: #222222; line-height: 18.3999996185303px;">MS: </span><span style="background: white; color: #222222; line-height: 18.3999996185303px;">Trading successfully is difficult. I remember a study from Taiwan stock exchange that highlight the % of successful traders from their account’s net results.<span class="apple-converted-space"> </span>It seems that 95% of futures traders after 6 months end burning up all their starting equity. It takes years of researching and testing to build a consistent<span class="apple-converted-space"> </span>and successful trading strategy. And when a trader succeeds with it, he keeps it a secret.</span><span style="color: #222222; line-height: 18.3999996185303px;"> </span><span style="background: white; color: #222222; line-height: 18.3999996185303px;">I started Nightly Patterns blog as a free trading diary, and then I decided to move it to a more professional service. There are so few really successful futures<span class="apple-converted-space"> </span></span><span style="color: #222222; line-height: 18.3999996185303px;">t</span><span style="background: white; color: #222222; line-height: 18.3999996185303px;">rading rooms that I decided to enter the business.<span class="apple-converted-space"> </span></span><br />
<span style="font-family: inherit;"><span style="color: #222222;"><span style="line-height: 18.3999996185303px;"><br /></span></span><span style="color: #222222; line-height: 115%;"><span style="background: white;">GM: What is something most people might find
surprising about you or your trading style?</span></span></span><br />
<span style="color: #222222; line-height: 18.3999996185303px;"><br /></span>
<span style="color: #222222; line-height: 18.3999996185303px;">MS: </span><span style="background: white; color: #222222; line-height: 18.3999996185303px;">Many traders and bloggers are surprised by my young age. Volatility has kept increasing during the last ten years. Huge algo-trading with “robot” volumes, high<span class="apple-converted-space"> </span>tech 2000 end bubble, subprime crisis, commodities crashes, sovereign debt crisis, zero interest behavior environment and currency wars<span class="apple-converted-space"> </span>completely changed many old market dynamics. Despite my ten year experience, I believe more in quantitative back testing than a thirty or forty year market<span class="apple-converted-space"> </span>experience. This is my "quant" view.</span><br />
<span style="background-color: white; color: #222222; font-family: inherit; line-height: 115%;"><br /></span>
<span style="background-color: white; color: #222222; font-family: inherit; line-height: 115%;">GM: Are there any
suggestions you would give to an aspiring trader?</span><br />
<span style="background-color: white; color: #222222; font-family: inherit; line-height: 115%;"><br /></span>
<span style="background-color: white; color: #222222; font-family: inherit; line-height: 115%;">MS: Just read
whatever you can about trading before trading. Do not trade first. Just read.
Read a lot. Don't get lost in the academic finance research,
but don't get lost in all those unhelpful trading books out there
either. Read one academic paper from a university teacher on markets
and investments and then one trading book from a trader. Do this again
and again. Look for your own trading style. Take something from all the
books and papers you read and collect everything. Then read blogs and all the
website can teach you something. Your trading success will come to you before
trading. And of course try to follow some well established traders and mirror
them, you will survive the markets.</span><br />
<span style="background-color: white; color: #222222; font-family: inherit; line-height: 115%;"><br /></span>
<span style="background-color: white; color: #222222; font-family: inherit; line-height: 115%;">GM: Thank you for sharing
with us today. Do you have any last thoughts?</span><br />
<span style="background-color: white; color: #222222; font-family: inherit; line-height: 115%;"><br /></span>
<span style="background-color: white; color: #222222; font-family: inherit; line-height: 115%;">MS: My last thoughts go to
consistency and systems diversification.</span><span class="apple-converted-space" style="background-color: white; color: #222222; font-family: inherit; line-height: 115%;"> </span><span style="background-color: white; color: #222222; font-family: inherit; line-height: 115%;">If
you have a system, follow it. This is vital. Follow your system, not your
gut. Do not cherry pick your strategies; this will lead you to bankruptcy.
You will find yourself taking the worst trades and missing the best
ones. If you have a winning system just trade it but always look for
another one to add to your trading arsenal. This will lead your results to
approach Shannons' Demon Effect. I wish you all a good trading. Thank you very
much for your attention.</span><br />
<span style="background-color: white; color: #222222; font-family: inherit; line-height: 115%;"><br /></span>
<span style="background-color: white; color: #222222; font-family: inherit; line-height: 115%;">GM: Thank you again Marco.
Thank you everyone else for reading, and please visit him at </span><a href="https://nightlypatterns.wordpress.com/" style="background-color: white; font-family: inherit; line-height: 115%;" target="_blank">Nightly Patterns</a><span style="background-color: white; color: #222222; font-family: inherit; line-height: 115%;">.</span><br />
implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com1tag:blogger.com,1999:blog-3007964440702200850.post-51469924308697633782015-01-11T22:20:00.000-08:002015-01-11T22:20:22.791-08:00Interviewing the Quants: An Inside Look With “Sanz Prophet”<div class="MsoNormal">
<span style="font-family: inherit;">Our first guest for the “Interviewing the Quants” series is
Vangelis Maderakis. Many of you probably know him as Sanz Prophet through the
eponymous blog (</span><a href="http://sanzprophet.blogspot.com/">sanzprophet.blogspot.com</a><span style="font-family: inherit;">). Recently, he became a partner at <a href="http://www.logical-invest.com/" target="_blank">Logical Invest</a>,
a successful business dedicated to logical, rules based investing.</span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">I’ve asked him if he would answer some questions, hopefully giving
readers in the quantosphere greater insight into the mind of an online quant. </span></span><span style="font-family: inherit;">So, even before we begin, please thank Vangelis Maderakis for
joining us today!</span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<i><span lang="EN-GB"><span style="font-family: inherit;">(Note: This interview takes place via
email and has been formatted as an article.)<o:p></o:p></span></span></i></div>
<div class="MsoNormal">
<i><span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></i></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">GM: What experience did you have with finance before you started?<o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">VM: Although I did start as an engineer and
hold a B.A. in Economics I ended up following my passion at the time, filmmaking.
I have trained as a director/producer and have directed commercials for a
living. I started trading and managing money during the 2007 crisis and
discovered I like the quant side of it. I have been developing systems and
trading ever since.<o:p></o:p></span></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">GM: How do you develop most of your ideas?<o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">VM: Reading books, research papers and blogs.
I like to backtest all kinds of ideas. From obscure fx strategies to option strategies
that sound ‘logical’ but do not stand up to scrutiny. At one point you realize
the market is just a sum of all participants and will adapt onto itself. Then
you start asking simple questions: How will the rise of Wealthfront and other
similar services affect the market? Or as a private investor what are my
advantages vis-à-vis the larger players or even ETF issuers? <o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">I believe there are good ideas to be found in the public sphere. Like
in many areas of life the difficulty is not in finding the idea. It is in
actually deploying it into the real world. Many people will stop at the first
stage.<o:p></o:p></span></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">GM: What inspired you to write a blog about finance?<o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">VM: The quant blogging community. People like Michael Stokes of
MarketSci who was not just enlightening but also entertaining. I also felt the
need to reach out to other independent minded investors and bloggers and
exchange ideas.<o:p></o:p></span></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">GM: What is the most essential or fundamental lesson you've
learned from the markets?<o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">VM: To have a plan or strategy. To simplify. To be patient. To
think outside the box. To try not to lose rather than trying to win. Also I
have learned that markets are counter-intuitive and extremely unjust. It’s like
flirting. Trying too much does not guarantee success. You need to be open
minded and flexible. You need to care. But you cannot be desperate.<o:p></o:p></span></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">GM: What is your favorite type of market and why?<o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">VM: A sideways moving market. From a risk management perspective, a
bull market is as challenging as a bear market. A system developer who is
conservative will likely underperform a straight bull run because they need to
protect the account from a possible unexpected event. <o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">In a sideways market one can trade with less risk since the
average participation time is usually low. So from a risk management point of
view, a side market can be desirable given one can find the system to exploit
it.<o:p></o:p></span></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">GM: What are ideas you find yourself returning to again and again?<o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">VM: Under-deployment of funds and mismanagement of the
‘big-picture’. That is my most common concern.<o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">A private investor will likely split up their available capital
and buy different kinds of assets, strategies, individual stocks, etc. It’s a
natural instinct to diversify and not put all eggs in one basket. The problem
is that there is no easy way to tell how her whole portfolio behaves. Is it
truly diversified? Is she deploying all available capital or risking too much
in small risky instruments while the bulk is sitting idle in cash? <o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">So what I find myself looking for is a ‘total’ strategy that
supersedes sub-strategies and can track and adaptively allocate to them. A
meta-strategy of strategies. That is one of our goals at Logical Invest.<o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">The other idea is more philosophical. I find myself thinking that
in trading as in life, the main truly scarce and non-tradable asset is time.
Since investment is really a compounding game, any mediocre young investor will
most likely outperform by a large margin anyone who is double his age. Of
course the young investor doesn’t know this. So the older person needs to guide
him.<o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">GM: So that the younger person is able to utilize those extra years
wisely?<span style="color: red;"><o:p></o:p></span></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="color: red;"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">VM: Yes. It would be smart to invest early on but one doesn’t even
need to be that smart an investor. I would argue that on a 30+ year timeline
being mediocre is sufficient. The assumption, of course, is that the money
survives the course of unpredictable foolish human passions, socio-political
changes and all the things that hopefully our kids will get to experience that
are more interesting than money</span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">.<o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">GM: How did you come to join <a href="http://www.logical-invest.com/" target="_blank">Logical Invest</a>?<o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">VM: As a quant blogger, I was exchanging ideas with Alex and
Scott. Part of those ideas were testing Frank Grossmann’s strategies.
Eventually we met Frank and he invited us to join Logical. So over at Logical, we
still develop strategies but now we can be challenged and disciplined into a
steady flow of production. It’s quite exciting. We feel we are operating in the
fringes of the financial world offering institutional+ level strategies for
$30. It’s quite outrageous, really. Some of our subscribers are wealth
managers. Recently a lot of physicians have joined. So you can say that at this
point we cater to self-minded, educated individuals. <o:p></o:p></span></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">GM: Do you have any final thoughts for readers?<o:p></o:p></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">VM: A quote by Dennis Richards:</span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">“I don’t think trading strategies are as
vulnerable to not working if people know about them, as most traders believe.
If what you are doing is right, it will work even if people have a general idea
about it. I always say you could publish rules in a newspaper and no one would
follow them. The key is consistency and discipline.”</span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;"><br /></span></span></div>
<div class="MsoNormal">
<span lang="EN-GB"><span style="font-family: inherit;">GM: Thank you very much for sharing with us
</span></span><span lang="EN-GB"><span style="font-family: inherit;">Vangelis
Maderakis. Please visit him at <a href="http://www.logical-invest.com/" target="_blank">Logical Invest </a>and his blog <a href="http://sanzprophet.blogspot.com/" target="_blank">Sanz Prophet</a>.</span> </span></div>
implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com0tag:blogger.com,1999:blog-3007964440702200850.post-37516856415717294252014-12-30T23:54:00.000-08:002014-12-30T23:54:46.891-08:00Free Full Version of Microsoft Visual Studio 2013It’s been a couple months since Microsoft released the professional version of Visual Studio 2013 for free under the title Community Edition. Meant for non-enterprise environments, it still has all the features of the professional version and is just as powerful.<br /><br />With <a href="http://quantconnect.com/" target="_blank">QuantConnect</a> just about to open sourcing their Lean engine (sign up for a hobbyist account to <a href="https://www.quantconnect.com/blog/open-source-algorithmic-trading-platform/" target="_blank">support this</a>!), it’s the perfect time to update and prepare your development environment. <br /><br /><div>
VS 2013 CE is somewhat hidden away on Microsoft's site. In order to download a copy go to <a href="http://www.visualstudio.com/en-us/downloads/">http://www.visualstudio.com/en-us/downloads/</a> and scroll down to “Community 2013 with Update 4”. Click on the title of the purple bar and go to “Install Now” (<a href="http://go.microsoft.com/?linkid=9863608">http://go.microsoft.com/?linkid=9863608</a>). Download and install, and you’re on your way to a professional dev environment!</div>
implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com0tag:blogger.com,1999:blog-3007964440702200850.post-5250465956685114772014-12-09T23:20:00.007-08:002014-12-09T23:23:38.877-08:00Barbell Strategy Backtest<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><a href="http://quantconnect.com/" target="_blank">QuantConnect </a>is working towards the support they need to open source their code. </span><span style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;">To promote their open source campaign, I’ll be converting a simple version of the barbell strategy into a QuantConnect backtest. If you want to run the code without reading anymore, feel free to sign up at QuantConnect and <a href="https://www.quantconnect.com/forum/discussion/371/barbell-investing" target="_blank">clone my code</a>! And if you enjoy QC, consider $10/mo a bargain for contributing to the next generation of backtesting. I personally can't wait to start looking at their code :)</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Onto the article!</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The following is a simple take on the idea presented over at “<a href="http://dontfearthebear.com/2014/07/02/barbell-investing-with-xiv-svxy/#comments" target="_blank">Don’t Fear the Bear</a>” (DFB from now on). Basically, we’re going to place approximately 20% of our portfolio in a high risk instrument and place the other 80% into a low risk instrument. DFB suggested XIV as the high risk instrument and cash as the low risk instrument. We’ll be doing the same.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">I suggest you clone my backtest on QC and play around with the dates, symbols and various other values to get an idea of how the code works. If you want to generate the backtest code from scratch (i.e. not clone mine), you can clone QC’s “Basic Template.” All of the barbell strategy code was derived from a “Basic Template” clone.</span></div>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">We’ll be using the following line to tell QC which symbol we want to trade:</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">public string symbol = "XIV";</span></div>
<br />
<span style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;">Now, wherever you see “MSFT” replace it with the word <i>symbol</i>. Do not use quotation marks. For example:</span><br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">AddSecurity(SecurityType.Equity, "MSFT", Resolution.Minute);</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">would become:</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">AddSecurity(SecurityType.Equity, symbol, Resolution.Minute);</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">What this replacement does is inserts whatever value we have symbol set to in each location. This will make it easier to change the symbol value if you ever wanted to. Instead of updating each of the locations, you can just update the line </span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: italic; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">public string symbol = “XIV”;</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> to </span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: italic; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">public string symbol = “SPY”; </span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">and the backtester will know you want to use “SPY” instead of “XIV”.</span></div>
<b style="font-weight: normal;"><br /></b>
<span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;">Changing the amount within the </span><span style="font-family: Arial; font-size: 15px; font-style: italic; vertical-align: baseline; white-space: pre-wrap;">setCash();</span><span style="font-family: Arial; font-size: 15px; vertical-align: baseline; white-space: pre-wrap;"> method call determines how much money you want the strategy to be tested with. I set mine to 100000. This doesn’t have to be changed, but it gives you an idea of how to change this value. Try starting with other values, if you’d like.</span><br />
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Next is the code that does contains all of the logic for our system. Two slashes tell the backtester to ignore those lines. They are called comments, and I’ve left you with several to give you an idea of what is going on.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Continue on after the code for more commentary.</span></div>
<b style="font-weight: normal;"><br /></b>
<b style="font-weight: normal;">======= Portion of Main.cs ===========</b><br />
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">//only get daily close</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> if (data[symbol].Time.ToString().Contains("3:59:00")) {</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"><span class="Apple-tab-span" style="white-space: pre;"> </span></span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">//prints out to console window</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> Debug(data[symbol].Time.ToString());</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> decimal currentPercentage = (Portfolio.TotalHoldingsValue / Portfolio.TotalPortfolioValue);</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> Debug("CurrentPercentage: " + currentPercentage);</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> //reconfigure for 20/80 holding; but give a little room (15%)</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> if (currentPercentage < (decimal).15 ){</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> //buy</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> decimal amtToBuy = ((((decimal).2 - (currentPercentage)) * Portfolio.TotalPortfolioValue)/data[symbol].Close);</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> amtToBuy = Math.Round(amtToBuy);</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> Order(symbol, amtToBuy);</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> Debug("Puchased " + amtToBuy + "of " + symbol);</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> return;</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> }</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> </span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> //reconfigure for 20/80; but give a little room (25%)</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> if (currentPercentage > (decimal).25) {</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> //sell</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> decimal amtToSell = (((currentPercentage - (decimal).2) * Portfolio.TotalPortfolioValue)/data[symbol].Close);</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> amtToSell = Math.Round(amtToSell);</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> Debug("Sold " + amtToSell + "of " + symbol);</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> Order(symbol, -1* amtToSell);</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> return;</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> }</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> //we don't care if it's not end of day, so return;</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> } else {</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> return;</span></div>
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> }</span></div>
<b style="font-weight: normal;"><br />==================</b><br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The first thing you’ll notice is that we’re only looking at minute bars that come from the end of day. That’s what the “3:59:00” check is for. This can be achieved several ways, but this way is fairly easy to understand. If it’s 3:59 PM we can go ahead and continue with our logic.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Anywhere you see “Debug();” is an easy method to print to QC’s console. Very helpful when you’re trying to figure out what is going on with your code!</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Next we’ll check the percentage of our holdings that our in XIV. We want this to be 20% at the start. And, we want to rebalance this using some sort of logic. I have chosen to rebalance when XIV gets to 15 or 25% of holdings. You can play with this number. If you’re feeling adventurous, try to rebalance only at the beginning of the month. :)</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">You’ll see amtToBuy and amtToSell, which are both variables holding the value created by the statements that follow. All I’m asking is for the backtesting program to determine how many shares (rounded to an integer) I need to buy or sell to bring myself back inline with our 20% goal. Nothing too complicated, although I do have to use the (decimal) cast in order to force .2 from float into decimal. Decimal is what’s used typically in the backtesting program (it’s good for money related activities like stocks).</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">After we find out how much we want to buy or sell, we can put in an order. This is as easy as using the Order(); method; which, is really, really easy. Just make sure your sell amount is negative. That is how QC knows the order is a sell.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">I hope the backtest code is otherwise fairly clear: </span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: italic; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">data[symbol].Close</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> gives the price of the symbol you are following. And, </span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: italic; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Portfolio.TotalHoldingsValue</span><span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;"> gives the value in dollars of your the symbol you own. Portfolio.TotalPortfolioValue lets you know how much your entire portfolio (cash included) is worth.</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; color: black; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">The outcome of the backtest isn't amazing, but it is a good start to your backtesting portfolio. Feel free to post questions and other backtests in the comments!</span></div>
<b style="font-weight: normal;"><br /></b>
<br />
<div dir="ltr" style="line-height: 1.15; margin-bottom: 0pt; margin-top: 0pt;">
<span style="background-color: transparent; font-family: Arial; font-size: 15px; font-style: normal; font-variant: normal; font-weight: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;">Also, please consider supporting <a href="http://quantconnect.com/" target="_blank">QuantConnect’s</a> drive to open source its code. Becoming a QC member is a small price for access to an amazing product! Just remember, the more people who have access and can work on it, the better the code will become!</span></div>
implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com0tag:blogger.com,1999:blog-3007964440702200850.post-17650666384727769312014-11-22T22:40:00.000-08:002014-11-22T22:40:11.985-08:00Unlimited Free Backtests with Large Tick Data Source<span id="docs-internal-guid-fa2154f0-db5b-d2f0-c0bb-223d7f19da23"><span style="font-family: Arial; font-size: 15px; line-height: 1.15; vertical-align: baseline; white-space: pre-wrap;">There are other services out there that allow free backtests. However, <a href="http://quantconnect.com/">QuantConnect </a>is the only service I know of that allows unlimited free backtests with </span><span style="font-family: Arial; font-size: 15px; font-style: italic; font-weight: bold; line-height: 1.15; vertical-align: baseline; white-space: pre-wrap;">tick</span><span style="font-family: Arial; font-size: 15px; line-height: 1.15; vertical-align: baseline; white-space: pre-wrap;"> data. Let me say that again: they have </span><span style="font-family: Arial; font-size: 15px; font-style: italic; font-weight: bold; line-height: 1.15; vertical-align: baseline; white-space: pre-wrap;">tick </span><span style="font-family: Arial; font-size: 15px; line-height: 1.15; vertical-align: baseline; white-space: pre-wrap;">(i.e. time and sales) data going back to 1998--something around 4TB of data. And, if you prefer, or want to utilize it along side the tick data, they also have minute and second data.</span></span><div>
<span><span style="font-family: Arial;"><span style="font-size: 15px; line-height: 17.25px; white-space: pre-wrap;"><br /></span></span><span style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;">QuantConnect even allows you to pull in the backtest data using their easy to use Rest API, or utilizing their (currently in beta) Visual Studio plugin. If you decide to utilize the backtesting portion of the Rest API more than 5 times a day, it’s a relatively cheap upgrade to unlimited api backtests: 19.95. You can always switch back to their Web IDE for continued unlimited backtesting.</span></span></div>
<div>
<span style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span><span style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;">Plus, even better, Visual Studio 2013 Community Edition has just been released. It has all the features of the Professional Edition for use with small teams and for personal use. As mentioned above, QuantConnect has a plugin (currently in beta, and <a href="https://github.com/QuantConnect/">open sourced on github</a>) that allows you to easily develop strategies directly from Visual Studio. Given Visual Studio is easily one of the best IDEs on Windows, this is beyond a blessing; and, definitely worth going along for the “beta” plugin ride. This is not available through other services that typically require API access through the browser. Although, I won't forget to mention that QC also has a nice browser-based IDE.</span><br /><span style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;">At a bare minimum, I would recommend signing up for free and trying out their web IDE. They use C# which is considerably more powerful than some of the languages used on other backtesting sites. This power comes in handy when you’re dealing with many, many data points and complex algos. It’ll keep things snappy while data is massaged across a number of cloud servers, allowing you to have your data points crunched in no time. And, if you’re new to C#, QC also has good documentation to help you along.</span><span><br /></span><span style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: Arial; font-size: 15px; line-height: 1.15; white-space: pre-wrap;">There are also many innate features (e.g. ability to have multiple files in a project) that you will come to appreciate as you delve into their system. They are adding things and upgrading frequently. I continue to expect more in the coming weeks and months. They have good documentation and are easy (and free) to sign up with. Here's another link to their site: <a href="http://quantconnect.com/">QuantConnect.com</a>.</span><br /><span style="font-family: Arial; font-size: 15px; white-space: pre-wrap;"><br /></span></div>
<div>
<span style="font-family: Arial; font-size: 15px; white-space: pre-wrap;">Please, feel free to post any backtests you run in the comments; I’d be interested to see!</span></div>
implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com1tag:blogger.com,1999:blog-3007964440702200850.post-62662811253725495732014-11-20T14:16:00.002-08:002014-11-20T14:16:37.077-08:00Gödel's Market on GithubThere are some snippets of useful code on this blog that could transform into even more useful code if placed in a more conducive environment. I've created a Github account for blog related code here: <a href="https://github.com/godelsmarket">https://github.com/godelsmarket</a>.<br />
<br />
Right now I've only had the time to put up a repository for the IB Historical Data Downloader. I had to clean up the code formatting a little bit and transfer over the article's content to a readme to make things a bit clearer.<br />
<br />
I'd really like to get all the code from the site transferred over. Then I would like to make it more user-friendly so that individuals who do not have as much experience with programming can get their feet wet. That's the goal :)<br />
<br />
If you have any questions on how to make a piece of code do something more interesting than it does, please ask. These are basic starting points. I love to help, but I'd rather not just give all the answers right up front. For one, it's no fun that way (check out <a href="http://projecteuler.net/">Project Euler</a> and tell me, with a straight face, that it would be fun to have the answers given to you...). Second, you'll learn far more trying to develop your own program, which will give you more confidence to create more complicated things. Third, if you want it all done for you, I'm available for hire. :)<br />
<br />
Now, back to the data.implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com4tag:blogger.com,1999:blog-3007964440702200850.post-74836311942534311332014-06-15T19:22:00.002-07:002014-06-15T19:24:37.990-07:00Average Return by Trading Day of Month<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjiOa06iqmOri2ia0VFlF1QU12qKAt_JKPYBYEt_NSEytuV9VOiDhWSMwUqn12kICnifO7xq-FYYtBYpdgw0y3EasJNgfycJXZFbYl5ceIpv-EXb0GG7gTi-_n_H3erY9EABahiCZEzDCLE/s1600/AverageReturnByTradingDayOfMonth.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjiOa06iqmOri2ia0VFlF1QU12qKAt_JKPYBYEt_NSEytuV9VOiDhWSMwUqn12kICnifO7xq-FYYtBYpdgw0y3EasJNgfycJXZFbYl5ceIpv-EXb0GG7gTi-_n_H3erY9EABahiCZEzDCLE/s1600/AverageReturnByTradingDayOfMonth.PNG" height="640" width="571" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Often people will say that the first of the month tends to be positive. By the above analysis (SPY from 07/01/2000 to 06/13/2014), that has--on average--been the case. You could also say go long for the 9th trading day of the month and short for the 14th. And if there are 23 trading days in the month, go long the close of the day before--well, if it happens to be in August.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Good luck and good trading. Please do not use the above for trading. Prepare your own analyses (the above could be flawed) and do in-depth investigations. I am not giving financial advice. I'm merely playing with numbers.</div>
<br />implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com4tag:blogger.com,1999:blog-3007964440702200850.post-90911794827254701062014-06-14T17:08:00.003-07:002014-06-14T17:08:39.176-07:00RegenerationIt's been more than a year since I last posted on this site, and a busy year for me. I've been trading regularly over the last four months or so, and have decided to get back into the blogging world.<br />
<br />
To start with, I've updated the links on the right to better reflect the state of the blogosphere, with blogs no longer posting placed into the "Of Interest, Archived" category.<br />
<br />
More to come.implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com0tag:blogger.com,1999:blog-3007964440702200850.post-34293322344883196012013-02-05T21:45:00.001-08:002013-02-07T13:50:24.769-08:00EURUSD Average Monthly GainI've been a little interested in foreign exchange pairs recently. They have a different flavor of trading from equities or derivatives. In any case, I ran the following test on data going back to <strike>1990</strike> November 1998. It looks at the opening of the month compared to the month's close (log changes).<br />
<br />
I highlighted some of the more interesting months. Looks like December has a nice strong long bias to it; much more consistent direction than any of the other months at 80% positive. Of course, I wouldn't trade any of this by itself.<br />
<br />
<div style="text-align: center;">
EDIT: (new chart using IMF.org's data <a href="http://www.imf.org/external/np/fin/ert/GUI/Pages/Report.aspx?CU='EUR'&EX=REP&P=DateRange&Fr=628929792000000000&To=634952736000000000&CF=UnCompressed&CUF=Period&DS=Ascending&DT=NA">found here</a>. Assumes purchase at first day of listed month and sale at first day of following month. This data goes back to November 1998 rather than 1990. Many thanks to Sanz Prophet for pointing at the mistake! Currently getting my hands on some better forex data! :D)<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEic7se0VlH_wXINTcauMOgUZSB0FI6U0gXKUypy4uokawR8bJz4cYmzJIkKtmi06TScrnzNnvhm6LWmmGh73B4D33bf1bkccDga6pdayatqPW0uUEQM997GSb_dGNLo085JlVgh7OumyYH_/s1600/eurusd.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEic7se0VlH_wXINTcauMOgUZSB0FI6U0gXKUypy4uokawR8bJz4cYmzJIkKtmi06TScrnzNnvhm6LWmmGh73B4D33bf1bkccDga6pdayatqPW0uUEQM997GSb_dGNLo085JlVgh7OumyYH_/s1600/eurusd.png" /></a></div>
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<i>The above is not meant to be trading advice. Furthermore, past results are not necessarily indicative of future returns. And, as seen in the earlier version of this post, there is no guarantee that the data is accurate.</i></div>
<br />implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com8tag:blogger.com,1999:blog-3007964440702200850.post-85277772149164512152013-01-29T00:00:00.000-08:002013-01-29T01:18:09.212-08:00Weather's Effect on Stock Prices: Part OneYesterday, I thought it would be neat to check out weather's affect on stock prices. So, I looked around for some historical temperature data and found <a href="http://www.wunderground.com/">Weather Underground, Inc.</a> Despite their name, they do provide weather information for above ground locations. They also, luckily for us, provide historical data in easy to download CSV format.<br />
<br />
So, I wrote this nifty Python script to help us download them.<br />
<pre style="background-image: URL(https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTJ3bZzZO44dRy268rcDj_umguWHIEwf1vDFD_kb9RqoCZIK1ZhNR7afgVhqspxb8xNT2RkJoEIcBzEzVGorb2JZjj6Z_Yyeh_ZPfe_7aS_XKfwWA_P9ZGd7mIUX5zOanS8FX11gnEKkIH/s320/codebg.gif); background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: 200px; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">import urllib
month = 1
year = 2000
while year < 2013:
while month < 13:
urllib.urlretrieve("http://www.wunderground.com/history/airport/KNYC/" + str(year) + "/" + str(month) + "/1/MonthlyHistory.html?req_city=NA&req_state=NA&req_statename=NA&format=1", "csvfiles/" + str(year) + "-" + str(month) + ".csv")
month = month + 1
month = 1
year = year + 1
</code></pre>
<br />
That should download data back to 2000. I checked on their data for 1990 and they didn't seem to have it. Furthermore, I'm not entirely sure of the accuracy of this data.<br />
<br />
This concludes Part 1. Yes, no analysis yet; but, half the fun of analyzing data is getting your hands on it in a format that you can process. Feel free to upload this data into a SQL database. I'll provide some data on the market's tendencies before and after snow, rain, temperature correlation, etc, in a few days.implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com0tag:blogger.com,1999:blog-3007964440702200850.post-11678289157098372522012-12-24T14:45:00.004-08:002012-12-24T14:53:18.510-08:00Simple Automated Trading SystemI put together some Python code for an automated trading program for use with Interactive Brokers. It's very simple, but with a little ingenuity, could easily be expanded to include multiple contracts and various buying or selling triggers.
<br />
<br />
Sometimes it's nice to have a simple template to work from. Hope this helps someone get started with their very own automated execution program.
<br />
<br />
It uses the ibPy module and Python 2.7 (although it probably works on previous versions). I tried commenting the most important aspects.<br />
<br />
What it does is send you a quote of the last price of the 'YM' 20130315 contract as long as that contract is between 13070 and 13100. Furthermore, (assuming you remove the '##'s) it will make a purchase if the contract is above 13083 and a sale if the contract is below 13082.<br />
<br />
I recommend using a paper trading account while testing trade execution code. Also, the strategy within the code, if you were curious, has no quantitative backing. It's merely an example. The point is, you can add your own system in place of the one in the code.<br />
<br />
I tried to comment on everything important in the code. If anything doesn't work or doesn't make sense, let me know! I'll try to reply quickly.<br />
<br />
Here's the code:
<br />
<br />
<pre style="background-image: URL(https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTJ3bZzZO44dRy268rcDj_umguWHIEwf1vDFD_kb9RqoCZIK1ZhNR7afgVhqspxb8xNT2RkJoEIcBzEzVGorb2JZjj6Z_Yyeh_ZPfe_7aS_XKfwWA_P9ZGd7mIUX5zOanS8FX11gnEKkIH/s320/codebg.gif); background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: 400px; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">from ib.ext.Contract import Contract
from ib.ext.Order import Order
from ib.opt import ibConnection, message
from time import sleep, strftime, localtime
from datetime import datetime
## Global variables
shares = 3
action = 'BUY'
orderID = 30
last = 0
prev_last = 0
sym = 'YM'
## Contract Creation Function
def makeStkContract(sym):
contract = Contract()
contract.m_symbol = 'YM'
contract.m_secType = 'FUT'
contract.m_expiry = '20130315'
contract.m_exchange = 'ECBOT'
contract.m_currency = 'USD'
return contract
## Order Creation Function
def makeStkOrder(shares,action):
order = Order()
order.m_minQty = shares
# order.m_lmtPrice = limit_price
order.m_orderType = 'MKT'
order.m_totalQuantity = shares
order.m_action = str(action).upper()
return order
## Tick Handler
def my_tick_handler(msg):
global last
global prev_last
#print msg
if msg.field == 4:
prev_last = last
last = float(msg.price)
print msg
##Connect
con = ibConnection()
con.register(my_tick_handler, message.TickSize, message.TickPrice)
con.connect()
## Make your contract
stkContract = makeStkContract(sym)
## Request tick data
con.reqMktData(orderID, stkContract, '', False)
sleep(1)
print last
## Prints last price as long as between 13070 and 13100
while (last > 13070 and last < 13100):
print last
#### A possible execution plan. Remove '##' to initiate.
#### A paper trading account is recommended for testing and debugging trade execution code.
## if (last > 13083):
## action = 'BUY'
## stkOrder = makeStkOrder(shares,action)
## con.placeOrder(orderID,stkContract,stkOrder)
## elif (last < 13082):
## action = 'SELL'
## stkOrder = makeStkOrder(shares,action)
## con.placeOrder(orderID,stkContract,stkOrder)
sleep(.5)
##Stop receiving tick values
con.cancelMktData(orderID)
##Disconnect from TWS
con.disconnect()
</code></pre>
implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com2tag:blogger.com,1999:blog-3007964440702200850.post-9407377547043550532012-12-19T15:24:00.000-08:002012-12-19T15:24:02.860-08:00TV is DeadAs I have yet to figure out what to do about my quantitative writings, I am back to discussing the shortfalls of articles I find largely through Abnormal Returns. <div>
<br /></div>
<div>
<a href="http://sumzero.com/headlines/media/CBS/147-special-report-another-pillar-of-mass-media-collapses">Here's the article I want to bash today.</a> </div>
<div>
<br /></div>
<div>
While still pertinent to understand, acquiring such knowledge now will not put you ahead of the game. I'm guessing this type of article is just an easy throw-away mimic article, in the vein of the "OMG the newspaper is dying!" article. Of course, it's kind of hard to write about newspapers demise anymore. Most are dead and gone, killed by up to the second free reporting. When it comes to people searching for deals and coupons in the paper, amazon and groupon (and its many clones) provide much better services--for both customer and business.</div>
<div>
<br /></div>
<div>
But, the newspaper sensation was fun to write about. People liked to talk about it. Now, that it is nearly dead and everyone has heard about it, I guess media outlets are moving onto the next thing. The next thing to watch die. </div>
<div>
<br /></div>
<div>
You should've seen it coming ten years ago. And for the last five, online video advertising has been exploding in revenue and profit. Plus, most shows are online, and anyone with an average computer know-how can download the latest episode of anything that is not online (or ask a friend to do it for them).<br /><br />The internet has replaced television for many people, and it will continue to decline. Besides habit and comfort of use, there's really no reason to access content through the television anymore. There will be even less reason to do so in ten years.</div>
<div>
<br /></div>
implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com0tag:blogger.com,1999:blog-3007964440702200850.post-67544764722822401232012-12-14T11:36:00.000-08:002012-12-14T11:36:22.775-08:00AAPL to Apple? Or APPL to orange?I never really liked these kind of comparisons: (<a href="http://www.marketanthropology.com/2012/12/the-universal-law-of-gravatation.html">http://www.marketanthropology.com/2012/12/the-universal-law-of-gravatation.html</a>). He could be right, but I don't think he's showing any data that is statistically significant. It's an analogy, alright, but not something I'd necessarily want to trade directly off of.implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com0tag:blogger.com,1999:blog-3007964440702200850.post-47436275649374083122012-09-27T09:35:00.002-07:002012-09-27T09:36:17.826-07:00SPY Intraday Mondays<div class="separator" style="clear: both; text-align: left;">
I have been doing a little research into the peculiarities of certain days of the week. For instance, Monday's tend to have very little economic news released intraday and very few earnings release or reports. You might expect 2008 to have some large intraday turbulence on Mondays, given government reports released over the weekends. But this is not the case. In face, the largest down day on a Monday since 1993 has been approx -4.7% and took place during 2009. The largest up day is 4.3% and also took place during 2009.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Here's a chart displaying the return of $100,000 placed into the SPY at its open on Monday morning and taken out of the SPY at its Monday close. Rather uneventful.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7ESbaFSNJ9o50gC-aXm8XxXSDwvn6zFS-20xXrC9K3z5KteEqHxHsJZJTuZcDsVgQ64vPADAEzqbEngU6DbTk3fUNQmUl5tHJUyr7qo4Y06-bI6ZMttwyXHTBYaUH2NODgBEZ9SMnvAgd/s1600/spymondays.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="426" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg7ESbaFSNJ9o50gC-aXm8XxXSDwvn6zFS-20xXrC9K3z5KteEqHxHsJZJTuZcDsVgQ64vPADAEzqbEngU6DbTk3fUNQmUl5tHJUyr7qo4Y06-bI6ZMttwyXHTBYaUH2NODgBEZ9SMnvAgd/s640/spymondays.png" width="640" /></a></div>
<br />implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com0tag:blogger.com,1999:blog-3007964440702200850.post-84912483561562462722012-09-26T11:33:00.001-07:002012-09-26T11:34:06.437-07:00Minutes per Price ChartTo compliment yesterday's post, here's a chart displaying the minutes spent at each price level.<br />
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHlqYa87rmmMe8Ybk0C5s_6Q3V37bue-JWl5c7_rdTMGu0qZr3JtkUAXAswrxoP8YgkM4WbUn6C5e_gdoZ8a2i1cqmOu7MFeXhYmzOKCYqY8J9c9EsRFHjHI6JcBjYmjh8Nr-NU7JWsn7j/s1600/aapl_minutes_per_dollar.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="352" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhHlqYa87rmmMe8Ybk0C5s_6Q3V37bue-JWl5c7_rdTMGu0qZr3JtkUAXAswrxoP8YgkM4WbUn6C5e_gdoZ8a2i1cqmOu7MFeXhYmzOKCYqY8J9c9EsRFHjHI6JcBjYmjh8Nr-NU7JWsn7j/s640/aapl_minutes_per_dollar.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
(Note: This is for AAPL between 7/17/2012 and 9/21/2012)</div>
<div>
<br /></div>
implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com0tag:blogger.com,1999:blog-3007964440702200850.post-278413929431379712012-09-25T13:24:00.001-07:002012-09-26T14:03:46.308-07:00Using SQL to Generate Volume by PriceSometimes you may want to know at which dollar level the most trades have taken place in your favorite stock. Here's an easy way to do this in MySQL using a simple SQL query:
<br />
<br />
<pre style="background-image: URL(https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhTJ3bZzZO44dRy268rcDj_umguWHIEwf1vDFD_kb9RqoCZIK1ZhNR7afgVhqspxb8xNT2RkJoEIcBzEzVGorb2JZjj6Z_Yyeh_ZPfe_7aS_XKfwWA_P9ZGd7mIUX5zOanS8FX11gnEKkIH/s320/codebg.gif); background: #f0f0f0; border: 1px dashed #CCCCCC; color: black; font-family: arial; font-size: 12px; height: 90px; line-height: 20px; overflow: auto; padding: 0px; text-align: left; width: 99%;"><code style="color: black; word-wrap: normal;">SELECT ROUND(close,0), SUM(volume) FROM stocks.stock_prices_minute
WHERE SYMBOL = 'AAPL'
AND DATE(date) = '2012-09-21'
GROUP BY ROUND(close, 0)
</code></pre>
<br />
Above, 'close' is the name of the column with 1-minute (or whatever you have) close prices, 'volume' is the name of the column with volume data, stocks is the database, and stock_prices_minute is the table.<br />
<br />
Here's a graph of a similar query (I removed the date requirement in order to get all data within the period I had available).<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyWSae2Lacf5CPeS8wPKn4d-xYVJ9uS4AvlsE71G4EBImkSP0WGYN8sMlWncDPHeURstBrUzZXnkos0J4i0cQIqI_B85Ee2vnN5kWQnIarlofyrod0_Z8Snye0-a7J8q6CwQmNqy7uxWcW/s1600/aapl_1min_volume_by_price.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="369" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgyWSae2Lacf5CPeS8wPKn4d-xYVJ9uS4AvlsE71G4EBImkSP0WGYN8sMlWncDPHeURstBrUzZXnkos0J4i0cQIqI_B85Ee2vnN5kWQnIarlofyrod0_Z8Snye0-a7J8q6CwQmNqy7uxWcW/s640/aapl_1min_volume_by_price.png" width="640" /></a></div>
implied volatilityhttp://www.blogger.com/profile/13281497787825298368noreply@blogger.com0