Finding Stock Splits and Ex-Dividends Using R

The following post shows you how to check for any stock splits and ex-dividends happening. It'll spit out a list of symbols. These can be used as alerts or piped into another script for updating your database as needed.

I want to thank pcavatore for suggesting this topic! Comments and suggestions help drive the content of this site. Now, onto the explanation. If you're just interested in the code, skip to the bottom or check out the github repo.


1. Requirements. We'll be using the "XML" library. In R, it makes it easy to extract tabular data from webpages. Running the following code will install "XML" if it isn't already installed. Then it will load the library for use.

if (!require(XML)) install.packages('XML')
library(XML)

2. URL selection. We'll need some online location from which to pull the data. Yahoo! has easy to grab stock split info; Nasdaq.com has easy to grab dividend info. So we'll use those.

#for stock split
url <- "http://biz.yahoo.com/c/s.html" 

#or, for dividend information
url <- "http://www.nasdaq.com/dividend-stocks/dividend-calendar.aspx"

3. Download the data. We use the readHTMLTable function in order to grab the data.

tables <- readHTMLTable(url)

4. Extract desired data. The previous command will return more tables than you're interested in. Within those tables is also more data than we're interested in. So, we'll print out just the specific table and data column we'd like, now that the tables are stored in memory.

#for stock split data you only want the 6th table and the 3rd data column
tables[[6]][3]

#for the ex-dividend data you only want the "Table1" table and its 1st data column
tables[['Table1']][1]

5. The complete code. It can also be found on github, here.

if (!require(XML)) install.packages('XML')
library(XML) 

# Check for stock splits
url <- "http://biz.yahoo.com/c/s.html"
tables <- readHTMLTable(url)
tables[[6]][3] 

# Check for ex-dividend
url <- "http://www.nasdaq.com/dividend-stocks/dividend-calendar.aspx"
tables <- readHTMLTable(url)
tables[['Table1']][1]

6.  Conclusion. You should now be able to get a printout of any stock splits or ex-dividend dates occurring. This can be used to help update your database or keep you alerted to potential trading opportunities.

7. Further. The above outline can be used to download other tabular data. The basic outline is: find a site with tabular date, download that data, and then extract the tables and columns you need.

(If you've enjoyed the above article, consider signing up for the Gödel's Market Newsletter. Every signup encourages me to write more. Thank you!)

All the gaps need to be filled: Marco Simioni

It's an old Wall Street saying--"All the gaps need to be filled"--and is widely known among traders both for its accuracy and its occasional fallacy. It holds true for many market indexes and many stocks. I'm primarily focusing on the S&P 500 etf: SPY. Gaps in SPX are not as clear as in the SPY graph, which is why I will be referring to the SPY.

Consider up gaps to occur when today's opening price is higher than yesterday's high; and, consider down gaps to be those with today's open lower than yesterday's low. It is clear how all down gaps have been closed by the market so far. Most of the gaps take just a few days to be covered, but some take years. Like big holes or magnets they attract prices back to them. Let's talk first about unfilled down gaps.

Unfilled down gaps have shown the most consistent behaviour. All of them have been closed by next markets' bullish moves. A bull market can take years to recover and close a gap, however, all bear markets have resumed to new all time highs. This is mainly due to the general upside bias of the economy.

The main problem is not if a down gap will be closed, but WHEN. Entering big bear markets like 1929, 2001 or 2008 too early results in huge losses because the down gap can take a lot of time, potentially years, to be filled. Past stock market behaviour has shown down gaps filling 100% of times and this must be considered as a strong potential warning with short trades during even the -50% or more bear markets. The down trend will resume to cover all unfilled down gaps. Concerning long side investments, down gap filling acts as an insurance: waiting for the highest down gap to be filled is neither a waste of money nor of time.

Moving on to unfilled up gaps, different results arise from their analysis.

I collected all the unfilled up gaps in the table below going back to SPY inception in 1993. The first unfilled up gap occurred on February the 3rd 1995. And since that time, it is still not covered. We can count 20 other unfilled gaps from then to the present. And we can count about 20 years since the first of them occurred. Filling in the lowest unfilled up gap, as it appears from the backtest below, would mean the SPY would have to lose more than 75% from today's close at 207,5 (time of writing). We could argue that neither 2001 nor 2008 bear markets managed to fill those gaps, and those 20 old gaps won't fill at all. Never.

Since SPY inception 268 up gaps were filled, with only 21 not filled. 92,73% of times the market acted as expected to. Markets act with symmetry, and I believe these gaps to be filled in order to equal 100% accuracy of down gaps; we can't know the time, but they will. In 1995 there were not only 1 but 6 unfilled gaps, and all of them pointing south more than 70%. I think about them as the most dangerous 6 holes in the Wall.

(If you're interested in more articles like this, subscribe to our newsletter! Newsletter subscribers received priority access to this article and will receive unique articles in the future. Thank you!)

The was a guest article written by Marco Simioni of Nightly Patterns. Many thanks to him for his insights on gaps in the SPY.

How to Download Code from Any Public Github Repository

Before we get into any more coding, this post may be of use to some of you. The following guide will show you how to clone, i.e. download, any public repository on github.

If you find it useful, or if you like my other posts, please consider subscribing to Gödel's Market Newsletter. Every sign up is a sign of support!

1. Download and install git. Choose one of the following based on your operating system.

Windows: git-scm.com/downloads 

Mac OSX: git-scm.com/downloads 

Linux (ubuntu): sudo apt-get install git

2. Find a repository you want to download off of github. For instance, the code for our last post is at https://github.com/godelsmarket/stock_database_updater.

3. Open up Git Bash if you're using Windows or Mac OSX. If you're using Linux you can use a standard terminal, because that's basically what Git Bash is.

4. Type the following:

git clone https://github.com/godelsmarket/stock_database_updater.git git_code

You can change "git_code" to whatever name you'd like. It is the directory (and path) that will be holding the code.

5. You're done! You now have all the code in that directory for your use. Try downloading some other repositories.

In the future I'll show you how to create your own repository on Github and how to contribute to public repositories. It's really pretty simple.

If you enjoy these posts, consider subscribing to Gödel's Market Newsletter. It encourages me to write more!