Using machine learning to predict LendingClub loan defaults

This spring I took some time to test out the free open source machine learning kit Scikit-learn, or SKlearn, recycling a project from a Data Analytics class from my MBA program.

The basic challenge: Given a bunch of loan application data from thousands of loans, can you predict which loans will default and which will not?

My code is in GitHub located here: https://github.com/dforrestwilson/albums/blob/master/lendingclub.py

 

Using Excel, figuring out what columns were important and which were not was a long and painful process. Once I had a basic script in Python, I was able to use something called a decision tree classifier to determine that for me, with greater accuracy. You can learn more about that here: http://scikit-learn.org/stable/modules/tree.html

In 39 lines of (amateur) code you can go from some basic cleaned .csv data to visual tree graphs. It should take less than 1 minute to run. Powerful stuff, and what I love about coding data analytics solutions like this is you can reuse it for many types of classification and regression problems.

You can get the data here: https://www.lendingclub.com/info/download-data.action

Feel free to install Python along with Pandas and SKLearn and give it a shot yourself! I am sure there are things I could improve, but it picked up on the proper drivers without any prompt from me. SKLearn has a lot of interesting features that I am starting to wrap my head around.

 

 

Great Reads for Tech, Finance, and Wantrepreneurs

All of these sites offer most articles for free, at most requiring registration and limiting the number you can read per month. NYT, WSJ, and FT articles of interest can always be read for free by Googling the article name and clicking on the first result.

 

General

http://fivethirtyeight.com/

http://www.economist.com/

http://qz.com/

 

Finance and the Economy

http://ftalphaville.ft.com/

http://theirrelevantinvestor.com/

http://conversableeconomist.com

https://blogs.wellesley.edu/jjoyce/

http://www.pragcap.com/

http://econbrowser.com/

http://www.nakedcapitalism.com/

http://www.advisorperspectives.com/

http://macromarketmusings.blogspot.com/

http://www.priceactionlab.com/Blog/

http://moslereconomics.com/

http://angrybearblog.com/

http://www.economicpopulist.org/

http://money.visualcapitalist.com/

http://economistsview.typepad.com/timduy/ 

https://piie.com/

http://www.valueuncovered.com/

http://basehitinvesting.com/

http://www.jasonzweig.com/blog/

 

Tech and VC News/Blogs

https://slashdot.org/

https://news.ycombinator.com/

http://techcrunch.com/

http://www.wired.com/ 

http://www.fastcompany.com/

http://themacro.com/

http://pointsandfigures.com/

https://freedom-to-tinker.com/

https://pando.com/

https://www.statnews.com/

http://www.recode.net/

https://stratechery.com/

https://github.com/gdb/stripe-prg/wiki/Papers

 

Resources for Wantrepreneurs

http://web.engr.illinois.edu/~jeffe/teaching/algorithms/

http://www.forentrepreneurs.com/saas-metrics-2/#Calculating_CAC_and_LTV

https://developers.google.com/startups/accelerator/ 

http://startupclass.samaltman.com/

https://www.reddit.com/r/Entrepreneur/wiki/faq

https://www.khanacademy.org/computing/computer-programming

https://cloud.google.com/why-google/

http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-00-introduction-to-computer-science-and-programming-fall-2008/video-lectures/

https://www.coursera.org/

https://www.udacity.com/

http://web.stanford.edu/class/cs101/

https://lagunita.stanford.edu/courses/Home/Databases/Engineering/about

https://www.entrepreneur.com/magazine 

https://cs50.harvard.edu/

 

Highly Opinionated/Interesting

https://www.propublica.org/

http://equitablegrowth.org/

http://promarket.org/

http://www.counterpunch.org/

http://brookings.edu/blogs

https://aeon.co/essays

 

Telco and Satellite Industry Coverage

https://www.runwaygirlnetwork.com/

http://tmfassociates.com/blog/

http://www.rabbitears.info/

http://www.lightreading.com/

Investing Basics for High Earning Millenials

Caveats:

  • Don’t rely solely on one opinion, take an aggregate and decide on what’s best for you. Nobody else knows what is best for your risk profile. They really don’t.
  • Your money is yours to lose. Shepherd it well so that it works for you, and not you for it!
  • I’m assuming you’ve worked out a basic monthly budget at this point and have a good idea what you can save.
  • Scheduling monthly or weekly automatic transfers makes investing feel a lot less painful and dollar-cost averages your investments for you.

Good References and Books

The Good

  • You have a very long investment horizon.
  • You’re planking on a mil as the rappers say ie: you have more income than you know what to do with.

The Bad

  • Bank savings and checking accounts do not keep up with inflation.
  • The U.S. stock market is overvalued and primed for flat-negative returns over the next decade.
  • You are probably in debt.

Your enemies are the tax man and changes in the value of your currency (inflation/deflation)

Do everything you can (401k, IRA, municipal bond investing) to avoid the tax man. If you avoid him you can significantly lower your reported income and boost your investment returns. I don’t make the rules here. You’ve already made big strides if you bought a house! Enjoy your mortgage interest deduction and bang on the system!

I worry a lot about out-of-control inflation or deflation. Lots of evidence exists that the Fed has driven the outlandish stock market rallies of the last 20 years by implying that it can protect the free market. This has led to aggressive growth investments (tech and bio stocks) and yield chasing (high yield bonds).I’m worried about a global deflationary cycle. Oil and commodity price declines are typically great for the global economy, but political upheaval tends to follow subsidy cuts, and that’s where all the major oil producers are headed. Meanwhile, all of the central banks are one-upping each other attempting to give their economies a booster shot through incredibly low rates. This is an undeclared currency war.

The way to guard against inflation/deflation historically is holding onto cash against deflation) and investing in stocks and alternative assets (timber, personal loans, other weird stuff) against both deflation/inflation.

Savings in order of importance assuming you have a job

  1. Rainy day funds. Keep 3-9 months of savings in a bank savings or checking account. That includes your mortgage and any other living expenses.
  2. Max out your 401k every year ($18,000 in contributions for 2016).
  3. Set aside another $5,500 cash for your IRA, it still has advantages despite you being in a higher tax bracket.

Once you have 6-12 months of cash savings for a rainy day and are investing tax-free

  1. Pay down your highest interest rate loans early, especially anything >5%. You are basically matching/beating the annual return of the stock market, and it is a guaranteed return on your investment.
  2. Roll some money into yield-bearing assets: CDs through your bank, online lenders like Lending Club, or a municipal bond ETF such as MUB: http://www.morningstar.com/etfs/arcx/mub/quote.html.
  3. Open an individual investment account with Fidelity, E-Trade, or Scottrade and start rolling small amounts of money in.
  4. Start looking at alternative assets you could buy. Want to buy a second house? Ever wanted to get into collectible items?

Roth vs 401K vs IRA

A lot of people have trouble understanding the various retirement investment vehicles out there. The core concept is simple: are you investing pre-tax (regular IRA or regular 401k) or post-tax (Roth IRA or Roth 401k)? While the vehicles differ, the biggest debate is typically pre vs post tax accounts. Which to choose a Roth or a regular IRA/401k?

The answer is that you should be doing both. Nobody knows what the tax code will look like in 30 years and your income could change quite a bit up or down. I live in NYC now. If I move to Denver where cost of living is much lower, I can make half of my old salary, live better, and it still puts me in a lower tax bracket.

And what about startups? If you take a job at a company you really like maybe you get more stock comp and lower cash salary. Maybe you start your own company and pay yourself in company stock gains. It’s too hard to predict, so invest a bit in both pre-tax and post-tax accounts.

People always say taxes will rise, but tax rates haven’t changed significantly in 30 years. Nobody knows what the tax code will look like when you retire or what your income will be in the interim, therefore diversify your tax-free investing strategies between both a Roth and a Standard 401k or IRA.

Me as an example

Lately I am trying to build up cash for the following reasons:

  1. Leaving my job here, building up a war chest to carry me, my girlfriend, and my daughter for 6 months if I am unemployed.
  2. Macro uncertainty.The stock market has been on a tear for the past 6 years. Statistically this is highly improbable. Cyclically adjusted ratios are abnormally high, and profit margins are peaking. Dividend yields on the S&P500 is low from a historical perspective as well. Fundamentally the market looks expensive. Bonds are no better. Fortunately the actual economy appears to be doing fine. People are working. This could be a finance market bubble not a true economy bubble.

I’m currently 50% cash, and don’t like the stock market at these levels, but I need to put it to work somehow until a buying opportunity arises.\

 

So I’m thinking more about preservation of capital than putting risk on at this point in the economic cycle, but I keep investing in my 401k and in Jane’s 529. This might seem odd, but I am skeptical of my own ability to call a market top. If you are 51% right in investing I would like to work for you. I’m not god and I could be wrong about the market, both investments are dollar cost averaged, tax-advantaged, and have very long horizons.

Retirement savings should not be gambled with.  So in short, no matter what the market does keep investing in your tax-free retirement accounts every month, and max out your 401 k every year. If you are in debt start chopping at those student loans.

Alternative Investment Ideas

A basket of personal loans are a safer investment for my spare cash than 2% muni bond ETFs or a volatile stock market. Muni bond ETFs correlate with the stock market, tending to decline alongside. I can’t think of anywhere else to taxably invest over 1-2 years than CDs yielding 1%. Yuck.

LendingClub is quite liquid, the interest after-tax is probably a good 5-7%, and LC loans performed far better than bonds or stocks in 2008-2009.The average return was very low 0.5-2%, but still positive. If the economy is fine I should earn a decent 7-9% annualized return through Lending Club. If things go badly, and maybe my Lending Club portfolio ends down 1-5%, that will still be far superior to the stock market which typically loses 15-30% during recessions. There has been recent bad press on LendingClub, which in my opinion is overblown, but it should be noted. I personally am putting more cash in. I have been investing in the platform for 5 years now and have only seen positive overall returns. LC has recently raised its lending standards, which means a higher potential return to lenders. All good things in my opinion. I’ve taken all of my cash out of my individual stock account and put it into LendingClub for the liquidity. I want to generate at least $800 of interest income per month in case I am not receiving a salary for 6-9 months.

Other ideas:

  1. Invest directly in small businesses via Funding Circle
  2. Join a Venture Capital investing syndicate through Angelist

Shuttering the Shop – A Brief Recap

I will soon start working in the investment management industry, and therefore, I have been forced to unwind almost all of my positions much earlier than anticipated.

From June 1st 2014 to May 31st 2015, my portfolio returned 5.15%. This is despite the massive cash holdings I have accumulated over the past year, now representing 43% of my portfolio as I have unwound multiple positions. Over the same timeframe the S&P 500 returned 9.56%. Considering the cash drag -43% of my portfolio earning <1% annually- I’m quite happy with the results. I’ve been wary of a market correction, and for personal reasons, have wanted greater liquidity. The rest is invested in passive ETFs as well as a handful of companies about which I feel truly compelling reasons to hold for the long-term.

Overall I came out of things quite happy. SWKS, JPM, GLW, and GILD all outperformed, while BP and FHCO lagged. My biggest mistakes come from small cap companies nobody has heard of. My Biotech Basket quickly became a roller coaster ride to hell, and I’ve always gotten a bit queasy on carnival rides. Lesson learned. Small Cap Biotech is outside of my circle of competence.

SWKS performed well beyond my wildest expectations, more than quintupling in value from my initial buy price. I still believe SWKS is a fantastic company riding a fantastic trend, and would recommend that investors continue holding for the long term despite the richer valuation.

Corning (GLW) has been another outperformer, beating the S&P 500 by 7% since my purchases in mid-2012 from prices between $10-12. This is another firm with a lot of long-term potential. Touch screens have only become bigger parts of our lives. I would probably still be holding if I could.

JPM handily beat the S&P500 by 30 percentage points since my initial $35-40 purchases in late 2011 and during the “London Whale” crisis in the spring of 2012. The stock yielded a 96% return for me, and I couldn’t be happier, though I would have liked to have held for a bit longer.

GILD, my most recent purchase from December at around $100 a share, is now trading around $120; while during the same time frame the S&P 500 is showing a 7% return. I’m most annoyed to be pushed out of this stock ahead of its time. I believe that Gilead will outperform over the next several years. Such is life.

FHCO tanked, and has yet to come back. I think the FC2 is a decent product, and that the new CEO has a good plan for recovery. Sales have picked back up, and while earnings are still extraordinarily lump, the current valuation is so low, that any uptick in earnings will make this stock attractive. I plan to continue holding over at least the next several months.

BP has resolved most of its legal woes, but got hit by the freight train of lower oil prices which has afflicted the entire industry. I also intend to continue holding BP for the rich dividend yield, which is currently north of 6%.

I continue to hold my Google stock, ACE, and SAN among other names. I remain bullish on tech, but due to the nature of my work, am forced to invest only in passive ETFs with my own money.

It’s been useful to me to record my thoughts in a public forum, and I will probably keep the site up for the foreseeable future. I love the domain name too much not to keep it.

Super-Sizing Skyworks (NASDAQ:SWKS), Selling IBM

I took advantage of the latest market swoon to triple the size of my position in SWKS, buying in at $50.17. The stock is currently up, trading at $53.93, and I expect it to continue to rise. Some might hesitate to purchase more of a stock which has risen 100% in value since the initial purchase, but tripling my position size is an extension of the original thesis.

What caused the dip in price? Microchip, a microcontroller vendor, issued an earnings warning and an opinion that the semiconductor industry had peaked. Semiconductor industry woes were once again, punishing a great company unfairly. I bought in, and will buy more if the price drops back below $50.

Four days later, SWKS raised it’s earnings guidance for the quarter upwards from $1.01 per share to $1.08. Shares rose, and have been trending upward since. Average analyst price targets are around $60.

I left my internship this summer covering the tech space with some trepidation around my IBM position. The latest earnings report has finalized my feelings on the stock. I exited my position with a 15% loss over the past 2 years. I’m definitely not thrilled about it, but IBM represents one of the last value traps which I had invested in prior to coming to school. For now, I intend to watch and wait from the sidelines on IBM. A turnaround is possible, but it’s going to take a new management team with a new strategy.

After a 26% Gain in 3 Months – Stepping to the Side on Angie’s List

After reaping a 26% gain short-selling ANGI (NASDAQ:ANGI) over the past 3 months, I have decided to buy-to-close.

Similar to when I last closed my Angie’s List short in December, board members have been making small open-market purchases. Until the business model changes, nothing really will here either, but in the past, board member open market purchases have been a good short-term indicator of a bounce. The stock subsequently rose 50% from December thru January.

I also believe that the recently installed Chairman could finally light a fire under management. The firm laid off 97 sales employees and re-structured the commissions system earlier this month. Perhaps this was an admission that the current business model is unsustainable? Time will tell.

During a period when most Internet sector stocks have risen to ludicrous heights, ANGI has been a great short for me. The stock is down 50% since I my short position in October, but I believe the board shakeup could portend changes to the Angie’s List subscription model. For now, I move back to the sidelines.

Adding Verizon (NYSE:VZ)

Six months ago, I threatened to switch to AT&T if  Verizon did not offer me lower rates. The customer service representative politely told me to kick rocks. When the customer service representative would not budge, I pulled the manager card. When the manager didn’t budge I resigned myself to expensive cell phone bills and hung up. I should have invested immediately. That is serious pricing power.

I am adding a position in Verizon Wireless (NYSE:VZ). There has been a lot of publicity about Berkshire Hathaway’s recent stake. I am honestly a bit ashamed to drag along after Buffett and Paulson. I was confident that I could find a better investment in the space, and Bezeq came pretty close. Ultimately, my telecom investment was about safety. VZ’s numbers are fantastic and reassuringly consistent. Cell phones today are as necessary to the average Joe as water or electricity. This is essentially a utility stock, which pays a dividend of 4.26% currently, and which consistently generates a high free cash flow yield of over 7%. VZ’s gross margin hovers around 50%, with an operating margin that stays in the high teens. The current Price/Free Cash Flow multiple is 8.02, well below it’s main rival AT&T, which is trading at a P/FCF of 14.55.

By altering its capital structure, Verizon takes on more debt at historically low rates, and sheds itself of a significant drain to dividends. Assuming that Verizon can continue to generate strong Operating Cash Flow, this act should increase Free Cash Flow Return on Invested Capital from a historical range of 8-11% to over 15%! If Verizon stays true to its history shareholders will see dividend hikes in the future. This is a classic value investor play.

I also looked hard at Vodafone (NASDAQ:VOD), but opted for the proven track record at Verizon. I think VOD has fantastic potential in the emerging markets, but I am worried about regulation in Europe. VOD also has a spottier track record of returning cash to shareholders. Lately, management has been quite generous with dividends, but I wonder if they can keep it up. I may invest in VOD in the future, but prefer to keep my options open for now. Price/Free Cash Flow appears low at 5.26, but whether management will invest or distribute cash wisely is another question altogether. Historically management hasn’t used free cash flow effectively.

My other prospect was Bezeq, the Israeli Telecommunications Company (TLV:BEZQ). The firm has declining sales volume in a fleshed out market. On the positive side, they pay a very attractive 7.9 % dividend after foreign residence tax, with a payout ratio of 100%. Bezeq appears to have a headlock on the local communications market (cable, internet, and cell phones) despite government-mandated market reforms. P/FCF here is also attractive at 6.55. What scares me away from Bezeq is the possibility of currency fluctuations. The whole point of a Telecom investment to me was to find a safe place to park some cash. VOD could work because of its international diversification. Verizon works because I live in the U.S.

It also helps that Verizon and AT&T represent a duopoly. Some investors have been skittish to invest because of the debt load. I would ask them to try going a day without their cell phone. Better yet, try to negotiate a discount.