Fred93's blog

I just wanted to mention that one of the best Prosper-related web sites has returned from the dead.  www.wallofsham.com  This ingenious site profiles Prosper deadbeats - borrowers who have defaulted on Prosper loans.  The stories of these individuals are real eye-openers, and sometimes just hilarious.  If you want to be a successful lender, it helps to understand the deadbeat. 

The site temporarily suffers from a few dead links, due to Prosper's recent removal of 2 years of forum content.  Don't let that bother you.  Much is still intact.

Note added 05/2008: Wall of Sham is again offline.  It is sad to see how people fear deadbeats and the laws that protect them.

Prosper no longer allows me to publish this material in the prosper.com discussion forum, so I will now publish it here on prospers.org .

Here's the 12/01/2007 update of the late loan chart.

 A larger version of this graph is available: here

The following chart presents the same data, except that the curves have been slid to a common starting point.  The horizontal axis now shows days since loan origination month.  The first three months Mar'06, Apr'06, May'06 have been left off of this chart, because the curves for those months (Prosper's early days) have a different shape.  With those curves left out you can easily see how similar the remaining curves are.

Look about 1 year out.  You see about 20% of the loans have gone bad at that point.  Individual months vary, but are quite similar.

Notes on methodology:
Each curve contains data on all the loans that originated in a particular month, as identified in the legend.  All data comes from Prosper's performance web page.  Data points are computed from data observed the 1st and 15th of each month.

Each data point is a ratio.  The numerator is the number of loans that have 1 month late or worse.  The denominator is the number of loans originated.  Thus this ratio is the fraction of loans that have "gone bad".

More specifically, the numberator is the sum of loans in the following states: defaulted + "1-3 months late" + "4+ months late" + repurchased.  The first three numbers are read directly from Prosper's performance web page.  Prosper does not explicitly display the number of loans repurchased.  I calculate the # loans repurchased by tracking changes in the number prosper shows as "loans originated".

The denominator is the number of loans originally shown by prosper as having originated in the designated month.  When prosper repurchases a loan found to have been fraudulent, they "unoriginate" the loan, in other words they subsequently show that a lower number of loans have originated.  I use the original (higher) number in the denominator of my ratio.  For example, in May of 2006, 458 loans originated, although they now show the number of originations as 446, after having repurchased 12 fraudulent loans.

Excluding the early few months of Prosper, these curves look very similar to one another.  Separating loans by age allows us to see the time evolution of delinquency.  Delinquency grows over time.  We can also see the rate at which it grows.  For the first year each group of loans shown here has gone bad at more than 20% per year.  Because almost all Prosper loans that go more than 1 month late go on to default, we can also see that the default rate on the all-prosper portfolio is approximately 20%/year. 

Given that this is so clear from Prosper's own data, it is sad to see the quality of Prosper loans misrepresented in the press over and over.  The most recent example is a November 27, 2007 Associated Press article http://www.msnbc.msn.com/id/21993720/ wherein Chris Larsen is quoted as saying:

Prosper’s default rate hovers at about 2.7 percent, Larsen said, but that figure is expected to rise as more loans mature.

Its not difficult to figure out how he comes up with 2.7%.  If you take the total $ worth of loans that prosper has declared defaulted so far, and divide it by the total $ worth of loans that Prosper has originated so far, you get approximately 2.7%. 

That ratio, however, is not meaningful to lenders. 

One problem with the 2.7% number is found in the choice of numerator.  The numerator in this fraction includes only loans that have been declared "default" by Prosper.  Prosper ages highly delinquent loans quite some time before declaring them "default".  Loans are subject to default at 4 months late, but Prosper holds such loans until a quarterly disposal, and only declares them default once they have passed the quarterly event.  Loans >4 months late would be in the default state except for the fact that they are waiting for the quarterly auction.  Today Prosper shows $2,864,899 of loans in the "default" state, and $3,694,261 in the "4+ months late" state.  More loans are in this suspended waiting-for-the-auction state than have so far completed the process and been declared "default"!  If the quarterly auction had been held yesterday, the calculation of Larsen's ratio would have come out 6.4% instead of 2.7%.

The denominator is also a bad choice.  Itcontains many loans so young that they could not possibly have defaulted yet.  (Loans must be 5 months old before they can default.)  The denominator also contains many loans old enough to default, but still low on the curve.    This methodology is simply defective.

Only when lenders understand the rate at which Prosper loans go bad will they have a chance to understand how selective they must be to succeed.  Prosper loans have been going bad on average at about 20% per year.

There was one small surprise in this update.  The Aug'07 curve took a jump.  This occured because the # of originated loans in Aug'07 dropped.  Was 979 originally, and was still 979 at my 11/15/07 reading.  As of 12/01/07 Prosper shows only 940 loans originated.  This (probably) means that some time during the last two weeks Prosper repurchased 39 loans from Aug'07.  Must have discovered a big batch of fraudulent loans.  If this theory is correct, it would be the largest batch of fraudulent loans ever detected.  Haven't heard anything about this event, but then it seems like it may have occurred at about the same time the Prosper forums were shut down, and there has been less sharing of information in the Prosper community since that time.  We should soon hear confirming information from lenders who were part of this repurchase, or else learn that it was a data glitch. 

A little over 2 months ago, Prosper presented their new VP of Operations, Doug Fuller.  He promised serious improvement in Prosper's disasterous collections department.  Now that he's been on the job two months, it seems like time for a status check.  How's Doug doin'?

Lets look at Prosper's collection statistics for the time since Doug's arrival.

As you can see from the chart, Prosper's statistic for late loans cured has increased by about 1% in the time since Doug joined. 

How should we think about that improvement?  Remember that in February '07, Prosper increased the lower cuttoff credit score for new borrowers,  Because of that change the average quality of borrowers since that time is substantially improved.  This change alone should cause the cure rate of late loans to increase slowly over time without any change in collection effort.  Indeed, the cure rate has increased gradually since that time.  The 1% improvement in the last 2 months is likely due to that effect alone.

What if the 1% improvement is due to Doug's effort.  Is that a good result?   Suppose he continues on this half-a-percent-per-month slope.  It will take two years to get to 26% . 

What about the "new agency test?"  Dunno.  Prosper has decided not  to share the data.  The only evidence of the test is a cryptic note on a few dozen loans.  Dunno 'bout you, but none of my loans with the cryptica "new agency test" note have been cured.  If there were great positive results, I'd expect Prosper would have shared them.  I'd recommend sharing the results in any case, as the lenders are after all the owners of the loans.

So far Doug is just another disappointment.  Hope he produces better results during the month of December.  Nothing would make me happier.  But frankly I believe structural changes are needed.  As long as lenders lose thousands, and the collection agency only loses the opportunity to make a few bucks, the right things aren't gonna happen .

See my writeup from May 2007: Open letter #2 to Prosper.com 

 Oops.

I've been critical of some of Prosper's misleading advertisements and newsletters in the past.  Last time I caught them making incorrect statements in the newsletter, a Prosper spokesman wrote "Going forward with the newsletter, we will diligently fact-check each story before publication."  Ok.  Great in fact.

They just published their November newsletters.  So how did they do this time?  If you'll pardon the slightly mangled and heavily overused words of Britney Spears, the title says it all.  How can this be?

The November lenders' newsletter contains the "stories" of two lenders.  I'll look briefly at each one. 

In one story, lender hcjack2208 writes

Right now I have 30 loans that I have invested in.

Even with a few defaulted loans, I am still making a 13% rate of return on my money.

Wow.  This dude is making 13%!  Cool.

Thanks to Prosper's data export, we can look at the status of hcjack2208's loans.  The lendingstats.com web site makes it easy for us.  We learn that he has 42 loans and 7 of these are late.  A pretty substantial fraction of lates.  If we estimate his rate of return with the assumption that loans in his portfolio will go bad in the future at the same rate they have in the past (the assumption that lendingstats ROI calculation makes), then hcjack2208 is achieving a rate of return of 4.6%, not 13%.  Big difference, eh?  Oops.

You can view the details of hcjack2208's portfolio here.

Next there's a story about lender davesax.  Davesax writes

At this point I've got $23,000 invested in 200 loans with a gross yield of 14.25% (less 1/2 point for Prosper) and only a few late pays, but none over 15 days at present.

Wow.  200 loans and nothing over 15 days late.  This gives the impression that davesax's return will be something close to that gross yield of 14.25% that he quotes.  That really sounded great, so figured I should look at his portfolio. 

Davesax now has 360 loans, of which 13 are late.  1 loan is 4+ months late.  3 are 3 months late.  2 are 2 months late.  3 are 1 month late.  4 are >15 days late.  The facts don't match the statement in the newsletter at all.  Well, that's misleading.

If you want to do the davesax rate of return calculation in your head, note that those 13 late loans are about 3.5% of his portfolio.  Because his portfolio is less than 6 months late, you've got to more than double that to get an annual default rate.  You end up with something around 7%/year estimated defaults.  He started with a little over 14% gross interest rate, so you would expect his return to be something a little less than 7%/year.  I won't bore you with the numerical details.  Lendingstats does a much more careful calculation, applying appropriate roll rates and loss rates. 

The lendingstats ROI esimate for davesax comes out 6.29% .

You can view the details of davesax's portfolio here.

The sin in the above examples is that Prosper took quotes from relatively new lenders, then used them more than 4 months later.  New lenders often think they are doing better than they really are, because they don't project losses appropriately.

Most lenders don't have the mathematical sophistication to compute how well they're doin'.  That's why the portfolio calculations at lendingstats are so very useful.  Prosper itself offers a similar calculation, on their "performance" web page, but they don't let you apply the calculation to your portfolio.  You can only apply the calculation to various subsets of the whole-prosper portfolio, based on a list of filter criteria (date, credit grade, etc).  I believe it is terribly important to make lenders understand their own performance.  To do anything less is misleading.

Toward that end, I have suggested that Prosper adjust their "performance" web page so that it will compute the ROI of a lender's portfolio.  In the meantime, lenders can use lendingstats.

Google

Google advertisements being run by Prosper recently deliver those who click to a landing page that contains the following graphic:

Unfortunately, the word "average" is pretty well defined, and the average portfolio on Prosper is not earning 9.4% as the ad implies.  The mean lender portfolio ROI is 4.39%. (again using the excellent Lendingstats calculations)  You can read a discussion of the distribution of Prosper portfolio returns here.

The 9.4% number is obtained by filtering for a small subset of Prosper loans, as explained in a footnote.  Showing the results of a cherry-picked set and calling it "average" is well...

I think the ad is misleading.  Regulators came down hard on the securities industry for this sort of shenanigans in the 60's and 70's.  Now there are quite well defined rules for how one calculates the return of a mutual fund, or an investment manager.  The SEC created uniform rules for how mutual funds would present their performance.  An industry self-reguating group known as CFA (formerly known as AIMR) created detailed rules describing how investment managers should report their performance.  (One of the rules says you don't cherry-pick.)

Prosper believes it is outside of such rules and regulations.  This is true only because Prosper's product is new and different, and the regulators aren't engaged.  I don't even know what regulator has oversight of the prosper & lender relationship.  The regulators don't know yet either.  Meanwhile, the best course of action for Prosper would be to very carefully avoid misleading people.

Some good news

Prosper has made some positive changes lately.  Until recently, Prosper presented lenders with a set of default rate statistics provided by Experian.  These statistics guided lenders by giving them default rate estimates for borrowers in each of the Prosper credit grades.  Unfortunately, as long as a year ago we could see that the default rates of Prosper borrowers were running several times higher than the Experian-provided statistics, but Prosper continued to present these misleading numbers to lenders.  In a recent upgrade, Prosper did away with the bogus Experian numbers and replaced them with stats derived from historical Prosper loans. 

Bravo.  One great big misleading thing has been removed.

But the newsletter... still misleading.

I've been charting Prosper's collections statistics ever since Prosper first made them avaiable.  Prosper sends loans to a collection agency when they are 1 month late, and the outcome after the collection agency's efforts are charted below.

The big picture is that Prosper's collection results continue to be scandalously bad.  Of those loans that go 1 month late, Prosper's data shows that 13.5% of the loans are cured, from which we can estimate that a horrible 86.5% go on to default.  (We are forced to estimate, because these stats include loans still in process.   However, as you can see from the breakdown, almost nothing is ever collected after the first month, so the estimate is pretty close.) Here's the data for Penncro, the collection agency handling most of Prosper's loans...

Notes:

The data for this chart comes from Prosper's Collection Agency Web Page.  I have shown data only for Penncro.  Although Prosper has two collection agencies, almost all late loans have gone to Penncro.  The other agency, FirstSource, is doing even worse.

I've hidden the red "net collected" curve prior to 7/4/07, because the data prosper provided for that period was obviously wrong.  Older versions of this chart and related discussions can be found in the collections thread on the prosper forum.

A more complete discussion of Prosper's collections can be found in my blog entry from May, "Open letter #2 - Collections is broken". 

A couple of weeks ago, Shira Levine published an interview with Doug Fuller, the new fellow in charge of Prosper collections.  At one point, Doug was talking about his success improving collections at a prior job, and Shira asked... 

Q:  Can you do the same thing with Prosper’s collections calls?

A:  Actually, for the month of September, we’ve seen greater than a 40% increase in the contact rates at our primary collection agency

Wow!  40% improvement!  Sounds great, but looking at the collections rates in the chart, I don't see a corresponding improvement in September.  How can it be that a 40% improvement in contact rates produces no results in collections? 

One way this could happen is if the contact rates were so small to begin with that a 40% increase is insignificant.  Prosper has never published "contact rates", so we don't know how big a 40% increase is.  If you were only contacting 5 people before, and you added 2, that would be a 40% increase, but the additional 2 contacts wouldn't affect the overal status of the 1148 loans in collections by much.

To Doug's credit, he has been the first Prosper employee to state publicly that collections has been mismanaged.  That's a start.  Now lets see some action.