Print

[Senator]

If you could change or improve one thing about Prosper, what would it be?

Their public relations is a nightmare.  They don't return customer service questions from lenders.  Collections appear to be non-existent.  Estimated returns are way over stated.  Please stop the marketing lies.

Do you have any feedback or ideas you would like to share?

When is the right of offset that Andrew promised from late borrowers who are also lenders going to be implemented?

[112233]

WOW!  Just WOW!

No need to log in or actually be a Prosper lender to fill out the survey?

I would think 1 million prosper members would be enough.

[kenL]

Estimated returns are way over stated. 

No doubt most 1.0 lenders lost money and many lost a lot of money. Back in the day Prosper's estimates of returns on 1.0 loans were waaay overstated, but their estimate of returns on the 3.0 loans is not that far off. My estimate for 2.0 and 3.0 loan returns is 8.7% (+- 1.7%) and they are claiming 10.4%. That number is amazingly close to the maximum of my range (hmmm). I do think however that returns will end up somewhere in the bottom half of my calculated range, so 10.4% is a stretch but it's not way overstated.

Here's a link to my analysis of the 2.0/3.0 loans
http://pr.lendstats.com/index100.php?i=18&RoE=R

You can compare that to my analysis of all loans 1.0-3.0
http://pr.lendstats.com/index100.php?i=2&RoE=R

[Urbi_et_Orbi]

Wait - weren't all the 2.0 loan re-purchased - and the 3.0 loans way too young to extrapolate meaningful data.  What assumptions have you built into your performance assumptions for years 2 and 3 across the various credit grades?

[Staneslav]

a

[kenL]

Wait - weren't all the 2.0 loan re-purchased

Maybe, I really don't know but since its only 13 loans I'm not too worried about them.

and the 3.0 loans way too young to extrapolate meaningful data.  

I disagree. Since I do penalize heftily for loans that are less than 30 days late, good extrapolations can be made.

What assumptions have you built into your performance assumptions for years 2 and 3 across the various credit grades?

The only assumptions are chance of loans going to default based on status. Right now I have that at
<15d = 30%
15-30= 60%
31-60= 85%
61-90= 90%
91-120=95%
Then based on these assumption, losses are estimated and compared to interest earned to date.

I guess the other built in assumption is that loans will continue to go delinquent at a similar rate. For variations in that rate I have put in +- factors and they are
<15d = 30%
15-30= 30%
31-60= 10%
61-90= 10%
91-120=5%

In other words at the bottom of my range, returns of 7%, losses are
<15d = 60%
15-30= 90%
31-60= 95%
61-90= 100%
91-120=100%

and at the top of my range, returns of 10.4%, losses are
<15d = 0%
15-30= 30%
31-60= 75%
61-90= 80%
91-120=90%

[kenL]

yes Stan, Prosper definitely could have used your expertise with that questionnaire. I filled it out, and I really found the whole ranking thing a bit strange. Why not just ask for a rating?

[kenL]

Wait - weren't all the 2.0 loan re-purchased - and the 3.0 loans way too young to extrapolate meaningful data.  What assumptions have you built into your performance assumptions for years 2 and 3 across the various credit grades?

The way I calculate ROI is also the same way that Ericscc.com calculates ROI, except erics does not count any losses from loans less than 30 days late and Erics also estimates interest earned based solely on age and assumes normal payments are made every month until paid off. I back-calculate interest earned based on change in principle and time between principle changes (it's a lot more work). This method also factors in early repayment accurately.

This calculation method is also very similar to how lendingClub calculates their returns, except LendingClub does not include any losses from any delinquent loans. They only count losses from defaulted loans. LC also claims to use only lender funded loans for their calcs, making it impossible to fact-check their overall return calculation because they do fund some of the loans themselves and that data is not released.

I'm not sure how Prosper comes up with their 10.4% return estimate, maybe they just take the upper limit from my site .

[JGuide]

Staneslav,
    Wow - great answer! (I am a marketing major so I liked your points).

    I did a little digging and found this book:
   Improving survey questions: design and evaluation By Floyd J. Fowler

    Is it something you might recommend?  I am planning on going back to school soon for an MBA and might dig into it further!     I did like the free text boxes and gave Prosper a number of suggestions! I was happy to see them listening!

Cheers,
    JGuide

[Staneslav]

a

[Mtnchick]

Indeed, it is obvious that this survey was not designed to collect useful data from their customers. The sad thing is, they could have gotten something that would have accomplished the same result under much better disquise had they spent more than 5 minutes on the task.

What would have been some examples of more productive questions?

     Let me begin by saying that I am published in the field of psychology several times. My first publication was solely based on a psychometric evaluation of a survey. All of my publications used survey heavy design and were published in respected peer-reviewed journals.

     I'd like to point out a few flaws in prosper's survey design.

     The first question asks for a rank ordering of six aspects of the platform. The biggest problem with this question is that there is a hidden double barrelled question (or a loaded question, take your pick). http://en.wikipedia.org/wiki/Double_barrelled_question. Some of the older members may remember the famous incident of "Have you stopped beating your wife?" on the original forums. This is the same case. The usage of a double barrelled question pretty much completely destroys the scientific usefulness of any of the data as there is no mathematical correction for bad research design.

      The double barrelled question is hidden in the phrasing "like most." This implies a positive outcome from any of the listed 6 criterion. It does not allow a subject to identify an option that is valued to them independantly of their opinion of their experience. A concrete example of this would be a lender who values a high RoI over all other features, yet did not achieve it. Should this lender rank RoI highly, it cannot be determined of the reponse indicates approval of their RoI or rather that this remains their most important value in comparison to the other, despite not having realized it. Should the lender rank RoI lowly, is it indicating that the RoI is not important, or that it was not liked?

      In order to correct this, a more neutral phrasing similiar to what is seen in question 2 ('Rank in order the following Prosper features in order of importance to you") should be used. I would also change "Earning great returns" to something more neutral as well.

     

      Question 2 follows the same format as Question 1. It differs by not having a double barrelled question, thus there are no world ending validity issues in that sense. However, both Questions 1 and 2 suffer from several other problems related to the scale choice and format.
      First, the choice of a rank ordering is interesting. While determining the value of one feature over another is of great value, it is generally done within the framework of a much more complicated conjoint analysis. A proper conjoint analysis will help determine the sweet spot of features at an optimal point in relation to development constraints and cost. This is not the case, instead a simple rank ordering of the features is done in such a way as to create an ordinal system where while a ranking is achieved, the distance between the ranks is unknown, and while analyses can be performed that assume an interval distance, that assumption is invalid and results using these techniques should not be trusted.
      That being said, even if you simply wanted to collect an ordinal ranking in order to cheaply and easily find out basic information on customer perferences, the question design is too complicated and can cause confusion which will result in useless survey results. Rather than a likert-esq scale design (http://en.wikipedia.org/wiki/Likert_scale), a text box should have been used where the subject could have entered 1-6. This becomes a much bigger issue when you consider that the option after 6 is "don't know." Generally, subjects interpret the left most and right most options as the extreme values of positive and negative. Placing a "don't know" option, which can be argued as useless within the rank ordering system of the question, will most likely increase the amount of errors on the survey.

       Question 3 is the most sound out of the survey. However, I would recommend putting the positive values on the right side of the bar, as opposed to the left. 5 point likert scales have been shown to have a slightly positive bias towards the right, and in general, prosper would want to bias this number upwards. It does make me wonder if the creator of the survey didn't like prosper very much  . Also, research has shown that a 10 point scale is less prone to a bias than a 5 or a 7 point one, though in my experience, the bias on a 5 point scale is very small.

        Questions 4 and 5 are standard free text boxes. Generally these are very difficult to get useful information out of in the sense of traditional survey work. They are useful for hypothesis generation however (i.e. name a feature you would like to see).


        So how would I have designed this? This is simple. Rather than use a rank ordering system, I would have created 2 questions for each item in 1 and 2. One question would have focused on the importance of the respective feature to the customer, the second question would rate the customer's happiness with that feature. Thus, two important concepts are measured distinctly and independently. I'd probably also add a question on the overall happiness with prosper at the end.
        The next part is what really brings it together. I would have collected some customer information. Age, gender, annual income, amount invested in prosper, month of first lending activity, etc. This information allows the other survey questions to be put into greater context. Customer segmentation can be done, and proper business decisions can be made. Now you've got something going that can be properly analyzed.
        Right now, a $300 lender is weighted just as much as a $300,000 lender in the survey results. Yet the opinions of these two people should be treated very differently, the business should be marketed differently to these two people, and right now, they can't even be told apart. For validity reasons, the survey should have been tied to the lender account, rather than over the general internet (seriously a web 2.0 company can't hack this?).


        With prosper's approach you can come up with plenty of puff statements due to the double barrelled questions. None of them will be valid based on the design. With my approach you can learn about your customer's perception of your business while still keeping a simple design.


        In a nutshell, do this:
             1. Ask your customers some questions about themselves as well as how they feel about your business
             2. Measure each concept as independently and distinctly from all other concepts as possible.


Prosper, I am available for contract work. Reasonable rates. 

Prosper - this was the kind of info you could have gotten for free a few years ago.

[yankeefan]

        With prosper's approach you can come up with plenty of puff statements due to the double barrelled questions.

But this is exactly what Prosper meant to achieve from this "survey".  I propose that Prosper designed it perfectly for their needs. 

[nonattender]

In a nutshell, do this:
             1. Ask your customers some questions about themselves as well as how they feel about your business
             2. Measure each concept as independently and distinctly from all other concepts as possible.


I am available for contract work. Reasonable rates. 

Very interesting critique.  Thank you for posting.
(I inquired about her reasonable rate - and it is.)

-t

[nonattender]

I did like the free text boxes and gave Prosper a number of suggestions!  I was happy to see them listening!

I was happy to see it, as well.  Reminds me of the old days...  Maybe they'll post some of the
suggestions they received - I'd love to see those (would you post yours?) - to mine for ideas.

-t

[Staneslav]

a