I always wondered where this colloquialism came from and whom used it first. As I was writing this blog post, I asked Mr Google who popularised this term. I was surprised by the response. For those of you uneducated like me, the term was first attributed it to the British Prime Minister Benjamin Disraeli who said: “There are three kinds of lies: lies, damned lies, and statistics.”
And you know in the SMSF industry, this is certainly the case.
Everyone seems to use data (much of it dubious in nature) to make themselves look like experts. Many of the interpretations however, seem to be more a matter of opinion and self interest than a reflection of the data. And I will say this concerns me somewhat.
BGL invests a lot of time and money extracting industry data from our client database. We know the rules – nothing can be identifiable – and that’s easy when you are holding over 250 million transactions in a single database.
So I know when I am quoting data it has a reasonable basis.
And this is why some of the reports about SMSFs concern me. We know, for example, the large super funds are scared of SMSFs because so many large balances are rolled out of large funds into SMSFs. So when a large fund says it is getting 60% more rollovers from SMSFs back into the fund, it means really nothing without the raw numbers. Unless you know the sample size – it could be from 1 rollover to 6 rollovers. So without the actual numbers, the interpretation is useless and possibly designed to be deliberately misleading.
And similarly, when I see some data in SMSF industry reports I just shake my head. An example: say a survey has 950 responses. And say one question in the survey has 60 response, of which 10% say something is likely or very likely – in other words 6 responses out of 950 say something is likely or very likely. And from those 6 responses, 2 responses pick option A and the other 4 responses pick options B, C, D and E. Do you think the 2 reponses for option A are so compelling you would include them in the survey report highlights ? Do you think this is a good statistical analysis ? Well some people in the industry sure do.
I have no issue with data but the data must be representative. This is why many polls really concern me.
In an industry with many 1,000’s of participants such as the SMSF industry, I would even doubt the validity of a survey with 950 responses, let alone 6 from 950. Even 2 organisations who each hold data for 25% of SMSFs often get different statistical results. So how representative are statistics from 950 participants in an industry with 25,000, 30,000, 40,000 or more participants ? I would say questionable.
So the next time you see an SMSF industry report you need to ask yourself a question – how valid is the data ? How representative are the results of the survey ? If you were looking at the data, would you interpret the data the same way ? There are certainly many cases where I would not.
