* Incompetent individuals tend to overestimate their own level of skill.
* Incompetent individuals fail to recognize genuine skill in others.
* Incompetent individuals fail to recognize the extremity of their inadequacy.
They found, for instance, that "individuals who scored in the 12th percentile believed that their general reasoning abilities fell at the 68th percentile". Curiously, even the top students "tended to overestimate" the incompetent ones. But at least they were better at assessing themselves. (I wouldn't be surprised if there was actually a reverse effect at higher levels -- cf. Colin Marshall's 'Why Smart People Often Think They're The Dumbest Person in the Room').
Should such research lead us to place less weight on subjective (self-)assessment, and more on external indicators of success?
Posts
Those findings are extremely accurate in my experience. It's scary to think about though -- what if I'm one of those self-deluded people? I try to keep that in mind and always stay humble, because I see so many self-aggrandizing people who clearly don't have a clue (especially among philosophy graduate students), and it's always a sad sight.
ReplyDeleteBut to answer your question: yes, I try to primarily judge myself by external indicators - perhaps excessively so - because I consider it more objective and therefore accurate, although in philosophy external indicators will never be completely objective. Sometimes I regret not going into math instead, where it's completely obvious to everyone who is a hack and who is not, and the rank order in ability is crystal clear.
This comment has been removed by the author.
ReplyDelete(But then I remember that I wouldn't have been able to cut it in math ;)
ReplyDeleteIt seems unlikely to me that external indicators would solve the problem. There is no particular reason to think that they correlate with competence, either; there are too many variables affecting them (luck, money, position in social networks, fit of interests with the interests of others, availability of opportunities of success, etc.).
ReplyDeleteBrandon, if by 'external indicators' you mean full-fledged real world success, then I agree with you that this is confounded by all sorts of variables besides ability and competence. But surely there are more modest external indicators than that -- say, winning a prize or a competition, having people compliment you on your productions, or perhaps simply doing well on a standardized test. (And conversely.)
ReplyDeleteWe knew it all along: the less they know, the less they know it.
ReplyDeleteLvb,
ReplyDeleteI'm inclined to think the same problem arises with more modest external indicators. I suppose to some extent, if the indicators were stable, then they would reliably underwrite an inference to a very limited form of competence. For instance, if I regularly did well on all sorts of standardized tests, I know I have a competence for standardized tests (barring the possibility of the tests being rigged, etc.). But I was thinking of things like the 'general reasoning abilities' mentioned in the post.
Reminds me of my undergrad psychometrics prof: "People once thought that IQ was a measure of general intelligence. We now define IQ more precisely as what the IQ tests measure." Heh.
ReplyDeleteThanks for the mention, Richard (I'm flattered!).
ReplyDeleteI'm guess I'm inclined to think that people are reliably bad at assessing their own intelligence or competence. At least sometimes, though, that's not just because we're ignorant. Being in philosophy grad school (for me, at least) has involved both situations where I have good reason to take myself to be much more competent than the evidence I have supports, and situations where there reverse is true.
So, since I don't know what my weaknesses are when I'm being trained, there's reason for me to have a general lack of confidence when (say) getting feedback on a paper, so that I'm open to whatever criticism my adviser or whoever is offering. But then there are also lots of situations where I'm only going to improve by over-confidently throwing myself into a situation seeing what happens (e.g. the more confidently I air my half-baked idea, the clearer it will be, and the more it will irritate people who hold different views, so that I'll get better feedback on whether to keep it baking or toss it out).
Richard, your psychometrics professor has a depressingly low opinion of his own field. There is every reason to believe that IQ tests (such as the GRE) are very good measures of what we intuitively call 'general intelligence'. No other type of test that psychologists have come up with has had anywhere near the predictive power of IQ tests.
ReplyDeleteRobin Hanson writes in: The literature after that "unskilled and unaware of it" article didn't really confirm its results, but that paper gets far more blog cites than all the others put together.
ReplyDelete