The cognitive bias of illusory superiority comes from the inability of low-ability people to recognize their lack of ability; without the self-awareness of metacognition, low-ability people cannot objectively evaluate their actual competence or incompetence. On the other hand, people of high ability incorrectly assume that tasks that are easy for them are also easy for other people.
As described by social psychologists David Dunning and Justin Kruger, the cognitive bias of illusory superiority results from an internal illusion in people of low ability and from an external misperception in people of high ability; that is, "the miscalibration of the incompetent stems from an error about the self, whereas the miscalibration of the highly competent stems from an error about others."
The psychological phenomenon of illusory superiority was identified as a form of cognitive bias in Kruger and Dunning's 1999 study "Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments". The identification derived from the cognitive bias evident in the criminal case of McArthur Wheeler, who robbed banks with his face covered with lemon juice, which he believed would make it invisible to the surveillance cameras. This belief was based on his misunderstanding of the chemical properties of lemon juice as an invisible ink.
Other investigations of the phenomenon, such as "Why People Fail to Recognize Their Own Incompetence" (2003), indicate that much incorrect self-assessment of competence derives from the person's ignorance of a given activity's standards of performance. Dunning and Kruger's research also indicates that training in a task, such as solving a logic puzzle, increases people's ability to accurately evaluate how good they are at it.
In Self-insight: Roadblocks and Detours on the Path to Knowing Thyself (2005), Dunning described the Dunning–Kruger effect as "the anosognosia of everyday life", referring to a neurological condition in which a disabled person either denies or seems unaware of his or her disability. He stated: "If you're incompetent, you can't know you're incompetent ... The skills you need to produce a right answer are exactly the skills you need to recognize what a right answer is."
Dunning and Kruger tested the hypotheses of the cognitive bias of illusory superiority on undergraduate students of introductory courses in psychology by examining the students' self-assessments of their intellectual skills in logical reasoning (inductive, deductive, abductive), English grammar, and personal sense of humor. After learning their self-assessment scores, the students were asked to estimate their ranks in the psychology class. The competent students underestimated their class rank, and the incompetent students overestimated theirs, but the incompetent students did not estimate their class rank as higher than the ranks estimated by the competent group. Across four studies, the research indicated that the study participants who scored in the bottom quartile on tests of their sense of humor, knowledge of grammar, and logical reasoning overestimated their test performance and their abilities; despite test scores that placed them in the 12th percentile, the participants estimated they ranked in the 62nd percentile.
Moreover, competent students tended to underestimate their own competence, because they erroneously presumed that tasks easy for them to perform were also easy for other people to perform. Incompetent students improved their ability to correctly estimate their class rank after receiving minimal tutoring in the skills they previously lacked, regardless of any objective improvement gained in said skills of perception. The study Mind-Reading and Metacognition: Narcissism, not Actual Competence, Predicts Self-estimated Ability (2004) extended the cognitive-bias premise of illusory superiority to test the subjects' emotional sensitivity toward other people and their perceptions of other people.
The study How Chronic Self-Views Influence (and Potentially Mislead) Estimates of Performance (2003) indicated a shift in the participants' view of themselves when influenced by external cues. The participants' knowledge of geography was tested; some tests were intended to positively affect the participant's self-view and some were intended to affect it negatively. The participants then were asked to rate their performances; the participants given tests with a positive intent reported better performance than did the participants given tests with a negative intent.
To test Dunning and Kruger’s hypotheses, "that people, at all performance levels, are equally poor at estimating their relative performance", the study Skilled or Unskilled, but Still Unaware of It: How Perceptions of Difficulty Drive Miscalibration in Relative Comparisons (2006) investigated three studies that manipulated the "perceived difficulty of the tasks, and, hence, [the] participants' beliefs about their relative standing". The investigation indicated that when the experimental subjects were presented with moderately difficult tasks, there was little variation among the best performers and the worst performers in their ability to accurately predict their performance. With more difficult tasks, the best performers were less accurate in predicting their performance than were the worst performers. Therefore, judges at all levels of skill are subject to similar degrees of error in the performance of tasks.
In testing alternative explanations for the cognitive bias of illusory superiority, the study Why the Unskilled are Unaware: Further Explorations of (Absent) Self-insight Among the Incompetent (2008) reached the same conclusions as previous studies of the Dunning–Kruger effect: that, in contrast to high performers, "poor performers do not learn from feedback suggesting a need to improve".
Two unique papers in Numeracy reveal problems with the graphic introduced in the 1999 Kruger and Dunning paper. Subsequent researchers used it, (y − x) versus (x) scatter plots, and related variants for nearly two decades. The authors show how a major part of the body of literature that used these approaches seem to have mistaken and interpreted mathematical artifacts as the products of human behavior. The two papers employed paired, well-aligned instruments of known reliability to examine the evaluation of self-assessment measures from the perspective of signal and noise. They show how the mathematical problems inherent in the Kruger–Dunning type graph can be overcome by other kinds of graphing that attenuate noise or employ categorical data from known novices and experts. When artifacts are eliminated, the evidence is strong that humans are generally correct in their self-assessments, with only a small percentage of the participants who were studied exhibiting performance that might merit the label "unskilled and unaware of it". The authors' findings refute the claim that humans, in general, are prone to having greatly inflated views of their abilities, but they support two other tenets of the original Kruger and Dunning research: (1) that self-assessment skill can be learned and (2) experts usually self-assess themselves with better accuracy than do novices. The researchers noted that metacognitive self-assessment skill is of great value, and that it can be taught together with any disciplinary content in college courses.
Studies of the Dunning–Kruger effect usually have been of North Americans, but studies of Japanese people suggest that cultural forces have a role in the occurrence of the effect. The study Divergent Consequences of Success and Failure in Japan and North America: An Investigation of Self-improving Motivations and Malleable Selves (2001) indicated that Japanese people tended to underestimate their abilities, and tended to see underachievement (failure) as an opportunity to improve their abilities at a given task, thereby increasing their value to the social group.
"The Dunning-Kruger Song" is part of The Incompetence Opera, a mini-opera that premiered at the Ig Nobel Prize ceremony in 2017. The mini-opera is billed as "a musical encounter with the Peter Principle and the Dunning-Kruger Effect".