L'illusione della conoscenza - The Knowledge Illusion
Perché non pensiamo mai da soli -- Why do we never think by our own
Steven Sloman, Philip Fernbach
Published: 2018
- L'illusione della conoscenza - The Knowledge Illusion
- ChatGPT's summary
- Personal notes & highlights
- Introduction
- 1 – What We Know
- 2 – Why Do We Think?
- 3 – How Do We Think?
- 4 – Why Do We Think Things That Aren’t True?
- 5 – Thinking with Our Body and the World
- 6 – Thinking with Other People
- 7 – Thinking with Technology
- 8 – Thinking About Science
- 9 – Thinking About Politics
- 10 – A New Definition of Intelligence
- 11 – Making People Smarter
- 12 – Making Smarter Decisions
- Conclusion
ChatGPT's summary
🔍 Core Premise
We think we understand much more than we actually do. Most of what we "know" resides not in our own minds but in the minds of others or in our environment. Human intelligence is largely collective—we think in communities.
🧠 Key Ideas & Concepts
1. The Illusion of Understanding
- People consistently overestimate their understanding of how things work (e.g., toilets, zippers, government policies).
- When asked to explain in detail, this illusion is exposed—this is known as the "illusion of explanatory depth."
2. The Mind Is Not a Hard Drive
- Human memory is not built to store detailed information.
- Instead, the brain is optimized for abstracting, approximating, and interacting with others.
3. Thinking Is Distributed
- We rely on others (experts, tools, books, communities) to know and remember.
- Intelligence is not individual; it is distributed across people and systems.
- This explains why societies can build complex technologies, even though few individuals understand them fully.
4. Reasoning Is for Communication
- We developed reasoning not to uncover truth, but to justify our actions and communicate with others.
- We are biased reasoners (prone to confirmation bias), but these biases often serve a social function.
5. Why Misinformation Spreads
- People form strong opinions with little understanding.
- Group dynamics, not individual reasoning, often drive political and scientific beliefs.
- Beliefs spread more through social identity than through facts or evidence.
6. Science and Democracy Rely on Shared Knowledge
- Since no one can know everything, society depends on trusted systems (e.g., science, expert communities).
- Effective democratic decisions require humility and deference to expert knowledge.
- Representative democracy is better than direct democracy for complex problems.
📘 Final Message
"We don’t think alone." Understanding our limits, recognizing our dependence on others, and building robust knowledge communities is essential in a complex world. True intelligence lies not in what we know individually, but in how well we collaborate, share, and question together.
Personal notes & highlights
Introduction
The mind is a flexible problem-solver; it extracts only the most useful information to guide our decisions. It doesn’t store detailed information -- this would take too much energy.
We live under the illusion that we grasp things, while in reality, we only have a partial idea of how they work. Our knowledge is superficial.
Thought is a master at selecting only the information that interests it. It extracts the essence (the power of abstraction). The mind is not designed to acquire and store every little detail about each object -- instead, we learn from experience in order to apply general principles to new objects and situations.
The importance of collaboration: "sharing intention" (paying attention together and dividing tasks) is a form of collective thinking not observed in other animal species.
Complexity is abundant in the world, and if more people were aware of it, there would likely be less polarization.
1 – What We Know
People tend to overestimate what they know. But just asking them to explain something can make them realize their illusion of understanding.
The estimated human memory capacity, according to some hypotheses, corresponds to around 500 MB.
Conclusion: humans are not “knowledge repositories” or hard drives.
2 – Why Do We Think?
Condition: Hyperthymesia, or superior autobiographical memory – people who remember every detail of every event.
“I relive my entire life every day and it’s driving me mad!” These individuals are often depressed.
What is the brain for?
Neural networks coordinate the complex actions of our body. The only animal without neurons is the sea sponge. A jellyfish has about 800 neurons. The brain is like a reprogrammable algorithm.
What the brain does best is abstraction. Hyperthymesia reduces the ability to abstract and to generalize rules from specific cases. Memorizing details turns out to be useless from an evolutionary perspective when it comes to effective action.
3 – How Do We Think?
Mice can associate stimuli when they are causally linked (Pavlov), but not otherwise. Causal thinking allows us to anticipate the future by understanding causes and effects. It’s easier for us to reason “forward” (into the future) due to this causal bias.
People often ignore alternative causes when reasoning about an effect, because reasoning backward (from effect to cause) requires more mental energy. Diagnostic reasoning (from effect to cause) is a uniquely human capacity.
4 – Why Do We Think Things That Aren’t True?
Experiment with a coin and an arrow rolling on another coin. 5 machinists build 5 objects in 5 minutes. How long would it take 100 machinists to build 100 objects?
Some processes are hardwired into the brain—this is what we call intuition. It’s an unconscious process that appears very quickly (e.g., certain anagrams, or words with jumbled letters replaced by numbers). Conscious thought, by contrast, takes more time and energy.
Deliberate thought (conscious reasoning) connects us to others. Explaining our reasoning makes it explicit and enables collective reasoning.
The illusion of knowledge is a product of intuitive reasoning.
5 – Thinking with Our Body and the World
How do we pass through a doorway without calculating complex trajectories (which a robot would need to do)? Answer: we ensure that the optical flow from the left and right sides of the doorway is symmetrical (both sides approach at the same speed). [Experiment: with VR goggles, one side of a hallway was artificially sped up. Result: people no longer walked down the center.]
Looking at a photo of a utensil pointing right makes it easier to use your right hand. The body plays an important role in cognition.
Brain activity does not occur only in the brain.
Common phobias relate to dangerous situations from our prehistoric past.
The mind is not in the brain—the brain is part of the mind (along with the body and other systems).
6 – Thinking with Other People
Group intelligence is greater than the sum of individual intelligences. Like bees, humans can collaborate to produce tremendous collective intelligence. Hunting was probably one of the first activities that required a high level of collaboration and task division—cognitive division of labor.
If everyone specializes in a different domain, the resulting collective intelligence exceeds the sum of its parts.
Modern humans have brains three times larger than our closest ancestors—this requires a huge amount of energy. Snowball effect: as groups grew, brains grew larger, group behaviors became more complex, and survival improved. Group size and brain size are closely correlated. Our brains are adapted to group life.
Example: sharing arbitrarily complex ideas through language.
Humans have a unique ability: shared attention. Example: an adult playing with a child hides something. If the adult points to a closed box, the child can infer the adult’s intention (that the object is hidden there)—no animal can do this.
Cumulative culture: the uniquely human ability to pass knowledge across generations.
The curse of knowledge: once we know something, it’s hard to imagine that others don’t. The illusion of knowledge is the opposite: believing what others know is also in our own heads.
7 – Thinking with Technology
No machine can share intention, and this affects how we interact with them.
Crowdsourcing: relying on voluntary knowledge from the public (e.g., Wikipedia). This only works well when contributors are domain experts. Superintelligence resides in communities.
8 – Thinking About Science
Anti-scientific behaviors are not based on rational thinking. Simply presenting information doesn’t change them. These behaviors stem from cultural and contextual factors, which make them resistant to change.
Individually, we don’t have enough knowledge to form conscious opinions on science. We rely on the opinions of trusted experts.
The first step to correcting false beliefs is to open people’s minds (and their communities) to the fact that they may be wrong. Nobody wants to be wrong.
9 – Thinking About Politics
We’re generally unaware of how little we know. A tiny bit of knowledge can make us feel like experts.
When group members know little but share an opinion, they reinforce each other’s sense of understanding. Groupthink: group discussions lead to greater polarization.
Explaining how a policy works reduces perceived understanding and depolarizes opinions.
Causal explanation helps us understand the effects of causes. Political choices are often based on principles (e.g., being pro-environment) rather than understanding the direct implications of decisions.
Strong moral reactions require no reflection. Strong political opinions don’t either. If opinions are based on sacred values rather than consequences, showing someone they’re wrong won’t change their mind. Rather than debating how to “do the best for everyone” (boring, technical debates), politicians prefer to emphasize sacred values and beliefs.
Politicians often take a stance before studying the pros and cons of the law they’re voting on.
The general public rarely has the tools to make informed decisions about complex social problems.
Current research on human ignorance supports representative democracy.
10 – A New Definition of Intelligence
Big success requires more than individual intelligence. Fluid vs. crystallized intelligence:
- Crystallized: the amount of knowledge a person has.
- Fluid: the ability to reason and solve problems quickly.
IQ tests: regardless of the method, results are similar. They measure attention and thinking ability. IQ is one of the best predictors of career success.
Group intelligence is the ability to contribute to collective reasoning and problem-solving. A group doesn’t need lots of high-IQ individuals, but rather a balance of diverse skills.
Popular belief: a startup needs a great idea. Reality: it needs a great team (investors bet on teams, not ideas).
We should give more credit to community in science, politics, business, and everyday life.
11 – Making People Smarter
Example: Brazilian children without schooling who sell goods in the street outperform schooled children in estimating large-number proportions. When survival is at stake, experience beats classical learning.
Static learning (lecture-style) is harder than active learning. The brain is made to learn through action.
Illusion of understanding: thinking you understand something until you try to explain it. Related to the illusion of explanatory depth.
We often confuse expert knowledge with our own.
We must avoid the false belief that education provides people with knowledge and reasoning skills to be intellectually independent. Intellectual independence doesn’t exist—knowledge depends on others. Example: a good mechanic accesses knowledge spread throughout the community (manufacturers, other garages, engineering know-how, etc.).
We must learn to pay attention to what we don’t know.
Scientists work at the frontier of their field, trying to turn the unknown into the known. Learning to act like a scientist means discovering what is not yet known.
Science is done by a community. Science is justified by direct observation, inference, or authority.
Science provides evidence where other justifications would be too slow or costly. What separates science from belief systems is the power of justification: results must be reproducible.
What I’m legally responsible for knowing may not be in my head.
How do I know if I’m talking to an expert and hearing the truth? Check whether the claim is based on replicable evidence. Was it published in a peer-reviewed journal? Have a critical mindset.
12 – Making Smarter Decisions
Most people are "enemies of explanation." A product for sale with too many details implies that the product is too complicated for most people. The "nudge" approach: it’s easier to change the context/environment than to change a person.
Conclusion
3 main arguments:
- Ignorance
- The illusion of knowledge
- The knowledge community
A well-measured dose of skepticism is necessary.