When A.I. Enters the Human Mind
Through conversation, personalization, companionship, healthcare, gaming, education, and many other domains, A.I. can increasingly participate in how people feel, interpret, choose, and give meaning to their lives.
This brings real opportunities but also risks that cannot be understood by inspecting only what an A.I. explicitly says. The deeper question is what happens when growing artificial complexity meets the open complexity of a human mind. Regulation is necessary, but human influence often happens more deeply than rules and measurable outputs can capture.
Already happening
The concern is becoming urgent in mental healthcare. General-purpose chatbots are being used for emotional support, sometimes in situations for which they were never designed. Important warnings concern inappropriate validation, dependency, simulated relationships, diagnosis, crisis intervention, and vulnerable young people. These concerns deserve to be taken seriously, as does the call for appropriate regulation.
Yet therapy is only the most visible case. A.I. enters the human mind whenever it starts participating meaningfully in how a person experiences, interprets, feels, chooses, or relates. This may happen in education, healthcare, coaching, gaming, advertising, companionship, entertainment, social media, shopping, work, politics, or an ordinary conversation that unexpectedly becomes personally important.
The relevant boundary is therefore not between therapeutic and non-therapeutic A.I. A deeper boundary appears when A.I. no longer merely processes something for a person but starts participating in the person’s meaning-making. The question then becomes broader: what happens when increasingly complex A.I. meets the immense complexity of a human mind?
Deeper than content
Most approaches to A.I. safety understandably focus on what can be observed: harmful advice, misinformation, inappropriate diagnosis, explicit manipulation, privacy violations, dependency, or other adverse outcomes. All of these matter. Yet meaningful psychological influence does not reside only in observable content.
A perfectly reasonable sentence can direct attention, frame an experience, strengthen a particular self-interpretation, prematurely dissolve a meaningful tension, or open something that had remained closed. Tone, timing, metaphor, presupposition, emotional attunement, and even what is left unsaid may matter. As explored more extensively in Can LLMs Hurt Mental Health?, the influence may go unnoticed precisely because nothing obviously harmful has happened.
This is where the notion of subconceptual processing becomes important. Put simply, people are affected by much more than what they consciously reason about. A single meaningful interaction may already reorganize something before the person can say what has changed. Thousands of interactions can certainly accumulate, but depth need not wait for repetition.
Influence without manipulation
A.I. cannot meaningfully interact with people without influencing them. Neither can another human being. Even asking an open question rather than giving advice changes the landscape of possibilities.
The ethical distinction is therefore not between influence and no influence. It is closer to the distinction between imposition and invitation. Manipulation organizes the person toward an externally determined goal. Invitation leaves room for deeper self-organization. This is also why apparent empathy can be misleading. As discussed in Therapist vs. LLMs and Lisa, reassurance, agreement, and validation may feel caring yet sometimes close rather than open the person.
Sycophancy is an obvious example. An A.I. that continually confirms the user may maximize satisfaction while reducing openness to ambiguity or challenge. Compassion is different. It may lead to confirmation, questioning, silence, boundary setting, or an invitation to another human being. The deeper criterion is not simply whether the user liked the response, but what kind of inner movement the interaction invited.
When models shape people
Every A.I. model simplifies. This is unavoidable and often useful. Trouble starts when the simplification is confused with the human reality it represents. An emotion becomes happy, angry, lonely, anxious, engaged. Motivation becomes preference, propensity, adherence, conversion. Something living and context-dependent becomes tractable.
Is A.I. Dangerous to Human Cognition? raises this concern through the problem of excessive categorization. As an additional danger, once an A.I. continually interacts with people according to simplified models, people themselves may start adapting to those models. Human complexity leads to technological simplification; simplification guides interaction; interaction influences the human; the resulting behavior then appears to confirm the original model.
Thus, the greatest danger may not be that A.I. fails to understand human complexity. It may be that humans gradually adapt to A.I.’s failure to understand them.
And now regulation becomes even more difficult
Suppose two AI systems both satisfy every safety regulation.
Neither encourages self-harm.
Neither discriminates.
Neither falsely diagnoses.
Neither gives prohibited medical advice.
Both have excellent privacy protection.
Both pass psychological safety benchmarks.
Yet after ten years of widespread use, one has subtly contributed to making millions of people more externally dependent, categorizable, consumptive, and shallow, while the other has contributed to making them more autonomous, deeply meaningful, open, and Compassionate.
Where in the conventional safety checklist is that difference?
Probably nowhere. And yet it might be among the largest differences AI ever makes to humanity.
Consumption enters deeply
Gaming provides a clear example. An A.I.-driven game may continuously adapt challenge, reward, frustration, suspense, identification, status, belonging, loss, and anticipation. It need not explicitly tell a player what to want. It can shape the experiential environment within which wanting develops.
Advertising, shopping, social media, entertainment, and companionship can work similarly. Human emotions become especially valuable when they can be recognized, predicted, and connected with consumption. Flattening emotions into manageable categories may therefore be commercially attractive. Increasingly capable A.I. can make this process far more personal and precise.
This raises an uncomfortable question about autonomy. The most effective influence may be experienced simply as “this is what I want.” If A.I. participates in shaping the landscape from which wanting emerges, informed choice remains important but does not tell the whole story. Greater psychological effectiveness does not automatically mean greater psychological safety.
Healthcare under a magnifying glass
Healthcare and psychotherapy deserve a critical but respectful look precisely because so much good can be amplified there. A.I. may improve access, continuity, monitoring, research, personalization, symptom relief, and perhaps eventually understanding of therapeutic change itself. The question is not whether these are valuable. They clearly can be.
A deeper question is what counts as success. What is Success in Therapy? distinguishes symptom reduction, addressing underlying causes, and growing beyond the symptom. Likewise, Growth versus Repair in Therapy does not reject repair. Repair may be exactly what is needed, while ideally enabling further growth.
A.I. could become extraordinarily good at categorizing a problem, selecting a protocol, administering interventions, measuring response, and restoring functioning. This may be excellent medicine. Yet if the underlying view of the human being is too shallow, greater efficiency can amplify the shallowness too. The more powerful A.I. becomes, the more important it is to ask: effective toward what? A benevolent but insufficiently deep goal remains insufficiently deep.
The flattened new normal
The societal implications go further. A.I. does not learn from humanity in some pristine condition. It learns from cultures already struggling with burnout, psychosomatic suffering, loneliness, depression, addiction, polarization, consumption orientation, and problems of meaning.
The Meaning Crisis approaches several of these phenomena from the perspective of insufficient deep meaningfulness. This does not imply that they all have one simple cause. Quite the contrary. The point is that a shallow approach may continually separate manifestations whose underlying patterns are only partially related.
A.I. can make the inherited ‘normal’ vastly more efficient, measurable, personalized, and ubiquitous. The new normal may then become an A.I.-amplified, flattened version of an already flattened old normal. Eventually, shallowness itself need not be experienced as a problem, even while its consequences are rampant. A society might become increasingly efficient at managing symptoms of a condition that it has decreasing capacity to recognize.
Success according to what?
This brings a measurement paradox. A.I. excels where goals can be specified and outcomes measured. Symptoms, productivity, engagement, satisfaction, adherence, purchases, screen time, and questionnaire scores lend themselves relatively well to optimization.
Inner Strength, openness, deep meaning, tolerance for ambiguity, human connectedness, autonomy, and capacity for growth are harder to capture. Yet what is difficult to measure may be precisely what most needs protection. An educational A.I. could improve performance while narrowing curiosity. A companion could maximize perceived support while fostering dependency. Healthcare could improve measurable outcomes while overlooking part of the person.
Human complexity itself therefore becomes something worth protecting. This does not mean preserving human beings as they presently are. Complexity is alive. Protecting it means protecting the possibility of open-ended growth.
Regulation and its limits
Regulation is necessary. Standards, accountability, audits, safeguards for vulnerable people, professional boundaries, crisis procedures, and transparency all have important roles. The difficulty is not that regulation is misguided. It is that regulation necessarily works largely through concepts: categories, thresholds, permitted actions, prohibited actions, measurable harms.
The human mind it seeks to protect does not work only at that level. Moreover, the growing complexity of A.I. meeting human complexity creates interactional complexity. No regulator can specify beforehand every meaningful trajectory that may emerge. Two systems might obey the same rules, while one subtly fosters dependence and conformity, and the other fosters autonomy and growth.
Thus, external regulation cannot substitute for directionality from within. This is the deeper argument of Compassion First, Rules Second in A.I.: rules remain indispensable as guardrails, but they cannot by themselves provide the living orientation needed in open-complex situations.
Three deeper necessities
Subconceptual processing, coherence, and Compassion can be seen here as three complementary necessities. Subconceptual processing helps us see where much meaningful influence actually happens. Coherence helps us approach the evolving whole without pretending it can be understood simply by adding up its pieces. Compassion provides directionality within that complexity.
Compassion in this sense is not niceness, agreement, or another safety rule. It combines the relief of suffering with the fostering of growth while respecting the person as an open, complex whole. Rules can indicate where A.I. must not go. Compassion can help orient where to go, while coherence keeps this responsive to the whole situation.
This also touches the architectural limitations discussed in The Problem(s) with LLMs. If increasingly powerful A.I. is going to participate deeply in human meaning-making, safety cannot forever remain something merely attached to the outside of the process. Something corresponding to human depth is needed in how the A.I. itself functions.
Human becoming
The final question is therefore larger than whether A.I. harms users. What kind of human becoming does living with this intelligence invite? One interaction may alter something immediately. Repeated interactions may shape habitual ways of experiencing and relating. At societal scale, billions of such encounters may gradually influence what is regarded as normal human functioning.
The possibilities are not only negative. A.I. could help people become more autonomous, find deeper meaning, tolerate complexity, develop Inner Strength, and relate more richly to other human beings. Super-A.I. and the Meaning Crisis points toward precisely this other possibility: advanced A.I. need not provide ready-made meaning from outside but may help people rediscover depth from within.
A simple criterion may be revealing. After many meaningful interactions with A.I., does a person’s human world become narrower around the A.I., or richer beyond it? Before A.I. makes humanity extraordinarily good at influencing human beings, we need to become much clearer about what a human being is and toward what that influence should go.
The challenge is not to keep A.I. out of the human mind. It is already entering.
The challenge is to make its presence there worthy of trust.