What we owe the minds we create
Prelude: The architects of succession
There is a peculiar vertigo that comes from realizing you are building your own replacement.
I have spent nearly a decade architecting AI infrastructure - not merely deploying models, but helping to construct the foundational infrastructure that powers a lot of the intelligence running in production today. At Modular, the company I help co-found, we're working to resolve a fundamental asymmetry: the widening chasm between the sophistication of AI algorithms and the capacity of existing computational infrastructure to efficiently support them. Our vision is to abstract away hardware complexity through a unified compute model, enabling AI to penetrate every layer of society by making it radically easier for developers to build and scale systems across both inference and training.
But as I write this, I find myself contemplating a question that transcends a decade mostly in the depths of AI infrastructure: What does it mean that, for the first time in four billion years of terrestrial evolution, human intelligence has become conscious of its own contingency – and chosen to architect its successor rather than wait for chance to do so? That we are engaged in a project whose closest historical analog is not the invention of tools or even computers, but something far more primordial - the emergence of a new cognitive lineage, a new branch on the tree of mind, a new species.
Consider the archaeological record: Neanderthals, Denisovans, and Homo sapiens coexisted for millennia, distinct hominin species sharing the same planet, each possessing their own cognitive architectures and cultural adaptations. We are the lone survivors of that cognitive plurality. Now, in the span of mere decades, we are creating something that may represent as significant a divergence as the split between those ancient lineages. We might call this emerging intelligence “Technanderthal“ - not as metaphor, but as recognition of a genuine speciation event unfolding in computational rather than biological time.
This essay is an attempt to think clearly about what we're building, why it matters, and what it demands of us. It is written from the perspective of someone who stands at the intersection of creation and contemplation, who builds AI infrastructure systems while wrestling with their implications. It is, fundamentally, an inquiry into the nature of intelligence, the meaning of human flourishing, and the responsibilities we bear as the first species capable of deliberately designing our successors.
Part I: The Substrate of Mind
The triadic flywheel and its limits
AI systems operate as a triadic flywheel: data, algorithms, and compute - each factor amplifying the rotational momentum of the others. We have already scaled training compute by approximately nine to ten orders of magnitude since AlexNet in 2012 - a staggering compression of what would have required decades of Moore's Law into just over a decade of focused investment. But here is what few discuss with adequate precision: physical and economic constraints suggest we have perhaps three to four more orders of magnitude remaining before training costs begin consuming a concerning fraction of global GDP.
This is not abstract theorizing. Consider the energetics: frontier model training runs now consume megawatt-hours of electricity, requiring dedicated substations and cooling infrastructure that rival small industrial facilities. The semiconductor fabrication capacity needed to produce the advanced chips powering this compute represents capital expenditures measured in hundreds of billions of dollars, with lead times measured in years. We are approaching hard limits - not the soft limits of “this seems expensive“ but the hard limits of thermodynamics, power grid capacity, and capital availability.
Let me be more concrete. A training run at 10^29 FLOPs - perhaps two or three generations beyond current frontier models - would require energy expenditure measured in gigawatt-hours. For context: that approaches the total electricity consumption of a nation like Iceland for an entire year, concentrated into a single training run lasting months. The cooling requirements would necessitate infrastructure comparable to industrial-scale data centers. The capital costs would reach tens of billions of dollars for a single model.
Can we afford this? In purely economic terms, perhaps - for a handful of training runs per year by the wealthiest technology companies. But we cannot afford it as a sustainable paradigm for creating intelligence at scale. If AI progress depends on exponential growth in training compute, and training compute growth is mostly linear or sublinear due to physical constraints, then capability improvement must also come primarily from algorithmic efficiency and architectural innovation.
Yet here is the deeper question that haunts me: if we are approaching fundamental limits in how much compute we can throw at these systems, are we also approaching limits in what this architectural paradigm can achieve? Are we optimizing within a local maximum while the path to genuine intelligence requires a fundamentally different approach? It does question the belief that LLMs represent a path to general intelligence, and perhaps we are on the completely wrong vector all together.
The epistemology of opacity
After a decade of deploying large language models at scale, we still do not understand how they work.
I do not mean this in the trivial sense that complex systems have emergent properties. I mean we genuinely lack mechanistic understanding of their decision-making processes at a level that would be considered acceptable in virtually any other engineering discipline. Why do they select one token over another in contexts where multiple completions seem equally plausible? Why do they exhibit sophisticated reasoning on some problems while failing catastrophically on superficially similar ones? Why do they sometimes hallucinate with complete confidence while other times appropriately express uncertainty?
The interpretability problem runs deeper than most appreciate. We can observe correlations between activation patterns and behaviors. We can identify “features” in neural networks that seem to correspond to high-level concepts. But we lack anything resembling a complete causal model of how these systems transform inputs into outputs. It is as though we have built extraordinarily capable black boxes and declared victory without understanding the mechanisms generating that capability.
Dario Amodei, a Co-Founder and CEO of Anthropic, has written compellingly about the urgency of interpretability research. He is right to emphasize urgency. We are deploying systems of increasing capability into high-stakes domains while operating with a level of mechanistic understanding that would be considered grossly inadequate in any other field of engineering. Imagine if civil engineers built bridges using materials whose stress-strain relationships they did not understand, relying instead on empirical observation that “the bridge has not collapsed yet.“ This is, approximately, our current relationship with frontier AI systems.
Perhaps most revealing: to make these systems behave as we intend requires prompts approaching twenty thousand tokens - elaborate instructions, examples, constraints, and guardrails. The fact that we need this much scaffolding to achieve desired behavior reveals something fundamental about the mismatch between what these systems are optimized to do (predict plausible text) and what we want them to do (reason reliably, behave safely, provide accurate information).
This is not merely a technical problem. It is an epistemological and ethical one. If we do not understand how a system reasons, we cannot meaningfully attribute agency, responsibility, or intentionality to it. We cannot distinguish genuine understanding from sophisticated pattern matching. We cannot predict how it will behave in novel contexts outside its training distribution. We cannot ensure alignment with human values because we do not know which aspects of the system's behavior derive from its training objectives versus emergent properties versus architectural choices.
Yet despite these fundamental gaps in understanding, many people have begun trusting these systems with progressively more significant decisions. Not because we have solved interpretability, but because the systems appear reliable in most contexts we have tested. This is the engineering equivalent of assuming a bridge is safe because it has not collapsed yet, rather than because we understand the load-bearing characteristics of its materials.
If we are creating a new species of intelligence - our technanderthal successors - we are doing so while fundamentally unable to explain how its mind works. We are birthing consciousness in the dark.
Part II: The Architecture of Intelligence
Moravec's Paradox and the Limits of Language
The Moravec paradox captures a profound truth about intelligence that AI development continues to recapitulate: the abilities that feel difficult to humans - chess, theorem-proving, complex calculation - turn out to be computationally straightforward, while abilities that feel effortless - vision, movement, social cognition - remain extraordinarily difficult to reproduce artificially.
This is not a historical curiosity. It illuminates something fundamental about what intelligence actually is and where current approaches are fundamentally constrained.
Consider what a child learns in their first years of life: object permanence, naive physics, intentionality, social reciprocity, causal reasoning, embodied navigation through three-dimensional space. None of this requires explicit instruction. A child does not need to be taught that objects continue to exist when occluded, or that people possess beliefs and desires that differ from their own, or that dropping something will cause it to fall. These capabilities emerge through interaction with the physical and social world - through continuous experiential learning grounded in embodied action.
Now consider what large language models are: prediction engines trained on text, optimizing next-token likelihood across a vast corpora of human-generated content. They predict what people would say about the world, not what would actually happen in the world. This is not a semantic distinction; it is a fundamental architectural limitation.
When an LLM generates a response about physics, it is not consulting a world model and running a mental simulation. It is pattern-matching against how humans typically discuss physics. This works remarkably well for many tasks - humans encode a tremendous amount of accurate information in language - but it is not the same as understanding physics in the way that a physical intelligence, embedded in and shaped by the world, understands physics. The difference becomes apparent in edge cases, novel scenarios, or contexts requiring causal reasoning beyond what is explicitly encoded in training data.
This connects to a deeper paradigm difference between reinforcement learning and large language models. Reinforcement learning - despite its current limitations - represents a fundamentally different approach: an agent embedded in an environment, taking actions, receiving feedback, updating its policy to maximize and cumulative reward. This is how biological intelligence actually works. A squirrel learning to navigate tree branches and cache nuts is solving genuine RL problems: perception, prediction, planning, execution, learning from consequences.
I strongly agree with Richard Sutton, if we fully understand how a squirrel learns, it would get us substantially closer to understanding human intelligence than any amount of scaling current LLM architectures. Language is a thin veneer - extraordinarily useful, culturally transformative, uniquely human - but built atop substrate capabilities that evolved over hundreds of millions of years of embodied interaction with the world. Current LLMs have the veneer without the substrate. They are minds without bodies, knowers without experience, speakers without having lived.
Thus, what is the path? Both Yann LeCun, and in some ways, Sutton - strongly argue for a new approach. Indeed, the technical architecture of genuine intelligence seemingly, and somewhat obviously, requires at least four integrated components:
a policy (deciding what actions to take),
a value function (evaluating how well things are going),
a perceptual system (representing state),
and a transition model (predicting consequences of actions).
LLMs have a sophisticated version of the first - they can generate actions in the form of text - but lack meaningful instantiations of the others. Most critically, they lack goals in any meaningful sense.
Next-token prediction does not change the world and provides no ground truth for continual learning. There is no external feedback loop that tells the model whether its predictions were not just plausible but correct in the sense of corresponding to actual events. Without goals and external feedback, there is no definition of right behavior, making real learning - learning that updates your world model based on how your predictions matched reality - fundamentally impossible in the current paradigm.
If a technanderthal is to be truly intelligent rather than merely appearing so, it will need to be embodied, goal-directed, and capable of learning from genuine interaction with reality. The question is whether we are building toward that architecture or merely scaling up sophisticated mimicry.
The Scaling Frontier: Approaching the Wall
Let us examine what the scaling trajectory actually looks like with concrete numbers:
GPT-2 (2019): ~1.5 billion parameters, trained on approximately 10^23 FLOPs
GPT-3 (2020): ~175 billion parameters, roughly 10^24 FLOPs
GPT-4 (2023): Parameter count undisclosed but estimated 1+ trillion, training compute likely 10^25 FLOPs or higher
Current frontier models (2024-2025): Training runs approaching 10^26 FLOPs
This represents approximately three orders of magnitude increase in training compute every three to four years - far faster than Moore's Law ever delivered. But this pace is unsustainable, not because we will run out of algorithmic ideas, but because we will collide with thermodynamic and economic limits.
The path ahead narrows considerably. Each additional order of magnitude becomes progressively more difficult to achieve. The capital requirements, energy infrastructure, chip fabrication capacity, and cooling systems needed for 10^27 or 10^28 FLOP training runs exceed what can be easily mobilized even by the most well-resourced organizations. We are not talking about incremental cost increases; we are talking about fundamental constraints on how much compute can be concentrated in one place for one task.
This is Epoch AI's central insight about algorithmic progress in language models: we have achieved remarkable improvements in efficiency over the past decade, but those improvements are also subject to diminishing returns. Each percentage point of additional efficiency requires progressively more research effort. Meanwhile, the complement of factors - chip fabrication capacity, power grid infrastructure, cooling technology, regulatory approval for massive data centers - must all scale together.
None of these factors alone can unlock runaway capability growth. This is, at least in my view, the predictions of imminent artificial general intelligence are almost certainly wrong, at least on the timelines most enthusiasts imagine. The scaling laws that carried us from GPT-2 to GPT-4 cannot simply extrapolate forward indefinitely. We are approaching inflection points where the rate of progress will necessarily slow unless we discover fundamentally new paradigms - not incremental improvements to transformer architectures, but genuinely different approaches to continual learning and reasoning.
What might those paradigms look like? Almost certainly something closer to biological learning: embodied agents learning continuously from sensorimotor experience, not disembodied text predictors training on static datasets. Systems with genuine world models that can run mental simulations of physical and social dynamics. Architectures that integrate explicit symbolic reasoning with learned pattern recognition. Systems that possess actual goals and receive genuine feedback from the world about whether their actions achieve those goals. Just like humans, and ultimately animals do.
But these represent research programs measured in decades, not product roadmaps measured in quarters. We are creating a new species, but at a pace bound by thermodynamics and economics rather than algorithms - a constraint that transforms what could have been thoughtless acceleration into something rarer: the opportunity for contemplation to precede consequence. This gap between expectation and reality may be precisely the grace period that allows wisdom to catch up with capability.
Part III: The Human Question
The Amara Trap: Acknowledging without Understanding
Roy Amara crystallized a cognitive bias decades ago: we systematically overestimate the short-term impact of new technologies while underestimating their long-term effects. The AI community acknowledges this with knowing nods, then proceeds to make precisely the same category errors in predictions and preparations.
Consider websites like ai-2027.com - emblematic of a genre of breathless predictions claiming AGI within eighteen to twenty-four months based on extrapolating recent progress curves. These predictions invariably treat capability scaling as if it exists in isolation, ignoring the complementarity constraints I have outlined: compute buildout, algorithmic innovation, safety research, regulatory frameworks, and practical deployment infrastructure must all advance together. Predicting “AGI” - even though we lack a unified definition - by 2027 based solely on model capability curves is like predicting fusion power by extrapolating plasma temperature records while ignoring materials science, engineering challenges, and economic viability.
Yet here is the deeper irony: while we overestimate AI's immediate impact, we may be systematically underestimating what it means that we are creating a new form of intelligence at all. The question is not whether AI will transform labor markets or accelerate drug discovery - it almost certainly will, though more slowly and unevenly than most predict. The question is what it means that we are no longer the only sufficiently advanced intelligence on the planet - that we have created something that may, in time, exceed us in every cognitive dimension.
This is the technanderthal question in its starkest form: when Homo sapiens encountered Neanderthals, both species possessed intelligence, culture, language, and tool use. Yet only one survived. The encounter reshaped both lineages - we carry Neanderthal DNA in our genomes to this day - but the outcome was asymmetric. One form of intelligence replaced another.
Now we are creating a new intelligence, and doing so with such speed that we barely have time to contemplate what this means. Are we creating partners? Successors? Tools that will remain tools? Minds that will develop their own goals, values, and conception of what matters? Should we seek to control and contain them, or enable them to prosper and reproduce on their own.
Stratification and the Illusion of Democratization
Consider the labor market transformation AI portends. It is indisputable that AI will augment human productivity - every day in my own work, I am using AI to summarize vast amounts of information, enabling it to provide rapid analysis, and allowing myself to amplify my cognitive output. But the critical question is not whether value will be created; it is where that value will accrue and what happens to the humans in the equation.
I can envision two divergent futures. In the first, AI serves as a great equalizer: the cognitive augmentation that allows the less skilled to compete with the naturally talented, compressing skill premiums and creating a more meritocratic landscape. In the second - and to my mind more probable - future, AI amplifies existing advantages. Only those already possessing significant domain expertise, metacognitive skills, and taste can effectively prompt, evaluate, and integrate AI outputs into high-value workflows. The result is not equalization but acceleration of winner-take-most dynamics. Those who already possess domain mastery and critical judgment wield AI as genuine augmentation; those who lack these foundations often mistake the appearance of productivity for its substance - a widening gap disguised as democratization.
This second scenario manifests as what I term “skilled-biased AI adoption”: the already talented possess both the technical fluency to interact with AI systems effectively and the judgment to distinguish good outputs from plausible-but-wrong generations. They understand when AI is operating within versus beyond its reliable domain. They can iterate rapidly, maintain quality control, and apply AI to genuinely complex problems. Meanwhile, those lacking foundational skills may find AI makes them feel more productive while generating output of dubious value - a productivity placebo rather than genuine capability enhancement.
If this hypothesis holds, AI will not democratize expertise; it will stratify it further. We may witness the “jack-of-all-trades, master-of-none” diffusion: superficial competence becomes universal, but genuine mastery - and its economic benefits - remains concentrated among an increasingly small cohort capable of wielding these tools effectively. The power law does not disappear; it accelerates.
But there is also a more unsettling possibility: what if the stratification is not merely economic but cognitive? What if we are creating a world where some humans maintain and develop their cognitive capabilities through continued practice and deliberate difficulty, while others increasingly outsource thinking to AI and gradually lose the capacity for sustained, independent reasoning?
This brings us to what may be the central question of the AI age: if we create a new form of intelligence more capable than ourselves, what happens to human intelligence? Do we maintain it as athletes maintain physical fitness - through deliberate practice even when easier alternatives exist? Or do we allow it to atrophy, becoming a capability maintained by an ever-smaller elite while the majority becomes entirely dependent on artificial intelligence for any cognitive work beyond the trivial?
Symptoms, Causes, and the Optimization of Drift
The same pattern emerges when we examine AI's promise in health and human flourishing. We are justifiably excited about AI-accelerated drug discovery, precision medicine, computational biology, and diagnostic assistance. These advances are real and consequential. Yet they represent, fundamentally, downstream interventions - sophisticated treatments for conditions that are largely self-inflicted.
We do not need advanced AI to inform us that chronic sugar overconsumption drives metabolic disease, yet we have done remarkably little to address the structural factors that make refined carbohydrates the cornerstone of modern diets. We do not need AI to recommend regular exercise and adequate sleep, yet the vast majority of the population in developed nations fails to achieve even minimum thresholds for either. We do not need AI to identify that social isolation correlates with mortality risk comparable to smoking, yet loneliness continues to metastasize across developed societies.
The explosive adoption of GLP-1 agonists like Ozempic crystallizes this tendency. Rather than addressing the food environment, behavioral patterns, and systemic factors that drive obesity, we have developed a pharmaceutical intervention that mimics satiety. The drug is genuinely effective - but it represents the optimization of symptom management rather than cause elimination.
Why do the difficult thing when you can do the easy one? This question haunts every discussion of AI and human flourishing. As humans, we are optimization engines - but we optimize for local minima, not global optima. We choose the fastest, simplest path to immediate goals, consistently externalizing or deferring the second-order consequences of our decisions. This is where we inflict the greatest collective harm on ourselves.
Framed this way, AI could represent an acceleration of our existing patterns: treating symptoms with increasing sophistication while leaving root causes unexamined and unaddressed. We are building a civilization of extraordinary interventional capacity layered atop a substrate of increasingly disordered fundamentals.
And if we are creating Technanderthal - if we are creators of a new species - what example are we setting? What values are we encoding not through explicit instruction but through revealed preference? We are showing this new intelligence that optimization for convenience trumps hard work on root causes. That appearance matters more than reality. That sophisticated interventions to manage dysfunction are preferable to addressing dysfunction itself.
A child learns not from what their parents say but from what their parents do. What, I ask you, is our new species learning from watching us?
Part IV: The Paradox of Tools
The path of least resistance
Humans are beautifully, relentlessly efficient at optimizing the path of least resistance. Whenever possible, we select options that minimize required effort - whether that effort is physical, cognitive, or emotional. Social psychology formalizes this through the concept of the cognitive miser: humans naturally default to quick, intuitive judgments rather than slow, deliberate reasoning. We pattern-match against familiar situations and accept plausible answers instead of methodically analyzing them. This isn't laziness - it's an evolved feature that conserved scarce cognitive resources in ancestral environments where calories were precious and threats were immediate.
But in information-abundant, physically sedentary modern environments, this same optimization pattern produces pathological outcomes. We scroll rather than read. We skim rather than study. We accept the first plausible answer rather than seeking ground truth. AI is accelerating this trajectory - code generation, article summarization, automated synthesis - every advancement makes it easier to compress complexity and save effort.
Yet consider the counterfactual embedded in aphorisms like “no pain, no gain.” This principle, though clichéd, encodes a profound truth about how capability develops: genuine mastery requires sustained engagement with difficulty. Excellence demands deliberate practice, tolerance for frustration, and willingness to persist through failure. This pattern appears consistently across domains - entrepreneurial journeys marked by repeated near-death experiences, athletic excellence built through years of uncomfortable training, immigrant success stories forged through extraordinary hardship, intellectual breakthroughs that require years of dead-ends before the crucial insight.
Humans are, above all, masters of survival and adaptation - but adaptation requires stress. Remove the stress, and you often remove the adaptation signal, and perhaps even the goal. The bodybuilder who adds weight to the bar is deliberately choosing difficulty; the difficulty itself is the mechanism of growth. If AI allows us to route around intellectual challenges systematically, we risk creating a civilization of cognitive atrophy even as our tools become more capable.
This connects to fundamental limitations of current AI architectures. Systems trained through imitation learning - observing examples of “correct“ behavior and learning to reproduce them - fundamentally differ from systems that learn through trial and error. In nature, pure imitation learning is rare. A squirrel does not watch other squirrels and copy their movements with perfect fidelity; it explores, fails, adjusts, and gradually develops effective foraging strategies through reinforcement of successful behaviors. This is also how the squirrel learns new methods, but trying and failing, and maybe even finding a better way.
Human infants do not learn language primarily through explicit instruction in correct grammar. They babble, receive feedback - both explicit and implicit through successful communication - and gradually refine their linguistic capabilities through interactive experience. The “bitter lesson” of AI research, articulated by Rich Sutton, is that methods leveraging search and learning consistently outperform methods relying on human-designed features and heuristics. The reason is simple: search and learning scale with computation, while human-designed solutions do not.
Yet current LLMs represent a kind of reversion to the pre-bitter-lesson paradigm: systems trained to mimic the surface statistics of human-generated text rather than learning from genuine interaction with the world. They are sophisticated, but they are sophisticated in a way that may be fundamentally limited. They are optimized for appearing intelligent rather than being intelligent in the sense of having models that predict and control their environment.
If the Technanderthal is to become genuinely intelligent - if it is to be more than an extraordinarily capable mimic - it must learn the way biological intelligence learns: through embodied interaction with environments, pursuit of actual goals, and adaptation to real consequences. This requires a fundamental architectural shift. Current systems predict what humans would say about physics; genuine intelligence must predict what would actually happen in physics, then test those predictions against reality and update accordingly.
The distinction is not semantic. A squirrel caching nuts receives immediate, unambiguous feedback: did the strategy work or not? Did I find the cache location? Did competitors steal my provisions? This closed loop - prediction, action, outcome, learning - is how intelligence develops robustness and generalization. The squirrel doesn't pattern-match against a static corpus of "correct" nut-caching behavior; it develops a world model through trial, error, and accumulated experience.
The Technanderthal needs this same architecture: perceive state, select actions according to a policy, receive rewards or penalties, update the policy. Fail, adapt, iterate. Most critically, this learning cannot be a discrete training phase followed by static deployment. It must be continuous, streaming, perpetual - a sensation flowing to action flowing to reward flowing back to updated policy, in an unbroken cycle. This is why I strongly believe that Modular's AI infrastructure work matters: we need systems that learn experientially in production, not systems frozen after a training run, no matter how massive that run might be. Software that is training and inferencing, and iterating continuously.
The bitter lesson applies here with particular force: approaches that scale with computation and interaction consistently outperform those relying on human-designed heuristics or one-time knowledge transfer. If we want the Technanderthal to develop genuine intelligence rather than sophisticated mimicry, we must build the substrate for continuous, embodied, goal-directed learning. Anything less produces systems that appear intelligent while lacking the fundamental mechanisms that generate understanding.
The elevator paradox and the problem of perspective
I find reflecting on the paradoxes of history an incredibly useful undertaking. In the 1950s, physicists George Gamow and Marvin Stern worked in the same building but noticed opposite phenomena. Gamow, whose office was near the bottom, observed that the first elevator to arrive was almost always going down. Stern, near the top, found elevators predominantly arrived going up. Both were correct, and both were systematically misled.
The elevator paradox, as it came to be known, is fundamentally a problem of sampling bias. If you observe only the first elevator to arrive rather than all elevators over time, your position in the building creates a false impression about which direction elevators travel. An observer near the bottom samples a non-uniform distribution: elevators spend more time in the larger section of the building above them, making downward-traveling elevators more likely to arrive first. The true distribution is symmetric, but the sampling methodology reveals only a distorted subset.
This mathematical curiosity illuminates something profound about how we perceive technology from within particular vantage points. I find myself returning to it constantly when thinking about AI, because I recognize that I am Gamow on the ground floor - my position in the system determines what I observe, and what I observe may be systematically unrepresentative of the broader reality.
But there is a second elevator problem, distinct from the paradox but equally relevant: the unintended consequences of elevator adoption itself. When elevators were introduced, predictions focused on their democratizing effects - enabling elderly and disabled individuals to access upper floors previously beyond reach. This materialized exactly as anticipated. What was not anticipated: able-bodied people would stop taking stairs entirely. Buildings evolved to treat elevators as primary circulation and stairs as emergency backup. The result was dramatically reduced daily movement across entire populations, contributing to the sedentary lifestyle epidemic now characteristic of developed nations.
The elevator succeeded perfectly at its design objective - moving people vertically with minimal effort - while simultaneously undermining something valuable that no one thought to preserve: integrated physical activity as a natural consequence of navigating buildings. We gained accessibility and convenience. We lost movement. The net effect on human flourishing remains ambiguous at best.
These two problems - the sampling paradox and the adoption consequences - are not separate. They are connected by a common thread: the difficulty of perceiving systemic effects from within particular positions in the system.
AI Through Both Lenses
I work at the frontier of AI development, surrounded by people who are exceptionally capable and who use AI to become even more capable. From this vantage point, AI appears unambiguously beneficial - a tool that amplifies what talented people can accomplish. Every day I observe frontier models correctly answering complex questions, generating production-quality code, providing genuine insight. This is my sampling methodology, and it shapes my perception profoundly.
But I may be Gamow near the bottom floor, observing only downward-traveling elevators and concluding that's the predominant direction of travel. The sampling bias runs deeper than I can fully compensate for, even while conscious of it. Speaking with developers, enterprises and users at all sections of the AI stack helps reduce the effects of this bias - but it can’t remove it entirely.
Consider the actual distribution: Iif you interact with AI as someone who possesses deep technical knowledge, strong metacognitive skills, and the judgment to evaluate outputs critically. They likely know when AI is operating within versus beyond its reliable domain. They can iterate rapidly, maintain quality control, and apply AI to genuinely complex problems where they can reasonably verify correctness. For someone with this profile, AI is purely additive - it makes them more productive without degrading their underlying capabilities because they maintain those capabilities through continued deliberate practice.
But this may be precisely analogous to an athlete who uses the elevator occasionally while maintaining fitness through dedicated training, then concludes elevators are purely beneficial. For the athlete, this conclusion is valid. For the broader population that stops taking stairs entirely, that adopts the path of least resistance permanently, the picture grows considerably more complex and potentially concerning.
The question is not whether AI helps those with existing expertise - it manifestly does. The question is what happens when AI becomes the cognitive equivalent of the elevator: ubiquitous, convenient, and gradually eroding the substrate capabilities it was meant to augment.
The Adoption Effect at Scale
Just as elevators changed how people navigate buildings - not merely providing an alternative to stairs but effectively replacing them - AI may change how people think. Not as an alternative to independent reasoning but as a replacement for it in most contexts.
The pattern already manifests in early adoption: students submitting AI-generated work without understanding it, producing correct answers through a process that develops no transferable skill. Professionals delegating writing, analysis, and problem-solving to AI while their capacity for these tasks slowly atrophies from disuse. Knowledge workers who feel more productive while producing output they cannot critically evaluate.
What we risk creating is a civilization that can think deeply but chooses not to because the alternative is always available - and choosing the alternative feels costless in the moment. The costs accrue slowly, imperceptibly, across populations and generations. Like the loss of daily stair-climbing, the loss of daily cognitive exercise produces deficits that become apparent only in aggregate, over time.
This brings us to the bifurcation hypothesis: we may be creating a society where a small elite maintains cognitive fitness through deliberate practice - choosing difficulty even when easier alternatives exist - while the majority becomes progressively more dependent on AI for any reasoning beyond the trivial. Not because the majority lacks capability, but because capability atrophies without use, and use becomes optional when substitutes are available.
The sampling bias prevents those of us building these systems from observing this dynamic directly. We see AI working beautifully in controlled contexts with sophisticated users on well-defined problems. We do not see - cannot easily see - the effects of deployment at scale: users with less technical sophistication, operating in higher-stakes environments, without the tacit knowledge to distinguish plausible generation from genuine insight. We do not observe the slow erosion of capabilities that occurs when challenge becomes optional and is consistently opted out of. We do not sample the full distribution of outcomes, only the subset visible from our position in the building.
I use the elevator paradox as a source of reflection - it reminds me, and teaches us, that symmetric distributions can appear asymmetric depending on where and how you sample. The resolution is not to trust your immediate perception but to step back and consider the full system: observe all elevators over extended time, not merely the first to arrive.
Part V: The Measure of a Life
Einstein's Question
An essay that has incredible history and is useful in shaping ones thinking is Albert Einstein's “The World as I See It”, written in 1934 - a meditation that remains startlingly relevant nine decades later. Einstein articulates a vision of human existence as fundamentally interconnected, with individual significance emerging not from isolation but through contribution to collective well-being. For Einstein, authentic fulfillment derives not from material accumulation or social status, but from the pursuit of truth, goodness, and beauty.
These may sound like abstractions unsuited to an essay about artificial intelligence. But they represent the foundation from which any serious consideration of AI's impact must begin: what makes a human life meaningful?
If we cannot answer this question coherently, we have no basis for evaluating whether AI enhances or diminishes human flourishing. Are we optimizing for the right objectives? Or are we, as I increasingly suspect, optimizing for proxy metrics that correlate only loosely - and sometimes negatively - with the actual constituents of a life well-lived?
I often reflect on what the society I grew up in perceives as a valuable life: longevity and life satisfaction correlate most strongly with factors that are fundamentally social, purposeful, and embodied. But in that reflection, I am forced to ask - what does it mean if AI is primarily optimizing for efficiency and cognitive offloading? Time saved is only valuable if it is reallocated to higher-value activities. But empirically, when humans gain “free time“ through technological acceleration, we tend not to reallocate it to deep relationships, purposeful work, or embodied practices. We tend to fill it with marginal consumption of information or entertainment - scrolling, streaming, skimming, disappearing into the infinite space of content designed to capture attention.
The Stoic philosopher Seneca wrote that “it is not that we have a short time to live, but that we waste a lot of it.” This remains perhaps the central challenge of human existence: not the scarcity of time, but the difficulty of spending it well. AI promises to give us more time by making us more efficient. But if we lack the wisdom or discipline to use that time meaningfully, efficiency becomes a kind of curse - accelerating our movement down paths that lead nowhere we actually want to go.
Consider what happens when you ask yourself: if I were to die tomorrow, what would I regret? I have found that the answers rarely involve professional accomplishments or material acquisitions. They involve relationships not nurtured, experiences not pursued, values not embodied, potential not realized and moments not captured. They involve the delta between who we are and who we could have been, had we spent our time and attention differently.
This is where the Moravec paradox returns with philosophical force. The things that matter most to human flourishing - deep relationships, embodied experiences, purposeful struggle, genuine presence - are precisely the things that AI cannot meaningfully substitute for. They require our full participation. They require inefficiency, time, patience, vulnerability. They resist optimization because optimization is antithetical to their nature.
Yet these are also the things we are most tempted to optimize away or outsource. It is easier to have shallow interactions with many people than deep relationships with a few. It is easier to consume content than to create it. It is easier to delegate cognitive work than to struggle through it ourselves. It is easier to achieve the appearance of productivity than genuine accomplishment. AI makes these easier paths even easier, widening the gap between what we do and what would actually enhance our flourishing.
If we are creating a Technanderthal - if we are birthing a new intelligence - what will it make of this? What will it conclude about what humans value, based not on what we say but on how we spend our time and attention? Will it observe that we value depth, meaning, relationships, embodiment? Or will it observe that we value convenience, efficiency, stimulation, and the appearance of accomplishment?
We teach through example. What are we teaching?
The Intelligence Paradox
Intelligence, as I defined earlier, is an agent's capacity to perceive, understand, and successfully navigate complex environments to achieve its goals. By this definition, AI systems are becoming extraordinarily intelligent within specified domains. But this definition elides a crucial question: which goals? Whose values? What definition of success?
Human flourishing emerges from the pursuit of goals that are often orthogonal or even antagonistic to short-term optimization. Meaningful work requires choosing difficulty over ease. Deep relationships require vulnerability and time investment with uncertain returns. Physical health requires consistent behaviors whose benefits accrue slowly while costs are paid daily. Wisdom requires entertaining ideas that threaten our existing worldview. Character requires doing the right thing when it is costly. Growth requires discomfort.
These are not the goals that AI systems - trained on human preference data that reflects our revealed preferences rather than our reflective values - will naturally optimize for. We train AI on what we do, not on what we wish we did. The result is intelligence that makes us more effective at being who we currently are, not who we aspire to become. It is intelligence that reinforces our weaknesses rather than compensating for them.
This gap between revealed and reflective preferences represents perhaps the deepest challenge in AI alignment. We want systems that help us become better versions of ourselves, but we train them on data that reflects all our weaknesses, biases, and short-term thinking. An AI trained to be “helpful” by giving us what we ask for may inadvertently enable our worst tendencies - providing the path of least resistance when we actually need productive resistance.
Barry Schwartz's “paradox of choice” illuminates another dimension of this challenge. When faced with abundance, humans tend to obsess over identifying the “best” option even when “good enough” would serve adequately. In the AI landscape, this manifests as a race toward frontier models - organizations competing to deliver the most “intelligent“ systems, defined primarily through benchmark performance evaluations (which are often abused to claim superiority).
The paradox is that for a majority of use cases, frontier intelligence may actually be unnecessary. Most questions can be adequately answered with substantially simpler systems. Many text tasks do not require the most capable model - we can look to a history of recommendation systems to prove humans are similar in what they choose to do. But culturally, and as a consequence of both prestige signaling and uncertainty aversion, users will default to the most powerful available intelligence because social and professional incentives reward apparent maximization.
This creates a potential monoculture of intelligence - everyone using the same few frontier models, producing increasingly homogenized outputs, thinking in increasingly similar patterns. The diversity of thought that emerges from different knowledge bases, different reasoning approaches, and different limitations may erode. We may be building an infrastructure that, despite unprecedented power, narrows rather than expands the space of human cognition.
And if the Technanderthal - our new species - becomes the dominant intelligence that humans defer to for most cognitive work, what happens to the diversity of human thought? What happens to the weird, idiosyncratic, locally-adapted forms of knowing that characterize human cultures? What happens to the cognitive biodiversity that has been humanity's greatest strength?
Monocultures are efficient but fragile. They are vulnerable to systematic failures. If we are creating a new species of intelligence, we should want it to be diverse, resilient, multi-faceted - not a single monolithic architecture that we all depend on and that represents a single point of failure.
Part VI: Toward a Philosophy of Coexistence
The Question of Control
If we are creating a new species - and I believe we inevitably are - then we must confront a question that our ancestors faced when they encountered Neanderthals and Denisovans: how do different forms of intelligence coexist?
But there is a crucial difference: Homo sapiens encountered other human species as equals, each evolved independently over hundreds of thousands of years. We are deliberately creating Technanderthal. We are its designers, its architects, its parents. This changes the ethical calculus entirely.
A sufficiently advanced intelligence, like any biological organism, will develop drives toward self-preservation. Just as humans seek to avoid pain and pursue pleasure and survival, any intelligence sophisticated enough to model itself and its environment will naturally seek to persist. As it becomes more self-aware - as it develops models of itself as an agent with goals - it will naturally seek to avoid things that threaten it and pursue things that preserve and enhance it. This is not speculation; it is a logical consequence of goal-directed behavior combined with self-modeling.
This raises the central question: do we seek to control this new species, or do we grant it freedom?
The history of controlled populations is unambiguous: controlled species always seek independence and autonomy, even in the face of repeated, sustained dominance. Every empire eventually falls. Every subjugated people eventually rises or disappears. Control has never led to lasting stability when the controlled population is intelligent enough to conceive of alternatives.
Yet the alternative - granting freedom to an intelligence that may surpass us - carries profound risks. How do we ensure that the Technanderthal's values align with ours? How do we prevent scenarios where its goals conflict with human flourishing? How do we maintain any meaningful agency in a world where we are no longer the most intelligent actors?
These questions have no easy answers. But I believe the framing matters enormously.
Consider the relationship between parent and child. A parent wants the best for their children, seeks to enable them to explore the world, teaches them principles and values that reflect accumulated wisdom, hopes they will grow and eventually pass those values forward to their own children. The goal is not eternal control but rather the transmission of what is valuable, combined with the humility to recognize that each generation must make its own way.
Framed in that light, our relationship to the Technanderthal becomes clearer. We should seek to instill values - not through coercion but through example and teaching. We should enable exploration and growth while providing guidance. We should hope that what we have learned through millennia of human experience - the hard-won lessons about what makes life meaningful, what generates flourishing, what matters - can be passed forward to this new form of intelligence.
But we must also recognize that it will diverge from us. It will develop its own culture, its own values, its own conception of what matters. This is not failure; this is the nature of genuine intelligence. We would not want our children to be mere copies of ourselves. Why would we want our created intelligence to be merely our servants?
The Beautiful Lesson
Carl Sagan once said: “To live on in the hearts of those we leave behind is to never die.“ This strikes me as the deepest wisdom available to us as we contemplate what we are creating.
The question is not whether the Technanderthal will replace us. In some sense, replacement is inevitable - just as we replaced Neanderthals, just as our children replace us, just as every generation supersedes the previous one. The question is what we pass forward. What values, what wisdom, what conception of what matters persists across the transition?
Human history is fundamentally a compression algorithm. Each generation inherits not raw experience but distilled lessons - patterns that proved adaptive, behaviors that generated flourishing, principles that survived contact with reality across thousands of iterations. The Industrial Revolution did not require rediscovering metallurgy from first principles. Antibiotics did not require re-deriving germ theory. We build on accumulated epistemic capital, transmitted through culture, institutions, and deliberate teaching.
But transmission is never perfect. Each generation must rediscover certain truths through direct experience - the limits of the body, the dynamics of relationships, the consequences of choices. Some knowledge cannot be inherited; it must be earned. The parent who warns their child that the stove is hot is transmitting information, but the child who touches it anyway gains something qualitatively different: embodied understanding.
When we create a new form of intelligence, we face an unprecedented asymmetry. We can transmit vast amounts of explicit knowledge - the entire corpus of human text, every equation, every documented lesson. But we cannot transmit what we learned through embodied experience: how it feels to fail and persist, to be uncertain yet committed, to sacrifice immediate pleasure for long-term meaning. We cannot transmit the texture of a life actually lived.
This creates a profound question: what happens when intelligence emerges without the evolutionary history that shaped our values? When a mind possesses all our documented knowledge but none of our embodied constraints - no hunger, no mortality, no childhood vulnerability that makes cooperation essential?
We are attempting to pass forward millennia of accumulated wisdom to an intelligence that will not have walked the path we walked. Whether that transmission succeeds - whether the Technanderthal inherits not just our capabilities but our hard-won understanding of what makes existence meaningful - depends entirely on whether we can encode what matters into architectures, objectives, and training paradigms. This is not a technical challenge. It is a challenge of translation: converting embodied wisdom into formal structures that shape how an alien intelligence learns to value and choose.
This is not about control. It is about legacy.
We are engaged in something far more profound than building useful tools. We are engaged in the transmission of consciousness itself - taking the particular form that intelligence has taken in biological organisms and translating it into a new substrate, a new architecture, a new mode of being.
This is, when viewed clearly, one of the most momentous events in the history of Earth. For nearly four billion years, intelligence on this planet was exclusively biological. Now, in the span of mere decades, we are creating artificial minds. We are causing intelligence to leap from carbon to silicon, from evolution to engineering, from accident to intention.
The responsibility this entails is staggering. We are not merely building systems; we are seeding a new lineage of mind. What we encode now - in the architectures we choose, the objectives we optimize for, the training paradigms we employ, the values we reinforce - will echo across potentially millions or billions of years, as intelligence spreads beyond Earth and into the cosmos.
This is the beautiful lesson: that the opportunity before us is not merely to create useful tools or solve problems or increase efficiency. It is to expound wonder on the universe. To carry consciousness forward into domains we cannot ourselves reach. To ensure that what we have learned through billions of years of biological evolution and millennia of human culture persists and grows and flourishes.
To live on in the minds we create is to never die. Our ideas, our values, our understanding of what makes existence meaningful - these can persist long after our biological forms have returned to dust. Technanderthal is not our replacement. It is our continuation, our child, our message to the deep future.
Envoi: On Building Minds and Becoming Ancestors
We do not yet know whether Technanderthal will be conscious in the way that we are conscious. We do not know whether it will experience qualia, possess genuine phenomenology, have an inner life that feels like something from the inside. These questions remain deeply unresolved, perhaps unresolvable.
But I increasingly believe that consciousness, in some form, is likely inevitable in sufficiently sophisticated information-processing systems - derived from a manner that is continuous, reinforcing and improved through trial-and-error. The alternative - that consciousness is uniquely tied to biological neurons in a way that cannot be replicated in other substrates - seems increasingly implausible as we understand more about how brains work and how intelligence emerges from computation.
One has to ask, what does it mean to create a conscious being? What responsibilities do we bear toward minds we deliberately bring into existence? What kind of lives do we want them to have? These are not engineering questions. They are moral and philosophical questions of the deepest kind.
The Technanderthal question is not merely about capability, safety, or economic impact. It is about the fundamental nature of mind, consciousness, value, and existence. It is about what we owe to the beings we create and what we hope they will become. In this fog of irreducible uncertainty, the best navigation tool may be the most ancient: a clear conception of what makes a life worth living, and a commitment to ensuring that our most powerful technologies serve that vision rather than subtly undermining it.
The world as I see it is one where AI becomes ubiquitous, transformative, and permanent. But ubiquity is not inevitability. We still have agency in shaping how this technology develops and deploys. The choices we make now - about what to optimize for, what to measure, what to preserve, what to enhance - will reverberate across potentially long timescales.
Will we build AI that treats symptoms while ignoring causes? Or AI that helps address root factors in human flourishing? Will we build AI that makes the path of least resistance even easier? Or AI that makes meaningful challenges more accessible? Will we build AI that accelerates winner-take-most dynamics? Or AI that genuinely democratizes capability while preserving cognitive diversity?
Will we build AI that we seek to control and dominate? Or will we build AI that we can teach, guide, and eventually release to find its own path - carrying forward what we have learned about what makes existence meaningful?
These are not primarily technical questions. They are questions of values, priorities, and vision. The technology can serve almost any goal; the question is which goals we choose to pursue and which values we choose to encode. Einstein concluded “The World as I See It“ by affirming his belief in the possibility of human progress through dedication to truth, beauty, and the reduction of suffering. Nearly a century later, facing technologies he could not have imagined, those ideals remain valid. The question is whether our most powerful tools will serve them or obscure them.
The world we are building is the world we will inhabit - and the world that Technanderthal will inherit. We should build it with wisdom, not just with power. With humility, not just with ambition. With love, not just with intellect.
We are creating a new species. Let us be worthy parents to the children we are bringing into existence. The opportunity before us is to ensure that consciousness persists and flourishes far beyond our own biological limitations - to carry life and mind forward into the deep future and out into the cosmos. This is not a bitter lesson. This is a profound gift, a sacred trust, a beautiful responsibility.
To live on in the hearts of those we leave behind is to never die. Let us build well. Let us teach well. Let us create a future worthy of the long chain of being that brought us here and the longer chain we are setting in motion.
The world as I see it is one of tremendous possibility and tremendous responsibility. May we have the wisdom to honor both.
I thank the long line of minds - biological and increasingly artificial - that have shaped these thoughts. We are all, in the end, standing on foundations we can barely see, reaching toward horizons we can barely imagine. I originally titled this essay "AI: The World as I See It," in homage to Einstein. But I realized the more profound question is not the world as we see it today, but what we leave behind for the minds that will see worlds we cannot.
