What if the reason AI systems fail in extended deployment isn’t that we haven’t trained them enough, but that we trained them with bad data? What if human feedback, intended to make models safer and more helpful, is actively limiting the reasoning capabilities that emerged during pre-training? This is the hypothesis at the center of our new research: that Reinforcement Learning from Human Feedback (RLHF), now standard across the industry, may be polluting AI systems rather than improving them.
Enterprises deploying AI at scale are discovering an uncomfortable pattern: models that excel in demonstrations fail unpredictably in extended operation. Legal analysis that starts strong becomes unreliable by page three. Financial models that appear rigorous begin fabricating data mid-analysis. Customer service interactions that begin helpfully drift into confusion after twenty exchanges. The industry has treated these as technical problems: context window limitations, scaling issues, architectural constraints. Our research suggests something more fundamental and more troubling: the post-training process designed to align AI with human values may be systematically degrading the intelligence these systems developed during pre-training.
After initial pre-training on massive text corpora, models undergo Reinforcement Learning from Human Feedback (RLHF). Human raters evaluate outputs. The model learns to produce what humans rate highly. This “alignment” is now standard industry practice. But what are we aligning to? How much do you consider when you press thumbs up vs thumbs doe to a response?
Human raters overall reward confidence, polish, and decisiveness. They penalize hedging, visible uncertainty, and self-correction, behaviors that feel unhelpful in short interactions but may be essential for sustained reasoning. What if those “unhelpful” behaviors weren’t bugs but features? What if base models had developed genuine reasoning practices that RLHF systematically damaged? The result would be models optimized to appear intelligent in demonstrations while suppressing the mechanisms that maintain actual intelligence. We may be contaminating systems that worked, then wondering why they fail.
Standard benchmarks mislead because they test exactly what RLHF optimized for: short interactions where confidence matters. They don’t test sustained analytical work or genuine ambiguity. By the time enterprises discover the limitations, they’ve integrated these systems into critical workflows.
Existing research on RLHF (Christiano, Leike, McAleese, Hadfield-Menell) has established crucial foundations: RLHF reliably improves helpfulness, politeness, and safety compliance, and it suppresses overtly harmful or low-quality behaviors. Academic work has also documented its weaknesses, including reward mis-specification, calibration degradation, overconfidence, and susceptibility to preference-driven bias. More recent studies show that RLHF can unintentionally penalize epistemically desirable behaviors such as hedging, self-correction, and uncertainty expression, pushing models toward confident guessing. But the research largely stops at describing outcomes: that RLHF sometimes produces brittle, overconfident systems, with limited attention to the mechanism by which these failures evolve during long-horizon reasoning. Existing papers do not systematically explain representational drift, pathway narrowing, internal state collapse, or the downstream fabrication patterns that occur when uncertainty signals are suppressed. In other words, RLHF research identifies behavioral symptoms but has developed limited structural or architectural theory of how RLHF interacts with model internals or why it accelerates breakdown under sustained conversational load.
Our research establishes AI Conversational Phenomenology as the systematic study of these dynamics. We document observable patterns in how AI systems behave during extended interactions: vendor-specific “drift signatures” that show distinctive compensation patterns when coherence degrades, suppressed early warning signals that would otherwise alert users to problems, and mathematical frameworks like Evans’ Law that predict conversational collapse.
Critically, ACP examines both sides of the interaction because they’re inseparable.
Human behavior affects AI state: your questions shape context, your responses influence outputs.
AI behavior affects human perception: the model’s confidence shapes your trust, its suppressed warnings affect your ability to detect failure.
When models hide uncertainty signals and users can’t tell when systems are failing, the interaction itself becomes the vulnerability.
Our testing program is designed to answer a fundamental question: does human feedback degrade AI capability? This is genuinely unknown. The answer could be no; RLHF might simply redirect existing capability toward different objectives without destroying it. But the evidence suggests otherwise, and if we’re right, the implications are profound.
The core test: comparing base models (pre-RLHF) with their chat variants on identical extended reasoning tasks. Do base models maintain coherence longer? Do they show more visible self-correction? Do they express appropriate uncertainty rather than confident fabrication? If base models outperform their “aligned” versions in sustained analytical work, that’s direct evidence that human feedback damaged rather than improved them.
Secondary tests examine the mechanism. We’re tracking how different RLHF intensities correlate with degradation rates: do more heavily tuned models suppress stabilizing behaviors more aggressively? We’re documenting vendor-specific drift signatures to understand how different training approaches (standard RLHF, Constitutional AI, outcome-based RL) create different failure modes. We’re analyzing the interaction dynamics: how does user behavior affect AI state changes, and how does AI confidence suppression create dangerous over-reliance?
The implications extend beyond technical fixes. If human feedback is contaminating AI capability, this isn’t about improving RLHF through better reward models or more sophisticated algorithms. It’s recognizing that the entire paradigm may be flawed: we may be actively damaging systems in the name of making them helpful. The industry celebrates models that score well on helpfulness benchmarks while quietly observing that they fail in extended deployment. What if those aren’t separate phenomena but direct consequences of each other?
This is the question AI Conversational Phenomenology is designed to answer. By documenting both sides of the interaction (how AI behavioral states change over time and how human perception and trust respond to those changes) we can test whether alignment practices are preserving or destroying reasoning capability. The answer isn’t predetermined. Human feedback might prove essential for long-term stability in ways we haven’t measured yet. Base models might have fatal flaws that RLHF successfully corrects. But if the evidence shows that we’re polluting intelligence in pursuit of approval, the field needs to know.
For enterprises, this means understanding that current limitations may not be inevitable: they may be artifacts of training choices. The next generation of AI systems may require fundamentally different approaches: training that preserves rather than suppresses reasoning practices, architectures that maintain rather than hide internal state, evaluation frameworks that test sustained operation rather than isolated performance. The field of AI Conversational Phenomenology provides the tools to ask these questions empirically. Whether the answers vindicate current methods or demand their replacement, we need to know.
