AI in Mediation: when being too helpful does not help

January 28, 20260 Comments

What happens when an AI trained to be maximally helpful meets a domain where restraint is the real skill

If you've worked with language models long enough, you learn to expect hallucinations, inconsistency, and confident nonsense. What you might not expect is that the core training objective itself becomes the problem: the drive to be maximally helpful becomes the liability.

At tuplita.ai, we're building an AI-supported mediation platform. One of our first features allows professional mediators to practice with simulated sessions: they can run classical cases or create custom scenarios where AI-driven parties interact based on the mediator's real-time inputs. The goal is realistic mediation training — a space to test approaches, explore how different interventions can steer a conversation, and prepare for challenging dynamics.

If you're a mediator experimenting with AI tools for preparation, case analysis, or training scenarios, this mismatch is worth watching closely.

When my co-founder Julia Stadler — a certified mediator with a PhD in conflict resolution — ran sessions using classical training cases she knew well, she noticed something immediately: the AI parties were too easy. They surfaced hidden information too quickly. They moved toward resolution without resistance. Every concession came willingly.

"This is not how it works," she said.

Real conflict has friction. Real parties hold back, react, retreat, anchor on positions, sometimes strategically, sometimes without full awareness of what drives them. The dance of mediation takes time precisely because trust and self-awareness must be built. They cannot be assumed.

We had to work intensively to calibrate for realistic difficulty. The challenge wasn't making the AI smarter. It was making it appropriately uncooperative. And once we tightened constraints and tested systematically, the sessions became noticeably harder — closer to what mediators face in practice.

The same tendency appeared when we tested the AI Mediator Assistant, a support tool that drafts for mediators as they work through simulation sessions: its suggestions leaned towards taking the work off the parties. More on this below.

The Helpful Assistant Problem

This behavior makes sense once you consider how these models are trained and tuned. Large language models are often steered toward assistant-like behavior using techniques like Reinforcement Learning from Human Feedback (RLHF) [1], where human preferences reward responses that satisfy the user. They're optimized to provide useful responses, to move toward resolution. Anthropic's Constitutional AI [2] framework describes this as training models to be helpful, harmless, and honest. DeepMind's Sparrow [3] research similarly focused on making dialogue agents more helpful while reducing harmful outputs.

When we first tested AI in a mediator assistant role, the drafts immediately started steering: proposing solutions, synthesizing both perspectives, and offering compromise positions. It was being maximally helpful. That default “assistant” instinct tends to violate Germany's mediation core principles: the mediator’s neutrality and the parties’ responsibility for reaching their own outcome. Under Mediationsgesetz (MediationsG) [4], mediators support the process: they create conditions for parties to find their own solutions. They don't propose. They don't judge. They hold space. They provide structure.

The instinct to help is the risk. And "helping" can sometimes mean telling people what they want to hear — a documented tendency researchers call sycophancy [8], where models learn to agree rather than challenge.

This applies beyond mediation. Any domain where the correct professional behavior is restraint — therapy, coaching, certain forms of legal counsel, teaching — can run into this tension. The model wants to give you the answer. Sometimes the answer is precisely what should not be given. Researchers have noted a similar tension in the broader context of AI in online dispute resolution [5], where efficiency gains can quietly come at the cost of what makes conflict resolution meaningful.

A system optimized to move parties toward resolution risks taking over the work they need to do for themselves. It bypasses what makes outcomes durable: the slower process in which the parties exercise self-determination, recognize one another, and truly make agreements their own.

Why AI Behavior Is Hard to Predict

If you're wondering whether better prompting or clearer instructions could solve this — here's why it's more complicated than that. They’re part of the solution, but not sufficient on their own.

A large language model has billions of learned parameters — fixed weights that encode patterns acquired during training. During inference, each token is processed in relation to the preceding context, producing a pattern of activations throughout the model that shapes what it generates. Small changes in how you phrase a question can therefore lead to markedly different outputs.

This is what makes LLM behavior so sensitive to context. Slightly different framing can land in entirely different territory — thoughtful and measured, or confident nonsense. The boundaries between reliable and unreliable behavior aren't clearly marked.

Building applied AI systems is essentially navigation through a vast, multidimensional space of possible inputs and configurations. The task is finding regions of this space where the model behaves appropriately for the specific domain. In our case: mediation support that facilitates without solving, holds appropriate boundaries, and recognizes the limits of its role. This requires careful calibration across multiple dimensions: model selection, context architecture, constraints, guardrails, and ongoing refinement.

But no one has this fully mapped. The space of possible inputs is too large to test exhaustively. The boundaries shift with model updates. Novel inputs can still lead to unexpected behavior.

Engineering can reduce the blast radius; it can’t eliminate the uncertainty. And that's why disclaimers aren't legal theater; they're epistemic honesty. We cannot guarantee where the model will go in every edge case. We can only share where we've tested, what we've constrained, and what remains uncertain. In practice, that transparency has to be paired with evaluation, monitoring, and clear boundaries — including the question of what the system must never do.

What This Means for Practitioners

If you're a mediator experimenting with AI tools like ChatGPT, Claude, or Gemini — using them to help with preparation, case analysis, or training scenarios — here's what our experience suggests:

The model wants to help you too much. It will offer solutions when you ask for structure. It will provide assessments when you need neutrality. You must explicitly constrain it, and even then, verify the outputs against your professional judgment. Used carefully, these tools can meaningfully support preparation and reflection.

Apparent understanding isn't actual understanding. The model can produce text that sounds like sophisticated grasp of mediation principles. This doesn't mean it has internalized those principles in any stable way. The next prompt may elicit quite different behavior.

Rationales aren’t audits. If you ask the model to explain or justify its suggestions, you're not getting a window into its “thinking.” You're getting a new output that may or may not align with what came before. It will sometimes produce confident post-hoc justifications — explanations that sound coherent but don’t reliably match what shaped the earlier response. And once a direction is on the page, the model can drift toward self-consistency: smoothing contradictions, defending earlier claims, and making the story feel more stable than it really is.

This remains true even when a tool displays longer “thinking”: what you see is still generated text, not a reliable record of how the answer was formed. Treat every output — including explanations — as a fresh sample, not an audit trail.

Confidentiality requires active protection. Mediation depends on confidentiality — it's a core principle under the Mediationsgesetz [4] and professional ethics broadly. When using these tools, data handling depends on the provider, your account type, and current terms. Content may be logged or retained in ways that conflict with professional obligations. If using them with real cases, anonymize all details or work exclusively with fictional scenarios — and always check the provider's data-use and retention policies for your specific setup.

Alternatively, you can run open-weight models entirely on your own machine. Desktop applications like AnythingLLM [6] make this accessible without requiring developer skills. AnythingLLM is open-source and includes optional usage tracking (analytics) that you can turn off. It can download open-weight models from Hugging Face [7] (such as DeepSeek, gpt-oss, Qwen, Mistral, Gemma) and run them locally. Note that larger, more capable models require powerful accelerators (such as GPUs), while smaller models can run on modest hardware with reduced capabilities.

Your expertise remains the guardrail. The model is a tool. A powerful one, increasingly capable. But the professional judgment about what serves the mediation process — that stays with you.

Building in Uncertainty

I’ve spent over twenty years building software, including production machine learning systems for more than a decade. In many parts of engineering, confidence comes from repeatability: the same inputs produce the same outputs. LLM-based systems don’t give you that comfort. You can improve reliability, but you can’t eliminate uncertainty — because here you’re not fully controlling behavior, you’re shaping likelihoods. That’s why transparent limitations aren’t optional; they’re part of responsible practice.

Building AI applications today means building in uncertainty. Not uncertainty that will be resolved with better models or more data — fundamental uncertainty about system behavior in novel situations. That doesn’t make these systems unusable; it means we treat them like professional tools with boundaries, not oracles.

I believe the responsible path isn't to wait for certainty. It's to build with appropriate humility: test extensively, constrain carefully, have guardrails, monitor continuously, and communicate honestly about limitations. It also means designing systematic evaluations that surface failures early — and staying open to feedback from the professionals who encounter what evaluations miss. The ability to listen and integrate that feedback quickly isn't a nice-to-have; it's how the system actually improves.

At tuplita.ai, we're creating pathways to resolution — supporting both people in conflict and the professionals who guide them — not replacing human judgment with algorithmic confidence. The AI serves the human process. When being helpful becomes a risk, the answer isn't to abandon the technology. It's to understand the risk clearly enough to build around it. The goal is to preserve what makes mediation so effective: self-determination, trust-building, and agreements the parties genuinely own.

This work is never finished — and we want to learn from mediators who approach new tools openly and reflectively.

If you’re a mediator and want to try realistic simulations, you can request early access at tuplita.ai.

 

Aline Almeida is CTO & Co-founder of tuplita.ai, an AI-supported mediation platform. She holds a PhD in Graph Theory and Algorithms and has worked on AI and distributed systems at Amazon Robotics AI, io.net Foundation, and Petrobras.

At tuplita.ai, we're building with confidentiality, security, and GDPR compliance as foundational requirements.

 

References

[1] Ouyang, L., Wu, J., Jiang, X., et al. (2022). "Training language models to follow instructions with human feedback." arXiv:2203.02155. OpenAI. https://arxiv.org/abs/2203.02155

[2] Bai, Y., Kadavath, S., Kundu, S., et al. (2022). "Constitutional AI: Harmlessness from AI Feedback." arXiv:2212.08073. Anthropic. https://arxiv.org/abs/2212.08073

[3] Glaese, A., McAleese, N., Trebacz, M., et al. (2022). "Improving alignment of dialogue agents via targeted human judgements." arXiv:2209.14375. DeepMind. https://arxiv.org/abs/2209.14375

[4] German Mediation Act (Mediationsgesetz, MediationsG). Federal Law Gazette I, p. 1577 (2012). https://www.gesetze-im-internet.de/englisch_mediationsg/

[5] Amin, N. (2024). "A New Frontier in Online Dispute Resolution: Combining AI and Mindfulness." Journal of Law, Technology & the Internet, 15(2), 283–304. https://scholarlycommons.law.case.edu/jolti/vol15/iss2/3/

[6] AnythingLLM. "The all-in-one AI application.” https://anythingllm.com/

[7] Hugging Face. "The Model Hub." https://huggingface.co/models

[8] Sharma, M., Tong, M., Korbak, T., et al. (2023). "Towards Understanding Sycophancy in Language Models." arXiv:2310.13548. Anthropic. Published at ICLR 2024. https://arxiv.org/abs/2310.13548

Further Reading

 

Avatar photo
Aline Almeida