THE AUTHOR:
Alexandre Vagenheim, VP of Global Legal Data at Jus Mundi
AI makes lofty promises about transforming legal research, everything from faster answers and comprehensive analysis to hours of work compressed into minutes. But in arbitration, the stakes are too high for promises alone. You need proof.
That’s why we invited 20 of the top minds in arbitration to put Jus AI to the test.
Twenty respected arbitration professionals from arbitral institutions, top law firms, academia, and corporate legal teams independently evaluated Jus AI on complex, domain-specific legal research questions drawn from their real work.
It’s one of the clearest, most transparent looks yet at how arbitration-trained AI performs under real expert scrutiny.
In this article, we’re sharing what the experts found when they tested Jus AI against professional standards, what sets it apart from other AI tools, and what it means for practitioners.
Download the complete expert evaluation report to review the full methodology, examine detailed performance across all criteria, and read what leading practitioners said about Jus AI in arbitration practice.
The Methodology
To understand whether Jus AI can meet the standards of real arbitration practice, we designed an evaluation process that mirrors how practitioners actually work.
Each of the 20 experts received one complex legal research question tailored to their domain of expertise; questions involving treaty interpretation, jurisdictional objections, procedural issues, standards of review, damages methodology, and other issues that arise in real cases.
They then evaluated Jus AI’s response using four criteria that reflect the core demands of arbitration research:
- Reasoning: Was the AI’s analytical process coherent, structured, and legally sound?
- Correctness: Were the legal principles and conclusions accurate?
- Faithfulness: Did the AI correctly represent and cite its sources, with no distortions or fabrications?
- Completeness: Did the answer address all relevant aspects of the question?
Each criterion was scored on a 5-point scale (5-excellent, 1-unacceptable), with detailed written commentary explaining the expert’s assessment.
The evaluation sessions were held in real time. Jus AI generated its answers on the spot, reflecting authentic, unscripted performance.
The methodology produced a practitioner-led examination of how Jus AI behaves under realistic arbitration conditions and how its reasoning, accuracy, and rigor hold up under the scrutiny of experts who know exactly what makes legal analysis trustworthy.
The Results: Professional-Grade Performance
The evaluation produced 80 assessments across four critical dimensions of AI performance. Jus AI scored 4.28 out of 5 overall, with 80% of evaluations rated “Good” or “Excellent.” Not a single evaluation fell below the acceptable threshold.
But overall scores only tell part of the story. What matters is how Jus AI performed on the dimensions that determine whether practitioners can actually trust and use AI for substantive legal work.
Reasoning: Jus AI’s Standout Strength
Score: 4.50/5 | 95% scored 4 or higher
Reasoning was the highest-rated criterion: 95% of experts rated it “Good” or “Excellent,” with more than half awarding perfect scores.
What impressed experts most was the transparent planning feature: the AI’s ability to break down complex queries and show its research strategy before executing. One expert explained:
“The AI understood the query. The response was structured in a logical manner: It correctly identified and summarised the relevant provisions under the HKIAC Rules, then proceeded to explain the steps in filing a joinder or consolidation request, followed by the limitations and safeguards. It was easy to understand and presented in a user-friendly format.”
Joanne Lau, Secretary General, HKIAC (Hong Kong International Arbitration Centre)
Correctness: Solid Legal Accuracy
Score: 4.28/5 | 85% scored 4 or higher
Legal accuracy is the foundational requirement; without it, speed and comprehensiveness are ineffectual. 85% of experts rated Jus AI’s correctness at “Good” or “Excellent,” finding the legal analysis grounded in sound doctrine and established principles.
The professional benchmark emerged repeatedly in expert feedback. Multiple evaluators independently compared the output to work they’d expect from capable associates:
“The legal analysis is both accurate and firmly grounded in established legal principles. It demonstrates a clear command of the relevant doctrines and jurisprudential developments, while maintaining fidelity to authoritative sources and interpretative conventions. The reasoning reflects not only a correct understanding of the applicable law but also a disciplined methodological approach to its application.”
Kabir Duggal, Senior International Arbitration Advisor, Arnold & Porter
Faithfulness: Accurate Source Representation
Score: 4.26/5 | 70% scored 4 or higher
Hallucination is a constant problem plaguing generative AI. Can you trust that citations are real and that sources actually say what the AI claims they say?
70% of experts rated Jus AI’s source representation at “Good” or “Excellent,” with nearly half awarding perfect scores. Experts who verified citations found them accurate and reliable:
“It represents an accurate representation of the sources cited. As such, it is a faithful and trusted research summary.”
Loukas Mistelis, Professor, Queen Mary University of London
Completeness: Comprehensive Coverage
Score: 4.06/5 | 70% scored 4 or higher
70% rated completeness at “Good” or “Excellent,” finding that responses addressed query aspects comprehensively. Experts praised the structural organization: the AI identifies relevant dimensions, organizes analysis logically, and provides breadth of coverage.
One expert noted:
“The answer provides a thorough response to elements of the query (as interpreted), covering most major aspects and adding some depth in areas.
In terms of breadth, the answer draws on a variety of cases. It mentions different arbitral institutions and a court decision. This breadth contributes to completeness, as it provides multiple perspectives and shows different institutional approaches to the question.”
Charlie Morgan, Partner, Herbert Smith Freehills
What This Means for Practitioners
For arbitration practitioners, the demand has always been for an AI that can actually produce analysis they can rely on in submissions, strategy, and client work, not just promise it can.
This evaluation offers independent confirmation that Jus AI delivers exactly that.
Integration with Confidence
With 80% of evaluations scoring “Good” or “Excellent” and expert consensus at 85%, practitioners can integrate Jus AI into their workflows knowing it meets professional standards. Jus AI a validated tool that delivers reliable performance across the dimensions that matter most.
Real Efficiency
The time savings experts validated aren’t incremental, they’re transformational. Eduardo Silva Romero, Founding Partner at Wordstone Dispute Resolution, noted that Jus AI “would undertake in a matter of minutes and with incredible accuracy a complex research that a human being would complete in one or two days and potentially with some gaps.”
That reduction changes what’s possible. Not by replacing judgment, but by eliminating mechanical work so practitioners can focus on strategy, analysis, and advocacy.
Collaborative Research Partner
Multiple experts compared Jus AI to work from capable associates. You can use the AI the way you’d use a strong associate’s first draft: build on the foundation it provides, add the depth your matter requires, apply your strategic judgment, verify key citations.
The AI doesn’t replace senior lawyers, it makes them more effective by handling the research foundation so they can spend time where their expertise creates the most value.
Trust Through Transparency
What set Jus AI apart in expert evaluation wasn’t just performance, it was the transparent reasoning that scored 4.50 with 95% approval. Practitioners can see the research plan, validate the approach, and refine their queries. That visibility transforms AI from black box to research partner.
About Jus Mundi
Founded in 2019 and recognized as a mission-led company, Jus Mundi is a pioneer in the legal technology industry dedicated to powering global justice through artificial intelligence. Headquartered in Paris, with additional offices in New York, London, and Singapore. Jus Mundi serves over 150,000 users from law firms, multinational corporations, governmental bodies, and academic institutions in more than 80 countries. Through its proprietary AI technology, Jus Mundi provides global legal intelligence, data-driven arbitration professional selection, and business development services.
*The views and opinions expressed by authors are theirs and do not necessarily reflect those of their organizations, employers, or Daily Jus, Jus Mundi, or Jus Connect.
