LawY and benchmarking AI

UK legal professionals embrace AI: how legal specific platforms are driving accuracy

Artificial intelligence (AI) use is no longer just a possibility for the legal profession; it is already supporting the daily work of thousands of lawyers across the UK. According to recent research*, 61% of lawyers are now using AI, up from 46% earlier this year. Only 6% now say they have no plans to adopt AI, down from 15% just six months ago. The real challenge is no longer whether lawyers will use AI, but how and whether they can do so safely, effectively and strategically.

The challenge of comparing accuracy levels of competing AI platforms

As more lawyers are turning to AI for assistance, one key challenge stands out; it is difficult to judge the accuracy of the competing AI platforms they use. Off-the-shelf Large Language Model (LLM) systems such as ChatGPT or Google Gemini are widely used for drafting and admin tasks, but their suitability for complex legal work remains in doubt. Errors in output can become professional mistakes with consequences for lawyer, law firm and client.

Half of lawyers surveyed are relying on general-purpose AI platforms, while the other half are using legal-specific AI systems. This difference in approach matters. General tools may save time but are not designed to give lawyers a trusted springboard for their legal work. In contrast, legal-specific tools, benchmarked against trusted legal sources, are proving more reliable. 88% of lawyers using legal AI exclusively said they felt more confident in its output.

Why accuracy of AI research must be measured

Confidence in AI cannot be built on perception or marketing claims alone; it must be earned through proof. By benchmarking the AI solutions used in law firms, you are providing that proof, giving legal professionals an objective standard to evaluate real performance.

LawY, the legal AI research assistant, recently published the first evaluation* of its system, as part of their commitment to radical transparency and ongoing innovation. This evaluation used automated benchmarking to compare its answers with those from two other commonly used platforms. Each response was assessed against a human-verified “golden answer”, a carefully defined source of truth.

When tested on a broad range of typical legal questions across multiple areas of law, LawY achieved 86% overall accuracy in the UK, compared with 57% for Gemini 2.5 Pro and 54% for ChatGPT 4.1. By only sharing comparative results, LawY aims to empower lawyers to make informed decisions about their AI research tools. They are committed to publishing future evaluations that will both empower users and also inform the future development of its platform.

Defining Safe AI in Practice

Evidence shows that safe adoption of AI already brings tangible benefits. Lawyers are using the time saved by AI to focus on higher-value work, and many report improvements in their work-life balance. Increasingly, there is also recognition that AI will reshape billing models, accelerating long-anticipated changes in legal pricing.

The risks of inaction are equally clear. Lawyers are beginning to question whether firms that fail to invest in AI will be able to support their careers or remain competitive in the long term.

Gareth Walker CEO of LEAP, a leading provider of legal practice management software and integration partner of LawY says “The legal sector is at a turning point. While AI adoption is accelerating, culture and confidence are struggling to keep pace. Firms that lean on generic tools without clear frameworks risk serious missteps.”

By contrast, firms that adopt AI safely, monitor appropriate internal usage rigorously, and integrate it strategically stand to improve client service, gain a healthier work-life balance, and a lasting competitive edge. However, to realise these benefits, firms need clear guiding principles:

• Choose sector-specific tools. Legal research requires platforms built for law, not generic LLMs.

• Insist on transparency. Demand published benchmarking results and accuracy data from AI providers.

• Embed a firm-wide strategy. Adoption cannot be left to individual discretion; it needs governance and clear policies.

• Invest in training. Equip lawyers to critically assess AI outputs and use them responsibly.

• Prioritise client outcomes. AI should deliver tangible benefits—greater efficiency, lower costs, and more accurate advice.

*Sources

1. The AI Culture Clash, published by LexisNexis, and covered in

https://www.lawgazette.co.uk/news/two-thirds-of-lawyers-using-ai-though-confidence-lags/5124323.article

2. LawY evaluation results and methodology - https://www.lawy.ai/accuracy#results