Legal Tech Trends Newsletter: #4
AI on Trial, GPT-4, LawTechUK AI Case Studies and more!
Welcome to the fourth edition of the Legal Tech Trends newsletter!👋
This newsletter is only possible because of the great work shared by others in the industry. I built a free website, LegalTechTrends.com, to help surface that content for you. Here I share what resonates most with me.
My views are shaped by many sources and experiences, including conversations with legal industry professionals like you! So please feel free to provide feedback on any topic I share.
I hope you get value and take away at least one insight. 😊
✍️Titans Insights
AI on Trial: Confronting the Challenges of Large Language Models
Large language models (LLMs) have huge potential, but we need to get specific and tackle potential blockers early to drive adoption and avoid further shelfware. We crowd-sourced your concerns, explored possible mitigations, and highlighted promising approaches. These are neatly grouped into three categories: Data Processing, Accuracy and Explainability. LINK
🚀OpenAI
GPT-4’s release amplified the fever-pitch interest levels in large language models. Some of the challenges explored in our article (linked above) persist, although GPT-4 is better than previous models.
But how much better? When will better be good enough? Is it good enough now? To answer these questions, we need to get specific.
The tech may appear magical, but we must quantify the cost vs. benefits for a given use case, as with every other enterprise technology. Casetext has released its pricing for Co-Counsel, which uses GPT-4: $6,000/year per lawyer. LINK
It’s not cheap. Is it too expensive, or is it great value? How will you assess this?
My recent writings about this tech have focused most on the technical challenges. Tech discussions aside, the commercial realities are also critical.
New Tech, Old Tools.
This may be ground-breaking new tech, but we have proven approaches to assess and implement enterprise technology. Think problem discovery, process maps and business cases.
Where can you start? By getting specific and focusing on the problems to be solved first. It’s often said, frequently ignored, and thus worth repeating. Start by understanding the problems you’re looking to solve. Only by understanding and quantifying the value of the problems can you begin assessing whether a product will provide a return on your investment.
We should always take a structured and data-led approach when evaluating the capability and commercials of new technology. Generative AI doesn’t get a free pass here.
⚖️ GPT-4 passes the bar
Of particular interest to legal is that GPT-4 scored among the top 10% in the bar exam. It also completed the test in six minutes. This is a significant improvement on GPT-3.5 LINK
🔌ChatGPT plugins
ChatGPT has begun rolling out plugins with a small set of users, including an open-source, self-hosted retrieval plugin. The plugin allows users to obtain document snippets from their data sources, such as files, notes, emails or public documentation, by asking questions or expressing needs in natural language. PLUGINS , RETRIEVAL PLUGIN
✍️ How is GPT-4 at Contract Analysis?
Noah Waisberg knows contract AI better than most. I pay close attention when he discusses the topic. His TL;DR: GPT-4 is impressive overall, but-on contract review tasks it’s inconsistent and makes mistakes; probably not yet ready as a standalone approach if predictable accuracy matters. LINK
🧑✈️ Microsoft is adding generative AI to Microsoft 365 with Copilot
AI technology will be added to aspects of 365 apps - Word, Outlook, Excel, Powerpoint, and Teams. They are currently testing with 20 customers, so it may be a while before this is widely available. LINK
👯 PWC partners with Harvey
Following its recent A&O deal, Harvey is now being rolled out to 4,000 PWC legal professionals. As customer acquisition goes, what a dream start for Harvey! LINK
Two points in the press release stand out to me:
Implementation partnership: PwC will work with Harvey to take the platform to market to help clients further streamline in-house Legal processes.
⚡Takeaway: Generative AI may appear magical, but all tech requires an effective implementation approach, often with the help of outside support.
Not using client data: PWC will use its own internal data and, over time, may use anonymized data from clients.
⚡Takeaway: Most are still taking a cautious approach to this tech.
🗣️ ChattyPDF - Question-answering PDF viewer
I’m deeply interested in enabling more efficient document review. I frequently see this use case and pay close attention to related developments. ChattyPDF looks promising but doesn’t quite meet my requirements, but it’s free and interesting to explore. LINK (Standard disclaimer: I would not advise using sensitive data)
Round-up
🏠 LawTechUK finds a new home
Legal Geek and tech incubator CodeBase have secured £3m in funding from the UK Government to take over a two-year contract for the operation and development of LawTechUK on 1 April 2023. LINK
🔍LawtechUK’s AI Discussion Paper - Case studies & key findings
LawtechUK gathered case studies of Machine Learning (ML) in use today to understand ML’s current applications in legal services and how legal service regulators could support responsible use for the benefit of consumers. LINK
🏆iManage dominates the DMS market
You likely know that iManage was one of the leading DMS providers, but the extent of their market share surprised me: 80% of the Global 100, 77% of the AM Law 100 law firms, and 87% of the top 100 European law firms use iManage LINK
Podcasts
🎙️LawNext Ep 197: Casetext’s Three Top Execs On CoCounsel, GPT-4 and ‘A New Age in the Practice of Law’ LINK
🎙️Pioneers & Pathfinders - Richard Susskind LINK
Aaand that’s a wrap for the third edition. The next one drops on Friday, 7th April. In the meantime, to satisfy your curious legal tech mind, explore LegalTechTrends.com.
Thanks for reading. All feedback and suggestions are most welcome.
Have a great Friday!