Last year’s launch of the Legal Services Innovation Index attracted much more attention than I anticipated. I knew that some would find the information useful, having been in the position of trying to gather data about innovation both as a lawyer in a law firm and a professor in a law school. But I underestimated the demand.

To date, there have been over 29,600 views of the three Tableau visualizations that make up the Index: the Law Firm Innovation Catalog, the Law Firm Innovation Index, and the prototype Law School Innovation Index.

Now that I’ve moved to Chicago and settled in at Northwestern Pritzker School of Law as a Visiting Professor of Law, I’ve begun work to update each component of the Index. Fortunately, three Northwestern second-year law students working with me as research assistants have already made substantial contributions to this project:

  • Lauren Diner
  • Doug Lavey
  • Yoon Hoo Lee

I’ve included their bios below, which will soon be added to the Index website.

Lauren, Doug, Yoon, and I drafted the following update.

The Index To Date

The Legal Services Innovation Index launched in August 2017 to describe and measure the state of legal-services delivery innovation. Version 1.0 launched with (1) a Catalog of more than 200 innovations from over 100 law firms of all sizes located in the US, UK, EU, Australia, and Canada and (2) a Law Firm Innovation Index measuring innovation in 260 law firms (the AmLaw Global 100, AmLaw 200, and Canadian 30).

In November 2017, we launched a prototype Law School Innovation Index and added 17 innovations in version 1.01 of the Catalog. Law firms had previously submitted each of these innovations via the submission form on the Catalog page.

Our mission is to measure and assessing innovation, thereby helping legal industry consumers and product and services providers better understand the legal innovation landscape and inform their decisions. Consumers include clients and law students. Providers include lawyers, law firms, alternative legal services providers, and legal technology companies, to name a few.

Our vision is that measurement and assessment of innovation will drive the legal industry forward, thereby increasing access to legal services and justice for all. Our mission and vision respond to the call to action by Jim Sandman, President of the Legal Services Corporation, that we most move from measuring revenue and profit to measuring technology adoption. (More about this on the Index Overview page.)

Interest in the Index and feedback received exceeded our highest expectations. Users saw the potential for the Index to become a key hub of information within a rapidly changing legal ecosystem.

Upcoming Index Updates

Where do we go from here? We are focused on the following goals in the near term:

  1. Updating the Catalog of law firm innovations. We plan to release version 1.02 of the Catalog by the end of October.
  2. Encouraging law firms and their collaborating partners to submit innovations for addition to the Catalog via a form available via a button at the bottom of the Catalog page.
  3. Beginning later in October, encouraging law schools to submit information for updates to the prototype Law School Innovation Index, which we plan to begin updating later this year.

Catalog of Law Firm Innovations Update Methodology

Version 1.02 of the Catalog of law firm innovations will include all innovation submissions received by October 19. Before adding an innovation to the Catalog, we will verify the information, including its categorization by practice area and discipline driving the innovation.

Additionally, we continue to review various public resources, such as The Financial Times, Legal Week, The Recorder, and Business Wire. We have also conducted uniform Google searches to locate law firm innovations.

When we find reference to a law firm innovation, we confirm that it appears on the law firm’s website before adding it to the Catalog. Limiting the innovations in the Catalog to those identified on law firm websites enables us to confirm the details, including that the innovation is currently offered to clients.

Once an innovation has been identified, we categorize and record each innovation across various factors, including: type of tool, area of law involved, and relationship (internal to the firm, a partnership, etc.). This allows users to analyze the data from a number of perspectives.

Law Firms: Submit Your Innovations by October 19

Law firms and their collaborating partners can help us by submitting information about their innovations by using the form available via a button at the bottom of the Catalog page. All submissions received by October 19 will be verified and added to the Catalog.

Who are we?

We are a team led by Dan Linna, Visiting Professor of Law at Northwestern Pritzker School of Law. The student team brings with them a diversity of backgrounds and interest areas including psychology, education, and public policy.

Lauren Diner

Lauren Diner

Lauren completed her undergraduate degree in psychology and elementary education in 2015. She then worked in an elementary school before earning her Master’s degree in Bioethics from New York University. Over the summer, she completed the Institute for the Future of Law Practice bootcamp and interned with Neota Logic. She will graduate from Northwestern Pritzker School of Law in May 2020. Lauren will join Allen & Overy’s New York office as a summer associate in 2019.

Lauren’s LinkedIn home page

Lauren’s Twitter home page

 

Doug Lavey

Doug Lavey

Doug plans to focus on the intersection of disruptive technologies, public policy, and the law. After completing his undergraduate degree, he worked as a consultant, implementing enterprise systems at large public institutions. He is currently pursuing a concurrent Master’s in Public Policy degree in addition to his JD. Doug will graduate from Northwestern Pritzker School of Law and the Harvard Kennedy School in May 2020. Doug will join Perkins Coie’s Chicago office as a summer associate in 2019.

Doug’s LinkedIn home page

Doug’s Twitter home page

Yoon Hoo Lee

Yoon Hoo Lee

Yoon plans to use her teaching background together with her interest in process improvement and technology to better serve clients. After completing her undergraduate degree in Plan II Honors and Latin at the University of Texas at Austin, she worked as a middle school Latin teacher at her alma mater. She will graduate from Northwestern Pritzker School of Law in May 2020. Yoon will join Weil, Gotshal & Manges’ Dallas office as a summer associate in 2019.

Yoon’s LinkedIn home page

Yoon’s Twitter home page

Algorithms have a substantial impact on everything happening around us. Most everyone would agree that the impact of algorithms, computational technologies, and artificial intelligence on everyday life, institutions, and society will only grow, and rapidly. Yet most law students and lawyers lack the foundational knowledge to generally explain how these technologies work, much less assess them.

One way to address this is to teach law students about artificial intelligence and computational technologies. I’m teaching several courses in this arena at Northwestern Pritzker School of Law this year, beginning with “Artificial Intelligence and Legal Reasoning” this fall.

Each class meeting includes discussion about the effectiveness of these technologies, in the proper context of the status quo, and related ethical questions. Additionally, we will spend two full class meetings discussing technical assessment (recall, precision, F1 score, etc.) and broader questions about the use of algorithms, computational technologies, and artificial intelligence in legal-services delivery and society (transparency, explainability, auditability, provenance, bias, fairness, etc.).

I had the opportunity to introduce these topics during multiple talks this summer, beginning with “Demystifying and Assessing Artificial Intelligence” at the launch of the LawAhead Hub think tank at IE Law School in Madrid. The event attracted a variety of leaders, primarily from international law firms and the legal departments of major corporations.

Continue Reading Training Lawyers to Assess Artificial Intelligence and Computational Technologies

How can we better train law students and lawyers for the future? Indiana University Law Professor Bill Henderson has been working on this problem for a long time. In 2017, Bill joined forces with Bill Mooz, an experienced lawyer and former general counsel who in 2014 established and led the Tech Lawyer Accelerator (TLA) program at the University of Colorado Law School.

In November 2017 at a Forum on Legal Evolution event hosted by Northwestern Pritzker School of Law, Henderson and Mooz presented their vision for expanding the successful TLA program. Shortly after that meeting, they invited me to join them as a co-founder of a nonprofit, the Institute for the Future of Law Practice (IFLP). Continue Reading Hire Tomorrow’s Lawyers Today! Don’t Miss the Opportunity to Hire an Institute for the Future of Law Practice Intern!

In July I had the honor of teaching for the inaugural Legal Technology and Operations summer program at Bucerius Law School in Hamburg, Germany. The program attracted 33 students from all over the globe: Australia, Belgium, Brazil, Chile, Colombia, Estonia, Egypt, Germany, Honduras, Hong Kong, India, Italy, Netherlands, Norway, Peru, Romania, Russia, Spain, Sweden, Turkey, and Ukraine. About 75% of the students were professionals taking leave to participate in the program, including two judges and more than a half dozen from major international law firms (all lawyers, with a couple now in operations roles).

The program began with my course, “Computational Law and Rules-Based Automation,” one of six 13.25-hour courses in the program. Classes met from 9 to 5 nearly every day, with several supplemental lectures in the evenings plus a study trip to Frankfurt the first weekend. Given this packed schedule, I worried that the students might not have sufficient time for outside reading and the team projects that I’d planned. But those concerns were quickly alleviated. The students were incredibly engaged throughout the course and delivered outstanding projects.

Team 1 – Validity of anti-compete clauses in employment contracts

Leading off the program, I started my course with an introduction to overarching concepts (e.g., the changing legal landscape; people, process, and data before technology). Next, I provided an overview of artificial intelligence, including rules- versus data-driven systems. (Assigned reading included Computational Law: The Cop in the Backseat, by Michael Genesereth, Codex – The Stanford Center for Legal Informatics.) From there, we jumped into an exercise that required each student to build an interactive question and answer system in QnA Markup that would guide a user to a determination of whether the General Data Protection Regulation (GDPR) applies to the user. To get the students started, I provided them with a couple of flowcharts of this GDPR analysis published by international law firms. Continue Reading Hands-On Learning: Bucerius Law School Computational Law and Rules-Driven Automation Course – #LegalTechBucerius

I am thrilled to share that I’ve joined Northwestern Pritzker School of Law as a Visiting Professor of Law for the 2018-19 academic year. I’m very excited to have this opportunity to contribute to Northwestern Law’s many initiatives to continuously improve the value of the education delivered to its students and prepare its students to have an impact in our society.

Northwestern Law’s Strategic Plan, “Leading Law,” recognizes that law schools are “educating lawyers for a profession that is undergoing enormous change.” The Strategic Plan says that Northwestern Law  “accept[s] the changing, dynamic profession not with resignation, but with excitement.” This excitement stems from the opportunity for Northwestern Law “to leverage our competitive advantages, to expand our impact and our reputation.” What are these competitive advantages?

  1. Northwestern Law is a place of innovation.
  2. Northwestern Law is student-centered.
  3. Northwestern Law is interdisciplinary.

I share this excitement about meeting the challenges of our changing profession. In the future, today’s law students will improve legal services-delivery, expand access to legal services, expand the rule of law, and contribute to multidisciplinary teams solving society’s “wicked” problems. We must innovate and think big, especially in law school. (I developed these thoughts a bit more in an Above the Law interview, alt.legal: Law Schools Can, Should, And Must Teach Innovation.)

Interdisciplinary opportunities at Northwestern Law include working with talented law faculty with PhDs and expertise in many domains and well-established relationships with other Northwestern schools. I’ve already benefited greatly from initial meetings and discussions about interdisciplinary research and classes. These opportunities are reflected in the classes that I will teach:

  • Artificial Intelligence and Legal Reasoning (Fall 2018; JD students)
  • Innovation Lab (Spring 2019; JD, Master of Science in Law, and Computer Science students; co-teaching with Law Professor David L. Schwarz and Professor of Electrical Engineering and Computer Science Kristian J. Hammond)
  • Law of Artificial Intelligence and Robotics (Spring 2019; JD students)
  • Assessing AI and Computational Technologies (Spring 2019; Master of Science in Law students)
  • Assessing AI and Computational Technologies (Spring 2019; San Francisco Immersion Program; JD and Kellogg School of Management students)

My connection to Northwestern Law began several years ago with an introduction to Alyson Carrel, Clinical Assistant Professor of Law. Alyson was very generous with her time and answered all of my questions during my early days of teaching Negotiation at the University of Michigan Law School, while I was still practicing at Honigman. Several of my Michigan Law classes completed negotiation simulations with Alyson’s Northwestern Law students.

Later, Dean Daniel B. Rodriguez appointed Alyson as the new Assistant Dean of Law and Technology at Northwestern Law. Since then, my interactions with Alyson have focused on improving legal-services delivery, technology, and legal education. While at Michigan State, I greatly appreciated and benefited from opportunities to collaborate with and the support provided by Alyson and Dean Rodriguez. They also see the enormous opportunities for collaboration and interdisciplinary work to generate both individual and collective value for law schools, the legal profession, the emerging legal industry, and society at large.

In addition to my work at Northwestern Law, I remain an affiliated faculty member at CodeX – the Stanford Center for Legal Informatics. I am also working on updates to the Legal Services Innovation Index. In future posts, I plan to write about these projects and complete a wrap-up post about LegalRnD projects completed during my final year at Michigan State. (Most have known about my move to Northwestern Law for awhile, including from a March 1, 2018 Dean Rodriguez tweet and March 15, 2018 Northwestern Law tweet.)

Now that I’ll be living in Chicago, I look forward to more frequent meetings of the Chicago Legal Innovation meetup as well. We’ve had strong attendance at meetings by Northwestern Law students in the past, and I look forward to seeing many more Northwestern Law students at future meetings!

Why do so many of today’s problems remain unsolved? Often, we have no shortage of viable solutions. Additionally, in many cases we have a critical mass of people committed to solving the problem. Yet despite abundant ideas, energy, and action, the problems persist.

How can we improve? Start by making scientific thinking a habit and using a Mission (or Business-Model) canvas.

Scientific thinking is embedded in lean thinking, as illustrated in Toyota Kata. Many similar approaches can also be reduced, at their essence, to the basic scientific method. The critical point is to identify conclusions and assumptions and recognize that we must experiment to test our ideas for solving problems—we do not know the answers, we must test our ideas and learn from experiments.

Scientific thinking involves more than prototyping and iterating. It’s critical to have a hypothesis and write it down. What did we expect? What actually happened? What will we try next? This is how we learn. (Entrepreneurs like to say “fail fast.” Lawyers have trouble embracing this. Thus, I try to emphasize that we want to “learn fast.”)

We get the greatest value from experiments when everyone has a shared understanding of the mission. There are many advantages of using a Business-Model (or, as described by Steve Blank, a Mission Canvas). A canvas organizes a group’s thinking around the critical issues. It’s also an excellent visual medium for capturing input from individuals and communicating it back to everyone. Additionally, it’s easy to quickly generate multiple canvases, depicting different models. This can be especially helpful to keep participants focused on generating ideas for testing rather than killing ideas before they can be further explored, tested, and improved.

#90minuteBlogPost

Legal industry leaders and analysts seem to suggest that United Kingdom law firms are out in front when it comes to legal innovation and technology adoption. The data my research team and I have assembled for the Legal Services Innovation Index lend some support to this.

Having just launched this project, I wanted to share a few Tableau Vizzes that summarize the LSII data on this topic. As discussed on the LSII website, it’s important to put this data into context and review and understand our methodology and the caveats that we’ve identified. That said, the Law Firm Index provides insight into whether and how law firms are innovating, especially when looking at the big picture, and the Catalog of Law Firm Innovations identifies concrete innovations that law firms have implemented.

We should not expect all law firms to look the same. Most would agree that we need greater differentiation across law firms, not copycat “innovation.” While one goal of this project is to define and generate discussion about specific categories of innovation, it is not intended to suggest that these categories make up the universe of innovation or that all law firms should be “innovating” across all categories. Additionally, we are not directly measuring critical aspects of organizational innovation, such as whether a firm has an innovative culture in which everyone from the bottom to top is engaged in innovation and efforts to continuously provide greater value to clients. All of this deserves further discussion.

We searched 260 law firm websites:

  • Am Law 200 – American Lawyer
  • Canadian Top 30 – Lexpert
  • Global 100 – American Lawyer

Note: The firms categorized in our results as “Global 100” include only those firms not already categorized as part of the Am Law 200 or Canadian 30. For this reason, we added an asterisk to “Global 100*” in the Tableau vizzes. Keep this in mind when comparing the Global 100 to other categories of law firms.

Finally, when comparing jurisdictions, consider that we’ve included only the UK firms that made the Global 100 while in the US we’ve included all Am Law 200 law firms. Consider this when comparing averages.

Again, please review the LSII website for necessary context, an overview, and our methodology. With that said, below I have added a few snapshots of the data.

Figure 1: Average Hits per Law Firm Website by Jurisdiction

Figure 1: Average hits per website with Google searches for innovation categories across law firm websites. For context, review overview and methodology at LegalTechIndex.com.
Figure 1: Average hits per website with Google searches for innovation categories across law firm websites. For context, review overview and methodology at LegalTechIndex.com.

 

Figure 2: Average Hits per Law Firm Website by Jurisdiction, Grouped

Figure 2: Average hits per website for innovation Google searches across law firm websites, grouped. Note that Global 100 does not include those Global 100 law firms that are a part of the Am Law 200 or Canadian 30. For context, review overview and methodology at LegalTechIndex.com.
Figure 2: Average hits per website for innovation Google searches across law firm websites, grouped. Note that Global 100 does not include those Global 100 law firms that are a part of the Am Law 200 or Canadian 30. For context, review overview and methodology at LegalTechIndex.com.

Figure 3: Catalog of Law Firm Innovations Grouped by Jurisdiction

Figure 3: Catalog of law firm innovations. A form on the LSII website allows law firms to submit innovations for inclusion in the Catalog. For context, review overview and methodology at LegalTechIndex.com.
Figure 3: Catalog of law firm innovations. A form on the LSII website allows law firms to submit innovations for inclusion in the Catalog. For context, review overview and methodology at LegalTechIndex.com.

I’m thrilled to launch the Legal Services Innovation Index, Phase 1, Version 1.0:

LegalTechInnovation.com

My student research team and I are looking forward to input and discussion, receiving submissions of law firm innovations, and working to improve this resource.

So far, we’ve received a great response:

If you care about improving legal-service delivery and increasing access to legal services, please take a look and share your thoughts:

LegalTechInnovation.com

In his May 2016 keynote at FutureLaw 2016 at Stanford Law School, Jim Sandman, Legal Services Corporation president, suggested that we rank and assess law firms on their use of technology. He argued that this could accelerate the adoption of technology in law firms and might stimulate investment in the development of new technology that could benefit all who need legal services. This was just one of ten excellent suggestions Jim provided for accelerating technology adoption to improve legal services and close the justice gap.

Click this image to see a video of my talk at the 2017 Legal Hackers International Summit.
Click this image to see a video of my talk at the 2017 Legal Hackers International Summit.

One year later, Jim spoke on a panel at FutureLaw 2017 (41:20 to 56:30 in the video) and again suggested that we assess law firms’ technology usage. No one had yet undertaken the effort to systematically assess law firm technology adoption. Right then, I decided to tackle this project.

This last Saturday at the 2017 Legal Hackers International Summit, I provided a preview of Phase 1, Version 1.0 of the Legal Services Innovation Index. A video of my twelve-minute talk is available on YouTube. It was an honor to immediately follow Jim Sandman’s keynote (starting at 8:25 in the video). In addition to previously providing the inspiration for me to begin this work, Jim provided input and support when I discussed the project with him earlier this summer.

In about two weeks, I will launch Phase 1, Version 1.0 of the Legal Services Innovation Index, consisting of:

  1. a catalog of legal-service delivery innovations that have been implemented by law firms, and
  2. measures of law firm innovation for 264 law firms (members of the Global 100, Am Law 200, and Lexpert 30 largest in Canada) based on searches of their websites using Google Advanced Search.

This release is intended to be a “minimum viable product.” I’m following the “Lean Startup,” Plan-Do-Study-Act process for innovation and product development that I’ve suggested for all legal innovators. This release curates available information, contributes additional analysis and information, and uses empirical methods to measure legal innovation. I’ve discussed this project with many in the legal industry and they’ve uniformly encouraged me to complete this initial research, make the information available, solicit input for improving the Index, and continue iterating to improve the Index.

The Index is not intended to be a ranking of law firms–at least not at this time. Like an index at the end of a book, this Index is intended to identify and categorize innovation and direct users to the places where legal-service delivery innovation can be found. After the launch of the Index, law firms will be able to propose innovations for inclusion in the catalog via a website form. As for the searches, this is but one measure of innovation, which–like nearly all measures–has its weaknesses, but nonetheless contributes something to our understanding.

In my talk, I shared more about the purpose of the Index, methodology, weaknesses, and future plans. I will share much more on these topics here, on my blog, when I launch the Legal Services Innovation Index website.

I want to thank the four Michigan State University law students on my research team:

Thank you as well to Blake, Cassels & Graydon LLP and Carla Swansburg (Director, Practice Innovation, Pricing & Knowledge) for contributing research that we used to seed the catalog of law firm innovations.

Lastly, thank you to LexBlog, which is contributing the development and hosting of the Legal Services Innovation Index website.

Again, I provided additional information about the Index in my talk at the 2017 Legal Hackers International Summit and will provide much more information when I launch the Index website. If you have questions, suggestions, or comments, please provide them below, tweet at me (@DanLinna), or send me an email at the address in my MSU bio.

August 15, 2017 update: In the original post, I linked to the session video that began with my talk. I’ve changed the links to a recently released excerpt of my talk at the 2017 Legal Hackers International Summit.

“It’s simple. Nice user interface. I like the questionnaire style.”

Comments like these, you assume, refer to an Apple iPhone, Google Search, or some other product hailed for its superior user experience. When is the last time you heard a client make comments like these when discussing a law firm’s legal services?

Okay, maybe you’ve never heard a law firm client make comments like these, particularly when talking about legal services of any complexity. But that is what I heard from a potential client who had just seen a demo of the Akerman Data Law Center, a client-facing expert system that provides data privacy and security advice. Continue Reading Collaborative Disaggregation: Law Firms Can Delight Clients with the Right Technology