|
|
Home
| Databases
| WorldLII
| Search
| Feedback
Computers and Law: Journal for the Australian and New Zealand Societies for Computers and the Law |
At the heart of creating legal technology that enables decision making is understanding how people use legal information. Understanding the reasons people seek legal help, and the methods lawyers use to provide legal information, leads to the development of more usable and sustainable technology solutions. At Portable,[1] our approach to developing guided decision-making technology is to use legal design to understand how people seek information or make decisions. We involve both lawyers and their clients in the design process to build technology solutions that create alternative pathways to suit more people online, and empower them to resolve their own issues. Technology, particularly the use of well-designed interfaces, can give decision-making more immediacy and presence, creating the possibility for effective nudges that provide guidance while minimising bias.[2] Our recently launched family law product, amica,[3] exemplifies our approach to enabling better decision-making through technology.
Legal design as a foundation for designing technology solutions
Applying the principles of human-centred design to the research, design and build phases of a project enables better technology solutions, due to the focus on understanding users, co-designing solutions, testing and iterating.[4] Focusing on co-designing and testing legal technology solutions is especially important, since the concepts that need to be communicated or processes that we need to enable can be more complex and have a greater impact on the lives of people using our products than in other industries. Legal design, which can be defined as ‘the application of human-centred design to the world of law, to make legal systems and services more human-centred, usable, and satisfying’,[5] can achieve better outcomes by reducing risks and increasing usability and desirability.[6]
Using artificial intelligence to enable better decision-making for ‘amica’
Exploring the problem space and prototyping solutions using machine learning
Creating an understanding of the problems people are facing is the first step in exploring opportunities to create better legal technology. When the Legal Services Commission of South Australia (on behalf of National Legal Aid) partnered with Portable to create ‘a simple, cost-effective and innovative way to negotiate family law issues online’, our first step as legal designers was to determine what challenges people were facing when separating their financial lives, and to use our understanding of the problem space as a jumping off point for solutions. We worked with family lawyers at the Legal Services Commission (LSC) to create workflows that replicated how lawyers conduct interviews to determine the financial position of both parties. By creating a step-by-step plain English questionnaire and progressively disclosing educational content throughout the interface, we allowed users to learn about their situation and enabled them to make better decisions about their legal and financial position.[7]
However, the most challenging aspect of replicating in-person legal advice online was providing a reference point for the parties to begin negotiating how they divide their assets and liabilities. By speaking with newly separated individuals, family mediators and lawyers, we worked out that in order to provide the most value to people using an online platform for their separation, we needed to inform them about how the process works and provide them with an objective overview of what to expect. We needed to find a way to automate answers to the most important question family lawyers heard from their clients, “What would you predict a fair division of assets for people in our situation would look like?”. Importantly, we are replicating the process of providing legal information, as opposed to providing an authoritative decision through an online court or arbitration. The suggested division offered by amica is meant to be the starting point for the parties to work out an agreement amongst themselves.
We decided that the need for greater certainty and consistency for separated couples involved in a property settlement could best be met by implementing a machine-learning model that takes into account user input, and provides a benchmark for their negotiation. The pioneering work of John Zeleznikow and Andrew Stranieri already paved the way for this through creating the Split Up system that attempted to standardise discretionary family law decisions,[8] and we saw the potential for a predictive algorithm--coupled with a user-interface designed to encourage agreement--to build on this work and enable better decision making. By introducing a simple AI tool to the negotiation process, our goal was to create an online dispute resolution system that followed best practice for online legal technology, by enabling people to ‘engage in informed and deliberate decision-making, introducing as little bias as possible, while enhancing (or at least preserving) litigants’ self-determination’.[9]
Crowdsourcing data collection and reducing bias
We believe AI offers significant value in this context because it provides the opportunity to use a statistical foundation in which to make decisions that are grounded in consensus opinion (which, in this case, includes the details of their financial situation, contributions, and future needs). In the context of asset division splits, there is potential for a great deal of subjectivity when using lawyers to negotiate settlement amounts, as each lawyer has their own individual biases with regards to making decisions. AI, on the other hand, can leverage the aggregate decision-making abilities of individual lawyers, and in doing so can ‘cancel out the noise’ that each individual lawyer elicits when making their own decisions regarding apportioning settlements to each party involved in this kind of legal dispute.
The AI model that we created was able to make informed predictions that represented the aggregated individual decisions of numerous lawyers that participated in the creation of our crowdsourced training dataset. We collected crowdsourced data from over a hundred different lawyers who specialise in divorce law by creating a tool that generated randomised scenarios and asked these lawyers to make their own predictions regarding the allocation of assets to each party involved in these scenarios. We were able to use a statistical model that natively supported and understood percentages.
The model chosen was a form of Beta Regression, which the literature has shown to be the paradigm of choice in these kinds of scenarios.[10] The AI offers the advantage of the cumulative knowledge of many expert lawyers; using a tool that leveraged only a single lawyer for making asset division settlements would have introduced a great deal of anthropocentric bias, which our approach helps to overcome. We also ensured that the randomised scenarios did not take into account gender – this was done to remove biases that could be introduced due to the gender of one party. We also standardised the features of the data, giving each feature the same scale. This helped to ensure that the model understood that the magnitude of a given feature (e.g. income) had no more bearing on the model than a feature that originally was of a smaller scale (e.g. number of children of each party).
Future iterations of the AI model
Refinements of the model in further iterations could include the use of more nuanced features, including using more complex ratings for both parties to describe their health as a way of representing the nuance inherent in assessing future needs. The crowdsourced data also suffered from an inconsistent methodology wherein certain lawyers contributed more predictions to the training dataset than others; this gives potential for individual lawyers’ contributions to the dataset to have more weight than others. Crowdsourcing future data in such a way as to allow all lawyers to contribute equally to the dataset would help alleviate this issue.
Another iteration of the model could also allow for the inclusion of a ‘random intercept’ term in the model, thereby including the variability pertaining to different lawyer’s contributions to the dataset as an explicit feature in the model. More rigorous removal of outliers from the dataset could also help to increase the stability of the model.[11]
Conclusion
The benefits and risks of AI, in a legal context, are well documented in the literature[12] .For amica, the benefits include the ability to leverage the collective wisdom of a select group of subject matter experts and by doing so, generate a model that has a strong statistical foundation. The chief drawback of the application of AI in this context is the lack of transparency; the mathematics behind the model is quite abstruse and its complexities are not easily translatable to a lay audience. Given that amica is a live product, we can continue to evaluate not only the statistical performance of the model, but also seek user feedback to evaluate how we are enabling them to achieve their goals and empowering them to make better, more informed decisions to help them move forward with their lives.
[1] Portable.com.au
[2] Ayelet Sela, ‘e-Nudging Justice: The Role of Digital Choice Architecture in Online Courts’ [2019] Journal of Online Dispute Resolution 127, 142.
[3] amica.gov.au
[4] Tim Brown, ‘Design Thinking’, [2008] Harvard Business Review 84.
[5] Margaret Hagan, Law by Design (Open Law Lab, 2017) pt 1.
[6] Lisa Toohey et al, ‘Meeting the access to civil justice challenge: Digital inclusion, algorithmic justice, and human-centred design’ [2019] (19) Macquarie Law Journal 133; Margaret Hagan, ‘Participatory Design for Innovation in Access to Justice’ [2019] (148) Dædalus, the Journal of the American Academy of Arts & Sciences 120.
[7] For more on the potential of self-help legal tools, see: D. James Greiner, Dalié Jiménez, Lois R. Lupica, ‘Self-Help, Reimagined’ (2017) 92(3) Indiana Law Journal.
[8] John Zeleznikow and Andrew Stranieri, ‘A Hybrid Rule – Neural Approach for the Automation of Legal Reasoning in the Discretionary Domain of Family Law in Australia’ [1999] (7) Artificial Intelligence and Law 153.
[9] Ayelet Sela (n 2) 138.
[10] Silvia L.P. Ferrari, Francisco Cribari-Neto, ‘Beta Regression for Modelling Rates and Proportions’ (2004) 31(7) Journal of Statistical Software 799.
[11] Jeanna Matthews, ‘Managing Bias in AI’, Conference: Companion The 2019 World Wide Web Conference, DOI: 10.1145/3308560.3317590, May 2019.
[12] Deven Desai and Joshua Kroll, ‘Trust But Verify: A Guide to Algorithms and the Law’ (2018) 31(1) Harvard Journal of Law & Technology 1; Leah Wing, ‘Artificial Intelligence and Online Dispute Resolution Systems Design’ (2017) 4(2) International Journal of Online Dispute Resolution 16; Monika Zalnieriute, Lyria Bennett Moses, and George Williams, ‘The Rule of Law and Automation of Government Decision-Making’ (2019) 82(3) Modern Law Review 425.
AustLII:
Copyright Policy
|
Disclaimers
|
Privacy Policy
|
Feedback
URL: http://www.austlii.edu.au/au/journals/ANZCompuLawJl/2021/8.html