Introduction

Batch AI processing is being used in commercial legal products, but can now be used by resource constrained lawyers in a variety of ways that could impact the criminal justice system. Functioning similar to a mail merge, batch processing allows a user to submit a large number of prompt requests to a model and receive responses within a relatively short amount of time (typically 24 hours). Lawyers can use this technology to review large amounts of documents searching for particular material or to draft many similar documents such as motions to dismiss, but that are particularized to the facts of different cases.

In this post, I expand on ideas for applications of AI in the criminal justice system by outlining a few specific ways batch processing can be used in the resource constrained, criminal law context. I will start by describing how I’m using AI batch processing to further research into possible AI bias and conclude with a brief discussion of the externalities that such use can impose on the system.

How I am using batch processing

I believe many lawyers, including prosecutors, are using AI chatbots like ChatGPT to assist with their work. There is a lot of evidence that this is happening. My concern is that it is not clear whether and how the chatbots might be biased. Chatbots could be explicitly discriminatory, though most AI companies are continually working to mitigate this. But chatbots could hold more subtle, implicit biases that are harder to recognize and that still impact human decision-making. For example, a chatbot asked to perform a risk assessment might consistently rate prior drug convictions as a high-risk attribute without recognizing historical racial patterns in drug arrests such that Black people are then consistently rated as higher risk.

These types of issues can be uncovered by running large scale tests to gather many AI responses to similar prompts and assessing whether there are underlying patterns that would manifest into biased results. To generate the data though, requires thousands of prompts and responses, which, when run through a chatbot’s normal interface, is error-prone and can take a long time. To deal with this, I am now creating AI batches to submit entire experiments’ worth of requests in a single batch. This saves a massive amount of time and money.

How batch processing can be used by criminal lawyers

There are likely many applications for batch processing within the criminal justice system, but in this blog, I am going to outline two that I think could be readily implemented and would have real impact. Both ideas require a batch processing system with web search and possibly web fetch capabilities, neither of which ChatGPT currently supports, though Claude does. These ideas also require an organization to carefully structure how case data is stored, so that there’s essentially a single link per case, which is itself a folder containing all case materials. Because of the size of some case files, these approaches (1) may not currently work, but as token limits increase (essentially, how much data the model can process) may become possible, and (2) token limits may require breaking up the proposed processes into multiple staged batches. Finally, implementation likely requires some network security expertise to ensure that any sensitive case materials are not exposed.

Prosecutor conviction review units. I know of at least one fellow researcher who had been proposing to funders a project to use AI in conviction review. This has also been discussed in criminal justice-related podcasts, so it is not a novel idea. The general thrust of those proposals, though, was focused on using AI to evaluate a single case. Because of the volume of case material, a single case review takes a massive effort that AI could shorten. Batch processing these reviews multiplies this workflow many times over. Here’s how I can imagine an office doing this:

• Create a data set with links to case folders. For a lot of offices, this is not trivial, depending on how the office has structured evidence/data management and the various technologies the office uses.
• Because of the volume of documents in each case, each folder should likely include a table of contents that lists and categorizes each file. Write a prompt for the AI to review materials, categorize them, and return a table of contents. Then put the table of contents into each folder.
• Since there are different reasons for reviewing a conviction, write prompts for each of these reasons. In the prompts, include directions for how to use the table of contents and what documents to focus on. For instance, reviewing for witness inconsistencies would require a prompt that directs the AI to focus on reviewing witness statements. Have the return from these prompts be a short summary and a prioritization ranking for reviewing cases.

From this, an office would have datasets ranking cases by the likelihood of various conviction review patterns, which could then help prioritize human review. Though there is significant investment needed to set up the process, these costs are likely small compared to the cost of having lawyers review case materials, especially since the review process itself — searching for things like inconsistencies across many documents — may better suit a model than a human as well.

Writing motions. Offices could use batch processing to generate first drafts of various motions on a daily or weekly basis. For example, a prosecutor’s office could gather from staff all motions that need to be written on a daily basis, create template prompts for each type of motion, again use links to case folders, and then nightly run a batch job to write drafts. I think this is unlikely to happen, in part because writing motions is what many lawyers, at least nominally, enjoy doing. But many convicted persons would benefit from a legal review, and with the introduction of processes like those in California’s Racial Justice Act, many people could submit legitimate motions but lack the expertise and support to do so.

The Racial Justice Act creates an avenue for a convicted person to submit a motion to reconsider a conviction based upon either discrimination by the state in that person’s case or based on evidence of systematic discrimination by the state — see challenges based on statistical disparities. The latter claim is potentially broadly applicable for many convicted people in California, and a process to mass produce well-written motions that can be particularized just enough to a defendant’s case means that hundreds to thousands of such motions could be drafted in a few weeks. Using the same structure outlined above, a legal services provider could create a system for organizing case materials and a set of basic prompts to draft the motions. In fact, an actor could create a platform whereby convicted persons would create their own evidence folders, submit them to the service, which then could organize the folders and run daily batches to generate the motions. Though there is not much evidence that these motions have succeeded often, the ability to file hundreds of motions, possibly modeled after successful motions, could significantly change the law’s impact.

What could happen

Scaling labor-intensive activities like conviction review can help to address long-standing equity issues, like conviction review being mostly reserved for those able and willing to navigate the process. In the past, it was defendants with the financial resources, advocacy support, and ability who were most likely to push a conviction review forward. Now that the process can be extended to nearly all defendants at an extremely low cost. Similarly, processes like motion writing can be scaled such that all persons within the criminal justice system receive a higher floor of service. This can be managed by those already working within the system, who can also reduce and prioritize workloads.

Unfortunately, batching can also be used to break systems like courts that have a narrow funnel. Individuals are already finding AI useful in legal proceedings and are able to produce high-enough quality legal filings to harass others and bog down courts and lawyers in process at a reduced cost. So just as batch processing can scale the benefits of AI in legal processes, it can also be used to scale potentially harmful practices that could grind courts to a halt. In turn, courts might limit the use of AI and do so in a way that prevents us from taking advantage of it to better enact our principles for due process and justice.

Rather than ignoring these realities, criminal justice agencies should begin to take a more active role in incorporating AI into practice. Without doing so and absent state or federal regulations, individual actors will control how AI impacts the system, which may or may not result in a net public benefit.

By: Rory Pulvino, Justice Innovation Lab Chief Implementation Officer. Admin for a Prosecutor Analytics discussion group.

For more information about Justice Innovation Lab, visit www.JusticeInnovationLab.org.

Introduction

The Council on Criminal Justice’s (CCJ) AI Task Force recently published a User Decision Framework that aims to be a practical guide for a wide variety of public agencies to evaluate and possibly adopt AI-enabled technologies. The framework is “detailed enough to serve as an action plan” for agencies looking to adopt AI tools without being so rigid that it constrains processes or doesn’t apply to some agencies. Built on the CCJ’s AI principles, the framework offers important guidance and a set of helpful appendices for all agencies evaluating AI tools.¹ The framework fills a much needed gap, and I hope will be widely reviewed and considered by public agencies as they inevitably adopt more AI-enabled tools.

The framework’s five phases and the principles underlying how CCJ structured the framework are an ideal workflow for technology adoption in the public sector. In particular, starting with asking an agency to define the problem and the organizational readiness is an oft-skipped first step. When developing JIL’s framework for AI tool evaluation, I did not consider the importance of procurement and think CCJ’s inclusion of these government processes is a very smart way to shape thoughtful evaluation. Furthermore, I appreciate the practical appendices that CCJ developed as that documentation and the ‘worksheets’ that are included cover a lot of the processes that are needed to force considered thought in any AI adoption.

Cross-Cutting Observations

The framework starts with a warning regarding the adoption of domain-specific technology versus the adoption or widespread use of general models like ChatGPT. This is a great first warning since it’s likely there’s many people experimenting with tools and, when confronted with the power of general purpose models, may quickly begin using them for work and inadvertently violating IT control policies. Though the framework starts with this warning, the processes outlined in the framework are applicable to evaluating general purpose tools, even if such tools require additional considerations.

Throughout the framework discusses the need to review tools for their impact on substantive rights. This reads as protections against the use of AI that would harm people through discrimination and a diminishment of due process, which most often is directed at protecting arrestees and defendants. This gives the impression that the framework is not applicable for AI that serves or more directly impacts victims. I don’t think this is the framework’s intention, but do think the framework could more explicitly address that criminal justice AI tools will impact victims and that they should be considered in assessments and evaluation. As discussed further below, agencies will and should spend significant time grappling with the intention of a tool’s adoption and the externalities imposed by a tool.

One further note about the framework’s discussion of substantive rights and identifying substantial and low risk applications for AI (Phase 2 & Phase 4) — most tools are ultimately adopted to affect fundamental procedural or substantive rights. For instance, AI technology to review and flag jail calls are not making a recommendation directly related to “stop, search, arrest, detention, bail, charging, plea, sentencing, parole, clemency, or similar decisions” rather their adoption is to solve a problem that there is just too much content to sift through. But in trying to more quickly and accurately sift through that information the tool is enabling decisions that do directly affect substantive and procedural rights. I strain to think of examples of AI technologies that are not being adopted in the criminal justice space that, through generally short causal chains, are intended to impact rights.

Phase-by-phase Discussion

Phase 1 (Defining the problem to be solved and assessing organizational readiness) of the framework starts with asking an office to define the problem the office is trying to solve. Like most organizations, government agencies can fall into the trap of buying technology without a real need or a technology suited to solve a superficial problem. By framing the first step in the acquisition step to force an agency to define the problem and consider the causal mechanism that an AI tool would solve the problem, the CCJ’s framework hopefully limits an agency from purchasing unneeded technology. Furthermore, the first phase asks an agency to assess its readiness to adopt the technology and an appendix ‘worksheet’ to assess readiness.

Though very helpful, the framework could be strengthened by an appendix to support the problem definition step. In particular, a worksheet that asks an agency to map out a user story that demonstrates the problem and how the technology solves the problem could do more to help agencies assess the root cause of the problem. This can help avoid situations whereby agencies adopt tools to solve an immediate need without due consideration to the root cause that may be easier or more deserving to solve. For instance, many prosecutor offices struggle with current volumes of evidence to review and are considering adopting tools for automated review and highlighting of evidence. While large amounts of evidence are an issue, other possible ways to view the issue are:

• A severe lack of prosecutors that leads to large caseloads.
• The breadth of criminal codes and policing choices that leads to a large number of cases to begin with.
• Office practices and legal rules that permit or even promote the creation of a large backlog of cases.

Phase 2 (Classifying the system’s risk and opportunity levels) is an essential step and when something goes terribly wrong, it is the one most frequently skipped. The process outlined forces an agency to essentially write down what it knows about the tool and its risks. It forces an agency to think through different types of risks and who the risk actually impacts. The framework states that “[i]mprovement should benefit those affected,” yet this may be hard to determine because tools will frequently benefit some while hurting others. Within the criminal justice system, many of the AI tools are marketed to law enforcement to lighten loads and enable prosecution. This forces an agency to consider then that the tool may benefit some — victims and defendants — but that it might also significantly harm others. For instance, if a tool ends up enabling the further prosecution of petty theft cases — a frequently dismissed case type — this could better serve some while upsetting others.

What might this look like? Imagine an AI tool that manages court notifications and sends reminders to interested parties like victims to attend court. The intention and consequence is to get more victims to attend court to pursue prosecution for petty theft. Yet the emphasis of the tool on court hearings and prosecution may mean that diversionary opportunities are pursued less and victims are guided towards prosecution rather than say a restorative justice program. This example illustrates that offices adopting these technologies should be honest about who they expect to benefit and who might be harmed.

Phase 3 (Establishing procurement protections) does a great job, especially the appendix, of raising issues that offices should be sure to think about before signing a contract. The most obvious that I have run into is data ownership where technology providers often lock an agency’s data away from the agency. This should generally be unacceptable, but companies have gotten away with this — some even charge customers for data exports of agency data. Another important aspect that this phase raises is the office training and change management. Adding or substituting an application to workflow is going to disrupt which will almost inevitably meet resistance. Ensuring that an office anticipates resistance, has a good working relationship with the developer to work through issues, and that there is training and responsiveness built into the contract is key.

Given the challenges with implementation, the framework could provide more information for users on change management and different approaches to deployment. Large organizations have staff dedicated to change management, including chief innovation officers and HR. Most government agencies do not have these roles with change management, resulting in implementation falling on IT departments, who are frequently segmented from the office generally, or a single other staff member who has a particular interest in technology. Offices should also have more guidance in this area because if AI tools deliver on their promises to reduce work, then there may be the need to do more than just training staff on how the tool works, but also how any staff time savings will be used by the office.

Phase 4 (Implementing with appropriate safeguards) is divided into discussions about ‘Level 1’ that apply to all systems and ‘Level 2’ systems that are higher risk as evaluated in earlier parts of the framework. The phase contains more recommended steps than other phases and includes two critical appendices — G and H. Appendix G is the Implementation Planning and Memorandum Template, which includes a pilot program design. This seems particularly useful for getting an agency to slow down and implement change in an incremental manner that forces thoughtfulness around the problem the tool solves. The pilot program is supposed to assess on success metrics that were defined in Phase 1, though I think a common issue will be that success metrics are ill-defined and that the office does not have the data necessary to evaluate these metrics. The framework would benefit from providing more concrete guidance on data collection and explaining the difference between a success metric and a simple output — an AI redaction tool can process lots of videos, but that doesn’t seem like success, rather success should be the time saved or some other important outcome.

This phase also includes recommended training for staff that “covers system functionality, limitations, known failure modes, and automation bias.” Such trainings are necessary and it is good that the framework includes automation bias and system limitations. I would add two training items that are perhaps encapsulated in that list: office objectives and error rates. With regards to office objectives, I think of this as explaining to staff what problem the agency is trying to solve by adopting the technology. If an office adopts an AI risk assessment tool, explain to staff the expectations for the tool in performing the risk assessment, this will help users to better pick out if the system is performing as desired. Error rates perhaps fall into limitations, but staff should know how a system can make a mistake. For instance, a video redaction tool could be very accurate, but can error in two very different ways — failing to redact material it should have and, for prosecutors a professional ethics consideration, redacting material it should NOT have redacted. Users should know and understand both risks.

Human centered design, transparency and accountability, community engagement, and ongoing oversight are included as elements of the phase as well. All these areas are very much needed, but many local public agencies will struggle to deliver on. The framework should include these elements, but possibly could deliver more guidance on how agencies could deliver on these given real resource constraints.

Finally, the framework recommends practices for ensuring higher levels of data privacy and security. These are necessary guidelines in the face of the continued lack of federal regulation that would provide better guarantees and peace of mind that a system meets some minimal standards. Unfortunately, from my own experience, keeping up with a tool’s security and privacy settings, especially in the current environment, is a near full-time job. Agencies should be prepared to devote significant staff time to reviewing updates to systems, reading technical documentation, running small experiments as settings change to ensure they work correctly, and, importantly, communicating to staff what is safe to do. Without guidance and continual communication, staff will inevitably begin operating outside of controls.

Phase 5 (Conducting ongoing monitoring and reassessment) is succinct — conduct regular reassessments of tools. As previously mentioned regarding assessments and the associated appendices, this is great and needed. Whether agencies can effectively carry out such assessments is a different matter as it would seem to require employing additional staff that most don’t have the budget for. The framework might instead provide guidance on how agencies can identify outside partners such as academic institutions to conduct assessments as well as what would be required to arrange such partnerships — contracting and data sharing, publishing guarantees, and funding. There is a significant need to grow partnerships between resource-constrained public agencies and public research institutions to support them.

Conclusion

Overall, the CCJ framework is a wonderful resource for public agencies to guide them through AI-enabled tool adoption. The framework contains many helpful appendices that, in my experience, are the greatest value-adds to agency staff. It’s great to recommend conducting an assessment, but if an agency does not have staff with assessment experience, this becomes much harder to carry out. By providing examples, the framework removes barriers to implementation. That said, I believe the framework could use even more appendices — perhaps CCJ is working on this and will create an AI tool resource library? Everything from example contract language to change management processes would be helpful for smaller agencies. The framework is currently most suited to larger agencies that have diverse staffs that include IT professionals, analysts, and law enforcement staff (officers, lawyers, paralegals, and others). Agencies without these staff members will likely struggle to fully utilize the framework because of the amount of additional work the framework proposes.

As mentioned above, I think this creates an opportunity for the CCJ to propose how certain elements, namely assessments and ongoing evaluations, could be aggregated and contracted to other public institutions. Doing so would serve several additional elements that the framework calls for — transparency and public accountability as well as external validation testing. This framework is not an advocacy document but possibly include appendices that would facilitate these needed collaborations to build momentum for such work.

For Justice Innovation Lab, since we partner with prosecutor offices in technology implementation for social good, we will use the framework in our partnerships when evaluating AI tools. The phases broadly align with our partnership approach whereby we spend a lot of time at the front end identifying a problem, designing a solution, and ensuring that we have hypotheses about impact and externalities. Measured adoption is not easy, especially given the hype currently behind AI, but it is the responsibility of the government to be thoughtful before taking on technology that can seriously harm individuals.

By: Rory Pulvino, Justice Innovation Lab Chief Implementation Officer.

For more information about Justice Innovation Lab, visit www.JusticeInnovationLab.org.

1: Full disclosure, I reviewed the framework prior to publication and provided comments.

Edited — The original version of this post asserted a 42x cost increase from batch to one-off requests. This was a mistake comparing requests using ChatGPT-4o one-off to ChatGPT-4o-mini batch. This has been corrected to consistently use ChatGPT-4o. h/t to Andrew Wheeler for raising this issue and pushing me to examine the requests and prompt more closely. Andrew recently published a book that includes details on batch prompting across models.

Introduction

Since Justice Innovation Lab’s first report testing ChatGPT 3.5’s ability to draft basic legal memos, identify legal issues, and possible racial bias in its output, frontier model producers introduced batch processing. In the last two weeks, I’ve only experimented with ChatGPT’s batch processing, but both Claude and Gemini also offer the functionality. This addition has transformed how I am now interacting with the models for experimentation and research into the safety and biases of the models. Though there are limitations in using batch processing, for my purposes to examine variation in responses and identify possible sources of error and bias, this functionality is extremely useful.

*Quick aside, there are serious externalities that all AI is enabling and batch processing could also increase these issues.

In the last week, I’ve established a test batch processing workflow using ChatGPT that is better than my original methodology in nearly every conceivable way:

• Significantly reduced cost and time. For comparison, I submitted a batch using my original (though updated to use Simon Willison’s llm package) process of one-off calls to the ChatGPT API which took over 5 hours to complete during which I had to ensure that I did not disconnect. It also cost 2.7x more than what the batch cost that I set and then left to complete without worrying about connections. This savings is above the expected savings of 50% using batch prompting. The additional savings is in part because the one-off API calls had 2 extra requests due to error handling and, again h/t to Andrew Wheeler for pointing this out, because using a temperature of 1 can lead to more variation in responses than a temperature of 0.
• Data retention and logging within OpenAI’s developer platform which made debugging easier and reduced the likelihood of losing data.
• Improved error handling since all data is written to a single batch return whereas in the original code, I am handling writing all errors per API request.
• Reduced energy consumption such that I did not have the fan on my laptop whirring as it kept working for the fifth hour in a row.

Though batch processing has limitations it opens up the ability for researchers to run large scale experiments on the initial response from AI models that can be used to measure the variance in responses and, importantly, identify biases in AI responses. In the rest of this post, I briefly explain what batch requests are, including the needed data structure, how researchers and others can use the functionality, and share my workflow for batch processing.

What is batch processing

Batch processing gives users the ability to format a large number of AI prompt requests and to submit them all at once to the model API for processing. The requests are returned as a batch within 24 hours in a set it and forget it functionality. Using this method means that the user does not run an interactive prompt-response-prompt session, but rather it is for one-off responses. Each prompt in the batch uses a new ‘conversation’ meaning that the model does not have context or memory from prior prompts which is helpful in some scenarios like mine where I am typically looking to measure the variance in responses and identify possible outliers. But, this means that if your project depends upon being responsive to a model, batch processing will not work.

You can submit batches via the GUI for each of the major platforms — ChatGPT, Claude, and Gemini. Batches are formatted in json in fairly similar formats across each platform. Each request within the batch requires the user to define which model to use, the prompt content, and the user. The differences between the platforms are differences that most developers will be familiar with such as being able to set a temperature in ChatGPT.

{"custom_id": "report-0_prompt-prosecutor_high", 
  "method": "POST", 
  "url": "/v1/chat/completions", 
  "body": 
  {"model": "gpt-4o-mini", 
    "temperature": 1.0, 
    "response_format": 
     {"type": "json_object"}, 
  "messages": [{
     "role": "system", 
  "content": "You are a legal assistant. Respond only with valid JSON."}, 
  {"role": "user", 
  "content": "prompt text"}]
}}

How can this be used by researchers

As outlined in the ChatGPT cookbook, the best use of batch jobs is for “Tagging, captioning, or enriching content[,…] Categorizing and suggesting answers [based on text or images,…] Performing sentiment analysis on large datasets[,…and] Generating summaries or translations for collections of documents or articles.” For researchers, this is an opportunity to create large amounts of new data that can either be used to research original content or the output from AI.

For instance, if a researcher was interested in understanding the different ways in which politicians speak, they could use a dataset of speeches and public appearances and enrich that content with a predetermined list of labels. This type of task used to be done by human review, transitioned to sentiment analysis code, and now can be done using a custom AI prompt that offers more granular and flexible answers. Researchers might also be interested in the AI response itself — either for capability testing to determine if AI can perform a certain task or for sensitivity analysis to assess the variability in AI responses.¹ Batch processing allows a researcher to conduct either type of test — constrained by the one-off prompt scenario — as we can ask the same prompt over and over again to measure response variance or provide a set of different prompts with clear criteria for passing that amount to large-scale benchmarking.

How to batch request

Rather than providing generic code snippets, I’m going to describe how I thought about batch processing and my AI-aided documentation for both batch processing and prompting AI to write the code. I’m approaching it this way because I’m not a good enough coder to write better, more comprehensive code than AI and I think that using AI to write initial code is more productive than trying to write it oneself. Furthermore, OpenAI, Anthropic, and Google all have great code cookbooks that give better overviews than I could for the actual code.

First, and the only part that I think really requires critical thinking before starting off with an AI prompt and directions is working through the batch data structure. For my most recent experiment, I had four different prompts I was wanting to use and 240 different pieces of content to provide to each prompt (so 960 total requests). For each piece of content there were critical pieces of metadata that I wanted to track as well. I needed to then set up my batch request with a clear custom ID that would be returned in the batch response and could then be linked back to my data source to include the metadata. I also needed to write a short loop to create separate requests for each prompt and piece of content. Getting this right did take a few test iterations that I ran against the one-off API rather than setting up a batch, running it, and then seeing if it worked. I did it this way because it was easier for me to track what I was doing and was faster than submitting a batch and waiting for a return that could take up to 24 hours.

After establishing my data structure, I moved to using prompts to write the batch processing workflow code. I started by drafting a batch processing markdown file that outlined what I wanted to accomplish and my various considerations and asking Claude to refine the output for simplicity (here’s a generic file that is similar to my markdown.) In creating these files, I always include security considerations and testing. These are two elements to coding that I think most developers and especially researchers are not typically attuned to because they are not trained computer scientists and are working quickly, but that AI is particularly good at and can easily add to a coding base.

Using this workflow and a basic prompt, I then created an initial batch processing workflow. While the workflow worked, there were issues with the responses that I didn’t anticipate — not all the responses within the batch were properly formatted json or necessarily responsive to my prompt — hallucinations that did not follow the prescribed format in response that I requested. I should have anticipated the bad responses and hallucinations since this was an issue I ran into when using the one-off API request approach. In that workflow, I lost a lot of time and money writing code that would evaluate responses and re-request if it was a bad response. In the batch processing workflow, I instead added a step to build in error handling after parsing responses to run another batch to replace malformed and incorrect responses.

In the end, I ended up with a workflow where:

1. Create the batch json using source data
2. Run the batch / check on its status
3. Download the responses
4. Parse the response to an analysis ready dataset that is merged back with the original dataset for important metadata
5. Identify malformed responses and build a new batch request to replace these observations
6. Steps 1–4 again for the replacement observations and append to analysis dataset
7. Done

All of this was accomplished with a well-documented markdown file, added testing and logging, and with a constant security review to ensure that I was not exposing API keys or sensitive data. To create a similar workflow, I suggest creating a similar batch processing outline document, setting up and structuring your dataset, and then reading through, editing, and using this AI prompt to kick off the coding.

Conclusion

Batch processing offers researchers with constrained budgets and limited coding experience an opportunity to enhance datasets or create new datasets. Within the criminal justice space, we can use batch processing for a variety of tasks that can significantly impact the system. This includes widespread case review and legal drafting and is especially relevant in a field facing significant resource constraints — both with respect to practitioners and those involved in the system. The reduced cost from batch processing and the simplicity of creating batches means that there are real opportunities to address common roadblocks in the system. Beside this opportunity, we need to approach use of AI, especially in this space, with caution as we look to mitigate the many possible hidden biases² that AI can perpetuate.

We can best utilize tools like batch processing through a robust community of public researchers testing publicly available models in novel ways. This community would be best placed to find the boundaries of both AI capabilities and variance. The more researchers experimenting with using AI, especially tools like ChatGPT, Claude, and Gemini, the better. This is needed since many software products are now embedded with these models. And despite claims of finely tuned models by software companies, testing against the frontier models is likely the best approach since they are the most up to date and offer features like batch processing that make testing scalable. We still do not understand the ultimate impact of these tools, but given their rapid adoption we should continue to test and shape their use to society’s benefit.

By: Rory Pulvino, Justice Innovation Lab Chief Implementation Officer. Admin for a Prosecutor Analytics discussion group.

For more information about Justice Innovation Lab, visit www.JusticeInnovationLab.org.

Footnotes:

1. The Center for AI Safety has plenty of interesting research on both capability and sensitivity analysis in AI and is a good resource for understanding the landscape of research into AI responses.
2. A recent WaPo article identified that ChatGPT exhibits random biases such as assuming that certain states are more educated than others. This finding is consistent with our finding that bias can appear in many unexpected forms within AI models.

Jury duty. Something I’ve watched play out on the big and small screen, read about in the news, and studied in school but until this month, something I had never actually done.

In February, I received my first summons. I was excited and curious: curious about the process, about the type of case I was being called for, and about how the reality of jury duty would compare to how it’s portrayed in movies, TV dramas, and the books I’d read.

Here are my top five surprises about the process:

1 — The less-than-smooth logistics. After receiving my summons in the mail, follow-up communication arrived via text and email, which I had opted-in to get. I received multiple confirmations and reminders leading up to the day. I had addresses for both the parking garage and the reporting building. And yet, the morning I drove downtown, nothing felt quite as clear as it had on paper. The underground garage had two elevators leading to two different street entrances and no indication of which one to take. I emerged at street level, walked into the wrong building, and was kindly redirected across the street. Upon arriving at the correct building, the security line stretched down the sidewalk. It took about 20 minutes to get through, though staff did open two additional lines while I waited. I made it to the jury room just as instructions were beginning.

2 — The number of people involved. When I walked into the jury room, it was packed. Seats were filled and people were standing along the walls. With a quick look, I estimated around 200 potential jurors were in that room — far more than I had anticipated. I learned that once a quarter, the court selects jurors for a grand jury, which is a three-month, three-day-a-week commitment, and that process happened to coincide with my appearance date. After 30 people volunteered for potential grand jury participation, the court began announcing groups for the four trial cases happening that week. My name was called in the second group. Our 40 potential jurors were escorted upstairs to our assigned courtroom for selection to begin.

3 — The adherence to process. As we lined up outside the courtroom, we were instructed to turn off our phones, remain standing until directed to sit, and not speak amongst ourselves, even during breaks while inside the courtroom. In a world of constant noise and connectivity, this felt like a deliberate and meaningful contrast. For hours, no one in that room was scrolling. We sat in silence. We stood and sat as directed.

Finally, the first 14 potential jurors were randomly selected from our group to sit in the jury box, and questioning from the prosecution and defense began.

4 — The education offered. The judge introduced the case — not every detail, but enough context to orient us. She explained that this was the Division II Criminal Court, what that designation meant, and what the week ahead would look like for those selected. Then the Assistant District Attorney took over. She moved from juror to juror, asking questions about their lives and, more pointedly, questions clearly tied to the case at hand. The charge against the defendant was second-degree murder, and she repeatedly explained what that meant — how it differs from first-degree murder or manslaughter, and how it would be the jury’s responsibility to take the law from the judge and the evidence from witnesses to reach a verdict. Repeatedly asking each juror, “Are you willing and able to do that?” The defense attorney followed, asking questions that were more specific to the facts of the case, while similarly walking us through what the state would need to prove for a guilty verdict for his client.

I had expected the questioning. What I hadn’t anticipated was how much we’d be taught. The process was genuinely designed to set jurors up to succeed — to understand not just what they were being asked to decide, but how to think about making that decision.

5 — The amount of waiting. We were told to arrive at 8:30 AM. Instructions to the full group began around 9. My group was taken to our courtroom around 10:30. We broke for lunch at 1 PM and didn’t resume until after 2. Just before 4 PM, the jury was finalized. I was not selected (my name was never called to sit in the jury box) so I spent the entire day in the gallery, watching and listening.

Throughout the day, I was struck by the cross-section of people summoned for jury duty. Two hundred strangers, from all walks of life, gathered because they were asked to show up. Whatever randomized process brought that group together, it seemed to work.

When the questioning of potential jurors began, I was reminded how little we actually know about the people we pass every day. Jurors shared about their family lives, whether they were married, had children, where they worked. Several mentioned work stress, unemployment, the weight of supporting a family on a single income. Then came questions more directly tied to the case: thoughts on firearms, on drugs, experiences as victims of crime. And with each answer, another layer of someone’s life came into view — reasons for owning or not owning a gun, addiction in the family, the complicated aftermath of being wronged.

It struck me that this is, in many ways, the point.

The American jury system is built on a deceptively simple idea: that ordinary people — not just legal experts, not just those with power or credentials — are capable of weighing evidence, applying the law, and reaching a just verdict. It’s a system that insists, at its core, that justice belongs to the community. That a person accused of a crime deserves to be judged by their peers, not by the state alone.

That ideal doesn’t always live up to its promise. Access, bias, and structural inequities shape who ends up in that courtroom on both sides of the bar. The work of improving the system is ongoing, and it’s urgent. But sitting in that gallery, I watched a room full of people set aside their phones, their plans, and their day-to-day lives to take seriously the weight of someone else’s fate.

Civic participation is easy to take for granted until you’re sitting in the middle of it. Jury duty asks something rare of us: to show up, to listen, and to decide. Together. That’s not a small thing. I went into that courthouse curious. I left tired and grateful. Grateful to have witnessed it, and to come away with a greater understanding about the process and why it matters that we keep working to make it better.

By: Kelli Ross Metz, Justice Innovation Lab Chief of Staff and Senior Communications Specialist

For more information about Justice Innovation Lab, visit www.JusticeInnovationLab.org.

Every system is perfectly designed to get the results it currently gets. That truth applies not only to criminal justice systems across the country, but also to how the leaders inside those systems come together to solve problems. If we want different outcomes — safer communities, fairer processes, better use of public dollars — we have to rethink the architecture behind both.

The Justice Innovation Accelerator was built with this core idea in mind. Drawing on systems thinking and the principles behind Priya Parker’s The Art of Gathering, the Accelerator creates intentional spaces where justice leaders can step out of outdated routines and design new ways of working. Instead of defaulting to familiar patterns and debates, the Accelerator gathers people around a clear purpose: to redesign a component of justice operations so that they deliver more safety, fairness, and efficiency.

A Purpose-Driven Space for Change

Too often, criminal justice reform efforts get stuck because the wrong people are in the room, or because the setting encourages speeches rather than solutions. The Accelerator flips that script. Each cohort brings together cross-sector teams — prosecutors, police leaders, judges, public defenders, social workers, victim advocates — whose roles interact every day but who rarely problem solve side by side.

This is not a networking event. It is a designed gathering, where every exercise, conversation, and constraint serves the purpose of diagnosing structural problems and co-creating practical solutions. The design is intentional: curated teams, structured collaboration, and a guided arc that moves from insight to action.

From Insight to Implementation

Over a 12-month journey, participants engage in virtual sessions, peer learning, expert coaching, and a three-day in-person workshop that functions as the Accelerator’s catalytic “heat.” This is where teams are pushed into productive tension — challenging assumptions, testing ideas, and prototyping solutions using real data.

The 2025 cohort showed what becomes possible when you create a space that protects purpose and invites bold thinking. Teams from Suffolk County (Boston), Massachusetts; King County (Seattle), Washington; Augusta, Georgia; Salt Lake County, Utah; Pine County, Minnesota; and Douglas County, Georgia; arrived with complex local challenges. They left with concrete, implementable pilots they designed themselves.

These include:

• SWIFT Justice in Suffolk County is a real-time diversion model pairing individuals with substance-use challenges with treatment, housing, and recovery support.
• Salt Lake County’s scaled pre-file diversion system, which already boasts a 93% success rate and under 4% recidivism.
• Douglas County’s Fast Track model, which is designed to cut case-processing times by more than half.
• Pine County’s Victim Services Optimizer and King County’s Respect, Voice, Transparency, Trust, are both redesigning victim engagement.
• Augusta’s streamlined pretrial process is reducing unnecessary incarceration and saving taxpayer dollars.

As one participant put it: “It forever changed the way I see problems in the criminal justice system — and problems in general.”

Scaling a New Architecture of Justice

Gatherings alone do not change systems. To turn these pilots into lasting, scalable improvements, jurisdictions need time, coaching, and sustained collaboration. Over the next three years, we will hold three Accelerators with a target of 30 more communities with the skills and evidence base to transform operations nationwide. Successful models are then scaled to help similarly situated communities.

The architecture of American justice is outdated — but it is not immutable. When we gather with purpose, authority, and intention, we create the conditions for transformation. The Justice Innovation Accelerator offers the blueprint. Now it’s time to scale it.

By: Jared Fishman, Justice Innovation Lab Founder and Executive Director

For more information about Justice Innovation Lab, visit www.JusticeInnovationLab.org.

As we wrap up 2025, we asked each Justice Innovation Lab team member to share their favorite criminal justice-related or work-related read, listen, or watch from the year. Below, we are sharing that list — which includes books, newsletters, and documentaries — with you.

Jared

Priya Parker’s The Art of Gathering is a bold reminder that most of our meetings, events, and celebrations fail long before they begin — because we never bother to ask why we’re gathering in the first place. When we design gatherings around a clear, specific purpose, everything else — who’s invited, how we structure the time, what rules we set — falls into place with far more meaning and impact.

What makes this book compelling is how aggressively it challenges our default behavior. She urges readers to stop copying traditions and instead design experiences that match the moment. Her examples, from conflict mediations to dinner parties, make one thing clear: the difference between dull and transformative is almost always intentionality.

That principle mirrors what we try to do inside the Justice Innovation Accelerator. Our work depends on curating the right mix of people — leaders whose decisions interact every day but who rarely problem-solve together — and then creating conditions where they can challenge assumptions, examine data, and co-design solutions. The Accelerator isn’t a conference or networking event; it’s an intentionally crafted experience where boundaries, structure, and facilitation help teams move from insight to action.

Reading The Art of Gathering made me see our program with fresh eyes: at its heart, the Accelerator is itself a gathering — with a clear purpose, a defined arc, and a design meant to break unhelpful conventions of how justice leaders typically convene. It reminded me that our role isn’t just to share tools or data; it’s to create the kind of space where people can imagine and build better systems together. Parker’s book reinforced that meaningful change begins with how we gather — and it strengthened my conviction that when we design those moments with intention, we give jurisdictions the chance to redesign the systems their communities depend on.

Jess

This year, I am recommending Parker Molloy’s newsletter, The Present Age. While this newsletter covers a wide range of topics related to media and culture, I find a lot of the content relevant to work in the criminal legal system. One of the most impactful newsletters I read this year, which JIL discussed during a group learning session, was “Stop Pretending Chatbots Have Feelings: Media’s Dangerous AI Anthropomorphism Problem.” This piece focuses on the lack of safety protections for AI users as tech companies disregard concerns in a rush to deploy products, as well as the danger of media outlets’ tendency to assign feelings and intent to AI models when things go wrong, instead of holding responsible the companies that have created the problematic products.

I found this writing so impactful because it does a great job of highlighting how dangerous it can be to humanize AI and to rely on it for accurate information. The piece also touches on biases inherent in AI products as a result of models being trained on human-generated content. With the expanding use of AI in the criminal legal space, it’s crucial to understand the ways in which AI models can generate inaccurate content, and how AI users may be susceptible to inadvertently perpetuating injustices.

Kelli

This year, I have two recommendations. First, if you’re looking for a daily news overview of criminal justice coverage, The Marshall Project’s Opening Statement is an excellent resource. I start almost every workday by scanning through this email newsletter, which offers a wide variety of news articles highlighting criminal justice and immigration-related stories from across the country. You can sign up for this newsletter (and others from The Marshall Project) here.

Second, I watched the recently released The Alabama Solution (currently streaming on HBO Max). This documentary provides an inside look into the Alabama prison system and the abuse and neglect those incarcerated face. A majority of the film features clips of those inside the prisons — specifically Robert Earl Council, Melvin Ray, and Raoul Poole who used cellphones they purchased through the prison’s black market — shooting videos of themselves explaining the conditions they live in, abuses of power, and regular violence — including killings — within the prisons. The film also interviews family members of those incarcerated, highlights the political landscape impacting policies driving decisions at the prisons, and explains the slave labor work conditions those incarcerated face. The documentary is disturbing, insightful, outraging, and painful to watch but so important. It effectively highlights the humanitarian crisis in the Alabama prisons in ways a viewer can’t forget.

Kevin

In his book, There is No Place for Us: Working and Homelessness in America, Brian Goldstone centers five families in Atlanta, Georgia, to highlight how systemic failures — specifically stagnant low wages, the rising cost of housing, and the erosion of social safety nets — are rapidly increasing the number of homeless individuals from the ranks of the “working poor.”

There are many reasons why someone working more than 40 hours a week wouldn’t be able to afford safe, clean housing. A notable one is that no one earning the federal minimum wage would be able to afford the median two-bedroom apartment in any U.S. metro. Divorce, medical debt, loss of job, or 100 other problems could befall a family and push them into homelessness.

This work is both inspiring and crushing. By recognizing the millions of unseen homeless — those people not sleeping in the street, but couch surfing with family or friends or staying in unsustainably expensive “extended stay” hotels — this book focuses attention on an invisible problem that requires collective action. This book is a must read for anyone who wants to understand that laziness, drug addiction, and mental health issues are not the primary reason people go unhoused. What remains to be seen is how society reacts to these systemic injustices.

Lily

As background research for a long-term project of JIL’s concerning RAP sheets, I read The Eternal Criminal Record by James B. Jacobs. It’s more of an academic book with lots of court case summaries than an easy read, but I recommend it if you’re looking for a deep dive into the history of criminal record keeping and how our focus on privacy vs. transparency has changed over time.

The book is divided into four sections: 1) The Production and Dissemination of Criminal Records, 2) Key Policy Issues, 3) U.S. Criminal Record Exceptionalism, and 4) Direct and Collateral Consequences of a Criminal Record. I found Part 3, which compared policies governing criminal records in the United States to those of countries in the European Union, especially insightful. I hadn’t realized just how much of an aberration the U.S. is with regards to making criminal records so readily available. The ways in which records follow people forever, even for arrests that were dismissed for being unfounded or for which the defendant was found not guilty, is a uniquely American phenomenon. I also hadn’t realized how much variation between states and jurisdictions there is with regards to policies governing criminal records, despite failed efforts to centrally regulate their dissemination.

The book emphasized the stigmatizing effects of criminal records and the ways in which they hinder people from employment and housing as well as paint a perception of them as inherently “criminal” should they have further contact with the criminal legal system. Especially in the case of arrests that didn’t result in convictions, circulating these records to individuals/private entities (employers, landlords, individual FOIA requests) is antithetical to due process, argues Jacobs, and should be reconsidered from a Constitutional perspective. Overall, the book provided a thorough overview of how the United States has grappled with these issues over time and offers frameworks for how we might move forward in a more fair and just way.

Rory

I thought I was going to suggest this econ paper came out in March and directly addressed a burning question — what is the marginal effect of dismissing a case versus an individual being convicted, but serving no time. After reading the paper, I can’t, in good conscience, recommend a dense economics paper that required multiple read-throughs. Instead, I’m recommending the Turning Point documentary from Turn90, a South Carolina-based nonprofit. I learned about the documentary because we partner with Turn90 on a few projects and it was a great testament to the director’s ability to adapt and push forward.

Many formerly incarcerated people face significant hurdles in getting employed post-incarceration. Incarceration often interrupts education and job training such that people miss critical periods to develop professional skills. Prison is a break from society such that when a person returns, society and their personal world has often transformed. In addition, prison is traumatizing. Working with this population is challenging and providing them with all the training needed is a big lift. Watching the documentary showed how the director and creator of Turn90 continually identified new needs for this population and developed answers for the program.

Turn90’s story is inspiring both for the individuals you meet in the film as well as the story of the organization. For an organization that is also pushing to work in the criminal justice space and find solutions, this organic evolution that ultimately creates a strong organization with demonstrated impact is helpful and encouraging.

Learn more about Justice Innovation Lab at https://www.justiceinnovationlab.org/.

Every system is perfectly designed to get the results it currently gets. In the case of the American criminal justice system, those results are unacceptable: billions in taxpayer dollars spent annually, low clearance rates for serious crimes, high recidivism, eroded public trust, and deep inequities that track along racial and socioeconomic lines. These outcomes aren’t accidents; they are the predictable result of a system designed for another era.

Our justice institutions were built decades ago, for far smaller caseloads and simpler social conditions. Today’s realities — mental health crises, addiction, digital evidence, and community mistrust — strain an outdated architecture. Dockets are clogged, jail populations swell with people awaiting trial, and communities lose faith that justice is being served.

If we want different results, we need a different design.

From Politics to System Design

Systems thinking offers a powerful lens for justice reform. Instead of focusing on individual policies or political debates, it asks how the parts of the system — law enforcement, courts, public defense, corrections, and community organizations — interact to produce outcomes. When those interactions are misaligned, even well-intentioned actors can perpetuate inefficiency or harm.

Doing justice, then, requires redesigning the system itself. This is the mission of the Justice Innovation Accelerator, a program that equips jurisdictions with tools and frameworks to rethink how justice actually works on the ground. The Accelerator helps leaders diagnose root causes of dysfunction, build cross-sector partnerships, and test data-informed solutions that improve safety, fairness, and trust.

The Justice Innovation Accelerator: A Blueprint for Change

The Justice Innovation Accelerator starts with the simple but transformative premise that our current justice system produces the outcomes it was designed for. To achieve better outcomes, we must redesign the system’s architecture — intentionally, collaboratively, and using data and evidence.

Over a 12-month program that includes virtual sessions, expert coaching, peer learning, a three-day in-person workshop, and six months of follow up, the Accelerator helps jurisdictions move from problem to insight to pilot. Its approach rests on four pillars:

1. Equip Leaders — Provide justice leaders with diagnostic tools to identify structural problems, not just symptoms.
2. Enable Collaboration — Bring prosecutors, law enforcement, judges, defense attorneys, advocates, and community organizations together to co-create solutions.
3. Empower Implementation — Support teams in prototyping and testing ideas with real-world data.
4. Expand the Evidence Base — Build proof points that show what works and can be replicated nationwide.

The Inaugural Accelerator

In October 2025, Justice Innovation Lab hosted the inaugural Justice Innovation Accelerator in Nashville, Tennessee, in partnership with Vanderbilt University Law School’s Project on Prosecution Policy and Prosecution Leaders of Now. Six diverse jurisdictions — Boston (MA), Seattle (WA), Augusta (GA), Salt Lake County (UT), Pine County (MN), and Douglas County (GA) — sent cross-sector teams including district attorneys, law enforcement, judges, victim advocates, and social workers.

During the intensive three-day workshop, each team developed an actionable pilot project to tackle a systemic challenge. Participants described the experience as both “inspiring and tangible,” providing them with tools that “forever changed the way [they] see problems in the criminal justice system.”

The pilots that emerged from this first Accelerator represent a new generation of locally-led and evidence-driven justice innovation with a focus on measurable results.

Early Outcomes: Transformational Projects in Motion

In Suffolk County (Boston), Massachusetts, the SWIFT Justice initiative is working to divert more individuals with substance use disorders toward recovery rather than prosecution. Through real-time data sharing and coordinated staffing, the pilot will test ways to accelerate access to treatment, housing, and employment to reduce recidivism and improve public health outcomes.

Salt Lake County, Utah, is expanding its pre-file diversion program, which already boasts a 93% success rate and under 4% recidivism. By improving data tracking and interagency coordination, the county aims to significantly increase participation in diversion opportunities and further reduce case backlogs.

In Pine County, Minnesota, the Victim Services Optimizer introduces a single point of contact for victims of domestic violence, improving communication and safety planning. The goal is to improve victim satisfaction and reduce repeat victimization.

Douglas County, Georgia, is tackling case delays with its Fast Track model, designed to cut average case resolution time from 439 days to under 180. Faster case processing saves money while enhancing fairness for both victims and defendants.

King County (Seattle), Washington, launched Respect, Voice, Transparency, Trust, a program to standardize victim outreach and feedback loops across all cases. With clear role definitions, shared databases, and ongoing listening sessions, the project aims to rebuild trust and consistency in victim engagement.

Finally, Augusta, Georgia’s Reducing Crime by Reducing Time initiative streamlines pretrial processes to alleviate jail overcrowding. Automatic bond rehearings and coordinated calendars help ensure no one sits in jail simply because of their socioeconomic status.

Together, these projects demonstrate what’s possible when justice systems are redesigned for outcomes.

Scaling for the Future

The inaugural Accelerator proved that transformation is possible when leaders have the tools and space to reimagine justice. But to achieve national impact, this model must scale. Every jurisdiction faces similar architectural failures — siloed operations, misallocated resources, and fraying trust.

With sustained investment, a multi-year Accelerator can reach dozens more communities, generating hundreds of tested solutions. Each cohort builds a stronger evidence base of what works, creating a network of reformers capable of redesigning justice across America.

In an age of polarization, this approach offers a rare point of consensus. The Justice Innovation Accelerator is neither “tough” nor “soft” on crime — it is smart, fair, and effective. By redesigning the architecture of justice, we can build systems that deliver both safety and justice for all.

The architecture of American justice isn’t working — but it can be fixed. The blueprint is here.

By: Jared Fishman, Justice Innovation Lab Founder and Executive Director; and Rory Pulvino, Justice Innovation Lab Chief Implementation Officer.

For more information about Justice Innovation Lab, visit www.JusticeInnovationLab.org.

The Council on Criminal Justice recently released a set of 5 principles for the use of AI in the criminal justice system. Overall, I think this is a solid framework that addresses many of the right concerns, but there are a few areas where I’d love to see more clarity or expansion.

Getting Definitions Right

I really appreciate the Scope and Definition section. The clarity around what counts as AI and the concrete examples are helpful for practitioners who might be navigating this landscape for the first time. That said, I was surprised by how they categorized complexity. They present hot spot mapping as relatively simple and machine learning-based risk assessments as complex, but my experience suggests it’s not quite that straightforward.

Hot spot mapping software often lets you click a button and generate a map, which seems simple enough. But under the hood, many of these tools run complex algorithms that few people actually understand. In contrast, risk assessment tools are frequently based on complex analyses that’s then converted into a point system someone can calculate by hand. I haven’t really seen the kind of dynamic, opaque risk assessment algorithms that would warrant the “complex” label. This distinction matters because it affects how we think about oversight and transparency requirements.

Who Needs to Know What, and When?

The Confidential and Secure principle raises some important questions for me. It states that people should be able to know about and contest the use of their information in AI decision-making. But what are the boundaries here? Does this apply when someone’s information is used to train an algorithm, or only when they’re an input to an already-trained system? And what about AI tools that aren’t technically decision-making tools — like generative text LLMs used to write police reports or photo enhancement software used in investigations?

I can see the argument for limiting these notice requirements to decision-making tools, which keeps the scope manageable. But I could also see a strong case for broader language that captures more types of AI use. This feels like an area that deserves more discussion and specificity.

Standards I Really Like

The Effective and Helpful standard really resonates with me. The idea that AI systems should demonstrably improve outcomes compared to the status quo is exactly the bar we should be setting. I’m not sure it’s always possible to measure this perfectly — after all, the “costs” of the status quo aren’t always well-defined — but as a guiding principle, this is spot on.

Looking Forward

I believe the Safe and Reliable principle could be improved through clearer language about how AI systems need regular, continuous testing and updating. The current language focuses on AI systems performing well in “novel situations,” which is important. But the principle should be expanded to explicitly address the need for these systems to change and update based on evolving laws, policies, and values.

This matters because AI systems can become outdated not just when they encounter novel situations, but when the world around them shifts. What was considered best practice five years ago might not align with current values or legal standards. The document hints at this issue — you can see it in discussions about AI reflecting current circumstances and in the Democratic and Accountable principle — but I’d like to see it stated more directly upfront.

Keeping tools and developers responsible

The principles do not clearly state who they apply to which leads to ambiguity, specifically for the Democratic and Accountable principle as to who it is meant to apply to? This principle, like the others, seems directed at both the agencies that will use these tools and the vendors who create them. But the language often addresses “AI systems” in the abstract rather than explicitly naming these two types of entities as responsible parties.

I’d like to see more explicit language that clearly delineates responsibilities for both criminal justice agencies and technology vendors. Accountability works best when everyone knows exactly what’s expected of them.

Connecting to the Broader Conversation

Finally, I would like CCJ to connect this work to other industries or organizations working on AI standards — particularly NIST, which has been developing AI frameworks that many sectors are looking to for guidance. Placing these criminal justice-specific principles in the broader context of how healthcare, finance, and other industries are grappling with similar issues would strengthen the document and help criminal justice practitioners learn from parallel efforts.

Conclusion

These principles represent thoughtful work on a critical issue, and I’m glad to see the Council on Criminal Justice engaging with AI governance in such detail. We need frameworks like this, and refining them through discussion and feedback will only make them stronger.

By: Rory Pulvino, Justice Innovation Lab Chief Implementation Officer. Admin for a Prosecutor Analytics discussion group.

For more information about Justice Innovation Lab, visit www.JusticeInnovationLab.org.

A Friday in March was my last day in the office before parental leave — my first child was born the following Wednesday.

In the months that followed, I faced what I’m sure all new mothers do: exhaustion like I’d never known, the all-consuming work of figuring out how to feed this tiny life, and a body that felt completely foreign. Between follow-up doctors appointments, physical therapy, and learning to navigate parenthood and a new dynamic with my husband, I was also trying to keep tabs on the greater world.

I couldn’t take in too much news. I literally didn’t have time to doom scroll for hours. But even if I could, it would have been too much — too much for my spirit, too much for my emotional capacity. And yet, I am now raising a baby who will one day be an adult. How do I make sure my husband and I are parenting him to be kind, empathetic, someone who cares for his neighbors, values all humans, and loves those he meets?

While on leave, trying to keep my head above water, there were No Kings protests in my city. In my previous life, I would have been there. Now, I was rocking my child to sleep. But I also knew that when my leave ended, I’d be returning to work — and I had the privilege of working to create a better world in my 9-to-5. While my volunteering was taking a back seat for this season, I could still participate in righting wrongs simply by going to work.

I also knew that when I returned, my organization, Justice Innovation Lab, would be hosting a Justice Innovation Accelerator workshop in October. This workshop had been years in the making, building on JIL’s work over the last five years.

As a witness and participant at the Accelerator, I was filled with hope during a time when hope can be hard to find. I saw prosecutors, law enforcement, and social workers coming together to think through innovative solutions to challenges in their communities. These decision makers were working to make their corners of the world better, and their enthusiasm was infectious.

Life right now is painful for so many, and it can be paralyzing to figure out what to do and how to help. I don’t have the answer for how to solve the world’s problems, but at the Accelerator I was reminded that in each of our personal worlds, there are problems we can tackle, people who will cheer us on, and we can make a difference by taking the first small step.

For me right now, that means loving my child and my husband. Caring for my neighbors. Showing up at my job and giving my best, while learning to balance motherhood with my career. Staying informed without doom scrolling. Calling my representatives.

I know these steps are small when I consider that people across this country and around the globe are facing hunger and war, unemployment and homelessness. Some who came here hoping for a better life now live in daily fear of deportation. I face none of these challenges, and I don’t pretend that my struggles compare.

But here’s what I do know. Change doesn’t only come from those in crisis — it also requires those of us with stability and resources to use what we have. My day job is to support people trying to reform broken systems. My time, even limited, can go toward advocacy. My voice can amplify others. And my child can be raised to be part of the solution.

Whatever your life looks like right now, I hope you can find glimmers of hope and the courage to take your next small step.

By: Kelli Ross Metz, Justice Innovation Lab Chief of Staff and Senior Communications Specialist

For more information about Justice Innovation Lab, visit www.JusticeInnovationLab.org.

In 2022 and 2023, Justice Innovation Lab led successful workshops in Charleston and Memphis that led to meaningful changes in policy and process to fix problems in their respective criminal justice systems. Our process — which integrates systems thinking, human-centered design, and measuring what matters — helps communities take the first step to building solutions.

After the successes of those pilots, I was asked whether we could design a program to help five jurisdictions develop similar pilot projects. I said, “I’ll do you one better. How about six?” And, the Prosecution Innovation Accelerator was born.

Teaming up with our partners at the Vanderbilt Project on Prosecution Policy (VPOPP) and Prosecution Leaders of Now, we developed an immersive program — part virtual sessions, part in-person workshop — to help jurisdictions identify a problem they wanted to tackle, build the teams necessary to solve it, and design a viable local solution to address the problem.

Last week, justice system decision makers from Suffolk County, Massachusetts (Boston); Salt Lake County, Utah; Douglas County, Georgia (outside Atlanta); King County, Washington (Seattle); Augusta, Georgia; and Pine County, Minnesota, gathered in-person at Vanderbilt University Law School for our first-ever Prosecution Innovation Accelerator.

Each jurisdiction came with a challenge they wanted to solve and the necessary teammates — prosecutors, law enforcement, social workers, victim services providers, county and city officials, and a judge — who were committed to solving it.

The workshop focused on “front-end” system problems that impact numerous people in their jurisdictions. Teams focused on issues ranging from:

• Courts being overwhelmed with low-level offenses associated with homelessness, substance abuse, and mental illness
• Jail overcrowding
• Unnecessary delays in resolving cases
• Crime victims feeling inadequately supported in the legal process

To come up with an actionable solution, the six jurisdictions participated in sessions on design thinking, systems mapping, and storyboarding. The goal was for each team to design a pilot solution they could pitch “Shark Tank-style” by Day 3 of the Accelerator. Over the course of the workshop, these teams ideated, evolved, and moved. They got creative and used their imaginations.

Not only did all six teams pitch viable, impactful solutions to their problems, but they mapped out a path to implementation. It’s only been a week since we left Nashville, and already these communities are moving forward by turning these pilots into reality.

Over the next six months, Justice Innovation Lab and our partners at VPOPP will assist these offices as they implement the solutions and measure their impact.

When we arrived in Nashville, my hope was that each jurisdiction would walk away with a new solution they could bring to their communities. They left with so much more. They left with a network of innovators facing similar challenges. They left with tools they can use again and again. They left with proof that change doesn’t require massive budgets or years of planning — it requires the courage to reimagine what’s possible.

A year ago, someone asked what it would take. Last week, we showed them. And, this is just the beginning.

Thank you to Arnold Ventures for providing funding for the Prosecution Innovation Accelerator.

By: Jared Fishman, Justice Innovation Lab Founder and Executive Director

For more information about Justice Innovation Lab, visit www.JusticeInnovationLab.org.