Model Justice — Part 1

A few of days ago, defense lawyer Mark Bennett tweeted an interesting thread about how loss aversion bias could affect a criminal defendant’s decision to take a plea deal:

(If you’re interested, read the whole thread.)

For as long as I’ve been following criminal defense lawyers on the web, they’ve been discussing the way the criminal justice system encourages defendants to accept plea bargains rather than insisting on trials to force the state to prove its case in court. Often it’s some pissed off lawyer who wants to eliminate plea bargaining or end pretrial detention for routine offenses.

I think these kinds of reform proposals raise some interesting questions, and I’d like to explore them. I have some of my own ideas about how plea bargaining works, and why the system is the way it is, and I’ve been thinking of writing a blog post or two about it. But then I got a crazy idea…

For a while now, I’ve been trying to teach myself how to do system dynamics modeling, and these kinds of questions seem like they would make an interesting subject. I’m going to try to build models for some of the strategic issues affecting trials and punishment.

Models of systems are necessarily much simpler than actual systems because the real world is too complex and messy to recreate in a hand-built computer model with perfect detail, and yet we build models because we want them to tell us useful things about the real world. Thus the art and science of model design involves choosing which details the model must capture and which details it can safely ignore.

Before you can do that, however, you have to know how the model will be used. If you are designing a new type of commercial airliner, and you want to know the predicted fuel efficiency under a variety of flight regimes, you may need to build a detailed simulation of the engine and flight control surfaces and connect them to an aerodynamics model parameterized by a weather model of the troposphere and stratosphere. On the other hand, if you work for an airline that is operating the new airplane, you may be able to plan flights around the country using little more than a few tables that relate speed, altitude, and fuel consumption.

A related concern is the accuracy of the model in making predictions. A company planning to build a package sorting facility may do extensive modeling to test the layout and operation of machinery and workstations to achieve optimum efficiency. This has to be good enough to generate quantitative results that will help project managers spot subtle problems, detect conflicting operations, capture transient events, and calculate costs.

At the other end of the accuracy spectrum are relatively simple models that capture broad relationships within a system without attempting to produce mathematically precise predictions. A lot of social science and economic models fall into this category. These are tools of exploration and learning, and we use them to test our intuition and refine our understanding.

That’s what I’m aiming for here. The model I’m building is going to start out extremely simple — it’s not even going to have plea bargaining at first. I hope to improve the model in future posts so it can exhibit more complex behavior including, eventually, plea bargaining. But before I can model plea bargaining, I have to model the circumstances under which plea bargaining arises, and that’s going to take a while.

I’m building this model using a simulation tool called Insight Maker. I chose it because it’s totally free, and because the simulation engine and development environment are written in JavaScript, which means it runs in a web browser. In addition, once I’ve got a model I like, I can share it on the Insight Maker website, and other people can run it directly in their browsers without needing to download and install specialized software.

I’ve decided to use an agent-based model. That means each defendant will be represented in the model by a separate simulation entity which goes through a series of state changes as the model runs. For this initial model, each defendant will progress through three states: Awaiting Trial, On Trial, and (if found guilty) Prison.

I’ve decided to use a simulated time interval of one month between each iteration of the model. That strikes me as a reasonable unit of time for describing the period spent awaiting trial and the period spent in prison. For the sake of some technical convenience, the period spent on trial will be modeled as lasting exactly one month. That’s unrealistic, but I don’t think it’s unrealistic in a way that matters.

I could switch to a shorter unit of simulated time, such as a single day, to make the model more flexible, but I decided to keep it at a month for three reasons: First, I don’t think a finer time resolution will produce a better model. Second, using a small time unit suggests that the model is more accurate than it ever could be. Third, iterating daily instead of monthly would increase the number of iterations by a factor of 30, greatly extending the simulation’s run time.

The complete model, or at least the first version of it, looks like this:

(The actual model is here on the Insight Maker website if you want to follow along or play with it yourself.)

The model has four variable parameters which can be controlled by sliders appearing on the page.

Here’s what they control:

• Arrest Rate: This is the number of defendants arrested each month. Every iteration of the simulation will fire the Arrest action, which will create this many defendant agent entities and add them to the Defendants pool, starting in the Awaiting Trial state.
• Time to Trial: This is the amount of time (in months) that defendants will spend waiting for their trial. In this version of the model, nothing interesting happens during this period, and I could have omitted it. However, I think it will be useful in future versions of the model.
• Probability of Guilt: This is the chance that a defendant will be found guilty at the end of the trial. In this version of the model, every defendant has the same probability of conviction. Future models will allow different probabilities of conviction, as a crude measure of the quality of the case. In preparation for that, each defendant agent in this model has an independent ProbGuilty property, but they are currently all set to the same value when the defendant agent is created, and that value is used in determining the trial outcome.
• MaxSentence: Specifies the maximum sentence for the defendant if convicted. As with probability of guilt, this is currently the same for every defendant, but the agents have independent NominalSentence properties that can be used in the future to adjust sentences on a per-defendant basis.

In addition, each agent has other properties associated with it to contain more detailed state information. I’ve added data items to represent the actual sentence (which is always the maximum sentence in this version of the model) and to help count the sentence down as it is served.

The meat of the model is the Defendant agent and its lifecycle, pictured above. Defendant agents begin their life in the model when the Arrest action fires, at which point they receive their probability of being convicted and their expected sentence. They sit in the Awaiting Trial state for however long Time to Trial specifies, and then they transition to the On Trial state and go through a one-month trial. In the Verdict action, they are found guilty or not guilty with a random draw, based on the probability of guilt assigned to their agent when it was created.

If found not guilty, they are removed from the defendant pool and from the model. If found guilty, they move to the Prison state, and the Serve Time action counts down their sentence with each iteration until their sentence is done, at which point they are removed from the model.

Running the model with the above parameters yields a graph displaying the number of agents in each model state over a simulated period of 10 years:

During the first four months, defendants are added to the model in the Awaiting Trial state (green line) at a rate of 30 per month, for a total population of 120. After that, as each new group of 30 defendants is added to the model, the group of 30 from four months ago stop waiting and enters the Trial state (red line). A month later, those 30 are remove from the Trial state, and a new batch of 30 begins trial. For the rest of the simulation run, these values remain in a steady equilibrium state, with 120 defendants awaiting trial, and 30 defendants on trial.

Starting at the 5th month, 30 defendants per month conclude their trials. Half of them, chosen randomly, are found not guilty and removed from trial. The other half — about 15 per month, on average — go to the Prison state. The prison population continues to build up at that rate until the 41st month, at which point the first batch of defendants is done serving their sentence. Since 15 prisoners per month serve for 36 months, the prison population stabilizes at around (30 x 0.5 x 36 =) 540 prisoners from there on out. Unlike the other populations in the model, the Prison population is influenced by the random events of trial, therefore the Prison population exhibits random rises and falls 

The advantage of using a model is that we can play around with it. If we “get tough on crime” by arresting an extra 10 defendants per month and sentencing the guilty ones to an extra year in prison, the steady state prison population booms to (40 x 0.5 x 48 =) 960 prisoners.

These results are not surprising, nor should they be, with such a simple model, and we didn’t really need an agent-based model to figure out these numbers, as demonstrated by the simple math in the preceding paragraphs. We could easily have modeled these same results with a few spreadsheet formulas. I think, however, that an agent-based model will be more suitable when we want to model more complex situations.

A few limitations or simplifications of this model are worth noting:

• The model assumes there are no resource limits. No matter how many defendants are arrested, there will always be enough prosecutors, defense lawyers, and criminal court rooms to try them all without delay. There’s also plenty of prison space for everyone found guilty.
• Everyone commits the same crime. If you wanted to model multiple crimes, or multiple degrees of severity, you’d have to re-run the model multiple times with different parameter settings and combine the results. Note that this implicitly assumes that cases of one type do not interfere with cases of another type, for which it is necessary, but not sufficient, to assume that there are no resource limits.
• As mentioned, all defendants have the same chance of being found guilty, which implicitly assumes that the cases against all defendants are of equal quality.
• If found guilty, all defendants are hit with the same sentence. There is no uncertainty about the sentence if found guilty.

As I develop future models, I’ll be relaxing some of those constraints.

Finally, you may be wondering why I’m bothering building models. After all, there are people who study this kind of thing for a living. They probably have much better models than I could ever build in my spare time. Why don’t I just use their models? Or better yet, why don’t I just read their criminology papers to find out what they’ve already learned?

The short answer is: Where’s the fun in that? I like exploring new ideas and learning new things.

And if you’re a criminal lawyer, prosecution or defense, who finds this project at all interesting, I could use some help. I want to build more elaborate versions of this model, and I need help figuring out what aspects of the criminal justice system belong in this model, and what aspects can be safely ignored. I need to figure out how to model plea bargain decision making on both sides, and I’m not sure where to begin. I’m a professional software developer, but my interest in the criminal justice system is strictly amateur. Nevertheless, I’d hate to put in so much work into the models — and these posts — only to discover that I’ve been making my readers stupider.

You don’t want that on your conscience, do you? Besides, it might be fun.