The debate on “human in the loop” governance models for AI focuses heavily on who should make decisions, yet it often treats decision-making as something that happens at the end of the process. Structuring the problem occurs much earlier and is a key factor in making decisions.

In fact, the way we structure a problem has a disproportionate impact on how well we understand it and our likelihood of solving it. In this article, I’ll unpack why problem structuring is so important and why I believe that it is something users should own in high-risk, AI-assisted decisioning models. An insurance claim I once managed illustrates the point.

While working for an insurance carrier, I once inherited a large-loss property claim that had been closed and re-opened for years. A period home had developed cracks through to its upper floors, and we had originally engaged our own contractor to perform repairs. When the cracks returned, we attributed them to ground movement, which the policy did not cover, and pointed to cracking elsewhere in the house to support the argument. However, we had carried out the initial repair, and the owners wanted us to resolve what they saw as faulty work. Doing so would have involved a far more substantial effort than a patch and fix. The evaluation of the claim became paralyzed around the question of whether the issue stemmed from our repair work or the ground movement. All of the reports on file were focused on this distinction.

It was only when we set this dispute aside and re-structured the problem that we investigated alternative causes, which led us to identify that structural alterations to the property, conducted well before the owners had purchased it and concealed behind later finishes, were the underlying cause. This evidence broke the deadlock and allowed for a resolution. This example highlights how structuring complex problems can be just as important as making the decisions themselves, because the structure informs how we frame the problem and what we see as worth investigating.

Human in the Loop Models and the Division of Labor

Regulators and lawmakers are increasingly likely to expect to see human in the loop designs for AI systems. In an earlier article, I highlighted the importance of a human-AI division of labor as a design principle for agentic systems. Some cases involve decisions that are too high risk to delegate to an AI, just as one would not delegate a decision to a team member who is not accountable for its outcome. Yet, this division needs to be more careful than a binary split between what a machine does best and what a person does best. Capability-based division-of-labor models are a mainstay because they are simple and intuitive, but they can oversimplify the problem. Dividing decision rights based on accountability and risk gives insurers a clearer view of who contributed what to the outcome.

If we consider a model whereby users make decisions based on information that has been prepared by an AI, a risk-focused division of labor leans toward human involvement in structuring the problem to solve as well as the determination when the decision is high risk. The AI may complement the user’s decision-making by collecting and collating data, informed by the user’s problem structure and specified rules for extraction, while leaving the structuring and analysis to the user. This helps to ensure the decision is owned by the user in full. The user remains accountable, the problem structure is transparent, and the AI adds efficiency and speed.

The Importance of Structure

Structuring a problem sounds like the kind of thing we pay consultants to do, but we all structure perceptual information, even if we don’t realize we’re doing it. Experienced claims adjusters and underwriters do this each time they form a working view of a file. If we didn’t do this, we would drown in the vast array of data around us. What we perceive – our surroundings, what we’re focused on, even the business problems we’re attending to – is always pre-framed. It’s a form of pre-judgment, and it informs how we interpret the data we perceive. Pre-conscious structure limits what we focus on, what we highlight or ignore, and can limit the potential solutions we see as available.

Intentional problem-structuring takes that further. It shapes how data is organized in ways that are intended to best solve a particular problem. (Tree diagrams or system maps are some basic examples; frameworks such as growth matrices or strategy diamonds are more complex ones.) These frameworks should be selected based on the context, available data and the aims of the analysis. They serve to help us see the same data differently, to see hidden relationships, or to surface patterns that weren’t evident, and they can expedite the process of problem-solving and arriving at a decision.

As an example, I once spent months focused on a complex manufacturing industry challenge involving the impact of digital platforms on parts procurement. On the surface, it was a problem of cost and speed—digital sourcing platforms had made procurement more efficient, and small local machine shops were struggling to compete.

The specific drivers of the issues that many small shops were experiencing were unclear to me until I re-framed the industry as a marketplace, which brought to the surface the relationships between machine shops and engineering buyers that had begun to atrophy, and the trust between market participants that had eroded.

A second frame, an analogy with the intensive forestry of 19th-century Germany, provided an explanation. Forests optimized for efficiency thrived for a generation and then declined because industrialized planting had used up long-term nutrients in the soil that no one had accounted for; trust had played the same role in local manufacturing. The analogy highlighted that naturally occurring systems are extremely complex, with many interdependent parts that are hard to isolate, and attempting to optimize such systems can cause them to break down in ways that are hard to predict. Further, it illustrated how top-down attempts to restore these forests had limited effect because they did not take into account how the mechanics of the system actually worked.

This led to my recommendation that rebuilding local manufacturing marketplaces ought to involve restoring the fundamental elements that allowed the system to self-manage. Together, these framings identified patterns that the cost view had hidden, and the analysis was well received by both manufacturing executives and senior military supply chain leaders.

Selecting a structure for data simplifies the process of making decisions. This is true for analytics as much as it is for operational problem-solving. Structure introduces constraints, and these constraints become the basis for inference. Each element of a structure is a choice that has an effect on how the data is represented, and it informs the range of plausible interpretations and decisions we can make. Given that’s the case, permitting an AI to structure problems for a user to review means delegating part of the decision. If the problem is routine, delegating this may be acceptable, but in higher-risk cases such as complex underwriting or claims decisions, expert users ought to decide on the problem-structure as well as making the decision. In the latter case, we can say that they own the decision in full.

Isn’t That Just Prompt Engineering?

One could argue that prompt engineering, through which users articulate the structure of a request, achieves the same thing as structuring a problem as I’ve described it. However, prompt engineering is a language-based set of commands that are open to interpretation. The AI model may introduce assumptions and fill gaps that are unexpected and can be difficult to trace afterward.

Instead, I argue that we ought to explore tools that allow users to lay out the structure of their reasoning, much as they may do on a whiteboard, along with guidance that informs how the AI is to collect and collate the data as content within the defined framework. Structure is more explicit than prose because it leaves less room for interpretation and can be inspected later. This approach puts the user in control, sets clearer requirements for the AI and leaves a record on file for decision audit.

Owning the Structure

The insurance claim I shared as an example had plenty of evidence and capable adjusters assigned to it, and yet it still took years to resolve because the problem had been defined too narrowly. How a problem is structured is a judgment about what’s most important, and in high-risk cases that judgment belongs to the user accountable for the outcome. It also needs to be transparent and auditable.

As AI governance requirements evolve, insurers are likely to be asked what their people are doing “in the loop.” Attending not only to how a final determination is made but to how the problem is structured and who is responsible for structuring it will put insurers in a much stronger position to respond.