AI is quickly being rolled out across the property/casualty insurance industry in various forms. Insurers are now actively embedding AI into underwriting, claims, customer service, fraud detection, software development and operational workflows. Large language models are summarizing claim files, generating code, assisting adjusters, drafting customer communications and helping underwriters analyze increasingly complex risks. Agentic AI promises to take this evolution a step further by coordinating work across multiple applications, data sources and business processes.

The excitement surrounding these capabilities is well deserved. AI has the potential to help insurers make better decisions, improve operational efficiency, accelerate software development and deliver better customer experiences.

Yet as insurers move into enterprise-scale deployments, a different challenge is beginning to emerge. The greatest obstacle is no longer determining whether AI can perform a task. Increasingly, the question is whether insurers can trust AI to perform that task consistently, responsibly and within the governance standards required in a highly regulated industry.

That shift fundamentally changes the role of quality engineering in insurance.

For decades, quality assurance focused primarily on one objective: verifying that software behaved exactly as expected. Applications were built around deterministic logic. Given the same inputs, software should always produce the same outputs. A premium calculation either matched the expected result or it did not. A policy endorsement either followed established business rules or it failed. Testing was largely about confirming correctness against predefined requirements.

AI operates differently.

Unlike traditional software, generative AI and many machine learning models produce probabilistic rather than deterministic results. Two identical prompts may generate slightly different responses. Recommendations may evolve as models improve, new data becomes available or contextual information changes. In many cases, multiple answers may be acceptable rather than a single correct response.

This variability is not necessarily a flaw. It is a characteristic of intelligent systems.

But it also means that many of the assumptions that have guided software testing for decades no longer apply.

Traditional pass-or-fail testing remains essential for core insurance systems, but it is no longer sufficient for evaluating AI-driven applications. Organizations must now determine whether outputs are reliable enough to support business decisions, whether recommendations remain consistent within acceptable boundaries, and whether the reasoning behind those recommendations can be understood and validated.

In other words, the focus expands from proving that software is always correct to establishing confidence that intelligent systems behave appropriately within clearly defined boundaries.

That distinction is particularly important in insurance. Core business functions such as premium calculations, policy rules, financial transactions and regulatory compliance remain deterministic by design. These systems must continue to produce consistent, verifiable and auditable results. There is little room for ambiguity when calculating a premium, applying filed rating rules or issuing a policy that represents a legally binding contract.

AI introduces a different class of capabilities. Tasks such as summarizing claim files, identifying potential fraud indicators, recommending next investigative steps, prioritizing underwriting information or assisting customer service representatives often involve judgment rather than a single correct answer. Multiple responses may be acceptable, provided they are accurate, explainable and consistent with business objectives.

Quality engineering therefore takes on a broader responsibility. It must ensure that deterministic systems continue to deliver provable correctness while also establishing confidence that AI-driven capabilities operate in ways that are trustworthy, explainable, observable and appropriately governed. Trust is not a replacement for proof; it is built upon it. The challenge for insurers is designing intelligent systems that combine the certainty required for core insurance operations with the flexibility that makes AI valuable in the first place.

Insurance has always been built on trust.

Policyholders trust insurers to honor contracts. Regulators trust carriers to make fair and compliant decisions. Executives trust the information used to manage risk and allocate capital. Every underwriting decision, claim settlement, pricing calculation and regulatory filing depends on confidence in the systems producing those outcomes.

As AI becomes part of those decisions, that same level of trust must extend to intelligent systems. Building that trust requires looking beyond the AI model itself and considering the broader enterprise environment in which AI operates.

One of the biggest misconceptions surrounding enterprise AI is that success depends primarily on selecting the best model. In reality, AI is only one component of a much larger technology ecosystem. For insurers, the quality of AI-generated recommendations depends just as much on the integrity of core systems, enterprise data, business rules and governance as it does on the underlying model itself.

AI rarely operates in isolation. An underwriting assistant may draw information from policy administration systems, pricing engines, geospatial data providers, document repositories, third-party data services, internal knowledge bases and regulatory rules. Claims assistants may rely on policy data, historical claims, payment systems, fraud models, repair estimates and external partners. The quality of AI-generated recommendations depends on every part of that ecosystem.

An advanced language model connected to incomplete, inconsistent or poorly governed data will simply generate unreliable answers with greater speed and confidence. AI does not eliminate weaknesses in data quality or governance magnifies them.

Conversely, when AI is grounded in trusted enterprise systems, governed business rules and high-quality data, it becomes significantly more valuable because it operates within an environment designed for accuracy, accountability, and consistency.

Data governance provides the framework that makes trustworthy AI possible. It establishes clear ownership of data; defines quality standards; ensures information remains consistent across enterprise systems; and supports the transparency, auditability and regulatory compliance required throughout the insurance life cycle. Rather than treating governance as a constraint on innovation, insurers are increasingly recognizing it as the foundation that allows AI to operate safely and reliably at enterprise scale.

This is one of the fundamental differences between enterprise AI and public AI tools. Consumer AI applications are often designed to answer questions or generate content in isolation. Enterprise AI, by contrast, operates within highly governed environments where every recommendation may influence financial decisions, regulatory compliance, customer outcomes or legally binding insurance contracts.

Quality engineering therefore becomes much broader than validating model outputs. It becomes the discipline responsible for validating the behavior of the entire intelligent system. That includes understanding where data originates, how information flows between applications, how AI interacts with business rules, how recommendations are monitored over time, and how organizations detect unexpected behavior before it affects customers.

Observability becomes as important as testing. Monitoring becomes as important as validation. Governance becomes as important as automation. These capabilities have always mattered, but AI raises the stakes because intelligent systems continue evolving, interacting with increasingly complex environments and influencing higher-value business decisions.

This evolution also changes where quality engineering delivers its greatest value.

Historically, quality assurance often focused on identifying defects after software had already been developed. While testing remains essential, trustworthy AI depends just as much on decisions made much earlier in the development life cycle. Clear business objectives, high-quality enterprise data, thoughtful system architecture, well-defined business rules, carefully designed AI interactions and strong governance all contribute more to trustworthy outcomes than simply adding additional testing at the end of development.

Quality, in other words, moves upstream.

The role of quality engineers evolves as well. AI can now generate test cases, review source code, create documentation, identify defects and automate many routine engineering activities. Rather than replacing experienced engineers, these capabilities elevate their responsibilities. Increasingly, quality engineers become architects of confidence. They establish evaluation strategies, define acceptable performance thresholds, validate data quality, monitor production behavior, investigate unexpected model responses and help ensure intelligent systems continue aligning with business objectives long after deployment.

This reflects a broader shift taking place across enterprise technology. AI will not replace the core insurance systems that carriers have spent decades building. Policy administration systems, claims platforms, billing systems and financial applications remain the authoritative systems of record responsible for executing deterministic business processes and maintaining regulatory compliance. AI delivers its greatest value when it extends to those trusted systems, not when it attempts to replace them.

The future of property/casualty insurance is therefore unlikely to be defined by standalone AI assistants making isolated decisions. Instead, it will be built around intelligent capabilities embedded within well-governed enterprise platforms, connected to trusted data, guided by established business rules and continuously monitored through disciplined quality engineering.

Ultimately, the defining question for enterprise AI is no longer simply, “Can it work?” It is, “Can we trust it?”

For insurers, that trust will not come from AI models alone. It will come from the data governance embedded across the organization, quality engineering practices, governance frameworks, enterprise architecture and disciplined software engineering that ensure intelligent systems operate reliably, transparently and responsibly.

In the AI era, trust is built on three foundations: quality engineering, trusted core systems and high-quality data governance models. Together, they enable insurers to rely on intelligent systems for some of their most important business decisions.