Eye on Policy

The more potential user organizations learn about artificial intelligence, the more they realize the importance of traditional, so-called “deterministic” software.

Deterministic simply means that for a given set of inputs, logic will produce the same results with 100% reliability. That is the way users expect software to operate. If, say, someone must submit six pieces of information to receive something such as a card or a campsite reservation, a deterministic program will greenlight all applications that meet the six criteria, or redlight those that meet five or fewer.

Sometimes robotic automation, which is deterministic, is insufficient to locate records or run verification checks. AI, or probabilistic, agents can help in such cases by saving people from the drudgery of, say, document hunting. Yet traditional RPA can orchestrate processes among new and legacy systems and between deterministic and probabilistic applications.

One strong use case demonstrates how deterministic and AI software works in tandem to solve something that continually vexes contractors. In a larger sense, it can also solve an analogous problem the government regularly faces.

To wit: changes in documents. The government regularly issues revisions to requests for information, solicitations, specifications, and contracts. Tracking them all generally produces headaches by itself. But, in an age of increasing compliance demands, knowing which of dozens or hundreds of alterations indicate material changes—that requires even more diligence. If things like prices, scope, quantities, specs, or delivery dates change, and you miss it, you could face real consequences.

I spoke with Fergal McGovern, the founder of Visible Thread, who published a comprehensive description of this problem. I used to interview him from time to time on my radio show when he first applied his software system to assess the readability of government documents. (Spoiler: It is mostly awful and does not get better.)

To those who think dumping the old and new versions of a lengthy document into an AI model prompt will give them usable insight, McGovern presents an argument.

“This feels like the modern answer, and everyone’s doing it,” McGovern said. “But it is the wrong tool, and in a way that is genuinely dangerous.” He told me, “AI gives the delusion of confidence. We believe it anyway.”

Algorithms designed to compare tend to merge documents, he said. That might give you a seemingly clean new copy, but they have not specifically revealed material changes. Probabilistic AI tools will also invent changes or highlight unimportant changes like paragraph renumbering. Turning on a tool like “track changes” lights up the document, but someone or a team must wade tediously through jumbled markups to suss out what matters.

McGovern recommends a two-step technique. First, use a deterministic program to produce side-by-side lists of everything that is different between the original and revised documents. This step applies pure, logical comparison, and it reduces the sheer volume of language an AI model will have to comb through. It does not merge documents but rather isolates the differences.

Only then do you apply an AI model trained on language about material matters—warranties, indemnity clauses, pricing, and the rest of it.

“You need to be certain you’ve found everything material,” McGovern said.

This approach can also work for the government in, for instance, evaluating revised proposals or grant applications. It can help pinpoint changes in program applications with fraudulent intent.

AI and the infrastructure it requires have absorbed trillions in investment capital. AI also drains IT budgets as agencies and companies alike wash up against computing token costs. The two-step, deterministic-then-probabilistic approach can improve results and keep costs in line by restricting AI to specific, delineated uses.

The bigger message might be this: Avoid AI for AI’s sake. Use it as a catalyst for improving processes derived from the paid-for and reliable software you already have.

Sea change or just ‘regular’ reform?

The “revolutionary” rewrite of the Federal Acquisition Regulation is complete. Contracting officers and offerors will spend the next few years figuring out its implications.

Meanwhile, the year-old Warfighting Acquisition System has accompanied the Pentagon’s shift from a program-centric to a portfolio-centric acquisition approach.

And just recently came the DoD’s memo to the defense industrial base. Besides a reinforced emphasis on fixed price contracts, the memo makes remarkable changes in the oversight complex surrounding defense suppliers. For example, it moves the department away from Cost Accounting Standards to GAAP—generally accepted accounting principles (except for large sole-source programs). It speeds up commerciality decisions. It seeks to modernize forward pricing agreements established by the Defense Contract Management Agency and the Defense Contracts Audit Agency.

Retired Air Force Major General David Abba, at a recent TechnoMile conference (where I was the paid emcee), asked rhetorically, “Is this just another round of acquisition reform, or a revolutionary change in how the Department of War buys and deploys military systems?”

Abba, who, among other things, directed the Air Force’s F-35 Integration office, believes it constitutes the latter, or aims to. Two conditions drive the change. One is the sheer slowness and expense of buying and deploying new systems when the pacing threats from China have essentially ended the era of undisputed U.S. dominance and deterrence. This problem has been talked to death, most recently in a sober GAO report from last July.

Second is the seeming inability of the DIB to replenish platforms and ordnance expended on campaigns such as the recent six-week war against Iran.

Abba said that, in essence, the government is acting on the principle that compliance does not equal effectiveness as it takes on the idea that to be compliant, you must be effective.

Sweeping reform has occurred before, at least as a statement of purpose. Effecting change, Abba said, usually happens when someone exerts sheer force of personality or after a catastrophic failure.

Me speaking: Defense Secretary Pete Hegseth is no Robert S. McNamara, Caspar Wienberger or Donald Rumsfeld. The country has avoided military catastrophe; Iran has not managed to sink a carrier. But the flurry of memos and executive orders concerning military acquisition will potentially result in a department that operates effectively in the digital age.