The overlooked challenge of reliable AI in engineering and regulatory assessment
By Dr Mansoor Zoveidavianpoor
Artificial intelligence is changing how organisations analyse information, interpret technical documents and support complex decisions. Yet, amid the rapid development of more capable AI models, one fundamental question deserves greater attention: How reliable is the information that reaches the AI in the first place?
AI systems can summarise lengthy reports, identify patterns, answer questions and assist with technical assessments. However, their outputs depend heavily on the quality, structure and completeness of the information they receive. When source information is extracted incorrectly, important context is lost or relationships between data points are misunderstood, even a sophisticated model can produce a convincing but unreliable answer.
This challenge is particularly important in engineering, environmental assessment and regulatory decision-making, where conclusions may depend on a specific numerical value, technical condition, regulatory clause or relationship between several pieces of evidence.
Dr Mansoor Zoveidavianpoor presenting his research. The work reflects a broader question for AI-assisted assessment: how can we ensure that machine-readable evidence faithfully represents the original technical document?
The hidden problem begins with the document
Much of the technical information used by organisations still arrives in PDF format. Regulatory submissions, environmental reports, engineering studies and operational records may contain searchable text, scanned pages, tables, figures, forms, footnotes and complex layouts.
PDF is effective at preserving how a document appears to a human reader. However, its visual presentation does not necessarily expose the meaning and relationships that a machine needs to interpret the content correctly.
Consider an engineering table containing measurements, units and operating conditions. A human reader can often recognise the column headings, follow a row and understand which value belongs to which parameter. During automated extraction, however, a scanned character may be misread, a decimal point may disappear, or a value may become separated from its corresponding heading.
A value of 0.07, for example, could be misread as 7. If that error passes unnoticed into a structured dataset, an AI system may use the incorrect value in an otherwise coherent explanation.
The resulting answer may sound authoritative. The problem is that the evidence supporting it has already been compromised.
This is not necessarily an AI hallucination in the conventional sense. It is an upstream data-quality failure that can lead to an incorrect AI-assisted conclusion.
Why converting PDF to JSON is not enough
Converting documents into structured formats such as JSON is an important step towards making information accessible to software and AI systems. Structured data can make individual fields, values and relationships easier to process.
But successful conversion does not automatically mean successful interpretation.
A JSON file can be syntactically valid while containing incorrect numbers, incomplete text, misplaced table cells or missing context. A clean-looking output may conceal uncertainty introduced during extraction.
The challenge, therefore, is not simply to convert a document into a machine-readable format. It is to establish whether the extracted content accurately represents the original source, identify where uncertainty remains, and preserve enough context to verify the information.
This distinction matters in regulatory assessment. An assessor may need to determine whether a submission satisfies a particular requirement, whether a reported value is consistent with a technical criterion, or whether the evidence supporting a conclusion can be traced to the original document.
In these circumstances, the quality of the evidence pipeline is as important as the capability of the AI model interpreting it.
A missing layer between documents and AI
Many AI workflows follow a straightforward path: upload a document, ask a question and receive an answer. This is convenient, but it can obscure what happens between the source document and the final response.
A more reliable approach introduces explicit controls before the information reaches AI reasoning.
The process begins with extracting content from the original document. The next step reconstructs its structure, including relationships between headings, paragraphs, tables, rows, columns and values. Validation then checks for potential errors, inconsistencies and uncertain content.
Where the evidence is ambiguous or critical information cannot be verified automatically, human review remains essential. Corrections should be recorded transparently, with the original content preserved and the revised content traceable to its source.
Only then can the system establish which information is suitable for downstream AI use and which items require caution or further verification.
The objective is not to eliminate every uncertainty automatically. It is to make uncertainty visible and prevent unresolved extraction problems from being treated as established facts.
AIXtract: focusing on the quality of evidence
This is the problem AIXtract is designed to address.
AIXtract is a document-to-data platform intended to transform technical and regulatory documents into structured, validated and traceable information for downstream analysis. Its approach combines document extraction, structure reconstruction, validation and AI-readiness controls.
Rather than treating every extracted value as equally reliable, the objective is to identify potential issues, retain source context and support human verification where necessary. The resulting structured evidence can then be supplied to AI systems for interpretation, analysis and decision support.
This creates a useful separation of responsibilities: AIXtract focuses on establishing what the source document contains and how reliably that information has been captured; the downstream AI system focuses on interpreting the evidence and answering the question.
The distinction is important. AIXtract does not eliminate hallucinations or guarantee that every AI-generated conclusion will be correct. AI reasoning can fail for many reasons. Improving the quality of its input addresses one important, and potentially preventable, source of error.
Towards more trustworthy AI-assisted assessment
The implications extend beyond document conversion. As organisations increasingly use AI to support technical reviews and regulatory workflows, they need ways to examine not only the final answer but also the evidence on which it depends.
This requires attention to source traceability, extraction quality, uncertainty, human verification and the distinction between documented facts and AI-generated interpretation.
It also creates an opportunity to evaluate AI systems more rigorously. The same question can be tested using an original PDF, unverified extracted data and validated structured evidence. Comparing the responses can reveal differences in factual accuracy, numerical consistency, references and traceability. Such comparisons should be controlled and documented rather than assuming that structured data will always produce a better answer.
Ultimately, reliable AI is not solely a model-development challenge. It is also an information-engineering challenge.
The objective is not to make AI more confident. It is to make the evidence supplied to AI more trustworthy.
As AI becomes more deeply integrated into engineering and regulatory assessment, the quality of the information pipeline deserves the same attention as the intelligence of the model itself. Before asking AI to make sense of complex evidence, we must first ensure that the evidence has been captured, structured and validated as faithfully as possible.
Explore AIXtract
Interested in improving the reliability of AI-assisted document analysis? Visit Carbon Strata to learn more about AIXtract and discuss how its document extraction, validation and AI-readiness capabilities could support your organisation’s technical and regulatory workflows.