AI Transparency and Governance: New Challenges for Reliable, Controllable Systems
Recent scientific research highlights significant gaps in the governance and transparency of artificial intelligence systems, raising crucial questions about their reliability and human control. These studies underscore how, despite technological advancements, the inherent complexity of AI demands increasing attention to oversight mechanisms and the principles of ethical AI.
What happened
A body of recent studies, published on ArXiv, has analyzed various facets of current challenges in AI. One such study, titled "Behavioral Consistency and Transparency Analysis on Large Language Model API Gateways", revealed a concerning lack of transparency in third-party Large Language Model (LLM) API gateways. These gateways, which serve as unified access points to models from multiple vendors, often do not disclose their internal routing, caching, and billing policies. This leaves users with limited visibility into which model is actually processing requests, whether responses remain faithful to the original APIs, and if billing is accurate. The research introduces GateScope, a framework for measuring these discrepancies.
Another study, "Structural Quality Gaps in Practitioner AI Governance Prompts", examined the quality of natural language prompts used to govern the behavior of AI agents. These prompts act as executable specifications, defining the agent's mandate and quality criteria. However, an analysis of a corpus of 34 publicly available governance prompts revealed significant structural gaps, indicating that many are incomplete or ambiguous, making it difficult to ensure AI operates as intended.
In the field of fraud detection, the research "TRAVELFRAUDBENCH: A Configurable Evaluation Framework for GNN Fraud Ring Detection in Travel Networks" introduced a new benchmark for evaluating Graph Neural Networks (GNNs). This study highlights how existing benchmarks are often insufficient for detecting complex, travel-specific fraud patterns, such as ticketing fraud or "ghost hotel" schemes, emphasizing the need for more robust and configurable evaluation tools to address evolving threats.
Finally, another publication, "Serialisation Strategy Matters: How FHIR Data Format Affects LLM Medication Reconciliation", explored the impact of data serialization on LLM-assisted medication reconciliation in clinical settings. The research demonstrated that the way structured health data (in FHIR format) is presented to the model can significantly affect its accuracy, with some serialization strategies leading to better outcomes than others. This underscores the importance of technical details in designing AI systems for critical applications like healthcare. These studies, while covering diverse areas, converge on a key point: the need for greater clarity, control, and reliability in AI implementation.
Why it matters
These findings are of paramount importance for society, the world of work, and trust in AI. The lack of transparency in LLM gateways, for instance, can lead to business decisions based on incorrect information or unexpected costs, undermining confidence in AI supply chains. For workers, reliance on AI systems with ambiguous governance prompts can create uncertainty about roles and responsibilities, as well as potential risks of unethical automation or unfair algorithmic decisions.
In critical sectors like healthcare, where LLMs are proposed for high-stakes tasks such as medication reconciliation, accuracy and reliability are non-negotiable. Errors due to improper data serialization could have direct consequences for patient health. Similarly, the inability of AI systems to effectively detect fraud can cause significant financial losses for businesses and consumers, highlighting how AI robustness is an economic and social requirement. The impact also extends to overall governance: if we cannot be certain of an AI system's behavior or its adherence to ethical principles, its widespread adoption risks generating more problems than solutions, eroding public trust and hindering responsible progress.
The HDAI perspective
The challenges highlighted by this research reinforce the philosophy of Human Driven AI, which places humans at the center of artificial intelligence development and governance. These are not merely technical problems, but deep issues of responsibility, transparency, and control. To ensure that AI serves humanity, it is imperative that companies and researchers adopt approaches that prioritize operational clarity and verifiability. The creation of robust evaluation frameworks, such as those proposed in the studies, is a step in the right direction, but it must be accompanied by a corporate culture that values documentation, auditability, and the disclosure of internal policies.
Integrating ethical AI principles from the design phase is crucial. This includes defining clear and unambiguous governance prompts, ensuring transparency in API gateway operations, and developing fraud detection systems that are fair and adaptable. It is essential that technological innovation is always balanced by solid governance and clear human accountability. Topics like these will be central to the HDAI Summit 2026, where experts from around the world will gather to discuss how to shape a future where AI is powerful, yet also reliable and controllable.
What to watch
The evolution of regulations, such as the EU AI Act, will play a key role in defining transparency and accountability standards for AI systems. It will be interesting to observe how companies adapt to these new requirements, developing tools and methodologies to make their systems more verifiable and reliable. Research into evaluation frameworks and best practices for prompt governance will continue to be a dynamic area, aiming to bridge the identified structural gaps. We also anticipate progress in the development of open standards and protocols for API gateways, which can ensure greater visibility and control for end-users. The focus will increasingly shift towards the practical implementation of these principles, transforming ethical guidelines into concrete operational requirements.