The New Reality of AI Procurement and Risk
For enterprise technology leaders, procurement has shifted from assessing service level agreements to auditing algorithmic integrity. With the formal adoption of the EU AI Act, the liability landscape for third-party software has fundamentally changed. Organizations can no longer treat AI as a standard SaaS purchase where the vendor carries all the risk. Instead, companies must perform rigorous AI vendor due diligence to ensure they do not inherit legal liabilities that could result in significant fines or forced system shutdowns.
The EU AI Act classifies AI systems based on risk levels, placing heavy burdens on ‘providers’ and ‘deployers’ of high-risk systems. Even if your organization is merely the user of a third-party tool, Article 26 mandates that deployers take specific measures to ensure the system is used within its intended parameters. Failing to verify a vendor’s compliance posture before integration is a strategic oversight that modern CTOs cannot afford.
Categorizing AI Systems Under Article 6
The first step in any due diligence process is determining the classification of the AI system according to Article 6 and Annex III of the EU AI Act. High-risk systems include those used in critical infrastructure, recruitment, credit scoring, and law enforcement. For e-commerce retailers, this often extends to AI used for pricing strategies or behavioral manipulation that could fall under prohibited practices defined in Article 5.
During the discovery phase, your team should document whether the vendor’s tool qualifies as high-risk. If it does, the vendor must meet stringent requirements regarding risk management, data governance, and technical documentation. If a vendor cannot provide a clear classification of their tool, it indicates a lack of regulatory maturity. To evaluate your current position, consider taking our AI Readiness Test to identify potential compliance gaps in your existing stack.
Technical Documentation and Article 11 Requirements
Transparency is a cornerstone of the EU AI Act. Article 11 requires that technical documentation for high-risk AI systems be drawn up before the system is placed on the market. This documentation must be detailed enough to demonstrate compliance with the regulation to national competent authorities. When conducting due diligence, you must demand more than a marketing whitepaper.
A compliant vendor should provide evidence of their quality management system and a detailed description of the AI system’s architecture, computational resources, and performance metrics. This is particularly critical when integrating AI into legacy IT environments where interoperability and data flow must be tightly controlled. Enterprises should request a ‘Bill of Materials’ for the AI model, including the base models used, any fine-tuning datasets, and the specific libraries employed in the inference pipeline.
Data Governance and Training Set Integrity
Article 10 of the AI Act outlines strict requirements for the data used to train, validate, and test high-risk AI systems. Vendors must demonstrate that their training sets are relevant, representative, and, to the best extent possible, free of errors and complete. In the context of AI vendor due diligence, this means auditing the vendor’s data acquisition process. You need to know if the data was lawfully sourced and if it reflects the diversity of the population the AI will affect.
For retailers using predictive analytics, biased training data can lead to discriminatory outcomes that violate not only the AI Act but also GDPR and local consumer protection laws. Practitioners should ask vendors for their bias detection and mitigation protocols. If a vendor claims their model is a ‘black box’ that cannot be audited for bias, they are effectively asking you to accept an unquantifiable legal risk.
Human Oversight and Article 14 Compliance
The EU AI Act rejects the notion of fully autonomous decision-making for high-risk applications without human intervention. Article 14 requires that high-risk AI systems be designed in a way that allows natural persons to oversee their functioning. This ‘human-in-the-loop’ or ‘human-on-the-loop’ requirement must be built into the interface of the tool you are purchasing.
Assess whether the vendor’s software provides a clear dashboard for human intervention. Can an operator override the AI’s output? Is there a mechanism to halt the system in the event of an anomaly? These features are not just UX preferences; they are legal requirements for compliance. You can explore how these principles are applied in practice through our AI Solutions, which prioritize audit-grade governance.
Post-Market Monitoring and Incident Reporting
Compliance does not end at the point of purchase. Article 61 mandates that providers of high-risk AI systems establish a post-market monitoring system to evaluate the performance of the AI throughout its lifecycle. As a deployer, you are part of this ecosystem. Your due diligence should include a review of the vendor’s incident response plan.
- How does the vendor notify customers of model drift or performance degradation?
- What is the process for reporting ‘serious incidents’ as defined in Article 3(44)?
- Does the vendor provide regular security patches and model updates to maintain compliance?
If a vendor lacks a structured post-market monitoring framework, your organization may be left holding the responsibility for system failures that occur months or years after deployment.
The Risks of General Purpose AI Models
While much of the AI Act focuses on high-risk systems, General Purpose AI (GPAI) models, such as those used in many generative AI applications, have their own set of rules under Title VIIIb. Vendors providing GPAI models must provide technical documentation and instructions for use to downstream providers like your organization. This is essential for ensuring that you can integrate these models into your own products while remaining compliant.
When vetting a GPAI provider, verify their adherence to the Regulation (EU) 2024/1689 (AI Act). Specifically, look for transparency regarding the training data and their commitment to copyright law compliance. This is a critical area for enterprises building custom agents or internal knowledge bases using Azure OpenAI or similar platforms.
Creating an AI Vendor Due Diligence Framework
To institutionalize these checks, enterprises should move beyond ad-hoc spreadsheets and develop a formal AI Governance Framework. This framework should involve legal, IT, and business stakeholders to evaluate every AI purchase through the lens of the EU AI Act. The objective is to transition from a ‘buyer beware’ mentality to a ‘buyer verified’ standard.
The checklist should prioritize evidence over assertions. If a vendor claims their system is compliant, they must provide the audit reports or certifications to prove it. In the absence of third-party certifications, which are still evolving, internal audits of the vendor’s technical documentation are the only viable path forward. This proactive approach ensures that your transition to an AI-native organization is built on a foundation of trust and legal certainty.
Conclusion: Secure Your AI Strategy
Buying AI without inheriting risk requires a shift in how we approach vendor relationships. By focusing on the specific requirements of the EU AI Act—from Article 6 classification to Article 14 human oversight—you can ensure your organization remains competitive without compromising on compliance. The complexity of these regulations means that standard procurement processes are no longer sufficient.
At CONAIS, we specialize in helping enterprises navigate these transitions with audit-grade governance and deep technical expertise. If you are ready to refine your procurement process or need assistance auditing your current AI portfolio, please Contact our advisory team to discuss your specific requirements.
Frequently asked questions
What is the most critical part of AI vendor due diligence under the EU AI Act?
The most critical step is identifying the system’s risk classification under Article 6, as high-risk systems require extensive technical documentation, data governance, and human oversight mechanisms.
Does the EU AI Act apply if my AI vendor is based outside the EU?
Yes, the AI Act applies to any AI system that is placed on the market or put into service within the EU, regardless of where the provider is located.
What documentation should I ask an AI vendor for?
You should request technical documentation required by Article 11, evidence of a Quality Management System, and details on data training sets as specified in Article 10.
