Technologies are not neutral, neither are choices in the public procurement of AI. The AI systems we deploy today are the systems we will live with tomorrow. Artificial intelligence is increasingly shaping the opportunities of European citizens and transforming their relationships with governments and public authorities. Today we have the opportunity to responsibly define the way AI will be implemented in the future.
Back to landing page of this white paper
This white paper provides a detailed map for public procurers to choose AI-based services and solutions that put data ethics, democracy and fundamental rights first. It is an initial step towards creating a standardized framework for the questions and considerations public procurement processes should include to adopt trustworthy AI and reward the European development of it.
WHY – the quest for a human-centric approach to AI
In late 2019, Ursula Von der Leyen pledged that within her first 100 days in office, she would propose legislation for a coordinated European approach on the human and ethical implications of AI. This white paper suggests integrating data ethics with the strategic approach to public procurement in the EU, making it supplementary to the green and social components that can be implemented as part of the overall legal framework on public procurement.
The social and ethical implications of the absence of a framework based on European values and norms for the adoption of AI in the public sector are already emerging. Automated systems to socially score families, automated systems with poorly-written code erroneously assigning public positions, black box big data analysis systems shared by different state institutions to track citizens etc.
If Europe is to truly innovate in terms of trustworthy AI systems, public institutions must lead the way. If we do not sufficiently address the trustworthy aspects of AI in public procurement, the cost for European societies, individuals and democracy will be high, with changing society forever and citizen’s rights potentially being negatively impacted by the systems we choose to implement.
Public procurement of AI technology is not just a matter of choosing between more or less efficient technical tools. It is also a prioritization of interests and values embedded in their design. Trustworthy human-centric AI is an alternative type of innovation that can be developed and thrive in Europe, and the public sector could spur that development with data ethics and principles being hard-wired into public procurement.
WHAT – public procurement as leverage for trustworthy AI
This white paper includes ‘data ethics’ as a horizontal theme that cuts across the components of trustworthy AI. Data ethics is the responsible and sustainable use of data. It is about doing the right thing for people and society. Data processes should be designed as sustainable solutions benefitting the interests of individual human beings first and foremost.
Data ethics is about efforts to create transparency and foreseeability in regard to the social and ethical implications of data processing, and it is about real accountability in governance and management structures. Its goal is actively developing privacy-by-design and privacy- enhancing products and infrastructures and stressing the need always to handle someone
else’s personal information in the same way as you wish your own data, or your children’s data, were handled.
WHERE – digitized public sector institutions
Strategic, guided public procurement processes could be implemented in public sectors such as:
▪ Justice and law enforcement
▪ E-government
▪ Government to Business (G2B)
▪ E-democracy
▪ Education
▪ Healthcare treatments and services
▪ Social security services
▪ Employment.
The process should involve all actors throughout the public procurement process from management, subject-matter experts, designers of systems, data scientists and engineers to civil servants, policy officials and governmental representatives.
HOW – a risk-based and systematic approach to public procurement
Trustworthy AI is possible to achieve by establishing an AI procurement framework that includes data ethics components and applying them within the context of existing legal obligations as well as demands for accountability, technical robustness and sustainability.
This white paper suggests a risk-based approach in public procurement that is aligned with both formalized and applied due diligence processes. It recommends a due diligence process consisting of five phases:
1. Preliminary risk assessment
Addressing any adverse impact on human beings or groups of people, their rights and freedoms, on democratic institutions and processes, and on society and the environment.
2. Preliminary screening of potential suppliers
This involves screening the market for potential suppliers that possess the necessary skills, competences and organizational structures to fulfil requirements for data ethics components in AI services and solutions.
Data ethical requirements relating to AI should be considered, defined and implemented from the very beginning of the design process. For example:
When AI systems interact with users directly (e.g. chatbots, virtual assistants) or indirectly (e.g. automated decision-making), they must reveal that they are not human AI systems must be traceable, explainable and include stakeholders
AI systems must avoid bias, be made according to universal design and include procedures for reviews
Technical robustness should be documented, as should explainability, fair communication and audits.
3. Contracting
General exclusion criteria should apply when assessing economic operators who have submitted tenders to provide AI-based services and solutions, including past participation in criminal organizations, corruption, fraud, child labour and human trafficking.
Selection criteria should cover relevant specialist technical competences and diverse, multidisciplinary teams that understand the interdependent disciplines that AI covers. In some circumstances, location within EU/EEA should be prioritized. Also, tenderers must guarantee their sub-suppliers comply with the same data ethics standards.
All the technical specifications for procurement of AI-based services or solutions should include requirements regarding the methodologies and processes foreseen in the development of the AI-based system or solution.
Award criteria Tenders should be assessed according to a set of economic and quality criteria and a best price-to-quality ratio. The quality criteria should reflect the technical specifications regarding applied standards and management systems for information security, data ethics, environmental aspects, privacy, universal design, etc.
4. Contract performance conditions
To reach the overall goal of sustainability and respect for fundamental rights, and the specific goal of data ethics in AI-based services and solutions, the contracting authority should include clauses in the contract performance conditions on these issues, along with possible sanctions and documentation requirements.
5. Contract implementation
A governance structure that includes top management in decision-making processes should support the project structure and identify roles and responsibilities in relation to all phases and levels of the AI project. The supplier should meet the requirements set out in public contracts under five headings: data ethics, legal compliance, accountability, technical robustness and sustainability. To do so, it should set up an organization with insight and overview of the contractual obligations and corresponding work processes.
Further, this white paper recommends (see Recommendations);
- An EU directive on public procurement of AI-based services and solutions for the public sector
- A guiding document on public procurement of AI-based services and solutions for the public sector
- Inclusion of data ethics as a strategic policy priority in the EU Public Procurement Strategy
- A training toolkit on data ethics in procurement of trustworthy AI-based services and solutions
- A handbook on data ethics in procurement of AI-based services and solutions
- An online help desk for public and private sector tenderers.
