Sittings · Compare

What changed

From · opinion parliamentary committee draft · 2021-12-10 ENVI-PA-699056 on the proposal for a regulation of the European Parliament and of the Council laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union Legislative Acts

→

To · opinion parliamentary committee draft · 2022-03-29 TRAN-PA-730085 on the proposal for a regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending certain union legislative acts

+283 added · −160 removed · 41 modified paragraphs hide unchanged text

SHORT JUSTIFICATION

The proposal for Regulation of the European Parliament and the Council laying down harmonized rules on Artificial Intelligence (hereinafter “AI Act”) is part of the broader agenda to boost Europe in the digital age and achieve its environmental and climate objectives . This stems from the fact that AI currently plays a role in all aspects of European daily life activities.

The Rapporteur supports the development of an EU legislative framework on Artificial Intelligence, a necessary step to ensure the Union becomes leader in the field, all while upholding our European values. The horizontal risk-based approach of the Artificial Intelligence Act will create a system in which rules will apply exclusively to applications of AI systems where threats may occur. This selective approach will allow the Union to continue developing its AI sector without hindering technological progress and digital transformation.

AI systems will become more and more embedded into products and services therefore requiring a horizontal legislative approach as set out in the AI Act. The Rapporteur is fully aligned with this as she considers that we must establish the common rules to provide a cross-cutting approach to all sectors, including the healthcare sector. By doing so, the European Union has a chance to lead and set the standards of AI worldwide, as it has already done with data protection through GDPR. The EU could also become a global leader in niche sectors that require a very forward-looking perspective such as the regulation of neurological rights.

Transport is a sector posed to benefit greatly from the integration of AI systems in its every day operations and logistics. The application of AI systems can help achieve the EU’s safety, environmental and, in some instances, societal objectives for the sector. The Union must therefore aim at removing those existing barriers that hinder development and investment, first among all, legal fragmentation and uncertainty, which hurts business and customers.

Overall the AI act should preserve European values, facilitating the distribution of AI’s benefits across society, protecting individuals, companies and the environment from risks while boosting innovation and employment and making Europe a leader in the field.

The key elements of the Rapporteur’s report are the following:

In this regard, the Rapporteur wants to emphasize the importance of sandboxes in certain areas (e.g. Health) and how it could be extended to other areas such as Hospitals, Health Authorities and research centers in order to reinforce and expand the leading position of the health system in all the Member States and at EU level. Health is wealth. By applying AI in health using interoperable health data we could further increase this wealth from health systems to society at large. The Rapporteur also highlights the potential implications of AI systems in mental health.

 Ensuring the AI Act does not overlap with sectoral legislation by imposing double/conflicting obligations on transport actors;

~~The Rapporteur for the opinion deems that the proposal insufficiently anticipates the risks of not having a common and consistent regulatory approach.~~

 Promoting the development of, and upholding, international standards, that are particularly important for the transport sector;

As a horizontal legislative initiative, the proposed AI Act is expected to intersect with several regulations currently in place (e.g GDPR or MDR) and several legislative initiatives that might intersect in the future such as the European Health Data Space. All these initiatives should be aligned with the AI Act to ensure a common and consistent regulatory approach therefore avoiding duplication of functions or discoordination among bodies and authorities at both the EU and Member State level.

 Fostering Research and Innovation to ensure the EU’s transport sector develops its own know-how in the implementation of AI, while upholding the highest ethical standards.

~~The Rapporteur for the opinion is concerned that the AI Act does not provide sufficient protection to the environment.~~

The general approach to the opinion by the Rapporteur has been to give a voice to the transport sector on the AI Act. For this reason, amendments had to be carefully drafted within the scope of the TRAN Committee, a complicated task when dealing with harmonized legislation. Still, the Rapporteur believes that in instances where horizontal provisions clearly affect transport, the TRAN Committee should have a say. In addition, some amendments have been included to clarify the key messages of the Commission’s report, such as in the case of the definition of AI systems and High-Risk, which are both essential for transport.

The Special Eurobarometer 513 Climate change published in 2021 shows that tackling climate and environmental-related challenges is one of the main concerns for European citizens. Therefore, the Rapporteur proposes that the AI Act shall include the environment among the areas that require a high level of protection. In order to do so, the environment has been included in all the recitals and articles together with health, safety and the protection of fundamental rights. This will entail the classification as “high risk AI” of all those systems that can have major negative implications on the environment. At the same time, the Rapporteur has reinforced the right to proper redress mechanisms in case of negative environmental impacts as set out in the Aarhus Convention, and has set the principle of “Do no significant harm” as established in the Taxonomy Regulation as a limit to ensure that AI systems abide with the EU´s high level of environmental standards and rights.

Following the consultation and research phase for this work, three clear issues emerged from a TRAN perspective. First, each segment of transport already presents, in different degrees and forms, sectoral legislation, provisions or initiatives to ensure the highest level of safety when it comes to AI system integration. Aviation, road, rail and maritime, all require measures specifically tailored to the sector to ensure the successful management of operations and services, while upholding the highest level of security. Harmonized legislation might fall short of the required measures to guarantee the aforementioned safety in the sector. For this reason, it was paramount to stress in the AI Act the need for sectoral legislation to be respected and prevail, in some instances, over harmonized rules. This was indeed the intention of the Commission as clear from Annex II, Part B. Yet, the text required further detail.

~~The Rapporteur for the opinion considers that the AI Act shall not just cover users but must expand its scope to end recipients too.~~

Secondly, due to the international nature of transport, a second key element that emerged from assessment is that international procedures are already in motion to develop global norms and guidelines for the safety of AI systems for each sector. It would be preferable therefore, that such global standards are respected and integrated in EU law and standards. Generally speaking, EU standards are developed through formal agreements between the European Standardization Organizations (ESOs) and International Standardization Organizations, which allows them in practice to work together. Nonetheless, in the interest of preserving the competitiveness of the EU’s transport sector vis-a-vis other regions, the language and provisions in the AI Act had to be strengthened to ensure provisions on AI for transport are respectful of the international norms and standards.

Many of the applications mentioned in the proposed AI Act will involve not just users but end recipients. In the case of healthcare applications this distinction is crucial as there is a clear differentiation between the intended use and capabilities of patients and doctors. Therefore, the draft report now includes a new definition of end recipients and grants them the appropriate degree of transparency and provision of specific information.

Lastly, Research and Development is the motor for the digitalization of each sector, let alone transport. New harmonized rules on AI must not impede research where such research is limited to controlled environments and the studied AI System is not placed onto the market. For this reason, language is added to clarify the scope in this regard. It must be noted, that the Commission proposal presents measures in support of innovation through specific articles on sandboxes and small-scale providers. Here, however, the Rapporteur amends to ensure Small and Medium Enterprises (SMEs) are able to benefit from such provisions.

Further minor amendments by the Rapporteur cover transparency of algorithms for transport work (simply reflecting Commission’s language in Annex III), an understanding of Human Oversight in the context of transport, and clarification on errors in data sets.

AMENDMENTS

The Committee on ~~the Environment, Public Health~~Transport and ~~Food Safety~~Tourism calls on the Committee on the Internal Market and Consumer ~~Protection,~~Protection and the Committee on Civil Liberties, Justice and Home Affairs, as the ~~committee~~committees responsible, to take into account the following amendments:

Amendment 1

Proposal for a regulation

~~Recital 1~~

Recital 5 a (new)

Text proposed by the Commission

Amendment

(1) The purpose of this Regulation is to improve the functioning of the internal market by laying down a uniform legal framework in particular for the development, marketing and use of artificial intelligence in conformity with Union values. This Regulation pursues a number of overriding reasons of public interest, such as a high level of protection of health, safety and fundamental rights, and it ensures the free movement of AI-based goods and services cross-border, thus preventing Member States from imposing restrictions on the development, marketing and use of AI systems, unless explicitly authorised by this Regulation.

(5a) Union legislation on artificial intelligence should be accompanied by actions aimed at addressing the main barriers hindering the digital transformation of the economy. Such measures should focus on education, upskilling and reskilling of workers, fostering investment in research and innovation, and boosting security in the digital sphere in line with initiatives aimed at achieving the targets of the Digital Decade. Digital transformation should occur in a harmonized manner across regions, paying particular attention to less digitally developed areas of the Union.

(1) The purpose of this Regulation is to improve the functioning of the internal market by laying down a uniform legal framework in particular for the development, marketing and use of artificial intelligence in conformity with Union values. This Regulation pursues a number of overriding reasons of public interest, such as a high level of protection of health, safety, the environment and fundamental rights, and it ensures the free movement of AI-based goods and services cross-border, thus preventing Member States from imposing restrictions on the development, marketing and use of AI systems, unless explicitly authorised by this Regulation.

Or. en

Proposal for a regulation

~~Recital 1 a (new)~~

Recital 6

Text proposed by the Commission

Amendment

~~(1a) This Regulation should serve as a basis to promote health, wellbeing, prevent diseases, and foster supportive environments for healthy lifestyles.~~

(6) The notion of AI system should be clearly defined to ensure legal certainty, while providing the flexibility to accommodate future technological developments. The definition should be based on the key functional characteristics of the software, in particular the ability, for a given set of human-defined objectives, to generate outputs such as content, predictions, recommendations, or decisions which influence the environment with which the system interacts, be it in a physical or digital dimension. AI systems can be designed to operate with varying levels of autonomy and be used on a stand-alone basis or as a component of a product, irrespective of whether the system is physically integrated into the product (embedded) or serve the functionality of the product without being integrated therein (non-embedded). The definition of AI system should be complemented by a list of specific techniques and approaches used for its development, which should be kept up-to–date in the light of market and technological developments through the adoption of delegated acts by the Commission to amend that list.

(6) The notion of AI system should be clearly defined to ensure legal certainty, while providing the flexibility to accommodate future technological developments. The definition should be based on the key functional characteristics of the software, and possibly also the hardware.1a. In particular, for the purpose of this Regulation, AI systems should be intended as having the ability, on the basis of machine- and/or human-based data and inputs, to infer the way to achieve a given set of human-defined objectives through learning, reasoning or modelling and generate specific outputs in the form of content for generative AI systems, as well as predictions, recommendations, or decisions which influence the environment with which the system interacts, be it in a physical or digital dimension. AI systems can be designed to operate with varying levels of autonomy and be used on a stand-alone basis or as a component of a product, irrespective of whether the system is physically integrated into the product (embedded) or serve the functionality of the product without being integrated therein (non-embedded). The definition of AI system should be complemented by a list of specific techniques and approaches used for its development, which should be kept up-to–date in the light of market and technological developments through the adoption of delegated acts by the Commission to amend that list.

__________________

1a https://digital-strategy.ec.europa.eu/en/library/definition-artificial-intelligence-main-capabilities-and-scientific-disciplines

Or. en

Proposal for a regulation

Recital 18 ba (new)

Text proposed by the Commission

Amendment

(1b) This Regulation should preserve the European values facilitating the distribution of AI benefits across society, protecting individuals, companies and the environment from risks while boosting innovation and employment and making Europe a leader in the field.

(8a) The use of biometrics in transport and tourism will vastly benefit user experience and overall safety and security. The application of fingerprint or retina scans to access cars could help prevent theft, while in-car biometrics could help detect drivers' stress levels and prevent intoxicated driving, directly contributing to the EU’s 2050 “VisionZero”. In the tourism sector, contactless check-ins, for example through facial recognition technology, will help attain a seamless travelling experience. This Regulation should accompany these developments by the highest/adequate level of protection, in particular when use of biometrics data is involved, in line with the data protection framework of the Union, while fostering research and investment for the development and deployment of AI systems that can positively contribute to society.

Or. en

Proposal for a regulation

~~Recital 3~~

Recital 11 a (new)

Text proposed by the Commission

Amendment

(3) Artificial intelligence is a fast evolving family of technologies that can contribute to a wide array of economic and societal benefits across the entire spectrum of industries and social activities. By improving prediction, optimising operations and resource allocation, and personalising digital solutions available for individuals and organisations, the use of artificial intelligence can provide key competitive advantages to companies and support socially and environmentally beneficial outcomes, for example in healthcare, farming, education and training, infrastructure management, energy, transport and logistics, public services, security, justice, resource and energy efficiency, and climate change mitigation and adaptation.

(11a) Due to risks associated with passenger and cargo transport, the sector has been carefully monitored and regulated to avoid incidents and loss of life. The Union legal framework for transport presents sectoral legislation for the aviation, road, rail and maritime transport. With the progressive integration of AI systems in the sector, new challenges could emerge in risk management. The provisions of this Regulation should only apply to high risk applications in the transport sector in so far as that they are not already covered by sectoral legislation and where they could have a harmful impact on the health, safety and fundamental rights of persons. Overlaps should therefore be avoided.

(3) Artificial intelligence is a fast evolving family of technologies that can contribute to a wide array of economic and societal benefits across the entire spectrum of industries and social activities. By improving prediction, optimising operations and resource allocation, and personalising digital solutions available for individuals and organisations, the use of artificial intelligence can provide key competitive advantages to companies and support socially and environmentally beneficial outcomes, for example in access and provision of healthcare, including mental health, protection of biodiversity, farming, education and training, infrastructure management, crisis management, energy, sustainable transport and logistics, public services, security, justice, resource and energy efficiency, and climate change mitigation and adaptation.

Or. en

Justification

The transport sector is a heavily regulated sector that demands tailored specific rules in order to correctly address its problems and requirements. The Artificial Intelligence Act must not overlap with sectoral legislation in place and avoid double obligations. When it comes to transport, sectoral legislation must prevail over harmonized legislation.

Amendment 5

Proposal for a regulation

Recital 311 ab (new)

Text proposed by the Commission

Amendment

(3a) According to the definition of the World Health Organisation (WHO), "Health is a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity." In order to improve the health of the population in the Union and reduce health inequalities, it is essential not to focus only on physical health. Digital technologies and especially Artificial Intelligence can have a direct negative impact on mental health. At the same time, we must unleash the full potential of AI in the development of prediction, detection, and treatment solutions for mental health.

(11b) Notes for example that the Union aviation sector, through the work of the European Aviation Safety Agency (EASA) and its stakeholders, is gradually developing its own guidance material and rules on the application and security management of AI systems in aviation. Takes note of EASA’s roadmap for AI, in which AI system application to aviation are categorized in three distinct levels, from assistance to human, to human-machine cooperation, to full machine automation. Stresses the need for a sector-specific oversight on AI systems laying out rules for the highest-level of safety for aviation while preserving the global competitiveness of Union businesses.

Or. en

Proposal for a regulation

Recital 312 ba (new)

Text proposed by the Commission

Amendment

~~(3b) The right to physical and mental health is a fundamental human right and universal health coverage is a Sustainable Development Goal that all signatories have committed to achieve by 2030.~~

(12a) This Regulation should support research and innovation for the application of AI systems in the transport and tourism sectors. For this reason, this Regulation should exclude from its scope applications of AI systems developed, applied and assessed in a controlled testing environment, for the sole purpose of evaluating their use and functionality. As regards product oriented research activity by providers, the provisions of this Regulation should apply insofar as such research leads to or entails placing an AI system on the market or putting it into service. All forms of research and development should be conducted in compliance with the highest ethical standards for scientific research.

Or. en

Justification

The integration of AI systems in transport and tourism will greatly benefit the sectors. To ensure we reap the benefit of innovation and digital transformation, the Artificial Intelligence Act should not apply to AI systems and their outputs used for the sole purpose of research and development, unless such systems are placed on the market or put into service.

Amendment 7

Proposal for a regulation

~~Recital 3 c (new)~~

Recital 13

Text proposed by the Commission

Amendment

~~(3c) The EU commits to progressing towards the recognition of the right to a clean, healthy and sustainable environment, as laid out in Resolution 48/13 of the U.N. Human Rights Council.~~

(13) In order to ensure a consistent and high level of protection of public interests as regards health, safety and fundamental rights, common normative standards for all high-risk AI systems should be established. Those standards should be consistent with the Charter of fundamental rights of the European Union (the Charter) and should be non-discriminatory and in line with the Union’s international trade commitments. This is of particular importance in the transport sector in order to ensure the highest level of interoperability among transport vehicles, infrastructure and intelligent systems and to guarantee safety and security. The Union and its standards organisations, in full respect of the Charter, should therefore strive to develop global standards for the different transport modes. Regular reviews of this Regulation should take into account updated standards for the transport sector.

Or. en

Proposal for a regulation

~~Recital 4~~

Recital 17 a (new)

Text proposed by the Commission

Amendment

(4) At the same time, depending on the circumstances regarding its specific application and use, artificial intelligence may generate risks and cause harm to public interests and rights that are protected by Union law. Such harm might be material or immaterial.

(17a) The use of AI in work can be beneficial to both the management and operations of an enterprise, supporting workers in their tasks and improving safety at the workplace. Still, AI systems applied to digital labour platforms, platforms for the management of workers, including in the field of transport, can entail risks of unjust/unnecessary social scoring, rooted in biased data sets, which can lead to violation of workers and fundamental rights. This Regulation should therefore aim at protecting the rights of transport workers managed by digital labour platforms and promote transparency, fairness and accountability in algorithmic management, to ensure that workers are aware of how algorithms work, which personal data is issued and how their behaviour affects decisions taken by the automated system.

Or. en

Justification

Adding language in support of the categorization as High Risk of AI systems in the field of employment, workers management and access to self-employment as included in Annex III.

Amendment 9

Proposal for a regulation

~~Recital 4 a (new)~~

Recital 29

Text proposed by the Commission

Amendment

(4a) Tackling climate change and environmental-related challenges and reaching the objectives of the Paris Agreement are at the core of the Communication on “The European Green Deal”, adopted on 11 December 2019, where the Commission recalled the role of digital technologies such as artificial intelligence, 5G, cloud and edge computing and the internet of things to accelerate and maximise the impact of policies to deal with climate change mitigation and adaptation, protect the environment and address biodiversity loss.

(29) As regards high-risk AI systems that are safety components of products or systems, or which are themselves products or systems falling within the scope of Regulation (EC) No 300/2008 of the European Parliament and of the Council39 , Regulation (EU) No 167/2013 of the European Parliament and of the Council40 , Regulation (EU) No 168/2013 of the European Parliament and of the Council41 , Directive 2014/90/EU of the European Parliament and of the Council42 , Directive (EU) 2016/797 of the European Parliament and of the Council43 , Regulation (EU) 2018/858 of the European Parliament and of the Council44 , Regulation (EU) 2018/1139 of the European Parliament and of the Council45 , and Regulation (EU) 2019/2144 of the European Parliament and of the Council46, it is appropriate, if required, to amend those acts to ensure that the Commission takes into account, on the basis of the technical and regulatory specificities of each sector, and without overlapping with existing governance, conformity assessment and enforcement mechanisms and authorities established therein, the mandatory requirements for high-risk AI systems laid down in this Regulation when adopting any relevant future delegated or implementing acts on the basis of those acts. Transport sectoral legislation should prevail over this Regulation and it should be ensured that no overlap exists between this Regulation and other current and upcoming legal acts (i.e. Data Act, ITS Review) to avoid duplication of obligations on providers and manufacturers, which would cause legal uncertainty for business and slow down the uptake of new technologies in the market. This Regulation should also provide for an efficient review mechanism in order to take into account future technological developments and to ensure fair, proportionate and targeted implementation.

__________________

39 Regulation (EC) No 300/2008 of the European Parliament and of the Council of 11 March 2008 on common rules in the field of civil aviation security and repealing Regulation (EC) No 2320/2002 (OJ L 97, 9.4.2008, p. 72).

40 Regulation (EU) No 167/2013 of the European Parliament and of the Council of 5 February 2013 on the approval and market surveillance of agricultural and forestry vehicles (OJ L 60, 2.3.2013, p. 1).

41 Regulation (EU) No 168/2013 of the European Parliament and of the Council of 15 January 2013 on the approval and market surveillance of two- or three-wheel vehicles and quadricycles (OJ L 60, 2.3.2013, p. 52).

42 Directive 2014/90/EU of the European Parliament and of the Council of 23 July 2014 on marine equipment and repealing Council Directive 96/98/EC (OJ L 257, 28.8.2014, p. 146).

43 Directive (EU) 2016/797 of the European Parliament and of the Council of 11 May 2016 on the interoperability of the rail system within the European Union (OJ L 138, 26.5.2016, p. 44).

44 Regulation (EU) 2018/858 of the European Parliament and of the Council of 30 May 2018 on the approval and market surveillance of motor vehicles and their trailers, and of systems, components and separate technical units intended for such vehicles, amending Regulations (EC) No 715/2007 and (EC) No 595/2009 and repealing Directive 2007/46/EC (OJ L 151, 14.6.2018, p. 1).

45 Regulation (EU) 2018/1139 of the European Parliament and of the Council of 4 July 2018 on common rules in the field of civil aviation and establishing a European Union Aviation Safety Agency, and amending Regulations (EC) No 2111/2005, (EC) No 1008/2008, (EU) No 996/2010, (EU) No 376/2014 and Directives 2014/30/EU and 2014/53/EU of the European Parliament and of the Council, and repealing Regulations (EC) No 552/2004 and (EC) No 216/2008 of the European Parliament and of the Council and Council Regulation (EEC) No 3922/91 (OJ L 212, 22.8.2018, p. 1).

46 Regulation (EU) 2019/2144 of the European Parliament and of the Council of 27 November 2019 on type-approval requirements for motor vehicles and their trailers, and systems, components and separate technical units intended for such vehicles, as regards their general safety and the protection of vehicle occupants and vulnerable road users, amending Regulation (EU) 2018/858 of the European Parliament and of the Council and repealing Regulations (EC) No 78/2009, (EC) No 79/2009 and (EC) No 661/2009 of the European Parliament and of the Council and Commission Regulations (EC) No 631/2009, (EU) No 406/2010, (EU) No 672/2010, (EU) No 1003/2010, (EU) No 1005/2010, (EU) No 1008/2010, (EU) No 1009/2010, (EU) No 19/2011, (EU) No 109/2011, (EU) No 458/2011, (EU) No 65/2012, (EU) No 130/2012, (EU) No 347/2012, (EU) No 351/2012, (EU) No 1230/2012 and (EU) 2015/166 (OJ L 325, 16.12.2019, p. 1).

Or. en

Proposal for a regulation

~~Recital 4 b (new)~~

Recital 34

Text proposed by the Commission

Amendment

(4b) In its White Paper on "Artificial Intelligence - A European approach to excellence and trust" of 19 February 2020, the Commission recalls that artificial intelligence can contribute to finding solutions to some of the most pressing societal challenges, including the fight against climate change, biodiversity loss and environmental degradation and highlights the potential benefits and risks of artificial intelligence in relation to safety, health and wellbeing of individuals.

(34) As regards the management and operation of critical infrastructure, it is appropriate to classify as high-risk the AI systems intended to be used as safety components in the management and operation of road traffic and the supply of water, gas, heating and electricity, since their failure or malfunctioning may put at risk the life and health of persons at large scale and lead to appreciable disruptions in the ordinary conduct of social and economic activities. Examples of critical infrastructure for road covered by Annex III should include but should not be not limited to traffic management control systems, intelligent transport systems and ICT infrastructure connected transport.

Or. en

Proposal for a regulation

~~Recital 4 c (new)~~

Recital 44

Text proposed by the Commission

Amendment

(4c) AI applications can bring environmental and economic benefits and strengthen predictive capabilities that contribute to the fight against climate change, to meeting the Sustainable Development Goals (SDGs) and to achieving our target of becoming the first climate-neutral continent. In this sense, the use of AI has the potential to reduce global greenhouse gas emissions by up to 4 % by 2030 but it is important that AI systems and associated machinery are designed sustainably to reduce resource usage and energy consumption, thereby limiting the risks to the environment;

(44) High data quality is essential for the performance of many AI systems, especially when techniques involving the training of models are used, with a view to ensure that the high-risk AI system performs as intended and safely and it does not become the source of discrimination prohibited by Union law. High quality training, validation and testing data sets require the implementation of appropriate data governance and management practices. Training, validation and testing data sets should be sufficiently relevant, representative and free of errors and complete in view of the intended purpose of the system. They should also have the appropriate statistical properties, including as regards the persons or groups of persons on which the high-risk AI system is intended to be used. In particular, training, validation and testing data sets should take into account, to the extent required in the light of their intended purpose, the features, characteristics or elements that are particular to the specific geographical, behavioural or functional setting or context within which the AI system is intended to be used. In order to protect the right of others from the discrimination that might result from the bias in AI systems, the providers shouldbe able to process also special categories of personal data, as a matter of substantial public interest, in order to ensure the bias monitoring, detection and correction in relation to high-risk AI systems.

(44) High data quality is essential for the performance of many AI systems, especially when techniques involving the training of models are used, with a view to ensure that the high-risk AI system performs as intended and safely and it does not become the source of discrimination prohibited by Union law. High quality training, validation and testing data sets require the implementation of appropriate data governance and management practices. Training, validation and testing data sets should be sufficiently relevant, representative and, to the best extent possible free of errors and complete in view of the intended purpose of the system and to ensure the highest level of security. They should also have the appropriate statistical properties, including as regards the persons or groups of persons on which the high-risk AI system is intended to be used. In particular, training, validation and testing data sets should take into account, to the extent required in the light of their intended purpose, the features, characteristics or elements that are particular to the specific geographical, behavioural or functional setting or context within which the AI system is intended to be used. In order to protect the right of others from the discrimination that might result from the bias in AI systems, the providers should be able to process also special categories of personal data, as a matter of substantial public interest, in order to ensure the bias monitoring, detection and correction in relation to high-risk AI systems.

Or. en

Proposal for a regulation

~~Recital 4 d (new)~~

Recital 47

Text proposed by the Commission

Amendment

(4d) Traditional identification of species has been time consuming and costly, which hinders real time biodiversity assessments. The integration of AI systems has the potential to move away from manual sorting and identification of species, which can play a role in animal conservation by allowing authorities to quickly identify, observe and monitor endangered species populations and help inform additional measures if needed for conservation purposes.

(47) To address the opacity that may make certain AI systems incomprehensible to or too complex for natural persons, a certain degree of transparency should be required for high-risk AI systems. Users should be able to interpret the system output and use it appropriately. High-risk AI systems should therefore be accompanied by relevant documentation and instructions of use and include concise and clear information, including in relation to possible risks to fundamental rights and discrimination, where appropriate.

(47) To address the opacity that may make certain AI systems incomprehensible to or too complex for natural persons, a certain degree of transparency should be required for high-risk AI systems, in particular when applied to digital labour platforms managing the activities of transport workers. Users should be able to interpret the system output and use it appropriately. Transparency, fairness, accountability and explanability of AI systems can also be a beneficial factor for their uptake by consumers in the market. High-risk AI systems should therefore be accompanied by relevant documentation and instructions of use and include concise and clear information, including in relation to possible risks to fundamental rights and discrimination, where appropriate.

Or. en

Proposal for a regulation

Recital 548

Text proposed by the Commission

Amendment

(5) A Union legal framework laying down harmonised rules on artificial intelligence is therefore needed to foster the development, use and uptake of artificial intelligence in the internal market that at the same time meets a high level of protection of public interests, such as health and safety and the protection of fundamental rights, as recognised and protected by Union law. To achieve that objective, rules regulating the placing on the market and putting into service of certain AI systems should be laid down, thus ensuring the smooth functioning of the internal market and allowing those systems to benefit from the principle of free movement of goods and services. By laying down those rules, this Regulation supports the objective of the Union of being a global leader in the development of secure, trustworthy and ethical artificial intelligence, as stated by the European Council33 , and it ensures the protection of ethical principles, as specifically requested by the European Parliament34 .

(5) A Union legal framework laying down harmonised rules on artificial intelligence is therefore needed to foster the development, use and uptake of artificial intelligence in the internal market that at the same time meets a high level of protection of public interests, such as health and safety, the environment and the protection of fundamental rights, as recognised and protected by Union law. To achieve that objective, rules regulating the placing on the market and putting into service of certain AI systems should be laid down, thus ensuring the smooth functioning of the internal market and allowing those systems to benefit from the principle of free movement of goods and services. By laying down those rules, this Regulation supports the objective of the Union of being a global leader in the development of secure, trustworthy and ethical artificial intelligence, as stated by the European Council33, and it ensures the protection of ethical principles, as specifically requested by the European Parliament34.

(48) High-risk AI systems should be designed and developed in such a way that natural persons can oversee their functioning. For this purpose, appropriate human oversight measures should be identified by the provider of the system before its placing on the market or putting into service. In particular, where appropriate, such measures should guarantee that the system is subject to in-built operational constraints that cannot be overridden by the system itself and is responsive to the human operator, and that the natural persons to whom human oversight has been assigned have the necessary competence, training and authority to carry out that role. With regards to the transport sector, a distinction should be made between automation and AI System applications since the classification of an AI system as high risk should depend on the different level of integration of AI within a vehicle and the task it performs. When physical security is at stake, for example when an accident is imminent, Union and international standards should determine in which case the possibility for a human operator to take back control should take prevalence over AI system’s decision.

~~__________________~~

~~33 European Council, Special meeting of the European Council (1 and 2 October 2020) – Conclusions, EUCO 13/20, 2020, p. 6.~~

34 European Parliament resolution of 20 October 2020 with recommendations to the Commission on a framework of ethical aspects of artificial intelligence, robotics and related technologies, 2020/2012(INL).

Or. en

Proposal for a regulation

Recital 659

Text proposed by the Commission

Amendment

(59) It is appropriate to envisage that the user of the AI system should be the natural or legal person, public authority, agency or other body under whose authority the AI system is operated except where the use is made in the course of a personal non-professional activity.

(6) The notion of AI system should be clearly defined to ensure legal certainty, while providing the flexibility to accommodate future technological developments, such as neurotechnology, which may put mental privacy at risk and require legislative proposals to protect neurodata. The definition should be based on the key functional characteristics of the software, in particular the ability, for a given set of human-defined objectives, to generate outputs such as content, predictions, recommendations, or decisions which influence the environment with which the system interacts, be it in a physical or digital dimension. AI systems can be designed to operate with varying levels of autonomy and be used on a stand-alone basis or as a component of a product, irrespective of whether the system is physically integrated into the product (embedded) or serve the functionality of the product without being integrated therein (non-embedded). The definition of AI system should be complemented by a list of specific techniques and approaches used for its development, which should be kept up-to–date in the light of market and technological developments through the adoption of delegated acts by the Commission to amend that list.

(59) It is appropriate to envisage that the user of the AI system should be the natural or legal person, public authority, agency or other body under whose authority the AI system is operated.

Or. en

Proposal for a regulation

Recital 1372

Text proposed by the Commission

Amendment

(72) The objectives of the regulatory sandboxes should be to foster AI innovation by establishing a controlled experimentation and testing environment in the development and pre-marketing phase with a view to ensuring compliance of the innovative AI systems with this Regulation and other relevant Union and Member States legislation; to enhance legal certainty for innovators and the competent authorities’ oversight and understanding of the opportunities, emerging risks and the impacts of AI use, and to accelerate access to markets, including by removing barriers for small and medium enterprises (SMEs) and start-ups. To ensure uniform implementation across the Union and economies of scale, it is appropriate to establish common rules for the regulatory sandboxes’ implementation and a framework for cooperation between the relevant authorities involved in the supervision of the sandboxes. This Regulation should provide the legal basis for the use of personal data collected for other purposes for developing certain AI systems in the public interest within the AI regulatory sandbox, in line with Article 6(4) of Regulation (EU) 2016/679, and Article 6 of Regulation (EU) 2018/1725, and without prejudice to Article 4(2) of Directive (EU) 2016/680. Participants in the sandbox should ensure appropriate safeguards and cooperate with the competent authorities, including by following their guidance and acting expeditiously and in good faith to mitigate any high-risks to safety and fundamental rights that may arise during the development and experimentation in the sandbox. The conduct of the participants in the sandbox should be taken into account when competent authorities decide whether to impose an administrative fine under Article 83(2) of Regulation 2016/679 and Article 57 of Directive 2016/680.

(13) In order to ensure a consistent and high level of protection of public interests as regards health, safety, the environment and fundamental rights, common normative standards for all high-risk AI systems should be established. Those standards should be consistent with the Charter of fundamental rights of the European Union (the Charter) and should be non-discriminatory and in line with the Union’s international trade commitments.

(72) The objectives of the regulatory sandboxes should be to foster AI innovation by establishing a controlled experimentation and testing environment in the development and pre-marketing phase with a view to ensuring compliance of the innovative AI systems with this Regulation and other relevant Union and Member States legislation; to enhance legal certainty for innovators and the competent authorities’ oversight and understanding of the opportunities, emerging risks and the impacts of AI use, and to accelerate access to markets, including by removing barriers for small and medium enterprises (SMEs) and start-ups, as well as to contribute to achieving the targets on AI as set in the Policy Programme “Path to the Digital Decade". To ensure uniform implementation across the Union and economies of scale, it is appropriate to establish common rules for the regulatory sandboxes’ implementation and a framework for cooperation between the relevant authorities involved in the supervision of the sandboxes. This Regulation should provide the legal basis for the use of personal data collected for other purposes for developing certain AI systems in the public interest within the AI regulatory sandbox, in line with Article 6(4) of Regulation (EU) 2016/679, and Article 6 of Regulation (EU) 2018/1725, and without prejudice to Article 4(2) of Directive (EU) 2016/680. Participants in the sandbox should ensure appropriate safeguards and cooperate with the competent authorities, including by following their guidance and acting expeditiously and in good faith to mitigate any high-risks to safety and fundamental rights that may arise during the development and experimentation in the sandbox. The conduct of the participants in the sandbox should be taken into account when competent authorities decide whether to impose an administrative fine under Article 83(2) of Regulation 2016/679 and Article 57 of Directive 2016/680.

Or. en

Proposal for a regulation

~~Recital 13 a (new)~~

Recital 73

Text proposed by the Commission

Amendment

(13a) AI systems shall fully respect the climate and environmental standards and priorities of the Union and the principle of ‘do no significant harm’ within the meaning of Article 17 of Regulation (EU) 2020/852 on the establishment of a framework to facilitate sustainable investment.

(73) In order to promote and protect innovation, it is important that the interests of small-scale providers and users of AI systems are taken into particular account. To this objective, Member States should develop initiatives, which are targeted at those operators, including on awareness raising and information communication. Moreover, the specific interests and needs of small-scale providers shall be taken into account when Notified Bodies set conformity assessment fees. Translation costs related to mandatory documentation and communication with authorities may constitute a significant cost for providers and other operators, notably those of a smaller scale. Member States should possibly ensure that one of the languages determined and accepted by them for relevant providers’ documentation and for communication with operators is one which is broadly understood by the largest possible number of cross-border users.

(73) In order to promote and protect innovation, it is important that the interests of small-scale providers, especially SMEs, and users of AI systems are taken into particular account. To this objective, Member States should develop initiatives, which are targeted at those operators, including on awareness raising and information communication. Moreover, the specific interests and needs of small-scale providers, including SMEs, should be taken into account when Notified Bodies set conformity assessment fees. Translation costs related to mandatory documentation and communication with authorities may constitute a significant cost for providers and other operators, notably those of a smaller scale. Member States should possibly ensure that one of the languages determined and accepted by them for relevant providers’ documentation and for communication with operators is one which is broadly understood by the largest possible number of cross-border users.

Or. en

Proposal for a regulation

~~Recital 13 b (new)~~

Article 1 – paragraph 1 – point e

Text proposed by the Commission

Amendment

(13b) AI in the field of health interventions has the potential to improve health outcomes, enhance the quality of care to patients respond to unmet needs, and also to foster the competitiveness of stakeholders and to improve the cost-effectiveness and sustainability of health services and medical care. The EU has the potential to become a leader in the application of AI in the healthcare sector.

(e) rules on market monitoring and surveillance.

(e) rules on market monitoring, market surveillance and governance.

Or. en

Proposal for a regulation

~~Recital 13 c (new)~~

Article 1 – paragraph 1 – point e a (new)

Text proposed by the Commission

Amendment

(13c) AI can unlock solutions in the health sector that could save millions of lives, improve our standard of living and improve patient care, especially in diagnosis, prognosis and treatment, patient engagement, adherence, management and follow-up, clinical decision-making, including predictive analytics, screening and optimization of clinical pathways, and pathology. AI can also improve prevention strategies, health system management and in the organization and provision of health services and medical care, including health promotion and disease prevention interventions.

(ea) Provision to foster and support research and development for innovation.

Or. en

Proposal for a regulation

~~Recital 16~~

Article 2 – paragraph 1 – point c a (new)

Text proposed by the Commission

Amendment

(16) The placing on the market, putting into service or use of certain AI systems intended to distort human behaviour, whereby physical or psychological harms are likely to occur, should be forbidden. Such AI systems deploy subliminal components individuals cannot perceive or exploit vulnerabilities of children and people due to their age, physical or mental incapacities. They do so with the intention to materially distort the behaviour of a person and in a manner that causes or is likely to cause harm to that or another person. The intention may not be presumed if the distortion of human behaviour results from factors external to the AI system which are outside of the control of the provider or the user. Research for legitimate purposes in relation to such AI systems should not be stifled by the prohibition, if such research does not amount to use of the AI system in human-machine relations that exposes natural persons to harm and such research is carried out in accordance with recognised ethical standards for scientific research.

(ca) importers and distributors of AI systems.

(16) The placing on the market, putting into service or use of certain AI systems intended to distort human behaviour, whereby physical, psychological harms or disruption of the sense of oneself are likely to occur, should be forbidden. Such AI systems deploy subliminal components individuals cannot perceive or exploit vulnerabilities of children and people due to their age, physical or mental incapacities. They do so with the intention to materially distort the behaviour of a person and in a manner that causes or is likely to cause harm to that or another person or remove ultimate control over personal decision-making, with unknown manipulation from external neurotechnologies. The intention may not be presumed if the distortion of human behaviour results from factors external to the AI system which are outside of the control of the provider or the user. Research for legitimate purposes in relation to such AI systems should not be stifled by the prohibition, if such research does not amount to use of the AI system in human-machine relations that exposes natural persons to harm and such research is carried out in accordance with recognised ethical standards for scientific research.

Or. en

Proposal for a regulation

~~Recital 27~~

Article 2 – paragraph 1 – point c b (new)

Text proposed by the Commission

Amendment

(27) High-risk AI systems should only be placed on the Union market or put into service if they comply with certain mandatory requirements. Those requirements should ensure that high-risk AI systems available in the Union or whose output is otherwise used in the Union do not pose unacceptable risks to important Union public interests as recognised and protected by Union law. AI systems identified as high-risk should be limited to those that have a significant harmful impact on the health, safety and fundamental rights of persons in the Union and such limitation minimises any potential restriction to international trade, if any.

(cb) product manufacturers placing on the market or putting into service an AI system together with their product and under their own name or trademark.

(27) High-risk AI systems should only be placed on the Union market or put into service if they comply with certain mandatory requirements. Those requirements should ensure that high-risk AI systems available in the Union or whose output is otherwise used in the Union do not pose unacceptable risks to important Union public interests as recognised and protected by Union law. AI systems identified as high-risk should be limited to those that have a significant harmful impact on the health, safety, the environment and fundamental rights of persons in the Union and such limitation minimises any potential restriction to international trade, if any.

Or. en

Proposal for a regulation

~~Recital 28~~

Article 2 – paragraph 2 – introductory part

Text proposed by the Commission

Amendment

(28) AI systems could produce adverse outcomes to health and safety of persons, in particular when such systems operate as components of products. Consistently with the objectives of Union harmonisation legislation to facilitate the free movement of products in the internal market and to ensure that only safe and otherwise compliant products find their way into the market, it is important that the safety risks that may be generated by a product as a whole due to its digital components, including AI systems, are duly prevented and mitigated. For instance, increasingly autonomous robots, whether in the context of manufacturing or personal assistance and care should be able to safely operate and performs their functions in complex environments. Similarly, in the health sector where the stakes for life and health are particularly high, increasingly sophisticated diagnostics systems and systems supporting human decisions should be reliable and accurate. The extent of the adverse impact caused by the AI system on the fundamental rights protected by the Charter is of particular relevance when classifying an AI system as high-risk. Those rights include the right to human dignity, respect for private and family life, protection of personal data, freedom of expression and information, freedom of assembly and of association, and non-discrimination, consumer protection, workers’ rights, rights of persons with disabilities, right to an effective remedy and to a fair trial, right of defence and the presumption of innocence, right to good administration. In addition to those rights, it is important to highlight that children have specific rights as enshrined in Article 24 of the EU Charter and in the United Nations Convention on the Rights of the Child (further elaborated in the UNCRC General Comment No. 25 as regards the digital environment), both of which require consideration of the children’s vulnerabilities and provision of such protection and care as necessary for their well-being. The fundamental right to a high level of environmental protection enshrined in the Charter and implemented in Union policies should also be considered when assessing the severity of the harm that an AI system can cause, including in relation to the health and safety of persons.

2. For high-risk AI systems that are safety components of products or systems, or which are themselves products or systems, falling within the scope of the following acts, only Article 84 of this Regulation shall apply:

(28) AI systems could produce adverse outcomes to health, the environment and safety of persons. Consistently with the objectives of Union harmonisation legislation to facilitate the free movement of products in the internal market and to ensure that only safe and otherwise compliant products find their way into the market, it is important that the safety risks that may be generated by a product as a whole due to its digital components, including AI systems, are duly prevented and mitigated. For instance, increasingly autonomous robots, whether in the context of manufacturing or personal assistance and care should be able to safely operate and performs their functions in complex environments. Similarly, in the health sector where the stakes for life and health are particularly high, increasingly sophisticated diagnostics systems and systems supporting human decisions such as neurological AI applications should be safe, reliable and accurate. The extent of the adverse impact caused by the AI system on the fundamental rights protected by the Charter and negative impact on the environment is of particular relevance when classifying an AI system as high-risk. Those rights include the right to human dignity, respect for private and family life, protection of personal data, neurological data, freedom of expression and information, freedom of assembly and of association, and non-discrimination, consumer protection, workers’ rights, rights of persons with disabilities, right to an effective remedy and to a fair trial, right of defence and the presumption of innocence, right to good administration and the right to a high level of environmental protection and the improvement of the quality of the environment. In addition to those rights, it is important to highlight that children have specific rights as enshrined in Article 24 of the EU Charter and in the United Nations Convention on the Rights of the Child (further elaborated in the UNCRC General Comment No. 25 as regards the digital environment), both of which require consideration of the children’s vulnerabilities and provision of such protection and care as necessary for their well-being. The fundamental right to a high level of environmental protection enshrined in the Charter and implemented in Union policies should also be considered when assessing the severity of the harm that an AI system can cause, including in relation to the health and safety of persons and the environment.

2. For AI systems classified as high-risk AI in accordance with Article 6(1) related to products covered by Union harmonisation legislation listed in Annex II, section B, only Article 84 of this Regulation shall apply.

Or. en

Proposal for a regulation

~~Recital 32~~

Article 2 – paragraph 2 – point a

Text proposed by the Commission

Amendment

(32) As regards stand-alone AI systems, meaning high-risk AI systems other than those that are safety components of products, or which are themselves products, it is appropriate to classify them as high-risk if, in the light of their intended purpose, they pose a high risk of harm to the health and safety or the fundamental rights of persons, taking into account both the severity of the possible harm and its probability of occurrence and they are used in a number of specifically pre-defined areas specified in the Regulation. The identification of those systems is based on the same methodology and criteria envisaged also for any future amendments of the list of high-risk AI systems.

(a) Regulation (EC) 300/2008;

(32) As regards stand-alone AI systems, meaning high-risk AI systems other than those that are safety components of products, or which are themselves products, it is appropriate to classify them as high-risk if, in the light of their intended purpose, they pose a high risk of harm to the health and safety, the environment or the fundamental rights of persons, taking into account both the severity of the possible harm and its probability of occurrence and they are used in a number of specifically pre-defined areas specified in the Regulation. The identification of those systems is based on the same methodology and criteria envisaged also for any future amendments of the list of high-risk AI systems.

deleted

Or. en

Proposal for a regulation

~~Recital 34~~

Article 2 – paragraph 2 – point b

Text proposed by the Commission

Amendment

(b) Regulation (EU) No 167/2013;

(34) As regards the management and operation of critical infrastructure, it is appropriate to classify as high-risk the AI systems intended to be used as safety components in the management and operation of road traffic, the supply of water and gas, healthcare systems, natural or man-made disaster control mechanisms, heating and electricity, since their failure or malfunctioning may put at risk the life and health of persons and environment at large scale and lead to appreciable disruptions in the ordinary conduct of social and economic activities.

deleted

Or. en

Proposal for a regulation

~~Recital 37~~

Article 2 – paragraph 2 – point c

Text proposed by the Commission

Amendment

(37) Another area in which the use of AI systems deserves special consideration is the access to and enjoyment of certain essential private and public services and benefits necessary for people to fully participate in society or to improve one’s standard of living. In particular, AI systems used to evaluate the credit score or creditworthiness of natural persons should be classified as high-risk AI systems, since they determine those persons’ access to financial resources or essential services such as housing, electricity, and telecommunication services. AI systems used for this purpose may lead to discrimination of persons or groups and perpetuate historical patterns of discrimination, for example based on racial or ethnic origins, disabilities, age, sexual orientation, or create new forms of discriminatory impacts. Considering the very limited scale of the impact and the available alternatives on the market, it is appropriate to exempt AI systems for the purpose of creditworthiness assessment and credit scoring when put into service by small-scale providers for their own use. Natural persons applying for or receiving public assistance benefits and services from public authorities are typically dependent on those benefits and services and in a vulnerable position in relation to the responsible authorities. If AI systems are used for determining whether such benefits and services should be denied, reduced, revoked or reclaimed by authorities, they may have a significant impact on persons’ livelihood and may infringe their fundamental rights, such as the right to social protection, non-discrimination, human dignity or an effective remedy. Those systems should therefore be classified as high-risk. Nonetheless, this Regulation should not hamper the development and use of innovative approaches in the public administration, which would stand to benefit from a wider use of compliant and safe AI systems, provided that those systems do not entail a high risk to legal and natural persons. Finally, AI systems used to dispatch or establish priority in the dispatching of emergency first response services should also be classified as high-risk since they make decisions in very critical situations for the life and health of persons and their property.

(37) Another area in which the use of AI systems deserves special consideration is the access to and enjoyment of certain essential private and public services, including healthcare, and benefits necessary for people to fully participate in society or to improve one’s standard of living. In particular, AI systems used to evaluate the credit score or creditworthiness of natural persons should be classified as high-risk AI systems, since they determine those persons’ access to financial resources or essential services such as housing, electricity, healthcare and telecommunication services. AI systems used for this purpose may lead to discrimination of persons or groups and perpetuate historical patterns of discrimination, for example based on racial or ethnic origins, disabilities, age, sexual orientation, or create new forms of discriminatory impacts. Considering the very limited scale of the impact and the available alternatives on the market, it is appropriate to exempt AI systems for the purpose of creditworthiness assessment and credit scoring when put into service by small-scale providers for their own use. Natural persons applying for or receiving public assistance benefits and services from public authorities are typically dependent on those benefits and services and in a vulnerable position in relation to the responsible authorities. If AI systems are used for determining whether such benefits and services should be denied, reduced, revoked or reclaimed by authorities, they may have a significant impact on persons’ livelihood, health and wellbeing, and may infringe their fundamental rights, such as the right to social protection, non-discrimination, human dignity or an effective remedy. Those systems should therefore be classified as high-risk. Nonetheless, this Regulation should not hamper the development and use of innovative approaches in the public administration, which would stand to benefit from a wider use of compliant and safe AI systems, provided that those systems do not entail a high risk to legal and natural persons. Finally, AI systems used to dispatch or establish priority in the dispatching of emergency first response services, disease prevention, diagnosis, control and treatment should also be classified as high-risk since they make decisions in very critical situations for the life and health of persons and their property.

deleted

Or. en

Proposal for a regulation

~~Recital 38~~

Article 2 – paragraph 2 – point d

Text proposed by the Commission

Amendment

(38) Actions by law enforcement authorities involving certain uses of AI systems are characterised by a significant degree of power imbalance and may lead to surveillance, arrest or deprivation of a natural person’s liberty as well as other adverse impacts on fundamental rights guaranteed in the Charter. In particular, if the AI system is not trained with high quality data, does not meet adequate requirements in terms of its accuracy or robustness, or is not properly designed and tested before being put on the market or otherwise put into service, it may single out people in a discriminatory or otherwise incorrect or unjust manner. Furthermore, the exercise of important procedural fundamental rights, such as the right to an effective remedy and to a fair trial as well as the right of defence and the presumption of innocence, could be hampered, in particular, where such AI systems are not sufficiently transparent, explainable and documented. It is therefore appropriate to classify as high-risk a number of AI systems intended to be used in the law enforcement context where accuracy, reliability and transparency is particularly important to avoid adverse impacts, retain public trust and ensure accountability and effective redress. In view of the nature of the activities in question and the risks relating thereto, those high-risk AI systems should include in particular AI systems intended to be used by law enforcement authorities for individual risk assessments, polygraphs and similar tools or to detect the emotional state of natural person, to detect ‘deep fakes’, for the evaluation of the reliability of evidence in criminal proceedings, for predicting the occurrence or reoccurrence of an actual or potential criminal offence based on profiling of natural persons, or assessing personality traits and characteristics or past criminal behaviour of natural persons or groups, for profiling in the course of detection, investigation or prosecution of criminal offences, as well as for crime analytics regarding natural persons. AI systems specifically intended to be used for administrative proceedings by tax and customs authorities should not be considered high-risk AI systems used by law enforcement authorities for the purposes of prevention, detection, investigation and prosecution of criminal offences.

(d) Directive 2014/90/EU;

(38) Actions by law enforcement authorities involving certain uses of AI systems are characterised by a significant degree of power imbalance and may lead to surveillance, arrest or deprivation of a natural person’s liberty as well as other adverse impacts on fundamental rights guaranteed in the Charter. In particular, if the AI system is not trained with high quality data, does not meet adequate requirements in terms of its accuracy or robustness, or is not properly designed and tested before being put on the market or otherwise put into service, it may single out people in a discriminatory or otherwise incorrect or unjust manner. Furthermore, the exercise of important procedural fundamental rights, such as the right to an effective remedy, including the right to access to justice for environmental matters as established in the Aarhus Convention (Regulation 1367/2006, as amended by Regulation (EU) 2021/1767) and to a fair trial as well as the right of defence and the presumption of innocence, could be hampered, in particular, where such AI systems are not sufficiently transparent, explainable and documented. It is therefore appropriate to classify as high-risk a number of AI systems intended to be used in the law enforcement context where accuracy, reliability and transparency is particularly important to avoid adverse impacts, retain public trust and ensure accountability and effective redress. In view of the nature of the activities in question and the risks relating thereto, those high-risk AI systems should include in particular AI systems intended to be used by law enforcement authorities for individual risk assessments, polygraphs and similar tools or to detect the emotional state of natural person, to detect ‘deep fakes’, for the evaluation of the reliability of evidence in criminal proceedings, for predicting the occurrence or reoccurrence of an actual or potential criminal offence based on profiling of natural persons, or assessing personality traits and characteristics or past criminal behaviour of natural persons or groups, for profiling in the course of detection, investigation or prosecution of criminal offences, as well as for crime analytics regarding natural persons. AI systems specifically intended to be used for administrative proceedings by tax and customs authorities should not be considered high-risk AI systems used by law enforcement authorities for the purposes of prevention, detection, investigation and prosecution of criminal offences.

deleted

Or. en

Proposal for a regulation

~~Recital 43~~

Article 2 – paragraph 2 – point e

Text proposed by the Commission

Amendment

(43) Requirements should apply to high-risk AI systems as regards the quality of data sets used, technical documentation and record-keeping, transparency and the provision of information to users, human oversight, and robustness, accuracy and cybersecurity. Those requirements are necessary to effectively mitigate the risks for health, safety and fundamental rights, as applicable in the light of the intended purpose of the system, and no other less trade restrictive measures are reasonably available, thus avoiding unjustified restrictions to trade.

(e) Directive (EU) 2016/797;

(43) Requirements should apply to high-risk AI systems as regards the quality of data sets used, technical documentation and record-keeping, transparency and the provision of information to users and end recipients, human oversight, and robustness, accuracy and cybersecurity. Those requirements are necessary to effectively mitigate the risks for health, safety, the environment and fundamental rights, as applicable in the light of the intended purpose of the system, and no other less trade restrictive measures are reasonably available, thus avoiding unjustified restrictions to trade.

deleted

Or. en

Proposal for a regulation

~~Recital 45~~

Article 2 – paragraph 2 – point f

Text proposed by the Commission

Amendment

(45) For the development of high-risk AI systems, certain actors, such as providers, notified bodies and other relevant entities, such as digital innovation hubs, testing experimentation facilities and researchers, should be able to access and use high quality datasets within their respective fields of activities which are related to this Regulation. European common data spaces established by the Commission and the facilitation of data sharing between businesses and with government in the public interest will be instrumental to provide trustful, accountable and non-discriminatory access to high quality data for the training, validation and testing of AI systems. For example, in health, the European health data space will facilitate non-discriminatory access to health data and the training of artificial intelligence algorithms on those datasets, in a privacy-preserving, secure, timely, transparent and trustworthy manner, and with an appropriate institutional governance. Relevant competent authorities, including sectoral ones, providing or supporting the access to data may also support the provision of high-quality data for the training, validation and testing of AI systems.

(f) Regulation (EU) 2018/858;

(45) For the development of high-risk AI systems, certain actors, such as providers, notified bodies and other relevant entities, such as digital innovation hubs, research and scientific institutes, health authorities, hospitals testing experimentation facilities and researchers, should be able to access and use high quality datasets within their respective fields of activities which are related to this Regulation. European common data spaces established by the Commission and the facilitation of data sharing between businesses and with government in the public interest will be instrumental to provide trustful, accountable and non-discriminatory access to high quality data for the training, validation and testing of AI systems. For example, in health, the European health data space will facilitate non-discriminatory access to health data and the training of artificial intelligence algorithms on those datasets, in a privacy-preserving, secure, timely, transparent and trustworthy manner, and with an appropriate institutional governance. Member States shall put in place incentives to ensure that the data is completely interoperable to unlock the full potential of Europe´s high quality healthcare services, while complying with the GDPR. Relevant competent authorities, including sectoral ones, providing or supporting the access to data may also support the provision of high-quality data for the training, validation and testing of AI systems.

deleted

Or. en

Proposal for a regulation

~~Recital 47~~

Article 2 – paragraph 2 – point g

Text proposed by the Commission

Amendment

(47) To address the opacity that may make certain AI systems incomprehensible to or too complex for natural persons, a certain degree of transparency should be required for high-risk AI systems. Users should be able to interpret the system output and use it appropriately. High-risk AI systems should therefore be accompanied by relevant documentation and instructions of use and include concise and clear information, including in relation to possible risks to fundamental rights and discrimination, where appropriate.

(g) Regulation (EU) 2018/1139;

(47) To address the opacity that may make certain AI systems incomprehensible to or too complex for natural persons, a certain degree of transparency should be required for high-risk AI systems. Users and end recipients should be able to interpret the system output and use it appropriately. High-risk AI systems should therefore be accompanied by relevant documentation and instructions of use and include concise and clear information, including in relation to possible risks to fundamental rights and discrimination, where appropriate.

deleted

Or. en

Proposal for a regulation

~~Recital 49~~

Article 2 – paragraph 2 – point h

Text proposed by the Commission

Amendment

(49) High-risk AI systems should perform consistently throughout their lifecycle and meet an appropriate level of accuracy, robustness and cybersecurity in accordance with the generally acknowledged state of the art. The level of accuracy and accuracy metrics should be communicated to the users.

(h) Regulation (EU) 2019/2144.

deleted

Or. en

Proposal for a regulation

~~Recital 50~~

Article 2 – paragraph 5 a (new)

Text proposed by the Commission

Amendment

(50) The technical robustness is a key requirement for high-risk AI systems. They should be resilient against risks connected to the limitations of the system (e.g. errors, faults, inconsistencies, unexpected situations) as well as against malicious actions that may compromise the security of the AI system and result in harmful or otherwise undesirable behaviour. Failure to protect against these risks could lead to safety impacts or negatively affect the fundamental rights, for example due to erroneous decisions or wrong or biased outputs generated by the AI system.

5a. This Regulation shall not apply to AI systems developed and put into service for the sole purpose of scientific research and development.

(50) The technical robustness is a key requirement for high-risk AI systems. They should be resilient against risks connected to the limitations of the system (e.g. errors, faults, inconsistencies, unexpected situations) as well as against malicious actions that may compromise the security of the AI system and result in harmful or otherwise undesirable behaviour. Failure to protect against these risks could lead to safety impacts, negative environmental implications, or negatively affect the fundamental rights, for example due to erroneous decisions or wrong or biased outputs generated by the AI system.

Or. en

Proposal for a regulation

~~Recital 59 a (new)~~

Article 2 – paragraph 5 b (new)

Text proposed by the Commission

Amendment

(59a) Considering the specific nature and potential uses of AI systems which can be addressed to natural persons who are not users or operators, it is important to ensure the protection of certain rights, notably regarding transparency and the provision of information, to end recipients such as patients of healthcare services, students, consumers, etc. The current legislation should aim at providing the appropriate type and degree of transparency as well as the provision of specific information to end recipients and establish a clear difference with users as it can increase the protection and usability of AI systems and components.

5b. This Regulation shall not apply to any research and development activity regarding AI systems in so far as such activity does not lead to or require placing an AI system on the market or putting it into service and is in full respect of approved scientific ethical standards.

Or. en

Proposal for a regulation

~~Recital 72~~

Article 3 – paragraph 1 – point 1

Text proposed by the Commission

Amendment

(1) ‘artificial intelligence system’ (AI system) means software that is developed with one or more of the techniques and approaches listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with;

(72) The objectives of the regulatory sandboxes should be to foster AI innovation by establishing a controlled experimentation and testing environment in the development and pre-marketing phase with a view to ensuring compliance of the innovative AI systems with this Regulation and other relevant Union and Member States legislation; to enhance legal certainty for innovators and the competent authorities’ oversight and understanding of the opportunities, emerging risks and the impacts of AI use, and to accelerate access to markets, including by removing barriers for small and medium enterprises (SMEs) and start-ups. To ensure uniform implementation across the Union and economies of scale, it is appropriate to establish common rules for the regulatory sandboxes’ implementation and a framework for cooperation between the relevant authorities involved in the supervision of the sandboxes. This Regulation should provide the legal basis for the use of personal data collected for other purposes for developing certain AI systems in the public interest within the AI regulatory sandbox, in line with Article 6(4) of Regulation (EU) 2016/679, and Article 6 of Regulation (EU) 2018/1725, and without prejudice to Article 4(2) of Directive (EU) 2016/680. Participants in the sandbox should ensure appropriate safeguards and cooperate with the competent authorities, including by following their guidance and acting expeditiously and in good faith to mitigate any high-risks to safety, health, the environment and fundamental rights that may arise during the development and experimentation in the sandbox. The conduct of the participants in the sandbox should be taken into account when competent authorities decide whether to impose an administrative fine under Article 83(2) of Regulation 2016/679 and Article 57 of Directive 2016/680.

(1) ‘artificial intelligence system’ (AI system) means a system that receives machine- and/or human-based data and inputs to infer how to achieve a given set of human-defined objectives using learning, reasoning or modelling implemented with the techniques and approaches listed in Annex I, and generates outputs in the form of content (generative AI systems), predictions, recommendations.

Or. en

Proposal for a regulation

~~Recital 74~~

Article 3 – paragraph 1 – point 4

Text proposed by the Commission

Amendment

(74) In order to minimise the risks to implementation resulting from lack of knowledge and expertise in the market as well as to facilitate compliance of providers and notified bodies with their obligations under this Regulation, the AI-on demand platform, the European Digital Innovation Hubs and the Testing and Experimentation Facilities established by the Commission and the Member States at national or EU level should possibly contribute to the implementation of this Regulation. Within their respective mission and fields of competence, they may provide in particular technical and scientific support to providers and notified bodies.

(4) ‘user’ means any natural or legal person, public authority, agency or other body using an AI system under its authority, except where the AI system is used in the course of a personal non-professional activity;

(74) In order to minimise the risks to implementation resulting from lack of knowledge and expertise in the market as well as to facilitate compliance of providers and notified bodies with their obligations under this Regulation, the AI-on demand platform, the European Digital Innovation Hubs, the European Institute of Innovation and Technology, and the Testing and Experimentation Facilities established by the Commission and the Member States at national or EU level should possibly contribute to the implementation of this Regulation. Within their respective mission and fields of competence, they may provide in particular technical and scientific support to providers and notified bodies.

(4) ‘user’ means any natural or legal person, public authority, agency or other body using an AI system under its authority;

Or. en

Proposal for a regulation

~~Recital 76 a (new)~~

Article 3 – paragraph 1 – point 5 a (new)

Text proposed by the Commission

Amendment

(76a) To ensure that there is a common and consistent approach regarding the deployment and implementation of artificial intelligence systems in the various areas and sectors concerned and to exploit potential synergies and complementarities, the Board should cooperate closely with other relevant sectoral advisory groups established at Union level, such as boards, committees and expert groups, including organisations from the civil society such as NGOs, consumer associations, and industry representatives with competence in areas related to digital technologies or artificial intelligence, such as governance, exchange, access or use and re-use of data, including health data or environmental information, while avoiding duplication of work.

(5a) ‘product manufacturer’ means a manufacturer within the meaning of any of the Union harmonisation legislation listed in Annex II;

Or. en

Proposal for a regulation

Article 3 – paragraph 1 – point 444 a– ~~(new)~~introductory part

Text proposed by the Commission

Amendment

~~(4a) ‘end recipient’ means any natural or legal person, other than an operator, to whom the output of an AI system is intended or to whom that output is provided;~~

(44) ‘serious incident’ means any incident that directly or indirectly leads, might have led or might lead to any of the following:

(44) ‘serious incident’ means any incident or malfunctioning of an AI system that directly or indirectly leads, might have led or might lead to any of the following:

Or. en

Proposal for a regulation

Article 3 – paragraph 1 – point 1444 a (new)

Text proposed by the Commission

Amendment

(14) ‘safety component of a product or system’ means a component of a product or of a system which fulfils a safety function for that product or system or the failure or malfunctioning of which endangers the health and safety of persons or property;

(44a) 'personal data' means data as defined in point (1) of Article 4 of Regulation (EU) 2016/679;

Or. en

Proposal for a regulation

Article 3 – paragraph 1 – point 1544 b (new)

Text proposed by the Commission

Amendment

(15) ‘instructions for use’ means the information provided by the provider to inform the user of in particular an AI system’s intended purpose and proper use, inclusive of the specific geographical, behavioural or functional setting within which the high-risk AI system is intended to be used;

(44b) ‘non-personal data’ means data other than personal data as defined in point (1) of Article 4 of Regulation (EU) 2016/679;

(15) ‘instructions for use’ means the information provided by the provider to inform the user and end recipient of in particular an AI system’s intended purpose and proper use, inclusive of the specific geographical, behavioural or functional setting within which the high-risk AI system is intended to be used;

Or. en

Proposal for a regulation

Article 35 – paragraph 1 – point 34d – point ii

Text proposed by the Commission

Amendment

~~(34) ‘emotion recognition system’ means an AI system for the purpose of identifying or inferring emotions or intentions of natural persons on the basis of their biometric data;~~

(ii) the prevention of a specific, substantial and imminent threat to the life or physical safety of natural persons or of a terrorist attack;

~~(34) ‘emotion recognition system’ means an AI system for the purpose of identifying or inferring emotions or intentions of natural persons on the basis of their biometric or neurological data;~~

(ii) the prevention of a specific and substantial and imminent threat to the critical infrastructure, life, health or physical safety of natural persons or of a terrorist attack;

Or. en

Proposal for a regulation

Article 76 – paragraph 1 – ~~point~~introductory bpart

Text proposed by the Commission

Amendment

(b) the AI systems pose a risk of harm to the health and safety, or a risk of adverse impact on fundamental rights, that is, in respect of its severity and probability of occurrence, equivalent to or greater than the risk of harm or of adverse impact posed by the high-risk AI systems already referred to in Annex III.

1. Irrespective of whether an AI system is placed on the market or put into service independently from the products referred to in points (a) and (b), that AI system shall be considered high-risk where both of the following conditions are fulfilled:

(b) the AI systems pose a risk of harm to the health, safety of persons, or a risk of adverse impact on fundamental rights or the environment, that is, in respect of its severity and probability of occurrence, equivalent to or greater than the risk of harm or of adverse impact posed by the high-risk AI systems already referred to in Annex III.

1. An AI system that is itself a product covered by the Union harmonisation legislation listed in Annex II shall be considered as high risk if it is required to undergo a third-party conformity assessment with a view to the placing on the market or putting into service of that product pursuant to the above mentioned legislation.

2. An AI system intended to be used as a safety component of a product covered by the legislation referred to in paragraph 1 shall be considered as high risk if it is required to undergo a third-party conformity assessment with a view to the placing on the market or putting into service of that product pursuant to above mentioned legislation. This provision shall apply irrespective of whether the AI system is placed on the market or put into service independently from the product.

3. AI systems referred to in Annex III shall be considered high-risk.

Or. en

Justification

The amendment on Article 6 is strictly to improve the language to ensure greater clarity. The content remains the same.

Amendment 40

Proposal for a regulation

Article 76 – paragraph 21 – ~~introductory~~point ~~part~~a

Text proposed by the Commission

Amendment

2. When assessing for the purposes of paragraph 1 whether an AI system poses a risk of harm to the health and safety or a risk of adverse impact on fundamental rights that is equivalent to or greater than the risk of harm posed by the high-risk AI systems already referred to in Annex III, the Commission shall take into account the following criteria:

(a) the AI system is intended to be used as a safety component of a product, or is itself a product, covered by the Union harmonisation legislation listed in Annex II;

2. When assessing for the purposes of paragraph 1 whether an AI system poses a risk of harm to the health and safety or a risk of adverse impact on fundamental rights or the environment that is equivalent to or greater than the risk of harm posed by the high-risk AI systems already referred to in Annex III, the Commission shall take into account the following criteria:

deleted

Or. en

Proposal for a regulation

Article 76 – paragraph 21 – point cb

Text proposed by the Commission

Amendment

(c) the extent to which the use of an AI system has already caused harm to the health and safety or adverse impact on the fundamental rights or has given rise to significant concerns in relation to the materialisation of such harm or adverse impact, as demonstrated by reports or documented allegations submitted to national competent authorities;

(b) the product whose safety component is the AI system, or the AI system itself as a product, is required to undergo a third-party conformity assessment with a view to the placing on the market or putting into service of that product pursuant to the Union harmonisation legislation listed in Annex II.

(c) the extent to which the use of an AI system has already caused harm to the health and safety or adverse impact on the fundamental rights and the environment or has given rise to significant concerns in relation to the materialisation of such harm or adverse impact, as demonstrated by reports or documented allegations submitted to national competent authorities;

deleted

Or. en

Proposal for a regulation

Article 76 – paragraph 2 ~~– point d~~

Text proposed by the Commission

Amendment

~~(d) the potential extent of such harm or such adverse impact, in particular in terms of its intensity and its ability to affect a plurality of persons;~~

2. In addition to the high-risk AI systems referred to in paragraph 1, AI systems referred to in Annex III shall also be considered high-risk.

~~(d) the potential extent of such harm or such adverse impact, in particular in terms of its intensity and its ability to affect a plurality of persons, the environment and biodiversity;~~

deleted

Or. en

Proposal for a regulation

Article 78 – paragraph ~~2 – point e~~1

Text proposed by the Commission

Amendment

(e) the extent to which potentially harmed or adversely impacted persons are dependent on the outcome produced with an AI system, in particular because for practical or legal reasons it is not reasonably possible to opt-out from that outcome;

1. High-risk AI systems shall comply with the requirements established in this Chapter.

(e) the extent to which potentially harmed or adversely impacted persons, including end recipients, are dependent on the outcome produced with an AI system, in particular because for practical or legal reasons it is not reasonably possible to opt-out from that outcome;

1. High-risk AI systems shall comply with the requirements established in this Chapter, taking into account sectoral legislation, harmonised standards and common specifications.

Or. en

Justification

Relevant amendments to ensure sectoral standards and specifications are respected, something key for the transport sector.

Amendment 44

Proposal for a regulation

Article 79 – paragraph 2 – point ga

Text proposed by the Commission

Amendment

~~(g) the extent to which the outcome produced with an AI system is easily reversible, whereby outcomes having an impact on the health or safety of persons shall not be considered as easily reversible;~~

(a) identification and analysis of the known and foreseeable risks associated with each high-risk AI system;

(g) the extent to which the outcome produced with an AI system is easily reversible, whereby outcomes having an impact on the health or safety of persons, the environment or biodiversity, shall not be considered as easily reversible;

(a) identification and analysis of the known and foreseeable risks associated with each high-risk AI system that might occur to health, safety and fundamental rights of persons in view of the intended purpose of or misuse of the high-risk AI system.

Or. en

Proposal for a regulation

Article 910 – paragraph ~~4 – subparagraph 1~~3

Text proposed by the Commission

Amendment

The risk management measures referred to in paragraph 2, point (d) shall be such that any residual risk associated with each hazard as well as the overall residual risk of the high-risk AI systems is judged acceptable, provided that the high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse. Those residual risks shall be communicated to the user.

3. Training, validation and testing data sets shall be relevant, representative, free of errors and complete. They shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons on which the high-risk AI system is intended to be used. These characteristics of the data sets may be met at the level of individual data sets or a combination thereof.

3. Training, validation and testing data sets shall be relevant, representative, and to the best extent possible free of errors and complete. They shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons on which the high-risk AI system is intended to be used. These characteristics of the data sets may be met at the level of individual data sets or a combination thereof.

Or. en

Proposal for a regulation

Article 1011 – paragraph 31 – subparagraph 1

Text proposed by the Commission

Amendment

The technical documentation shall be drawn up in such a way to demonstrate that the high-risk AI system complies with the requirements set out in this Chapter and provide national competent authorities and notified bodies with all the necessary information to assess the compliance of the AI system with those requirements. It shall contain, at a minimum, the elements set out in Annex IV.

3. Training, validation and testing data sets shall be relevant, representative, free of errors and complete. They shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons on which the high-risk AI system is intended to be used, including end recipients. These characteristics of the data sets may be met at the level of individual data sets or a combination thereof.

Or. en

Proposal for a regulation

Article 1014 – paragraph 41

Text proposed by the Commission

Amendment

4. Training, validation and testing data sets shall take into account, to the extent required by the intended purpose, the characteristics or elements that are particular to the specific geographical, behavioural or functional setting within which the high-risk AI system is intended to be used.

1. High-risk AI systems shall be designed and developed in such a way, including with appropriate human-machine interface tools, that they can be effectively overseen by natural persons during the period in which the AI system is in use.

4. Training, validation and testing data sets shall take into account, to the extent required by the intended purpose, the characteristics or elements that are particular to the specific geographical, environmental, behavioural or functional setting within which the high-risk AI system is intended to be used.

Or. en

Justification

The degree of human oversight should be adapted to the specific risks, the level of automation, context and intended use to avoid hindering automated processes in transport. The addition is important for the sector as human oversight might not always be the preferable option to guarantee safety.

Amendment 48

Proposal for a regulation

~~Article 13 – title~~

Article 14 – paragraph 4 – point e a (new)

Text proposed by the Commission

Amendment

~~Transparency and provision of information to users~~

(ea) be able to comprehend when a high risk AI system decision is preferable to human oversight.

~~Transparency and provision of information to users and end recipients~~

Or. en

Proposal for a regulation

Article 1315 – paragraph 1

Text proposed by the Commission

Amendment

1. High-risk AI systems shall be designed and developed in such a way ~~to ensure~~ that ~~their operation is sufficiently transparent to enable users~~they toachieve, ~~interpret~~in the ~~system’s output and use it appropriately. An appropriate type and degree~~light of ~~transparency shall be ensured, with a view to achieving~~their ~~compliance~~intended ~~with~~purpose, ~~the~~an ~~relevant~~appropriate ~~obligations~~level of ~~the~~accuracy, ~~user~~robustness and ~~of the~~cybersecurity, ~~provider~~and ~~set~~perform ~~out~~consistently in ~~Chapter~~those 3respects ofthroughout ~~this~~their ~~Title.~~lifecycle.

1. High-risk AI systems shall be designed and developed in such a way ~~to ensure~~ that ~~their operation is sufficiently transparent to enable users and end recipients~~they toachieve, ~~interpret~~in the ~~system’s output and use it appropriately. An appropriate type and degree~~light of ~~transparency shall be ensured, with a view to achieving~~their ~~compliance~~intended ~~with~~purpose, the ~~relevant~~highest ~~obligations~~level of ~~the user, end~~accuracy, ~~recipient~~robustness and ofcybersecurity ~~the~~possible, ~~provider~~and ~~set~~perform ~~out~~consistently in ~~Chapter~~those 3respects ofthroughout ~~this~~their ~~Title.~~lifecycle.

Or. en

Proposal for a regulation

Article 1315 – paragraph 3 a– ~~(new)~~subparagraph 2

Text proposed by the Commission

Amendment

~~3a.~~ High-risk AI systems ~~shall~~that becontinue ~~designed,~~to ~~developed~~learn ~~and~~after ~~used~~being placed on the market or put into service shall be developed in such a way to ensure that ~~the~~possibly biased outputs ~~are~~due ~~sufficiently~~to ~~transparent,~~outputs ~~relevant,~~used ~~accessible~~as ~~and~~an ~~comprehensible~~input tofor ~~the~~future ~~end~~operations ~~recipients~~(‘feedback inloops’) ~~accordance~~are duly addressed with ~~the~~appropriate ~~intended~~mitigation ~~purpose.~~measures.

High-risk AI systems that continue to learn after being placed on the market or put into service shall be developed in such a way to ensure that possibly biased outputs influencing an input for future operations (‘feedback loops’) are duly addressed with appropriate mitigation measures.

Or. en

Proposal for a regulation

Article 1428 – paragraph 21 – point c a (new)

Text proposed by the Commission

Amendment

2. Human oversight shall aim at preventing or minimising the risks to health, safety or fundamental rights that may emerge when a high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, in particular when such risks persist notwithstanding the application of other requirements set out in this Chapter.

(ca) they modify the intended purpose of an AI system which is not high-risk and is already placed on the market or put into service, in a way which makes the modified system a high-risk AI system.

2. Human oversight shall aim at preventing or minimising the risks to health, safety or fundamental rights or the environment that may emerge when a high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, in particular when such risks persist notwithstanding the application of other requirements set out in this Chapter.

Or. en

Proposal for a regulation

Article 5340 – paragraph 31 a (new)

Text proposed by the Commission

Amendment

3. The AI regulatory sandboxes shall not affect the supervisory and corrective powers of the competent authorities. Any significant risks to health and safety and fundamental rights identified during the development and testing of such systems shall result in immediate mitigation and, failing that, in the suspension of the development and testing process until such mitigation takes place.

2. When issuing a standardisation request to European standardisation organisations in accordance with Article 10 of Regulation 1025/2012, the Commission shall specify that standards are coherent, easy to implement and drafted in such a way that they aim to fulfil in particular the following objectives:

3. The AI regulatory sandboxes shall not affect the supervisory and corrective powers of the competent authorities. Any significant risks to health and safety, fundamental rights and the environment, identified during the development and testing of such systems shall result in immediate mitigation and, failing that, in the suspension of the development and testing process until such mitigation takes place.

a) ensure that AI systems placed on the market or put into service in the Union are safe and respect Union values and strengthen the Union's digital sovereignty;

b) promote investment and innovation in AI, as well as competitiveness and growth of the Union market;

c) enhance multi-stakeholder governance, representative of all relevant European stakeholders (e.g. industry, SMEs, civil society, researchers).

d) contribute to strengthening global cooperation on standardisation in the field of AI that is consistent with Union values and interests.

The Commission shall request the European standardisation organisations to provide evidence of their best efforts to fulfil the above objectives.

Or. en

Justification

The Artificial Intelligence Act must promote the uptake of international standards for the transport sector to ensure innovation is promoted and the EU's industry global competitiveness supported.

Amendment 53

Proposal for a regulation

Article 5341 – paragraph 41

Text proposed by the Commission

Amendment

4. Participants in the AI regulatory sandbox shall remain liable under applicable Union and Member States liability legislation for any harm inflicted on third parties as a result from the experimentation taking place in the sandbox.

1. Where harmonised standards referred to in Article 40 do not exist or where the Commission considers that the relevant harmonised standards are insufficient or that there is a need to address specific safety or fundamental right concerns, the Commission may, by means of implementing acts, adopt common specifications in respect of the requirements set out in Chapter 2 of this Title. Those implementing acts shall be adopted in accordance with the examination procedure referred to in Article 74(2).

4. Participants in the AI regulatory sandbox shall remain liable under applicable Union and Member States liability legislation for any harm inflicted on third parties or the environment as a result from the experimentation taking place in the sandbox.

1. Where harmonised standards referred to in Article 40 do not exist or where the Commission considers that the relevant harmonised standards are insufficient or that there is a need to address specific safety or fundamental right concerns, the Commission may, after consulting the AI Board referred to in Article 56 and the responsable authorities and organizations for a given sector, by means of implementing acts, adopt common specifications in respect of the requirements set out in Chapter 2 of this Title. Those implementing acts shall be adopted in accordance with the examination procedure referred to in Article 74(2).

Or. en

Proposal for a regulation

Article 5441 – paragraph ~~1 – point a – point ii~~2

Text proposed by the Commission

Amendment

~~(ii) public safety and public health, including disease prevention, control and treatment;~~

2. The Commission, when preparing the common specifications referred to in paragraph 1, shall gather the views of relevant bodies or expert groups established under relevant sectorial Union law.

~~(ii) public safety and public health, including disease prevention, diagnosis, control and treatment;~~

2. When preparing the common specifications referred to in paragraph 1, the Commission shall fulfil the objectives referred of Article 40(2) and gather the views of relevant bodies or expert groups established under relevant sectorial Union law.

Or. en

Proposal for a regulation

Article 54 – paragraph 1 – point a – point iii a (new)

Text proposed by the Commission

Amendment

~~(iii) a high level of protection and improvement of the quality of the environment;~~

(iiia) safety and resilience of transport systems, infrastructure and networks.

~~(iii) a high level of protection and improvement of the quality of the environment, protection of biodiversity as well as climate change mitigation and adaptation;~~

Or. en

Proposal for a regulation

~~Article 56 – paragraph 2 – point b~~

Article 55 – title

Text proposed by the Commission

Amendment

(b) coordinate and contribute to guidance and analysis by the Commission and the national supervisory authorities and other competent authorities on emerging issues across the internal market with regard to matters covered by this Regulation;

Measures for small-scale providers and users

(b) coordinate and contribute to guidance and analysis by the Commission and the national supervisory authorities as well as advisory and expert groups, including organisations from the civil society such as NGOs, consumer associations, and industry representatives and other competent authorities on emerging issues across the internal market with regard to matters covered by this Regulation;

Measures for SMEs and users

Or. en

Proposal for a regulation

Article 5755 – paragraph 1 – point a

Text proposed by the Commission

Amendment

1. The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority, and the European Data Protection Supervisor. Other national authorities may be invited to the meetings, where the issues discussed are of relevance for them.

(a) provide small-scale providers and start-ups with priority access to the AI regulatory sandboxes to the extent that they fulfil the eligibility conditions;

1. The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority, and the European Data Protection Supervisor. Other national authorities including those which are members of relevant advisory and expert groups at Union level, may be invited to the meetings, where the issues discussed are of relevance for them.

(a) provide SMEs and start-ups with priority access to the AI regulatory sandboxes to the extent that they fulfil the eligibility conditions;

Or. en

Proposal for a regulation

Article 5755 – paragraph 41 – point b

Text proposed by the Commission

Amendment

4. The Board may invite external experts and observers to attend its meetings and may hold exchanges with interested third parties to inform its activities to an appropriate extent. To that end the Commission may facilitate exchanges between the Board and other Union bodies, offices, agencies and advisory groups.

(b) organise specific awareness raising activities about the application of this Regulation tailored to the needs of the small-scale providers and users;

4. The Board may invite external experts and observers to attend its meetings and may hold exchanges with interested third parties including organisations from the civil society such as NGOs, consumer associations and industry representatives to inform its activities to an appropriate extent. To that end the Commission may facilitate exchanges between the Board and other relevant Union bodies, offices, agencies and expert advisory groups.

(b) organise specific awareness raising activities about the application of this Regulation tailored to the needs of SMEs and users;

Or. en

Proposal for a regulation

Article 5855 – paragraph 1 – point c ~~a (new)~~

Text proposed by the Commission

Amendment

(ca) ensure that there is a common and consistent approach among the different advisory and expert groups established at Union level on matters covered by this Regulation or related to artificial intelligence systems.

(c) where appropriate, establish a dedicated channel for communication with small-scale providers and user and other innovators to provide guidance and respond to queries about the implementation of this Regulation.

(c) where appropriate, establish a dedicated channel for communication with SMEs and user and other innovators to provide guidance and respond to queries about the implementation of this Regulation.

Or. en

Proposal for a regulation

Article 5955 – paragraph 12 a (new)

Text proposed by the Commission

Amendment

1. National competent authorities shall be established or designated by each Member State for the purpose of ensuring the application and implementation of this Regulation. National competent authorities shall be organised so as to safeguard the objectivity and impartiality of their activities and tasks.

2a. Where appropriate, Member States shall find synergies and cooperate with relevant instruments funded by EU programmes, such as the European Digital Innovation Hubs.

Or. en

Proposal for a regulation

Article 5975 – paragraph 41

Regulation (EC) No 300/2008

Article 4 – paragraph 3

Text proposed by the Commission

Amendment

4. Member States shall ensure that national competent authorities are provided with adequate financial and human resources to fulfil their tasks under this Regulation. In particular, national competent authorities shall have a sufficient number of personnel permanently available whose competences and expertise shall include an in-depth understanding of artificial intelligence technologies, data and data computing, fundamental rights, health and safety risks and knowledge of existing standards and legal requirements.

When adopting detailed measures related to technical specifications and procedures for approval and use of security equipment concerning Artificial Intelligence systems in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Chapter 2, Title III of that Regulation shall be taken into account.”

4. Member States shall ensure that national competent authorities are provided with adequate financial and human resources to fulfil their tasks under this Regulation. In particular, national competent authorities shall have a sufficient number of personnel permanently available whose competences and expertise shall include an in-depth understanding of artificial intelligence technologies, data and data computing, fundamental rights, health and safety risks, environmental risks and knowledge of existing standards and legal requirements.

When adopting detailed measures related to technical specifications and procedures for approval and use of security equipment concerning Artificial Intelligence systems in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, without interfering with existing governance, the requirements set out in Chapter 2, Title III of that Regulation shall be taken into account.”

Or. en

Proposal for a regulation

Article 6276 – paragraph 1 ~~– subparagraph 1~~

Regulation (EU) No 167/2013

Article 17 – paragraph 5

Text proposed by the Commission

Amendment

Providers of high-risk AI systems placed on the Union market shall report any serious incident or any malfunctioning of those systems which constitutes a breach of obligations under Union law intended to protect fundamental rights to the market surveillance authorities of the Member States where that incident or breach occurred.

When adopting delegated acts pursuant to the first subparagraph concerning artificial intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Providers of high-risk AI systems placed on the Union market shall report any serious incident or any malfunctioning of those systems which constitutes a breach of obligations under Union law intended to protect health, safety, fundamental rights and the environment to the market surveillance authorities of the Member States where that incident or breach occurred.

When adopting delegated acts pursuant to the first subparagraph concerning artificial intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, without interfering with existing governance, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Or. en

Proposal for a regulation

Article 6578 – paragraph 1

Directive 2014/90/EU

Article 8 – paragraph 4

Text proposed by the Commission

Amendment

1. AI systems presenting a risk shall be understood as a product presenting a risk defined in Article 3, point 19 of Regulation (EU) 2019/1020 insofar as risks to the health or safety or to the protection of fundamental rights of persons are concerned.

4. “For Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, when carrying out its activities pursuant to paragraph 1 and when adopting technical specifications and testing standards in accordance with paragraphs 2 and 3, the Commission shall take into account the requirements set out in Title III, Chapter 2 of that Regulation.

4. “For Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, when carrying out its activities pursuant to paragraph 1 and when adopting technical specifications and testing standards in accordance with paragraphs 2 and 3, and without interfering with existing governance, the Commission shall take into account the requirements set out in Title III, Chapter 2 of that Regulation.

Or. en

Proposal for a regulation

Article 6779 – paragraph 1

Directive (EU) 2016/797

Article 5 – paragraph 12

Text proposed by the Commission

Amendment

1. Where, having performed an evaluation under Article 65, the market surveillance authority of a Member State finds that although an AI system is in compliance with this Regulation, it presents a risk to the health or safety of persons, to the compliance with obligations under Union or national law intended to protect fundamental rights or to other aspects of public interest protection, it shall require the relevant operator to take all appropriate measures to ensure that the AI system concerned, when placed on the market or put into service, no longer presents that risk, to withdraw the AI system from the market or to recall it within a reasonable period, commensurate with the nature of the risk, as it may prescribe.

12. “When adopting delegated acts pursuant to paragraph 1 and implementing acts pursuant to paragraph 11 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

1. Where, having performed an evaluation under Article 65, the market surveillance authority of a Member State finds that although an AI system is in compliance with this Regulation, it presents a risk to the health or safety of persons or the environment, to the compliance with obligations under Union or national law intended to protect fundamental rights or to other aspects of public interest protection, it shall require the relevant operator to take all appropriate measures to ensure that the AI system concerned, when placed on the market or put into service, no longer presents that risk, to withdraw the AI system from the market or to recall it within a reasonable period, commensurate with the nature of the risk, as it may prescribe.

12. “When adopting delegated acts pursuant to paragraph 1 and implementing acts pursuant to paragraph 11 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, without interfering with existing governance, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Or. en

Proposal for a regulation

~~Annex~~Article ~~III~~80 – paragraph 1 ~~– point 5 – point a~~

Regulation (EU) 2018/858

Article 5 – paragraph 4

Text proposed by the Commission

Amendment

(a) AI systems intended to be used by public authorities or on behalf of public authorities to evaluate the eligibility of natural persons for public assistance benefits and services, as well as to grant, reduce, revoke, or reclaim such benefits and services;

4. “When adopting delegated acts pursuant to paragraph 3 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council *, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

(a) AI systems intended to be used by public authorities or on behalf of public authorities to evaluate the eligibility of natural persons for public assistance benefits and services, including healthcare services, as well as to grant, reduce, revoke, or reclaim such benefits and services;

4. “When adopting delegated acts pursuant to paragraph 3 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council *, without interfering with existing governance, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Or. en

Proposal for a regulation

~~Annex~~Article IV81 – paragraph 1 – point 1 ~~– point g~~

Regulation (EU) 2018/1139

Article 17 – paragraph 3

Text proposed by the Commission

Amendment

~~(g) instructions of use for the user and, where applicable installation instructions;~~

3. “Without prejudice to paragraph 2, when adopting implementing acts pursuant to paragraph 1 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

~~(g) instructions of use for the user and end recipient and, where applicable installation instructions;~~

3. “Without prejudice to paragraph 2, and to the certification, oversight and enforcement system referred to in Article 62 of this Regulation, when adopting implementing acts pursuant to paragraph 1 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

__________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).

Or. en

Proposal for a regulation

~~Annex~~Article IV81 – paragraph 1 – point 32

Regulation (EU) 2018/1139

Article 19 – paragraph 4

Text proposed by the Commission

Amendment

3. Detailed information about the monitoring, functioning and control of the AI system, in particular with regard to: its capabilities and limitations in performance, including the degrees of accuracy for specific persons or groups of persons on which the system is intended to be used and the overall expected level of accuracy in relation to its intended purpose; the foreseeable unintended outcomes and sources of risks to health and safety, fundamental rights and discrimination in view of the intended purpose of the AI system; the human oversight measures needed in accordance with Article 14, including the technical measures put in place to facilitate the interpretation of the outputs of AI systems by the users; specifications on input data, as appropriate;

4. When adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

3. Detailed information about the monitoring, functioning and control of the AI system, in particular with regard to: its capabilities and limitations in performance, including the degrees of accuracy for specific persons or groups of persons on which the system is intended to be used and the overall expected level of accuracy in relation to its intended purpose; the foreseeable unintended outcomes and sources of risks to health and safety, fundamental rights, the environment and discrimination in view of the intended purpose of the AI system; the human oversight measures needed in accordance with Article 14, including the technical measures put in place to facilitate the interpretation of the outputs of AI systems by the users; specifications on input data, as appropriate;

4. Without prejudice to the certification, oversight and enforcement system referred to in Article 62 of this Regulation, when adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Or. en

Amendment 68

Proposal for a regulation

Article 81 – paragraph 1 – point 3

Regulation (EU) 2018/1139

Article 43 – paragraph 4

Text proposed by the Commission

Amendment

4. When adopting implementing acts pursuant to paragraph 1 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

4. Without prejudice to the certification, oversight and enforcement system referred to in Article 62 of this Regulation, when adopting implementing acts pursuant to paragraph 1 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Or. en

Amendment 69

Proposal for a regulation

Article 81 – paragraph 1 – point 4

Regulation (EU) 2018/1139

Article 47 – paragraph 3

Text proposed by the Commission

Amendment

3. When adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

3. Without prejudice to the certification, oversight and enforcement system referred to in Article 62 of this Regulation, when adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Or. en

Amendment 70

Proposal for a regulation

Article 81 – paragraph 1 – point 5

Regulation (EU) 2018/1139

Article 57 – paragraph 3

Text proposed by the Commission

Amendment

When adopting those implementing acts concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Without prejudice to the certification, oversight and enforcement system referred to in Article 62 of this Regulation, when adopting those implementing acts concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account."

Or. en

Amendment 71

Proposal for a regulation

Article 81 – paragraph 1 – point 6

Regulation (EU) 2018/1139

Article 58 – paragraph 3

Text proposed by the Commission

Amendment

3. When adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] , the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account..

Or. en

Amendment 72

Proposal for a regulation

Article 82 – paragraph 1

Regulation (EU) 2019/2144

Article 11 – paragraph 3

Text proposed by the Commission

Amendment

3. “When adopting the implementing acts pursuant to paragraph 2, concerning artificial intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

3. “When adopting the implementing acts pursuant to paragraph 2, concerning artificial intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, without interfering with existing governance, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Or. en

Amendment 73

Proposal for a regulation

Article 84 – paragraph 7 a (new)

Text proposed by the Commission

Amendment

7a. Any relevant future delegated or implementing acts to regulations listed in Annex II, section B, introducing mandatory requirements for High-Risk AI systems laid down in this Regulation, shall take into account the regulatory specificities of each sector and shall not overlap with existing governance, conformity assessment, and enforcement mechanisms and authorities established therein.

Or. en

ANNEX: LIST OF ENTITIES OR PERSONS FROM WHOM THE RAPPORTEUR FOR THE OPINION HAS RECEIVED INPUT

 BEUC

 ETF

 Google

 Amazon

 Airbus

 Hitachi

 DG MOVE

 DG Connect

 EASA

 AMCHAM

 ACEA

 CLEPA

 Ericsson