Sittings · Compare

What changed

From · opinion parliamentary committee · 2022-09-12 JURI-AD-719827 on the proposal for a regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain Union Legislative Acts

→

To · opinion parliamentary committee · 2022-07-12 TRAN-AD-730085 on the proposal for a regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending certain union legislative acts

+400 added · −420 removed · 115 modified paragraphs hide unchanged text

PA_Legam

SHORT JUSTIFICATION

The Rapporteur supports the development of an EU legislative framework on Artificial Intelligence, a necessary step to ensure the Union becomes leader in the field, all while upholding our European values. The horizontal risk-based approach of the Artificial Intelligence Act will create a system in which rules will apply exclusively to applications of AI systems where threats may occur. This selective approach will allow the Union to continue developing its AI sector without hindering technological progress and digital transformation.

Transport is a sector posed to benefit greatly from the integration of AI systems in its every day operations and logistics. The application of AI systems can help achieve the EU’s safety, environmental and, in some instances, societal objectives for the sector. The Union must therefore aim at removing those existing barriers that hinder development and investment, first among all, legal fragmentation and uncertainty, which hurts business and customers.

The key elements of the Rapporteur’s report are the following:

 Ensuring the AI Act does not overlap with sectoral legislation by imposing double/conflicting obligations on transport actors;

 Promoting the development of, and upholding, international standards, that are particularly important for the transport sector;

 Fostering Research and Innovation to ensure the EU’s transport sector develops its own know-how in the implementation of AI, while upholding the highest ethical standards.

The general approach to the opinion by the Rapporteur has been to give a voice to the transport sector on the AI Act. For this reason, amendments had to be carefully drafted within the scope of the TRAN Committee, a complicated task when dealing with harmonized legislation. Still, the Rapporteur believes that in instances where horizontal provisions clearly affect transport, the TRAN Committee should have a say. In addition, some amendments have been included to clarify the key messages of the Commission’s report, such as in the case of the definition of AI systems and High-Risk, which are both essential for transport.

Following the consultation and research phase for this work, three clear issues emerged from a TRAN perspective. First, each segment of transport already presents, in different degrees and forms, sectoral legislation, provisions or initiatives to ensure the highest level of safety when it comes to AI system integration. Aviation, road, rail and maritime, all require measures specifically tailored to the sector to ensure the successful management of operations and services, while upholding the highest level of security. Harmonized legislation might fall short of the required measures to guarantee the aforementioned safety in the sector. For this reason, it was paramount to stress in the AI Act the need for sectoral legislation to be respected and prevail, in some instances, over harmonized rules. This was indeed the intention of the Commission as clear from Annex II, Part B. Yet, the text required further detail.

Secondly, due to the international nature of transport, a second key element that emerged from assessment is that international procedures are already in motion to develop global norms and guidelines for the safety of AI systems for each sector. It would be preferable therefore, that such global standards are respected and integrated in EU law and standards. Generally speaking, EU standards are developed through formal agreements between the European Standardization Organizations (ESOs) and International Standardization Organizations, which allows them in practice to work together. Nonetheless, in the interest of preserving the competitiveness of the EU’s transport sector vis-a-vis other regions, the language and provisions in the AI Act had to be strengthened to ensure provisions on AI for transport are respectful of the international norms and standards.

Lastly, Research and Development is the motor for the digitalization of each sector, let alone transport. New harmonized rules on AI must not impede research where such research is limited to controlled environments and the studied AI System is not placed onto the market. For this reason, language is added to clarify the scope in this regard. It must be noted, that the Commission proposal presents measures in support of innovation through specific articles on sandboxes and small-scale providers. Here, however, the Rapporteur amends to ensure Small and Medium Enterprises (SMEs) are able to benefit from such provisions.

Further minor amendments by the Rapporteur cover transparency of algorithms for transport work (simply reflecting Commission’s language in Annex III), an understanding of Human Oversight in the context of transport, and clarification on errors in data sets.

AMENDMENTS

The Committee on ~~Legal~~Transport ~~Affairs~~and Tourism calls on the Committee on the Internal Market and Consumer Protection and the Committee on Civil Liberties, Justice and Home Affairs, as the committees responsible, to take into account the following amendments:

Amendment 1

(1) The purpose of this Regulation is to improve the functioning of the internal market by laying down a uniform legal framework in particular for the development, marketing and use of artificial intelligence in conformity with Union values. This Regulation pursues a number of overriding reasons of public interest, such as a high level of protection of health, safety and fundamental rights, and it ensures the free movement of AI-based goods and services cross-border, thus preventing Member States from imposing restrictions on the development, marketing and use of AI systems, unless explicitly authorised by this Regulation.

(1) The purpose of this Regulation is to improve the functioning of the internal market by laying down a uniform legal framework in particular for the development, marketing and use of artificial intelligence in conformity with Union ~~principles and democratic~~ values. This Regulation pursues a number of overriding reasons of public interest, such as a high level of protection of health, ~~safety~~safety, fundamental rights and ~~fundamental~~the ~~rights,~~environment, and it ensures the free movement of AI-based goods and services cross-border, thus preventing Member States from imposing restrictions on the development, marketing and use of AI systems, unless explicitly authorised by this Regulation.

Amendment 2

Proposal for a regulation

Recital 35

Text proposed by the Commission

Amendment

(3) Artificial intelligence is a fast evolving family of technologies that can contribute to a wide array of economic and societal benefits across the entire spectrum of industries and social activities. By improving prediction, optimising operations and resource allocation, and personalising digital solutions available for individuals and organisations, the use of artificial intelligence can provide key competitive advantages to companies and support socially and environmentally beneficial outcomes, for example in healthcare, farming, education and training, infrastructure management, energy, transport and logistics, public services, security, justice, resource and energy efficiency, and climate change mitigation and adaptation.

(5) A Union legal framework laying down harmonised rules on artificial intelligence is therefore needed to foster the development, use and uptake of artificial intelligence in the internal market that at the same time meets a high level of protection of public interests, such as health and safety and the protection of fundamental rights, as recognised and protected by Union law. To achieve that objective, rules regulating the placing on the market and putting into service of certain AI systems should be laid down, thus ensuring the smooth functioning of the internal market and allowing those systems to benefit from the principle of free movement of goods and services. By laying down those rules, this Regulation supports the objective of the Union of being a global leader in the development of secure, trustworthy and ethical artificial intelligence, as stated by the European Council33 , and it ensures the protection of ethical principles, as specifically requested by the European Parliament34 .

(3) Artificial intelligence is a fast evolving family of technologies that can contribute to a wide array of economic and societal benefits across the entire spectrum of industries and social activities if developed in accordance with relevant general principles in line with the EU Charter of Fundamental Rights and the values on which the Union is founded. By improving prediction, optimising operations and resource allocation, and personalising digital solutions available for individuals and organisations, the use of artificial intelligence can provide key competitive advantages to companies and support socially and environmentally beneficial outcomes, for example in healthcare, farming, education and training, infrastructure management, energy, transport and logistics, public services, security, justice, resource and energy efficiency, and climate change mitigation and adaptation.

(5) A Union legal framework laying down harmonised rules on artificial intelligence is therefore needed to foster the development, use and uptake of artificial intelligence in the internal market that at the same time meets a high level of protection of public interests, such as health and safety, the environment, and the protection of fundamental rights, as recognised and protected by Union law. To achieve that objective, rules regulating the placing on the market and putting into service of certain AI systems should be laid down, thus ensuring the smooth functioning of the internal market and allowing those systems to benefit from the principle of free movement of goods and services. By laying down those rules, this Regulation supports the objective of the Union of being a global leader in the development of secure, trustworthy and ethical artificial intelligence, as stated by the European Council33 , and it ensures the protection of ethical principles, as specifically requested by the European Parliament34.

__________________

33 European Council, Special meeting of the European Council (1 and 2 October 2020) – Conclusions, EUCO 13/20, 2020, p. 6.

34 European Parliament resolution of 20 October 2020 with recommendations to the Commission on a framework of ethical aspects of artificial intelligence, robotics and related technologies, 2020/2012(INL).

Amendment 3

Proposal for a regulation

~~Recital 4~~

Recital 5 a (new)

Text proposed by the Commission

Amendment

(4) At the same time, depending on the circumstances regarding its specific application and use, artificial intelligence may generate risks and cause harm to public interests and rights that are protected by Union law. Such harm might be material or immaterial.

(5a) Union legislation on artificial intelligence should contribute to the dual green and digital transition. The artificial intelligence can contribute positively to the green transition but also has significant environmental impacts due to the critical raw material required to design and build its infrastructure and microprocessors and the energy used for its development, training, tuning and use. Development and use of AI should therefore be compatible with sustainable environmental resources at all stages of the lifecycle of AI systems. Also, unnecessary data acquisition and processing should be avoided. Moreover, Union legislation on artificial intelligence should be accompanied by actions aimed at addressing the main barriers hindering the digital transformation of the economy. Such measures should focus on education, upskilling and reskilling of workers, fostering investment in research and innovation, and boosting security in the digital sphere in line with initiatives aimed at achieving the targets of the Digital Decade. Digital transformation should occur in a harmonized manner across regions, paying particular attention to less digitally developed areas of the Union.

Amendment 4

Proposal for a regulation

Recital 65 ab (new)

Text proposed by the Commission

Amendment

(6a) Building on the seven key requirements set out by the High-Level Expert Group on Artificial Intelligence, it is important to note that AI systems should respect general principles establishing a high-level framework that promotes a coherent human-centric approach to ethical and trustworthy AI in line with the Charter of Fundamental Rights of the European Union and the values on which the Union is founded, including the protection of fundamental rights, human agency and oversight, technical robustness and safety, privacy and data governance, transparency, non-discrimination and fairness and societal and environmental wellbeing.

(5b) Harmonised Union legislation on artificial intelligence can contribute to create legal certainty and coherence across the Union. However, due to risks associated with passenger and goods transport, the sector has been carefully monitored and regulated to avoid incidents and loss of life. The Union legal framework for transport presents sectoral legislation for the aviation, road, rail and maritime transport. With the progressive integration of AI systems in the sector, new challenges could emerge in risk management. This Regulation should only apply to high risk applications in the transport sector in so far as that they are not already covered by sectoral legislation and where they could have a harmful impact on the environment or health, safety and fundamental rights of persons. Double regulation should therefore be avoided.

Amendment 5

Proposal for a regulation

~~Recital 13~~

Recital 5 c (new)

Text proposed by the Commission

Amendment

(13) In order to ensure a consistent and high level of protection of public interests as regards health, safety and fundamental rights, common normative standards for all high-risk AI systems should be established. Those standards should be consistent with the Charter of fundamental rights of the European Union (the Charter) and should be non-discriminatory and in line with the Union’s international trade commitments.

(5c) The Union aviation sector, for example, through the work of the European Aviation Safety Agency (EASA) and its stakeholders, is gradually developing its own guidance material and rules on the application and security management of AI systems in aviation. In the EASA’s roadmap for AI, AI systems with application to aviation are categorised in three distinct levels, from assistance to human, to human-machine cooperation, to full machine automation. A sector-specific oversight on AI systems laying out rules for the highest-level of safety for aviation while preserving the global competitiveness of Union businesses is needed.

(13) In order to ensure a consistent and high level of protection of public interests as regards health, safety, fundamental rights and the environment, common normative standards for all high-risk AI systems should be established. Those standards should be consistent with the Charter of fundamental rights of the European Union (the Charter), the European Green Deal (The Green Deal) and the Joint Declaration on Digital Rights of the Union (the Declaration) and should be non-discriminatory and in line with the Union’s international commitments.

Amendment 6

Proposal for a regulation

~~Recital 14 a (new)~~

Recital 6

Text proposed by the Commission

Amendment

(14a) For this Regulation to be effective, it is essential to address the issue of the digital divide and, therefore, it should be accompanied by a policy of education, training and awareness as regards these technologies that ensures a sufficient level of AI literacy.

(6) The notion of AI system should be clearly defined to ensure legal certainty, while providing the flexibility to accommodate future technological developments. The definition should be based on the key functional characteristics of the software, in particular the ability, for a given set of human-defined objectives, to generate outputs such as content, predictions, recommendations, or decisions which influence the environment with which the system interacts, be it in a physical or digital dimension. AI systems can be designed to operate with varying levels of autonomy and be used on a stand-alone basis or as a component of a product, irrespective of whether the system is physically integrated into the product (embedded) or serve the functionality of the product without being integrated therein (non-embedded). The definition of AI system should be complemented by a list of specific techniques and approaches used for its development, which should be kept up-to–date in the light of market and technological developments through the adoption of delegated acts by the Commission to amend that list.

(6) The notion of AI system should be clearly defined to ensure legal certainty, while providing the flexibility to accommodate future technological developments. The definition should be based on the key functional characteristics of the software, and possibly also the hardware.1a In particular, for the purpose of this Regulation, AI systems should be intended as having the ability, on the basis of machine- and/or human-based data and inputs, to infer the way to achieve a given set of human-defined objectives through learning, reasoning or modelling and generate specific outputs in the form of content for generative AI systems, as well as predictions, recommendations, or decisions which influence the environment with which the system interacts, be it in a physical or digital dimension. AI systems can be designed to operate with varying levels of autonomy and be used on a stand-alone basis or as a component of a product, irrespective of whether the system is physically integrated into the product (embedded) or serve the functionality of the product without being integrated therein (non-embedded). The definition of AI system should be complemented by a list of specific techniques and approaches used for its development, which should be kept up-to–date in the light of market and technological developments through the adoption of delegated acts by the Commission to amend that list.

__________________

1a https://digital-strategy.ec.europa.eu/en/library/definition-artificial-intelligence-main-capabilities-and-scientific-disciplines

Amendment 7

Proposal for a regulation

Recital 148 ba (new)

Text proposed by the Commission

Amendment

(14b) ‘AI literacy’ refers to skills, knowledge and understanding that allows providers, users and affected persons, taking into account their respective rights and obligations in the context of this Regulation, to make an informed deployment of AI systems, as well as to gain awareness about the opportunities and risks of AI and possible harm it can cause and thereby promote its democratic control. AI literacy should not be limited to learning about tools and technologies, but should also aim to equip providers and users with the notions and skills required to ensure compliance with and enforcement of this Regulation. It is therefore necessary that the Commission, the Member States as well as providers and users of AI systems, in cooperation with all relevant stakeholders, promote the development of a sufficient level of AI literacy, in all sectors of society, for citizens of all ages, including women and girls, and that progress in that regard is closely followed.

(8a) The use of biometrics and high technologies in transport and tourism may vastly benefit user experience and overall safety and security. This Regulation should accompany these developments by setting the highest level of protection, in particular when use of biometrics data is involved, in line with the data protection framework of the Union, while fostering research and investment for the development and deployment of AI systems that can positively contribute to society.

Amendment 8

Proposal for a regulation

~~Recital 15~~

Recital 12 a (new)

Text proposed by the Commission

Amendment

(15) Aside from the many beneficial uses of artificial intelligence, that technology can also be misused and provide novel and powerful tools for manipulative, exploitative and social control practices. Such practices are particularly harmful and should be prohibited because they contradict Union values of respect for human dignity, freedom, equality, democracy and the rule of law and Union fundamental rights, including the right to non-discrimination, data protection and privacy and the rights of the child.

(12a) This Regulation should support research and innovation for the application of AI systems in the transport and tourism sectors while ensuring a high level of protection of public interests, such as health, safety, fundamental rights, the environment and democracy. For this reason, this Regulation should exclude from its scope applications of AI systems developed, applied and assessed in a controlled testing environment, for the sole purpose of evaluating their use and functionality. As regards product oriented research activity by providers, the provisions of this Regulation should apply insofar as such research leads to or entails placing an AI system on the market or putting it into service. All forms of research and development should be conducted in compliance with the highest ethical standards for scientific research.

Amendment 9

Proposal for a regulation

Recital 1613

Text proposed by the Commission

Amendment

(16) The placing on the market, putting into service or use of certain AI systems intended to distort human behaviour, whereby physical or psychological harms are likely to occur, should be forbidden. Such AI systems deploy subliminal components individuals cannot perceive or exploit vulnerabilities of children and people due to their age, physical or mental incapacities. They do so with the intention to materially distort the behaviour of a person and in a manner that causes or is likely to cause harm to that or another person. The intention may not be presumed if the distortion of human behaviour results from factors external to the AI system which are outside of the control of the provider or the user. Research for legitimate purposes in relation to such AI systems should not be stifled by the prohibition, if such research does not amount to use of the AI system in human-machine relations that exposes natural persons to harm and such research is carried out in accordance with recognised ethical standards for scientific research.

(16) The placing on the market, putting into service or use of certain AI systems intended to distort human behaviour, whereby physical or psychological harms are likely to occur, should be forbidden. Such AI systems deploy subliminal components individuals cannot perceive or exploit vulnerabilities of children and people due to their age, physical or mental incapacities. They do so with the intention to materially distort the behaviour of a person and in a manner that causes or is likely to cause harm to that or another person. The intention may not be presumed if the distortion of human behaviour results from factors external to the AI system which are outside of the control of the provider or the user. Research for legitimate purposes in relation to such AI systems should not be stifled by the prohibition, if such research does not amount to use of the AI system in non-supervised human-machine relations that exposes natural persons to harm and such research is carried out in accordance with recognised ethical standards for scientific research. If necessary and in accordance with this Regulation, further flexibilities in order to foster research, and thereby European innovation capacities, should be introduced by Member States.

(13) In order to ensure a consistent and high level of protection of public interests as regards health, safety and fundamental rights and the environment, common normative standards for all high-risk AI systems should be established. Those standards should be consistent with the Charter of fundamental rights of the European Union (the Charter) and should be non-discriminatory and in line with the Union’s international trade commitments. This is of particular importance in the transport sector in order to ensure the highest level of interoperability among transport vehicles, infrastructure and intelligent systems and to guarantee safety and security. The Union and its standards organisations should participate actively in the development of global standards for the different transport modes with a view to align them as much as possible with any applicable European standards and to ensure that they are in compliance with Union law. Regular reviews of this Regulation should take into account updated standards for the transport sector.

Amendment 10

Proposal for a regulation

~~Recital 28~~

Recital 17 a (new)

Text proposed by the Commission

Amendment

(28) AI systems could produce adverse outcomes to health and safety of persons, in particular when such systems operate as components of products. Consistently with the objectives of Union harmonisation legislation to facilitate the free movement of products in the internal market and to ensure that only safe and otherwise compliant products find their way into the market, it is important that the safety risks that may be generated by a product as a whole due to its digital components, including AI systems, are duly prevented and mitigated. For instance, increasingly autonomous robots, whether in the context of manufacturing or personal assistance and care should be able to safely operate and performs their functions in complex environments. Similarly, in the health sector where the stakes for life and health are particularly high, increasingly sophisticated diagnostics systems and systems supporting human decisions should be reliable and accurate. The extent of the adverse impact caused by the AI system on the fundamental rights protected by the Charter is of particular relevance when classifying an AI system as high-risk. Those rights include the right to human dignity, respect for private and family life, protection of personal data, freedom of expression and information, freedom of assembly and of association, and non-discrimination, consumer protection, workers’ rights, rights of persons with disabilities, right to an effective remedy and to a fair trial, right of defence and the presumption of innocence, right to good administration. In addition to those rights, it is important to highlight that children have specific rights as enshrined in Article 24 of the EU Charter and in the United Nations Convention on the Rights of the Child (further elaborated in the UNCRC General Comment No. 25 as regards the digital environment), both of which require consideration of the children’s vulnerabilities and provision of such protection and care as necessary for their well-being. The fundamental right to a high level of environmental protection enshrined in the Charter and implemented in Union policies should also be considered when assessing the severity of the harm that an AI system can cause, including in relation to the health and safety of persons.

(17a) The use of AI in work can be beneficial to both the management and operations of an enterprise, supporting workers in their tasks and improving safety at the workplace. Still, AI systems applied to the management of workers, in particular by digital labour platforms, including in the field of transport, can entail a number of risks such as unjust/unnecessary social scoring, rooted in biased data sets or intrusive surveillance practice which can lead to violation of workers’ and fundamental rights. This Regulation should therefore aim at protecting the rights of transport workers managed with the assistance of AI systems, including those working via digital labour platforms and promote transparency, fairness and accountability in algorithmic management, to ensure that workers have a broad understanding of how algorithms work, which personal data is issued and how their behaviour affects decisions taken by the automated system.

(28) AI systems could produce adverse outcomes to health and safety of persons, in particular when such systems operate as components of products. Consistently with the objectives of Union harmonisation legislation to facilitate the free movement of products in the internal market and to ensure that only safe and otherwise compliant products find their way into the market, it is important that the safety risks that may be generated by a product as a whole due to its digital components, including AI systems, are duly prevented and mitigated. For instance, increasingly autonomous robots, whether in the context of manufacturing or personal assistance and care should be able to safely operate and performs their functions in complex environments. Similarly, in the health sector where the stakes for life and health are particularly high, increasingly sophisticated diagnostics systems and systems supporting human decisions should be reliable and accurate. The extent of the adverse impact caused by the AI system on the fundamental rights protected by the Charter is of particular relevance when classifying an AI system as high-risk. Those rights include the right to human dignity, respect for private and family life, protection of personal data, freedom of expression and information, freedom of assembly and of association, and non-discrimination, education, consumer protection, workers’ rights, gender equality, rights of persons with disabilities, right to an effective remedy and to a fair trial, right of defence and the presumption of innocence, right to good administration, right to protection of intellectual property, cultural diversity. In addition to those rights, it is important to highlight that children have specific rights as enshrined in Article 24 of the EU Charter and in the United Nations Convention on the Rights of the Child (further elaborated in the UNCRC General Comment No. 25 as regards the digital environment), both of which require consideration of the children’s vulnerabilities and provision of such protection and care as necessary for their well-being. The fundamental right to a high level of environmental protection enshrined in the Charter and implemented in Union policies should also be considered when assessing the severity of the harm that an AI system can cause, including in relation to the health and safety of persons.

Amendment 11

Proposal for a regulation

Recital 4717 ab (new)

Text proposed by the Commission

Amendment

(47a) Such requirements on transparency and on the explicability of AI decision-making should also help to counter the deterrent effects of digital asymmetry and so-called ‘dark patterns’ targeting individuals and their informed consent.

(17b) In addition, users and individuals should have the right to object to a decision taken solely by an AI system, or relying to a significant degree on the output of an AI system, which produces legal effects concerning them, or similarly significantly affects them.

Amendment 12

Proposal for a regulation

~~Recital 48 a (new)~~

Recital 27

Text proposed by the Commission

Amendment

(48a) Human oversight aims at serving human-centric objectives. The individuals to whom human oversight is assigned should be provided with adequate education and training on the functioning of the AI system, its capabilities to influence or make decisions, the possible harmful effects it can cause, notably on fundamental rights, and its probability of occurrence. The persons in charge of the assignment of these individuals should provide them with the necessary staff and psychological support and authority to exercise their function.

(27) High-risk AI systems should only be placed on the Union market or put into service if they comply with certain mandatory requirements. Those requirements should ensure that high-risk AI systems available in the Union or whose output is otherwise used in the Union do not pose unacceptable risks to important Union public interests as recognised and protected by Union law. AI systems identified as high-risk should be limited to those that have a significant harmful impact on the health, safety and fundamental rights of persons in the Union and such limitation minimises any potential restriction to international trade, if any.

(27) High-risk AI systems should only be placed on the Union market or put into service if they comply with certain mandatory requirements. Those requirements should ensure that high-risk AI systems available in the Union or whose output is otherwise used in the Union do not pose unacceptable risks to important Union public interests as recognised and protected by Union law. AI systems identified as high-risk should be limited to those that have a significant harmful impact on the health, safety and fundamental rights of persons in the Union or the environment and such limitation minimises any potential restriction to international trade, if any.

Amendment 13

Proposal for a regulation

~~Recital 57 a (new)~~

Recital 29

Text proposed by the Commission

Amendment

(57a) AI systems, which have been placed on the market but require further training or the use of a model not provided by the provider should be considered as general purpose AI system. The training of these systems after they have been placed in the market should be considered as adapting them to a specific purpose;

(29) As regards high-risk AI systems that are safety components of products or systems, or which are themselves products or systems falling within the scope of Regulation (EC) No 300/2008 of the European Parliament and of the Council39 , Regulation (EU) No 167/2013 of the European Parliament and of the Council40 , Regulation (EU) No 168/2013 of the European Parliament and of the Council41 , Directive 2014/90/EU of the European Parliament and of the Council42 , Directive (EU) 2016/797 of the European Parliament and of the Council43 , Regulation (EU) 2018/858 of the European Parliament and of the Council44 , Regulation (EU) 2018/1139 of the European Parliament and of the Council45 , and Regulation (EU) 2019/2144 of the European Parliament and of the Council46 , it is appropriate, if required, to amend those acts to ensure that the Commission takes into account, on the basis of the technical and regulatory specificities of each sector, and without overlapping with existing governance, conformity assessment and enforcement mechanisms and authorities established therein, the mandatory requirements for high-risk AI systems laid down in this Regulation when adopting any relevant future delegated or implementing acts on the basis of those acts. Transport sectoral legislation should prevail over this Regulation and it should be ensured that no conflicting overlap exists between this Regulation and other current and upcoming legal acts (i.e. Data Act, ITS Review) to avoid duplication of obligations on providers and manufacturers, which would cause legal uncertainty for business and slow down the uptake of new technologies in the market. This Regulation should also provide for an efficient review mechanism in order to take into account future technological developments and to ensure fair, proportionate and targeted implementation. In order to avoid substantial legal uncertainty, and to ensure that this Regulation applies to all sectors concerned by it without undue delays, those acts should be amended to integrate the provisions of this Regulation no later than 24 months after its entry into force.

__________________

39 Regulation (EC) No 300/2008 of the European Parliament and of the Council of 11 March 2008 on common rules in the field of civil aviation security and repealing Regulation (EC) No 2320/2002 (OJ L 97, 9.4.2008, p. 72).

40 Regulation (EU) No 167/2013 of the European Parliament and of the Council of 5 February 2013 on the approval and market surveillance of agricultural and forestry vehicles (OJ L 60, 2.3.2013, p. 1).

41 Regulation (EU) No 168/2013 of the European Parliament and of the Council of 15 January 2013 on the approval and market surveillance of two- or three-wheel vehicles and quadricycles (OJ L 60, 2.3.2013, p. 52).

42 Directive 2014/90/EU of the European Parliament and of the Council of 23 July 2014 on marine equipment and repealing Council Directive 96/98/EC (OJ L 257, 28.8.2014, p. 146).

43 Directive (EU) 2016/797 of the European Parliament and of the Council of 11 May 2016 on the interoperability of the rail system within the European Union (OJ L 138, 26.5.2016, p. 44).

44 Regulation (EU) 2018/858 of the European Parliament and of the Council of 30 May 2018 on the approval and market surveillance of motor vehicles and their trailers, and of systems, components and separate technical units intended for such vehicles, amending Regulations (EC) No 715/2007 and (EC) No 595/2009 and repealing Directive 2007/46/EC (OJ L 151, 14.6.2018, p. 1).

45 Regulation (EU) 2018/1139 of the European Parliament and of the Council of 4 July 2018 on common rules in the field of civil aviation and establishing a European Union Aviation Safety Agency, and amending Regulations (EC) No 2111/2005, (EC) No 1008/2008, (EU) No 996/2010, (EU) No 376/2014 and Directives 2014/30/EU and 2014/53/EU of the European Parliament and of the Council, and repealing Regulations (EC) No 552/2004 and (EC) No 216/2008 of the European Parliament and of the Council and Council Regulation (EEC) No 3922/91 (OJ L 212, 22.8.2018, p. 1).

46 Regulation (EU) 2019/2144 of the European Parliament and of the Council of 27 November 2019 on type-approval requirements for motor vehicles and their trailers, and systems, components and separate technical units intended for such vehicles, as regards their general safety and the protection of vehicle occupants and vulnerable road users, amending Regulation (EU) 2018/858 of the European Parliament and of the Council and repealing Regulations (EC) No 78/2009, (EC) No 79/2009 and (EC) No 661/2009 of the European Parliament and of the Council and Commission Regulations (EC) No 631/2009, (EU) No 406/2010, (EU) No 672/2010, (EU) No 1003/2010, (EU) No 1005/2010, (EU) No 1008/2010, (EU) No 1009/2010, (EU) No 19/2011, (EU) No 109/2011, (EU) No 458/2011, (EU) No 65/2012, (EU) No 130/2012, (EU) No 347/2012, (EU) No 351/2012, (EU) No 1230/2012 and (EU) 2015/166 (OJ L 325, 16.12.2019, p. 1).

Amendment 14

Proposal for a regulation

~~Recital 57 b (new)~~

Recital 32

Text proposed by the Commission

Amendment

(57b) Open Source software licences allow users to run, copy, distribute, study, change and improve software freely. By default the use of Open Source software in this manner attributes liability to the user, whereas when a provider provides Open Source software commercially under a Software as a Service (SaaS) or Professional Services model, then the provider may retain the liability instead of the user. Research by the European Commission shows that Open Source software contributes between €65bn -€95bn to the European Union’s GDP, and provides significant growth opportunities for the Union economy. Open Source providers should be able to adopt the same economic model for AI systems. Hence, the provisions of this Regulation should not apply to Open Source AI systems until those systems are put into service. To ensure that AI systems cannot be put into service without complying with this Regulation, when an Open Source AI System is put into service, the obligations associated with providers should be transferred to the person putting the system into service.

(32) As regards stand-alone AI systems, meaning high-risk AI systems other than those that are safety components of products, or which are themselves products, it is appropriate to classify them as high-risk if, in the light of their intended purpose, they pose a high risk of harm to the health and safety or the fundamental rights of persons, taking into account both the severity of the possible harm and its probability of occurrence and they are used in a number of specifically pre-defined areas specified in the Regulation. The identification of those systems is based on the same methodology and criteria envisaged also for any future amendments of the list of high-risk AI systems.

(32) As regards stand-alone AI systems, meaning high-risk AI systems other than those that are safety components of products, or which are themselves products, it is appropriate to classify them as high-risk if, in the light of their intended purpose, they pose a high risk of harm to the health and safety or the fundamental rights of persons or the environment, taking into account both the severity of the possible harm and its probability of occurrence and they are used in a number of specifically pre-defined areas specified in the Regulation. The identification of those systems is based on the same methodology and criteria envisaged also for any future amendments of the list of high-risk AI systems.

Amendment 15

Proposal for a regulation

Recital 7334

Text proposed by the Commission

Amendment

(73) In order to promote and protect innovation, it is important that the interests of small-scale providers and users of AI systems are taken into particular account. To this objective, Member States should develop initiatives, which are targeted at those operators, including on awareness raising and information communication. Moreover, the specific interests and needs of small-scale providers shall be taken into account when Notified Bodies set conformity assessment fees. Translation costs related to mandatory documentation and communication with authorities may constitute a significant cost for providers and other operators, notably those of a smaller scale. Member States should possibly ensure that one of the languages determined and accepted by them for relevant providers’ documentation and for communication with operators is one which is broadly understood by the largest possible number of cross-border users.

(73) In order to promote and protect innovation, it is important that the interests of small-scale providers and users of AI systems are taken into particular account. To this objective, Member States should develop initiatives, which are targeted at those operators, including on AI literacy, awareness raising and information communication. Moreover, the specific interests and needs of small-scale providers shall be taken into account when Notified Bodies set conformity assessment fees. Translation costs related to mandatory documentation and communication with authorities may constitute a significant cost for providers and other operators, notably those of a smaller scale. Member States should possibly ensure that one of the languages determined and accepted by them for relevant providers’ documentation and for communication with operators is one which is broadly understood by the largest possible number of cross-border users.

(34) As regards the management and operation of critical infrastructure, it is appropriate to classify as high-risk the AI systems intended to be used as safety components in the management and operation of road traffic and the supply of water, gas, heating and electricity, since their failure or malfunctioning may put at risk the life and health of persons at large scale and lead to appreciable disruptions in the ordinary conduct of social and economic activities. Some examples of critical infrastructure management systems for road covered by Annex III should include traffic management control systems, intelligent transport systems and ICT infrastructure connected transport.

Amendment 16

Proposal for a regulation

Recital 7637

Text proposed by the Commission

Amendment

(76) In order to facilitate a smooth, effective and harmonised implementation of this Regulation a European Artificial Intelligence Board should be established. The Board should be responsible for a number of advisory tasks, including issuing opinions, recommendations, advice or guidance on matters related to the implementation of this Regulation, including on technical specifications or existing standards regarding the requirements established in this Regulation and providing advice to and assisting the Commission on specific questions related to artificial intelligence.

(37) Another area in which the use of AI systems deserves special consideration is the access to and enjoyment of certain essential private and public services and benefits necessary for people to fully participate in society or to improve one’s standard of living. In particular, AI systems used to evaluate the credit score or creditworthiness of natural persons should be classified as high-risk AI systems, since they determine those persons’ access to financial resources or essential services such as housing, electricity, and telecommunication services. AI systems used for this purpose may lead to discrimination of persons or groups and perpetuate historical patterns of discrimination, for example based on racial or ethnic origins, disabilities, age, sexual orientation, or create new forms of discriminatory impacts. Considering the very limited scale of the impact and the available alternatives on the market, it is appropriate to exempt AI systems for the purpose of creditworthiness assessment and credit scoring when put into service by small-scale providers for their own use. Natural persons applying for or receiving public assistance benefits and services from public authorities are typically dependent on those benefits and services and in a vulnerable position in relation to the responsible authorities. If AI systems are used for determining whether such benefits and services should be denied, reduced, revoked or reclaimed by authorities, they may have a significant impact on persons’ livelihood and may infringe their fundamental rights, such as the right to social protection, non-discrimination, human dignity or an effective remedy. Those systems should therefore be classified as high-risk. Nonetheless, this Regulation should not hamper the development and use of innovative approaches in the public administration, which would stand to benefit from a wider use of compliant and safe AI systems, provided that those systems do not entail a high risk to legal and natural persons. Finally, AI systems used to dispatch or establish priority in the dispatching of emergency first response services should also be classified as high-risk since they make decisions in very critical situations for the life and health of persons and their property.

(76) In order to avoid fragmentation and ensure the optimal functioning of the Single Market, it is essential to guarantee an effective and harmonised implementation of this Regulation. To this end, a European Artificial Intelligence Board should be established and entrusted with a number of advisory tasks, including issuing opinions, recommendations, advice or guidance on matters related to the implementation of this Regulation, including on technical specifications or existing standards regarding the requirements established in this Regulation and providing advice to and assisting the Commission on specific questions related to artificial intelligence. However, such a solution might prove not to be sufficient to ensure a fully coherent cross-border action and, therefore, [within three years after the date of application of this Regulation], the Commission should be required to consider whether the creation of an EU Agency is necessary to ensure a consistent application of this Regulation at Union level.

(37) Another area in which the use of AI systems deserves special consideration is the access to and enjoyment of certain essential private and public services and benefits necessary for people to fully participate in society or to improve one’s standard of living. In particular, AI systems used to evaluate the credit score or creditworthiness of natural persons should be classified as high-risk AI systems, since they determine those persons’ access to financial resources or essential services such as housing, electricity, and telecommunication services. AI systems used for this purpose may lead to discrimination of persons or groups and perpetuate historical patterns of discrimination, for example based on racial or ethnic origins, gender, disabilities, age, sexual orientation, or create new forms of discriminatory impacts. Considering the very limited scale of the impact and the available alternatives on the market, it is appropriate to exempt AI systems for the purpose of creditworthiness assessment and credit scoring when put into service by small-scale providers for their own use. Natural persons applying for or receiving public assistance benefits and services from public authorities are typically dependent on those benefits and services and in a vulnerable position in relation to the responsible authorities. If AI systems are used for determining whether such benefits and services should be denied, reduced, revoked or reclaimed by authorities, they may have a significant impact on persons’ livelihood and may infringe their fundamental rights, such as the right to social protection, non-discrimination, human dignity or an effective remedy. Those systems should therefore be classified as high-risk. Nonetheless, this Regulation should not hamper the development and use of innovative approaches in the public administration, which would stand to benefit from a wider use of compliant and safe AI systems, provided that those systems do not entail a high risk to legal and natural persons. Finally, AI systems used to dispatch or establish priority in the dispatching of emergency first response services should also be classified as high-risk since they make decisions in very critical situations for the life and health of persons and their property.

Amendment 17

Proposal for a regulation

~~Recital 76 a (new)~~

Recital 43

Text proposed by the Commission

Amendment

(76a) The Commission should re-establish the High Level Expert Group or a similar body with a new and balanced membership comprising an equal number of experts from SMEs and start-ups, large enterprises, academia and Research, social partners and civil society. This new High Level Expert Group on Trustworthy AI should not only act as advisory body to the Commission but also to the Board. At least every quarter, the new High Level Expert Group on Trustworthy AI must have the chance to share its practical and technical expertise in a special meeting with the Board.

(43) Requirements should apply to high-risk AI systems as regards the quality of data sets used, technical documentation and record-keeping, transparency and the provision of information to users, human oversight, and robustness, accuracy and cybersecurity. Those requirements are necessary to effectively mitigate the risks for health, safety and fundamental rights, as applicable in the light of the intended purpose of the system, and no other less trade restrictive measures are reasonably available, thus avoiding unjustified restrictions to trade.

(43) Requirements should apply to high-risk AI systems as regards the quality of data sets used, technical documentation and record-keeping, transparency and the provision of information to users, human oversight, and robustness, accuracy and cybersecurity. Those requirements are necessary to effectively mitigate the risks for health, safety, fundamental rights and the environment, as applicable in the light of the intended purpose of the system, and no other less trade restrictive measures are reasonably available, thus avoiding unjustified restrictions to trade.

Amendment 18

Proposal for a regulation

Recital 7744

Text proposed by the Commission

Amendment

(44) High data quality is essential for the performance of many AI systems, especially when techniques involving the training of models are used, with a view to ensure that the high-risk AI system performs as intended and safely and it does not become the source of discrimination prohibited by Union law. High quality training, validation and testing data sets require the implementation of appropriate data governance and management practices. Training, validation and testing data sets should be sufficiently relevant, representative and free of errors and complete in view of the intended purpose of the system. They should also have the appropriate statistical properties, including as regards the persons or groups of persons on which the high-risk AI system is intended to be used. In particular, training, validation and testing data sets should take into account, to the extent required in the light of their intended purpose, the features, characteristics or elements that are particular to the specific geographical, behavioural or functional setting or context within which the AI system is intended to be used. In order to protect the right of others from the discrimination that might result from the bias in AI systems, the providers shouldbe able to process also special categories of personal data, as a matter of substantial public interest, in order to ensure the bias monitoring, detection and correction in relation to high-risk AI systems.

(77) Member States hold a key role in the application and enforcement of this Regulation. In this respect, each Member State should designate one or more national competent authorities for the purpose of supervising the application and implementation of this Regulation. In order to increase organisation efficiency on the side of Member States and to set an official point of contact vis-à-vis the public and other counterparts at Member State and Union levels, in each Member State one national authority should be designated as national supervisory authority. In order to facilitate a consistent and coherent implementation of this Regulation, national supervisory authorities should engage in substantial and regular cooperation not only with the Board, but also among themselves to promote the exchange of relevant information and best practices. In this regard and also taking into account that, given the current lack of AI experts, it might be difficult to ensure at national level that the supervisory authorities are provided with adequate human resources to perform their tasks, Member States are also strongly encouraged to consider the possibility of creating transnational entities for the purpose of ensuring joint supervision of the implementation of this Regulation.

(44) High data quality is essential for the performance of many AI systems, especially when techniques involving the training of models are used, with a view to ensure that the high-risk AI system performs as intended and safely and it does not become the source of discrimination prohibited by Union law. High quality training, validation and testing data sets require the implementation of appropriate data governance and management practices. Training, validation and testing data sets should be sufficiently relevant, representative, up to date and, to the best extent possible free of errors and as complete as possible in view of the intended purpose of the system and to ensure the highest level of security. They should also have the appropriate statistical properties, including as regards the persons or groups of persons on which the high-risk AI system is intended to be used. In particular, training, validation and testing data sets should take into account, to the extent required in the light of their intended purpose, the features, characteristics or elements that are particular to the specific geographical, behavioural or functional setting or context within which the AI system is intended to be used. In order to protect the right of others from the discrimination that might result from the bias in AI systems, the providers should ensure that databases contain adequate data on groups which are more vulnerable to discriminatory effects posed by AI, such as people with disabilities, and be able to process also special categories of personal data, as a matter of substantial public interest, in order to ensure the bias monitoring, detection, update, and correction in relation to high-risk AI systems.

Amendment 19

Proposal for a regulation

~~Recital 80 a (new)~~

Recital 47

Text proposed by the Commission

Amendment

(80a) Natural or legal persons affected by decisions made by AI systems which produce legal effects that adversely affect their health, safety, fundamental rights, socio-economic well-being or any other of their rights deriving from the obligations laid down in this Regulation, should be entitled to an explanation of that decision. Such an explanation is to be provided to the affected persons and, therefore, when providing such an explanation, providers and users should duly take into account that the level of expertise and knowledge of the average consumer or citizen regarding AI systems is limited and much lower than the one that they possess. On the other hand, some AI systems cannot provide an explanation for their decisions beyond the initial input data. When AI systems are required to provide an explanation and cannot, they should clearly state that an explanation cannot be provided. This should be taken into account by any administrative, non-administrative or judicial authority dealing with complaints from affected persons.

(47) To address the opacity that may make certain AI systems incomprehensible to or too complex for natural persons, a certain degree of transparency should be required for high-risk AI systems. Users should be able to interpret the system output and use it appropriately. High-risk AI systems should therefore be accompanied by relevant documentation and instructions of use and include concise and clear information, including in relation to possible risks to fundamental rights and discrimination, where appropriate.

(47) To address the opacity that may make certain AI systems incomprehensible to or too complex for natural persons, a certain degree of transparency should be required for high-risk AI systems, in particular when applied to digital labour platforms managing the activities of transport workers. Users should be able to interpret the system output and use it appropriately. Transparency, fairness, accountability and explanability of AI systems can also be a beneficial factor for their uptake by consumers in the market. High-risk AI systems should therefore be accompanied by relevant documentation and instructions of use and include concise and clear information, including in relation to possible risks to fundamental rights and discrimination, where appropriate.

Amendment 20

Proposal for a regulation

~~Recital 85~~

Recital 47 a (new)

Text proposed by the Commission

Amendment

(85) In order to ensure that the regulatory framework can be adapted where necessary, the power to adopt acts in accordance with Article 290 TFEU should be delegated to the Commission to amend the techniques and approaches referred to in Annex I to define AI systems, the Union harmonisation legislation listed in Annex II, the high-risk AI systems listed in Annex III, the provisions regarding technical documentation listed in Annex IV, the content of the EU declaration of conformity in Annex V, the provisions regarding the conformity assessment procedures in Annex VI and VII and the provisions establishing the high-risk AI systems to which the conformity assessment procedure based on assessment of the quality management system and assessment of the technical documentation should apply. It is of particular importance that the Commission carry out appropriate consultations during its preparatory work, including at expert level, and that those consultations be conducted in accordance with the principles laid down in the Interinstitutional Agreement of 13 April 2016 on Better Law-Making 58 . In particular, to ensure equal participation in the preparation of delegated acts, the European Parliament and the Council receive all documents at the same time as Member States’ experts, and their experts systematically have access to meetings of Commission expert groups dealing with the preparation of delegated acts.

(47a) Based on previous experience, it is particularly important to ensure clear requirements and guidelines for interoperability between AI systems both within and amongst different economic sectors, contributing to foster innovation and providing favourable conditions for small and medium enterprises (SMEs).

(85) In order to ensure that the regulatory framework can be adapted where necessary, the power to adopt acts in accordance with Article 290 TFEU should be delegated to the Commission to amend the techniques and approaches referred to in Annex I to define AI systems, the Union harmonisation legislation listed in Annex II, the high-risk AI systems listed in Annex III, the provisions regarding technical documentation listed in Annex IV, the content of the EU declaration of conformity in Annex V, the provisions regarding the conformity assessment procedures in Annex VI and VII and the provisions establishing the high-risk AI systems to which the conformity assessment procedure based on assessment of the quality management system and assessment of the technical documentation should apply. It is of particular importance that the Commission carry out appropriate consultations during its preparatory work, including at expert level, and that those consultations be conducted in accordance with the principles laid down in the Interinstitutional Agreement of 13 April 2016 on Better Law-Making58. These consultations should involve the participation of a balanced selection of stakeholders, including consumer organisations, associations representing affected persons, business representatives from different sectors and of different sizes, trade unions as well as researchers and scientists. In particular, to ensure equal participation in the preparation of delegated acts, the European Parliament and the Council receive all documents at the same time as Member States’ experts, and their experts systematically have access to meetings of Commission expert groups dealing with the preparation of delegated acts.

Amendment 21

Proposal for a regulation

~~Recital 86 a (new)~~

Recital 48

Text proposed by the Commission

Amendment

(86a) Given the rapid technological developments and the required technical expertise in conducting the assessment of high-risk AI systems, the powers delegated to the Commission and the implementing powers conferred on it should be exercised with as much flexibility as possible. The Commission should regularly review Annex III without undue delay while consulting with the relevant stakeholders.

(48) High-risk AI systems should be designed and developed in such a way that natural persons can oversee their functioning. For this purpose, appropriate human oversight measures should be identified by the provider of the system before its placing on the market or putting into service. In particular, where appropriate, such measures should guarantee that the system is subject to in-built operational constraints that cannot be overridden by the system itself and is responsive to the human operator, and that the natural persons to whom human oversight has been assigned have the necessary competence, training and authority to carry out that role.

(48) High-risk AI systems should be designed and developed in such a way that natural persons can oversee their functioning, unless there is clear evidence that it doesn't add value and could even be detrimental to the protection of health, safety and fundamental rights. For this purpose, appropriate human oversight measures should be identified by the provider of the system before its placing on the market or putting into service. In particular, where appropriate, such measures should guarantee that the system is subject to in-built operational constraints that cannot be overridden by the system itself and is responsive to the human operator, and that the natural persons to whom human oversight has been assigned have the necessary competence, training and authority to carry out that role. With regards to the transport sector, the AI system applications should respect the sector-specific legislation in place. When physical security is at stake, Union standards, and where applicable international standards, should determine in which case the possibility for a human operator to take back control should take prevalence over AI system’s decision.

Amendment 22

Proposal for a regulation

~~Article 1 – paragraph 1 – point a~~

Recital 51

Text proposed by the Commission

Amendment

~~(a) harmonised rules for the placing on the market, the putting into service and the use of artificial intelligence systems (‘AI systems’) in the Union;~~

(51) Cybersecurity plays a crucial role in ensuring that AI systems are resilient against attempts to alter their use, behaviour, performance or compromise their security properties by malicious third parties exploiting the system’s vulnerabilities. Cyberattacks against AI systems can leverage AI specific assets, such as training data sets (e.g. data poisoning) or trained models (e.g. adversarial attacks), or exploit vulnerabilities in the AI system’s digital assets or the underlying ICT infrastructure. To ensure a level of cybersecurity appropriate to the risks, suitable measures should therefore be taken by the providers of high-risk AI systems, also taking into account as appropriate the underlying ICT infrastructure.

(a) harmonised rules for the development, placing on the market, the putting into service and the use of human-centric and trustworthy artificial intelligence systems (‘AI systems’) in the Union in compliance with democratic values;

(51) Cybersecurity plays a crucial role in ensuring that AI systems are resilient against attempts to alter their use, behaviour, performance or compromise their security properties by malicious third parties exploiting the system’s vulnerabilities. Cyberattacks against AI systems can leverage AI specific assets, such as training data sets (e.g. data poisoning) or trained models (e.g. adversarial attacks), or exploit vulnerabilities in the AI system’s digital assets or the underlying ICT infrastructure. To ensure a level of cybersecurity appropriate to the risks, suitable measures should therefore be taken by the providers of high-risk AI systems, as well as the notified bodies, competent national authorities and market surveillance authorities accessing the data of providers of high risk AI systems, also taking into account as appropriate the underlying ICT infrastructure.

Amendment 23

Proposal for a regulation

~~Article 1 – paragraph 1 – point d~~

Recital 54

Text proposed by the Commission

Amendment

(d) harmonised transparency rules for AI systems intended to interact with natural persons, emotion recognition systems and biometric categorisation systems, and AI systems used to generate or manipulate image, audio or video content;

(54) The provider should establish a sound quality management system, ensure the accomplishment of the required conformity assessment procedure, draw up the relevant documentation and establish a robust post-market monitoring system. Public authorities which put into service high-risk AI systems for their own use may adopt and implement the rules for the quality management system as part of the quality management system adopted at a national or regional level, as appropriate, taking into account the specificities of the sector and the competences and organisation of the public authority in question.

~~(d) harmonised transparency rules for certain AI systems;~~

(54) The provider should establish a sound quality management system, ensure the accomplishment of the required conformity assessment procedure, draw up the relevant documentation in the language of the Member State concerned and establish a robust post-market monitoring system. All elements, from design to future development, should be made transparent for the user. Public authorities which put into service high-risk AI systems for their own use may adopt and implement the rules for the quality management system as part of the quality management system adopted at a national or regional level, as appropriate, taking into account the specificities of the sector and the competences and organisation of the public authority in question.

Amendment 24

Proposal for a regulation

~~Article 1 – paragraph 1 – point e~~

Recital 59

Text proposed by the Commission

Amendment

~~(e) rules on market monitoring and surveillance.~~

(59) It is appropriate to envisage that the user of the AI system should be the natural or legal person, public authority, agency or other body under whose authority the AI system is operated except where the use is made in the course of a personal non-professional activity.

~~(e) rules on governance, market monitoring, market surveillance and enforcement;~~

(59) It is appropriate to envisage that the user of the AI system should be the natural or legal person, public authority, agency or other body under whose authority the AI system is operated.

Amendment 25

Proposal for a regulation

~~Article 1 – paragraph 1 – point e a (new)~~

Recital 71

Text proposed by the Commission

Amendment

~~(ea) a high level protection of public interests, such as health, safety, fundamental rights and the environment, against potential harms caused by artificial intelligence;~~

(71) Artificial intelligence is a rapidly developing family of technologies that requires novel forms of regulatory oversight and a safe space for experimentation, while ensuring responsible innovation and integration of appropriate safeguards and risk mitigation measures. To ensure a legal framework that is innovation-friendly, future-proof and resilient to disruption, national competent authorities from one or more Member States should be encouraged to establish artificial intelligence regulatory sandboxes and make such regulatory sandboxes widely available throughout the Union, in order to facilitate the development and testing of innovative AI systems under strict regulatory oversight before these systems are placed on the market or otherwise put into service. It is especially important to ensure that SMEs and start-ups can easily access these sandboxes, are actively involved and participate in the development and testing of innovative AI systems, in order to be able to contribute with their knowhow and experience. Their participation should be supported and facilitated.

Amendment 26

Proposal for a regulation

~~Article 1 – paragraph 1 – point e b (new)~~

Recital 72

Text proposed by the Commission

Amendment

(eb) measures in support of innovation with a particular focus on SMEs and start-ups, including but not limited to setting up regulatory sandboxes and targeted measures to reduce the compliance burden on SME’s and start-ups;

(72) The objectives of the regulatory sandboxes should be to foster AI innovation by establishing a controlled experimentation and testing environment in the development and pre-marketing phase with a view to ensuring compliance of the innovative AI systems with this Regulation and other relevant Union and Member States legislation; to enhance legal certainty for innovators and the competent authorities’ oversight and understanding of the opportunities, emerging risks and the impacts of AI use, and to accelerate access to markets, including by removing barriers for small and medium enterprises (SMEs) and start-ups. To ensure uniform implementation across the Union and economies of scale, it is appropriate to establish common rules for the regulatory sandboxes’ implementation and a framework for cooperation between the relevant authorities involved in the supervision of the sandboxes. This Regulation should provide the legal basis for the use of personal data collected for other purposes for developing certain AI systems in the public interest within the AI regulatory sandbox, in line with Article 6(4) of Regulation (EU) 2016/679, and Article 6 of Regulation (EU) 2018/1725, and without prejudice to Article 4(2) of Directive (EU) 2016/680. Participants in the sandbox should ensure appropriate safeguards and cooperate with the competent authorities, including by following their guidance and acting expeditiously and in good faith to mitigate any high-risks to safety and fundamental rights that may arise during the development and experimentation in the sandbox. The conduct of the participants in the sandbox should be taken into account when competent authorities decide whether to impose an administrative fine under Article 83(2) of Regulation 2016/679 and Article 57 of Directive 2016/680.

(72) The objectives of the regulatory sandboxes should be to foster AI innovation by establishing a controlled experimentation and testing environment in the development and pre-marketing phase with a view to ensuring compliance of the innovative AI systems with this Regulation and other relevant Union and Member States legislation; to enhance legal certainty for innovators and the competent authorities’ oversight and understanding of the opportunities, emerging risks and the impacts of AI use, and to accelerate access to markets, including by removing barriers for small and medium enterprises and start-ups, as well as to contribute to achieving the targets on AI as set in the Policy Programme “Path to the Digital Decade". To ensure uniform implementation across the Union and economies of scale, it is appropriate to establish common rules for the regulatory sandboxes’ implementation and a framework for cooperation between the relevant authorities involved in the supervision of the sandboxes. This Regulation should provide the legal basis for the use of personal data collected for other purposes for developing certain AI systems in the public interest within the AI regulatory sandbox, in line with Article 6(4) of Regulation (EU) 2016/679, and Article 6 of Regulation (EU) 2018/1725, and without prejudice to Article 4(2) of Directive (EU) 2016/680. Participants in the sandbox should ensure appropriate safeguards and cooperate with the competent authorities, including by following their guidance and acting expeditiously and in good faith to mitigate any high-risks to safety and fundamental rights that may arise during the development and experimentation in the sandbox. The conduct of the participants in the sandbox should be taken into account when competent authorities decide whether to impose an administrative fine under Article 83(2) of Regulation 2016/679 and Article 57 of Directive 2016/680.

Amendment 27

Proposal for a regulation

~~Article 1 – paragraph 1 – point e c (new)~~

Recital 73

Text proposed by the Commission

Amendment

~~(ec) provisions on the establishment of an independent ‘European Artificial Intelligence Board’ and on its activities supporting the enforcement of this Regulation.~~

(73) In order to promote and protect innovation, it is important that the interests of SMEs and users of AI systems are taken into particular account. To this objective, Member States should develop initiatives, which are targeted at those operators, including on awareness raising and information communication. Moreover, the specific interests and needs of SMEs should be taken into account when Notified Bodies set conformity assessment fees. Translation costs related to mandatory documentation and communication with authorities may constitute a significant cost for providers and other operators, notably those of a smaller scale. Member States should possibly ensure that one of the languages determined and accepted by them for relevant providers’ documentation and for communication with operators is one which is broadly understood by the largest possible number of cross-border users.

Amendment 28

Proposal for a regulation

~~Article 2 – paragraph 1 – point b~~

Recital 76

Text proposed by the Commission

Amendment

~~(b) users of AI systems located within the Union;~~

~~(b) users of AI systems who are located or established within the Union;~~

Amendment 29

Proposal for a regulation

~~Article 2 – paragraph 1 – point c~~

Recital 77 a (new)

Text proposed by the Commission

Amendment

~~(c) providers and users of AI systems that are located in a third country, where the output produced by the system is used in the Union;~~

(77a) To encourage knowledge sharing from best practices, the Commission should organise regular consultative meetings for knowhow exchange between different Member States' national authorities responsible for notification policy.

(c) providers and users of AI systems that are located in a third country, where the output, meaning predictions, recommendations or decisions produced by the system and influencing the environment it interacts with, is used in the Union and puts at risk the environment or the health, safety or fundamental rights of natural persons physically present in the Union, insofar as the provider or user has permitted, is aware or can reasonably expect such a use;

Amendment 30

Proposal for a regulation

Article 21 – paragraph 1 – point ~~c a (new)~~e

Text proposed by the Commission

Amendment

~~(ca) importers, distributors, and authorised representatives of providers of AI systems;~~

(e) rules on market monitoring and surveillance.

(e) rules on market monitoring, market surveillance and governance.

Amendment 31

Proposal for a regulation

Article 21 – paragraph 21 – ~~introductory~~point ~~part~~e a (new)

Text proposed by the Commission

Amendment

2. For high-risk AI systems that are safety components of products or systems, or which are themselves products or systems, falling within the scope of the following acts, only Article 84 of this Regulation shall apply:

(ea) provision to foster and support research and development for innovation.

2. For high-risk AI systems that are safety components of products or systems, or which are themselves products or systems and that fall within the scope of the listed acts in Annex II, section B, only Article 84 of this Regulation shall apply.

~~(a) Regulation (EC) 300/2008;~~

~~(b) Regulation (EU) No 167/2013;~~

~~(c) Regulation (EU) No 168/2013;~~

~~(d) Directive 2014/90/EU;~~

~~(e) Directive (EU) 2016/797;~~

~~(f) Regulation (EU) 2018/858;~~

~~(g) Regulation (EU) 2018/1139;~~

~~(h) Regulation (EU) 2019/2144.~~

Amendment 32

Proposal for a regulation

Article 2 – paragraph 32 – introductory part

Text proposed by the Commission

Amendment

~~3. This Regulation shall not apply to AI systems developed or used exclusively for military purposes.~~

~~deleted~~

2. For AI systems classified as high-risk AI in accordance with Article 6 related to products covered by Union harmonisation legislation listed in Annex II, section B, only Article 84 of this Regulation shall apply:

Amendment 33

Proposal for a regulation

Article 2 – paragraph 32 a– ~~(new)~~point a

Text proposed by the Commission

Amendment

3a. This Regulation shall not affect research, testing and development activities regarding an AI system prior to this system being placed on the market or put into service, provided that these activities are conducted respecting fundamental rights and the applicable Union law. The Commission is empowered to adopt delegated acts in accordance with Article 73 to specify this exemption. The Board shall provide guidance on the governance of research and development pursuant to Article 56 (2) (cc), also aiming at coordinating the way this exemption is put in place by the Commission and the national supervisory authorities.

(a) Regulation (EC) 300/2008;

deleted

Amendment 34

Proposal for a regulation

Article 2 – paragraph 32 b– ~~(new)~~point b

Text proposed by the Commission

Amendment

3b. Title III of this Regulation shall not apply to AI systems that are used in a strictly business-to-business environment and provided that those systems do not pose a risk of harm to the environment, health or safety or a risk of adverse impact on fundamental rights.

(b) Regulation (EU) No 167/2013;

deleted

Amendment 35

Proposal for a regulation

Article 2 – paragraph 42 a– ~~(new)~~point c

Text proposed by the Commission

Amendment

4a. This regulation shall not apply to Open Source AI systems until those systems are put into service or made available on the market in return for payment, regardless of if that payment is for the AI system itself, the provision of the AI system as a service, or the provision of technical support for the AI system as a service.

deleted

Amendment 36

Proposal for a regulation

Article 32 – paragraph 12 – point ~~1 a (new)~~d

Text proposed by the Commission

Amendment

(1a) general purpose AI system' means an AI system that - irrespective of the modality in which it is placed on the market or put into service including as open source software - is intended by the provider to perform generally applicable functions such as image or speech recognition, audio or video generation, pattern detection, question answering, translation or others; a general purpose AI system may be used in a plurality of contexts and may be integrated in a plurality of other AI systems;

(d) Directive 2014/90/EU;

deleted

Amendment 37

Proposal for a regulation

Article 32 – paragraph 12 – point ~~1 b (new)~~e

Text proposed by the Commission

Amendment

~~(1b) ‘open source AI systems’ means AI systems, including test and training data, or trained models, distributed under open licenses.~~

(e) Directive (EU) 2016/797;

deleted

Amendment 38

Proposal for a regulation

Article 32 – paragraph 12 – point 2f

Text proposed by the Commission

Amendment

(2) ‘provider’ means a natural or legal person, public authority, agency or other body that develops an AI system or that has an AI system developed with a view to placing it on the market or putting it into service under its own name or trademark, whether for payment or free of charge;

(f) Regulation (EU) 2018/858;

deleted

Amendment 39

Proposal for a regulation

Article 32 – paragraph 12 – point ~~4 a (new)~~g

Text proposed by the Commission

Amendment

~~(4a) ‘affected person’ means any natural person or a group of persons who are subject to or affected by an AI system~~

(g) Regulation (EU) 2018/1139;

deleted

Amendment 40

Proposal for a regulation

Article 32 – paragraph 12 – point 43h

Text proposed by the Commission

Amendment

~~(43) ‘national competent authority’ means the national supervisory authority, the notifying authority and the market surveillance authority;~~

(h) Regulation (EU) 2019/2144.

deleted

Proposal for a regulation

Article 32 – paragraph ~~1 – point 44~~5 a (new)

Text proposed by the Commission

Amendment

~~(44a) 'AI literacy' means the skills, knowledge and understanding regarding AI systems that are necessary for the compliance with and enforcement of this Regulation.~~

5a. This Regulation shall not apply to AI systems, including their output, developed and put into service for the sole purpose of research and development.

Amendment 42

Proposal for a regulation

~~Article 4 a (new)~~

Article 2 – paragraph 5 b (new)

Text proposed by the Commission

Amendment

~~Article 4a~~

5b. This Regulation shall not apply to any research and development activity regarding AI systems in so far as such activity does not lead to or require placing an AI system on the market or putting it into service and is in full respect of approved scientific ethical standards.

~~General principles applicable to all AI systems~~

1. All AI operators shall respect the following general principles that establish a high-level framework that promotes a coherent human-centric European approach to ethical and trustworthy Artificial Intelligence, which is fully in line with the Charter as well as the values on which the Union is founded:

• ‘human agency and oversight’ means that AI systems shall be developed and used as a tool that serves people, respects human dignity and personal autonomy, and that is functioning in a way that can be appropriately controlled and overseen by humans.

• ‘technical robustness and safety’ means that AI systems shall be developed and used in a way to minimize unintended and unexpected harm as well as being robust in case of unintended problems and being resilient against attempts to alter the use or performance of the AI system so as to allow unlawful use by malicious third parties.

• ‘privacy and data governance’ means that AI systems shall be developed and used in compliance with existing privacy and data protection rules, while processing data that meets high standards in terms of quality and integrity.

• ‘transparency’ means that AI systems shall be developed and used in a way that allows appropriate traceability and explainability, while making humans aware that they communicate or interact with an AI system as well as duly informing users of the capabilities and limitations of that AI system and affected persons about their rights.

• ‘diversity, non-discrimination and fairness’ means that AI systems shall be developed and used in a way that includes diverse actors and promotes equal access, gender equality and cultural diversity, while avoiding discriminatory impacts and unfair biases that are prohibited by Union or national law.

• ‘social and environmental well-being’ means that AI systems shall be developed and used in a sustainable and environmentally friendly manner as well as in a way to benefit all human beings, while monitoring and assessing the long-term impacts on the individual, society and democracy.

~~2. Paragraph 1 is without prejudice to obligations set up by existing Union and national law.~~

For high-risk AI systems, the general principles are translated into and complied with by providers or users by means of the requirements set out in Articles 8 to 15 of this Regulation. For all other AI systems, the voluntary application on the basis of harmonised standards, technical specifications and codes of conduct as referred to in Article 69 is strongly encouraged with a view to fulfilling the principles listed in paragraph 1.

3. The Commission and the Board shall issue recommendations that help guiding providers and users on how to develop and use AI systems in accordance with the general principles. European Standardisation Organisations shall take the general principles referred to in paragraph 1 into account as outcome-based objectives when developing the appropriate harmonised standards for high risk AI systems as referred to in Article 40(2b).

Amendment 43

Proposal for a regulation

~~Article 4 b (new)~~

Article 2 – paragraph 5 c (new)

Text proposed by the Commission

Amendment

~~Article 4b~~

5c. This Regulation is without prejudice to the rules laid down by other Union legal acts regulating the protection of personal data, in particular Regulation (EU) 2016/679, Regulation (EU) 2018/1725, Directive 2002/57/EC and Directive (EU) 2016/680.

~~AI literacy~~

1. When implementing this Regulation, the Union and the Member States shall promote measures and tools for the development of a sufficient level of AI literacy, across sectors and taking into account the different needs of groups of providers, users and affected persons concerned, including through education and training, skilling and reskilling programmes and while ensuring proper gender and age balance, in view of allowing a democratic control of AI systems.

2. Providers and user of AI systems shall promote tools and take measures to ensure a sufficient level of AI literacy of their staff and other persons dealing with the operation and use of AI systems on their behalf, taking into account their technical knowledge, experience, education and training and the environment the AI systems are to be used in, and considering the persons or groups of persons on which the AI systems are to be used.

3. Such literacy tools and measures shall consist, in particular, of the teaching and learning of basic notions and skills about AI systems and their functioning, including the different types of products and uses, their risks and benefits and the severity of the possible harm they can cause and its probability of occurrence.

~~4. A sufficient level of AI literacy is one that contributes, as necessary, to the ability of providers and users to ensure compliance and enforcement of this Regulation.~~

Amendment 44

Proposal for a regulation

Article 93 – paragraph ~~4 - subparagraph 2~~1 – point c1

Text proposed by the Commission

Amendment

~~(c) provision of adequate information pursuant to Article 13, in particular as regards the risks referred to in paragraph 2, point (b) of this Article, and, where appropriate, training to users.~~

(1) ‘artificial intelligence system’ (AI system) means software that is developed with one or more of the techniques and approaches listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with;

(c) provision of adequate information pursuant to Article 13, in particular as regards the risks referred to in paragraph 2, point (b) of this Article, and training to users, as appropriate to ensure a sufficient level of AI literacy in line with Article 4b.

(1) ‘artificial intelligence system’ (AI system) means a system that:

i) receives machine and/or human-based data and inputs,

ii) infers how to achieve a given set of human-defined objectives using learning, reasoning or modelling implemented with the techniques and approaches listed in Annex I, and

iii) generates outputs in the form of content (generative AI systems), predictions, recommendations or decisions, which influence the environments it interacts with;

Amendment 45

Proposal for a regulation

Article 93 – paragraph 91 – point 4

Text proposed by the Commission

Amendment

9. For credit institutions regulated by Directive 2013/36/EU, the aspects described in paragraphs 1 to 8 shall be part of the risk management procedures established by those institutions pursuant to Article 74 of that Directive.

(4) ‘user’ means any natural or legal person, public authority, agency or other body using an AI system under its authority, except where the AI system is used in the course of a personal non-professional activity;

9. For providers of AI systems already covered by other acts of Union law that require them to put in place specific risk management systems, including credit institutions regulated by Directive 2013/36/EU, the aspects described in paragraphs 1 to 8 shall be part of the risk management procedures established by those acts of Union law.

(4) ‘user’ means any natural or legal person, public authority, agency or other body using an AI system under its authority;

Amendment 46

Proposal for a regulation

~~Article 13 – title~~

Article 3 – paragraph 1 – point 5 a (new)

Text proposed by the Commission

Amendment

~~Transparency and provision of information to users~~

(5 a) ‘product manufacturer’ means a manufacturer within the meaning of any of the Union legislation listed in Annex II;

~~Transparency and provision of information~~

Amendment 47

Proposal for a regulation

Article 133 – paragraph 1 – point 13

Text proposed by the Commission

Amendment

1. High-risk AI systems shall be designed and developed in such a way to ensure that their operation is sufficiently transparent to enable users to interpret the system’s output and use it appropriately. An appropriate type and degree of transparency shall be ensured, with a view to achieving compliance with the relevant obligations of the user and of the provider set out in Chapter 3 of this Title.

(13) ‘reasonably foreseeable misuse’ means the use of an AI system in a way that is not in accordance with its intended purpose, but which may result from reasonably foreseeable human behaviour or interaction with other systems;

1. High-risk AI systems shall be designed and developed in such a way to ensure that their operation is sufficiently transparent to enable providers and users to reasonably understand the system’s functioning. Appropriate transparency shall be ensured in accordance with the intended purpose of the AI system, with a view to achieving compliance with the relevant obligations of the provider and user set out in Chapter 3 of this Title.

(13) ‘reasonably foreseeable misuse’ means the use of an AI system in a way that is not in accordance with its purpose as indicated in instruction for use or technical specification, but which may result from reasonably foreseeable human behaviour or interaction with other systems;

Transparency shall thereby mean that, at the time the high-risk AI system is placed on the market, all technical means available in accordance with the generally acknowledged state of art are used to ensure that the AI system’s output is interpretable by the provider and the user. The user shall be enabled to understand and use the AI system appropriately by generally knowing how the AI system works and what data it processes, allowing the user to explain the decisions taken by the AI system to the affected person pursuant to Article 68(c).

Amendment 48

Proposal for a regulation

Article 133 – paragraph 21 – point 14

Text proposed by the Commission

Amendment

2. High-risk AI systems shall be accompanied by instructions for use in an appropriate digital format or otherwise that include concise, complete, correct and clear information that is relevant, accessible and comprehensible to users.

(14) ‘safety component of a product or system’ means a component of a product or of a system which fulfils a safety function for that product or system or the failure or malfunctioning of which endangers the health and safety of persons or property;

2. High-risk AI systems shall be accompanied by intelligible instructions for use in an appropriate digital format or made otherwise available in a durable medium that include concise, correct, clear and to the extent possible complete information that helps operating and maintaining the AI system as well as supporting informed decision-making by users and is reasonably relevant, accessible and comprehensible to users .

(14) ‘safety component of a product or system’ means a component of a product or of a system the failure or malfunctioning of which endangers the health and safety of persons or property;

Amendment 49

Proposal for a regulation

Article 133 – paragraph 31 – ~~introductory~~point ~~part~~35

Text proposed by the Commission

Amendment

~~3. The information referred to in paragraph 2 shall specify:~~

(35) ‘biometric categorisation system’ means an AI system for the purpose of assigning natural persons to specific categories, such as sex, age, hair colour, eye colour, tattoos, ethnic origin or sexual or political orientation, on the basis of their biometric data;

~~3. To achieve the outcomes referred to in paragraph 1, information referred to in paragraph 2 shall specify:~~

(35) ‘biometric categorisation system’ means an AI system for the purpose of assigning natural persons to specific categories, such as sex, age, disability, hair colour, eye colour, tattoos, ethnic origin or sexual or political orientation, on the basis of their biometric data;

Amendment 50

Proposal for a regulation

Article 133 – paragraph 31 – point a44 – introductory part

Text proposed by the Commission

Amendment

~~(a) the identity and the contact details of the provider and, where applicable, of its authorised representative;~~

(44) ‘serious incident’ means any incident that directly or indirectly leads, might have led or might lead to any of the following:

~~(a) the identity and the contact details of the provider and, where applicable, of its authorised representatives;~~

(44) ‘serious incident’ means any incident or malfunctioning of an AI system that directly or indirectly leads, might have led or might lead to any of the following:

Amendment 51

Proposal for a regulation

Article 133 – paragraph 31 – point a44 a (new)

Text proposed by the Commission

Amendment

~~(aa) where it is not the same as the provider, the identity and the contact details of the entity that carried out the conformity assessment and, where applicable, of its authorised representative;~~

(44a) 'personal data' means data as defined in point (1) of Article 4 of Regulation (EU)2016/679;

Amendment 52

Proposal for a regulation

Article 133 – paragraph 31 – point ~~b –~~44 ~~introductory~~b ~~part~~(new)

Text proposed by the Commission

Amendment

~~(b) the characteristics, capabilities and limitations of performance of the high-risk AI system, including:~~

(44b) ‘non-personal data’ means data other than personal data as defined in point (1) of Article 4 of Regulation (EU) 2016/679;

~~(b) the characteristics, capabilities and limitations of performance of the high-risk AI system, including, where appropriate:~~

Amendment 53

Proposal for a regulation

Article 134 – paragraph ~~3 – point b – point ii~~1

Text proposed by the Commission

Amendment

(ii) the level of accuracy, robustness and cybersecurity referred to in Article 15 against which the high-risk AI system has been tested and validated and which can be expected, and any known and foreseeable circumstances that may have an impact on that expected level of accuracy, robustness and cybersecurity;

The Commission is empowered to adopt delegated acts in accordance with Article 73 to amend the list of techniques and approaches listed in Annex I, in order to update that list to market and technological developments on the basis of characteristics that are similar to the techniques and approaches listed therein.

(ii) the level of accuracy, robustness and cybersecurity referred to in Article 15 against which the high-risk AI system has been tested and validated and which can be expected, and any clearly known and foreseeable circumstances that may have an impact on that expected level of accuracy, robustness and cybersecurity;

The Commission is empowered to adopt delegated acts in accordance with Article 73 to amend the list of techniques and approaches listed in Annex I within the scope of the definition of an AI system as provided for in Article 3(1), in order to update that list to market and technological developments on the basis of characteristics that are similar to the techniques and approaches listed therein

Amendment 54

Proposal for a regulation

Article 136 – paragraph ~~3 – point b~~1 – ~~point~~introductory ~~iii~~part

Text proposed by the Commission

Amendment

(iii) any known or foreseeable circumstance, related to the use of the high-risk AI system in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, which may lead to risks to the health and safety or fundamental rights;

1. Irrespective of whether an AI system is placed on the market or put into service independently from the products referred to in points (a) and (b), that AI system shall be considered high-risk where both of the following conditions are fulfilled:

(iii) any clearly known or foreseeable circumstance, related to the use of the high-risk AI system in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, which may lead to risks to the health and safety, fundamental rights or the environment, including, where appropriate, illustrative examples of such limitations and of scenarios for which the system should not be used;

1. An AI system that is itself a product covered by the Union harmonisation legislation listed in Annex II shall be considered as high risk if it is required to undergo a third-party conformity assessment with a view to the placing on the market or putting into service of that product pursuant to the above mentioned legislation.

Amendment 55

Proposal for a regulation

Article 136 – paragraph ~~3 – point b~~1 – point ~~iii~~ a ~~(new)~~

Text proposed by the Commission

Amendment

~~(iiia) the degree to which the AI system can provide an explanation for decisions it takes;~~

(a) the AI system is intended to be used as a safety component of a product, or is itself a product, covered by the Union harmonisation legislation listed in Annex II;

2. An AI system intended to be used as a safety component of a product covered by the legislation referred to in paragraph 1 shall be considered as high risk if it is required to undergo a third-party conformity assessment with a view to the placing on the market or putting into service of that product pursuant to abovementioned legislation. This provision shall apply irrespective of whether the AI system is placed on the market or put into service independently from the product.

Amendment 56

Proposal for a regulation

Article 136 – paragraph ~~3 – point b – point v~~2

Text proposed by the Commission

Amendment

(v) when appropriate, specifications for the input data, or any other relevant information in terms of the training, validation and testing data sets used, taking into account the intended purpose of the AI system.

2. In addition to the high-risk AI systems referred to in paragraph 1, AI systems referred to in Annex III shall also be considered high-risk.

(v) relevant information about user actions that may influence system performance, including type or quality of input data, or any other relevant information in terms of the training, validation and testing data sets used, taking into account the intended purpose of the AI system.

3. AI systems referred to in Annex III shall be considered high-risk.

Amendment 57

Proposal for a regulation

Article 136 – paragraph ~~3 –~~2 ~~point~~a e(new)

Text proposed by the Commission

Amendment

~~(e) the expected lifetime of the high-risk AI system and any necessary maintenance and care measures to ensure the proper functioning of that AI system, including as regards software updates.~~

2a. The classification as high-risk as a consequence of Article 6(1) 6(2) and 6(3) shall be disregarded for AI systems whose intended purpose demonstrates that the generated output is a recommendation requiring a human intervention to convert this recommendation into a decision and for AI systems, which do not lead to autonomous decisions or actions of the overall system.

~~(e) any necessary maintenance and care measures to ensure the proper functioning of that AI system, including as regards software updates, through its expected lifetime.~~

Amendment 58

Proposal for a regulation

Article 137 – paragraph 31 – point ~~e a (new)~~b

Text proposed by the Commission

Amendment

~~(ea) a description of the mechanisms included within the AI system that allows users to properly collect, store and interpret the logs in accordance with Article 12(1).~~

(b) the AI systems pose a risk of harm to the health and safety, or a risk of adverse impact on fundamental rights, that is, in respect of its severity and probability of occurrence, equivalent to or greater than the risk of harm or of adverse impact posed by the high-risk AI systems already referred to in Annex III.

(b) the AI systems pose a risk of harm to the health or safety, or a risk of adverse impact on fundamental rights or the environment, that is, in respect of its severity and probability of occurrence, equivalent to or greater than the risk of harm or of adverse impact posed by the high-risk AI systems already referred to in Annex III.

Amendment 59

Proposal for a regulation

Article 137 – paragraph 32 – ~~point e b~~introductory ~~(new)~~part

Text proposed by the Commission

Amendment

~~(eb) The information shall be provided at least in the language of the country where the AI system is used.~~

2. When assessing for the purposes of paragraph 1 whether an AI system poses a risk of harm to the health and safety or a risk of adverse impact on fundamental rights that is equivalent to or greater than the risk of harm posed by the high-risk AI systems already referred to in Annex III, the Commission shall take into account the following criteria:

2. When assessing for the purposes of paragraph 1 whether an AI system poses a risk of harm to the health and safety or a risk of adverse impact on fundamental rights or on the environment that is equivalent to or greater than the risk of harm posed by the high-risk AI systems already referred to in Annex III, the Commission shall take into account the following criteria:

Amendment 60

Proposal for a regulation

Article 137 – paragraph 32 a– ~~(new)~~point c

Text proposed by the Commission

Amendment

~~3a. In order to comply with the obligations laid down in this Article, providers and users shall ensure a sufficient level of AI literacy in line with Article 4b.~~

(c) the extent to which the use of an AI system has already caused harm to the health and safety or adverse impact on the fundamental rights or has given rise to significant concerns in relation to the materialisation of such harm or adverse impact, as demonstrated by reports or documented allegations submitted to national competent authorities;

(c) the extent to which the use of an AI system has already caused harm to the health and safety or adverse impact on the fundamental rights or on the environment or has given rise to significant concerns in relation to the materialisation of such harm or adverse impact, as demonstrated by reports or documented allegations submitted to national competent authorities;

Amendment 61

Proposal for a regulation

Article 147 – paragraph 12 – point d

Text proposed by the Commission

Amendment

(d) the potential extent of such harm or such adverse impact, in particular in terms of its intensity and its ability to affect a plurality of persons;

1. High-risk AI systems shall be designed and developed in such a way, including with appropriate human-machine interface tools, that they be effectively overseen by natural persons as proportionate to the risks associated with those systems. Natural persons in charge of ensuring human oversight shall have sufficient level of AI literacy in accordance with Article 4b and the necessary support and authority to exercise that function, during the period in which the AI system is in use and to allow for thorough investigation after an incident.

(d) the potential extent of such harm or such adverse impact, in particular in terms of its intensity and its ability to affect a plurality of persons or the environment;

Amendment 62

Proposal for a regulation

Article 147 – paragraph 2 – point g

Text proposed by the Commission

Amendment

2. Human oversight shall aim at preventing or minimising the risks to health, safety or fundamental rights that may emerge when a high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, in particular when such risks persist notwithstanding the application of other requirements set out in this Chapter.

(g) the extent to which the outcome produced with an AI system is easily reversible, whereby outcomes having an impact on the health or safety of persons shall not be considered as easily reversible;

2. Human oversight shall aim at preventing or minimising the risks to health, safety, fundamental rights or environment that may emerge when a high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, in particular when such risks persist notwithstanding the application of other requirements set out in this Chapter and where decisions based solely on automated processing by AI systems produce legal or otherwise significant effects on the persons or groups of persons on which the system is to be used.

(g) the extent to which the outcome produced with an AI system is easily reversible, whereby outcomes having an adverse impact on the health or safety of persons, or on the environment shall not be considered as easily reversible;

Amendment 63

Proposal for a regulation

Article 148 – paragraph ~~3 – introductory part~~1

Text proposed by the Commission

Amendment

~~3. Human oversight shall be ensured through either one or all of the following measures:~~

1. High-risk AI systems shall comply with the requirements established in this Chapter.

~~3. Human oversight shall take into account the specific risks, the level of automation, and context of the AI system and shall be ensured through either one or all of the following types of measures:~~

1. High-risk AI systems shall comply with the requirements established in this Chapter, taking into account sectoral legislation where applicable, harmonised standards and common specifications.

Amendment 64

Proposal for a regulation

Article 149 – paragraph 42 – ~~introductory~~point ~~part~~a

Text proposed by the Commission

Amendment

~~4. The measures referred to in paragraph 3 shall enable the individuals to whom human oversight is assigned to do the following, as appropriate to the circumstances:~~

(a) identification and analysis of the known and foreseeable risks associated with each high-risk AI system;

4.(a) ~~For~~identification ~~the~~and ~~purpose~~analysis of ~~implementing~~the ~~paragraphs~~known 1and toforeseeable 3,risks ~~the~~associated with each high-risk AI system ~~shall~~that bemight ~~provided~~cause harm or damage to the ~~user~~environment inor ~~such~~to athe ~~way~~health, ~~that~~safety ~~natural~~and ~~persons~~fundamental torights ~~whom~~of ~~human~~persons ~~oversight~~in isview ~~assigned~~of ~~are~~the ~~enabled,~~intended aspurpose ~~appropriate~~of ~~and~~or ~~proportionate~~misuse toof the ~~circumstances:~~high-risk AI system.

Amendment 65

Proposal for a regulation

Article 149 – paragraph 42 – point ac

Text proposed by the Commission

Amendment

(a) fully understand the capacities and limitations of the high-risk AI system and be able to duly monitor its operation, so that signs of anomalies, dysfunctions and unexpected performance can be detected and addressed as soon as possible;

(c) evaluation of other possibly arising risks based on the analysis of data gathered from the post-market monitoring system referred to in Article 61;

(a) be aware of and sufficiently understand the relevant capacities and limitations of the high-risk AI system and be able to duly monitor its operation, so that signs of anomalies, dysfunctions and unexpected performance can be detected and addressed as soon as possible;

(c) evaluation of other possibly arising risks based on the analysis of data gathered from the post-market monitoring system;

Amendment 66

Proposal for a regulation

Article 149 – paragraph 4 – ~~point~~subparagraph e1

Text proposed by the Commission

Amendment

~~(e) be able to intervene on the operation of the high-risk AI system or interrupt the system through a “stop” button or a similar procedure.~~

The risk management measures referred to in paragraph 2, point (d) shall be such that any residual risk associated with each hazard as well as the overall residual risk of the high-risk AI systems is judged acceptable, provided that the high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse. Those residual risks shall be communicated to the user.

(e) be able to intervene on the operation of the high-risk AI system or interrupt, the system through a “stop” button or a similar procedure that allows the system to come to a halt in a safe state, except if the human interference increases the risks or would negatively impact the performance in consideration of generally acknowledged state-of-the-art.

The risk management measures referred to in paragraph 2, point (d) shall be such that any residual risk associated with each hazard as well as the overall residual risk of the high-risk AI systems is judged acceptable, provided that the high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, subject to terms, conditions as made available by the provider, and contractual and license restrictions. Those residual risks shall be communicated to the user.

Amendment 67

Proposal for a regulation

Article 1410 – paragraph 51

Text proposed by the Commission

Amendment

5. For high-risk AI systems referred to in point 1(a) of Annex III, the measures referred to in paragraph 3 shall be such as to ensure that, in addition, no action or decision is taken by the user on the basis of the identification resulting from the system unless this has been verified and confirmed by at least two natural persons.

1. High-risk AI systems which make use of techniques involving the training of models with data shall be developed on the basis of training, validation and testing data sets that meet the quality criteria referred to in paragraphs 2 to 5.

5. For high-risk AI systems referred to in point1(a) of Annex III, the measures referred to in paragraph 3 shall be such as to ensure that, in addition, no action or decision is taken by the user on the basis of the identification resulting from the system unless this has been verified and confirmed by at least two natural persons with the necessary competence, training and authority.

Amendment 68

Proposal for a regulation

Article 1610 – paragraph 12 – point ag

Text proposed by the Commission

Amendment

~~(a) ensure that their high-risk AI systems are compliant with the requirements set out in Chapter 2 of this Title;~~

(g) the identification of any possible data gaps or shortcomings, and how those gaps and shortcomings can be addressed.

~~(a) ensure that their high-risk AI systems are compliant with the requirements set out in Chapter 2 of this Title, before placing them on the market or putting them into service;~~

(g) the identification of any other data gaps or shortcomings that materially increase the risks of harm to the health, environment and safety or the fundamental rights of persons, and how those gaps and shortcomings can be addressed.

Amendment 69

Proposal for a regulation

Article 1610 – paragraph ~~1 – point a a (new)~~3

Text proposed by the Commission

Amendment

(aa) indicate their name, registered trade name or registered trade mark, the address at which they can be contacted on the high-risk AI system or, where that is not possible, on its packaging or its accompanying documentation, as applicable;

3. Training, validation and testing data sets shall be relevant, representative, free of errors and complete. They shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons on which the high-risk AI system is intended to be used. These characteristics of the data sets may be met at the level of individual data sets or a combination thereof.

3. Training, validation and testing data sets shall be relevant, representative, free of errors and to the best extent possible and as complete as possible. They shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons on which the high-risk AI system is intended to be used. These characteristics of the data sets may be met at the level of individual data sets or a combination thereof.

Amendment 70

Proposal for a regulation

Article 1610 – paragraph ~~1 – point c~~4

Text proposed by the Commission

Amendment

~~(c) draw-up the technical documentation of the high-risk AI system;~~

4. Training, validation and testing data sets shall take into account, to the extent required by the intended purpose, the characteristics or elements that are particular to the specific geographical, behavioural or functional setting within which the high-risk AI system is intended to be used.

~~(c) keep the documentation and, where not yet available, draw up the technical documentation referred to in Article 18;~~

4. Training, validation and testing data sets shall be sufficiently diverse to accurately capture, to the extent required by the intended purpose, the characteristics or elements that are particular to the specific geographical, behavioural or functional setting within which the high-risk AI system is intended to be used.

Amendment 71

Proposal for a regulation

Article 1611 – paragraph 1 – ~~point~~subparagraph d2

Text proposed by the Commission

Amendment

~~(d) when under their control, keep the logs automatically generated by their high-risk AI systems;~~

~~(d) when under their control, keep the logs automatically generated by their high-risk AI systems, in accordance with Article 20;~~

The technical documentation shall be drawn up in such a way to demonstrate that the high-risk AI system complies with the requirements set out in this Chapter and provide national competent authorities and notified bodies with all the necessary information to assess the compliance of the AI system with those requirements. It shall contain, at a minimum, the elements set out in Annex IV or, in the case of SMEs and start-ups, any equivalent documentation meeting the same objectives, subject to approval of the competent authority. Documentation shall be kept up to date throughout its entire lifecycle.

Amendment 72

Proposal for a regulation

Article 1612 – paragraph ~~1 – point e~~2

Text proposed by the Commission

Amendment

~~(e) ensure that the high-risk AI system undergoes the relevant conformity assessment procedure, prior to its placing on the market or putting into service;~~

2. The logging capabilities shall ensure a level of traceability of the AI system’s functioning throughout its lifecycle that is appropriate to the intended purpose of the system.

~~(e) carry out the relevant conformity assessment procedure, as provided for in Article 19, prior to its placing on the market or putting into service;~~

2. The logging capabilities shall ensure a level of traceability of the AI system’s functioning while the AI system is used within its lifecycle that is appropriate to the intended purpose of the system.

Amendment 73

Proposal for a regulation

Article 1613 – paragraph ~~1 – point g~~2

Text proposed by the Commission

Amendment

~~(g) take the necessary corrective actions, if the high-risk AI system is not in conformity with the requirements set out in Chapter 2 of this Title;~~

~~(g) take the necessary corrective actions as referred to in Article 21, if the high-risk AI system is not in conformity with the requirements set out in Chapter 2 of this Title;~~

2. High-risk AI systems shall be accompanied by instructions for use in an appropriate digital format or made otherwise available, that include concise, complete, correct and clear information that is relevant, accessible and comprehensible to users to assist them in operating and maintaining the AI system, taking into consideration the system’s intended purpose and the expected audience for the instructions.

Amendment 74

Proposal for a regulation

Article 1613 – paragraph 13 – point jb – point ii

Text proposed by the Commission

Amendment

~~(j) upon request of a national competent authority, demonstrate the conformity of the high-risk AI system with the requirements set out in Chapter 2 of this Title.~~

(j) upon reasoned request of a national competent authority, provide the relevant information and documentation to demonstrate the conformity of the high-risk AI system with the requirements set out in Chapter 2 of this Title.

(ii) the level of accuracy, robustness and cybersecurity referred to in Article 15 against which the high-risk AI system has been tested and validated and which can be expected, and any known and reasonably foreseeable circumstances that could materially impact that expected level of accuracy, robustness and cybersecurity;

Amendment 75

Proposal for a regulation

~~Article 23 a (new)~~

Article 13 – paragraph 3 – point b – point iii

Text proposed by the Commission

Amendment

~~Article 23a~~

~~Conditions for other persons to be subject to the obligations of a provider~~

1. Concerning high risk AI systems, any natural or legal person shall be considered a new provider for the purposes of this Regulation and shall be subject to the obligations of the provider under Article 16, in any of the following circumstances:

(a) they put their name or trademark on a high-risk AI system already placed on the market or put into service, without prejudice to contractual arrangements stipulating that the obligations are allocated otherwise;

~~(b) they make a substantial modification to or modify the intended purpose of a high-risk AI system already placed on the market or put into service;~~

~~(c) they modify the intended purpose of a non-high-risk AI system already placed on the market or put it to service, in a way which makes the modified system a high-risk AI system;~~

(d) they place on the market or make available on the market, with or without modification and in return for payment an Open Source AI system, an AI system derived from an Open Source AI system, or Technical Support Services for any such Open Source AI systems;

~~(e) they adapt a general purpose AI system, already placed on the market or put into service, to a specific intended purpose.~~

2. Where the circumstances referred to in paragraph 1, points (a), (b), (c) or (d), occur, the former provider that initially placed the high-risk AI system on the market or put it into service shall no longer be considered a provider for the purposes of this Regulation. The former provider shall upon request and respecting its own intellectual property rights or trade secrets, provide the new provider with all essential, relevant and reasonably expected information that is necessary to comply with the obligations set out in this Regulation.

3. The original provider of a general purpose AI system as referred to in paragraph 1, point (e), shall, respecting its own intellectual property rights or trade secrets and taking into account the risks that are specifically linked to the adaption of the general purpose AI system to a specific intended purpose:

~~(a) ensure that the general purpose AI system which may be used as high-risk AI system complies with the requirements established in Articles 9, 10, 11, 13(2) and (3), 14(1) and 15 of this Regulation;~~

~~(b) comply with the obligations set out in Articles 16aa, 16e, 16f, 16g, 16i, 16j, 48 and 61 of this Regulation;~~

(c) assess the reasonable foreseeable misuses of the general purpose AI system that may arise during the expected lifetime and install mitigation measures against those cases based on the generally acknowledged state of the art;

(d) provide the new provider referred to in paragraph 1, point (d), with all essential, relevant and reasonably expected information that is necessary to comply with the obligations set out in this Regulation.

4. For high-risk AI systems that are safety components of products to which the legal acts listed in Annex II, section A apply, the manufacturer of those products shall be considered the provider of the high-risk AI system and shall be subject to the obligations referred to in Article 16 under either of the following scenarios:

~~(i) the high-risk AI system is placed on the market together with the product under the name or trademark of the product manufacturer; or~~

~~(ii) the high-risk AI system is put into service under the name or trademark of the product manufacturer after the product has been placed on the market.~~

5. Third parties involved in the sale and the supply of software including general purpose application programming interfaces (API), software tools and components, or providers of network services shall not be considered providers for the purposes of this Regulation.

Amendment 76

Proposal for a regulation

Article 2913 – paragraph 13 – point e

Text proposed by the Commission

Amendment

~~1. Users of high-risk AI systems shall use such systems in accordance with the instructions of use accompanying the systems, pursuant to paragraphs 2 and 5.~~

(e) the expected lifetime of the high-risk AI system and any necessary maintenance and care measures to ensure the proper functioning of that AI system, including as regards software updates.

1. Users of high-risk AI systems shall take appropriate organisational measures and ensure that the use of such systems takes place in accordance with the instructions of use accompanying the systems pursuant to paragraphs 1a to 5 of this Article. Users shall bear responsibility in case of any use of the AI system that is not in accordance with the instructions of use accompanying the systems.

(e) the expected lifetime of the high-risk AI system, the description of the procedure of withdrawing it from use and any necessary maintenance and care measures to ensure the proper functioning of that AI system, including as regards software updates.

Amendment 77

Proposal for a regulation

Article 2914 – paragraph 1 ~~a (new)~~

Text proposed by the Commission

Amendment

~~1a. To the extent the user exercises control over the high-risk AI system, that user shall assign human oversight to natural persons who have the necessary AI literacy in accordance with Article 4b.~~

1. High-risk AI systems shall be designed and developed in such a way, including with appropriate human-machine interface tools, that they can be effectively overseen by natural persons during the period in which the AI system is in use, unless there is clear evidence that human intervention compromises the safety of the high risk AI system concerned.

Amendment 78

Proposal for a regulation

Article 2914 – paragraph 2

Text proposed by the Commission

Amendment

2. The obligations in paragraph 1 are without prejudice to other user obligations under Union or national law and to the user’s discretion in organising its own resources and activities for the purpose of implementing the human oversight measures indicated by the provider.

2. The obligations in paragraphs 1 and 1a are without prejudice to other obligations of the user under Union or national law and to the user’s discretion in organising its own resources and activities for the purpose of implementing the human oversight measures indicated by the provider.

2. Human oversight shall aim at preventing or minimising the risks to health, safety or fundamental rights or the environment that may emerge when a high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, in particular when such risks persist notwithstanding the application of other requirements set out in this Chapter.

Amendment 79

Proposal for a regulation

Article 2914 – paragraph 34 – introductory part

Text proposed by the Commission

Amendment

3. Without prejudice to paragraph 1, to the extent the user exercises control over the input data, that user shall ensure that input data is relevant in view of the intended purpose of the high-risk AI system.

4. The measures referred to in paragraph 3 shall enable the individuals to whom human oversight is assigned to do the following, as appropriate to the circumstances:

3. Without prejudice to paragraph 1, to the extent the user exercises control over the input data, that user shall ensure that input data is relevant and sufficiently representative in view of the intended purpose of the high-risk AI system.

4. The measures referred to in paragraph 3 shall enable the individuals to whom human oversight is assigned to do the following, as appropriate and proportionate to the circumstances:

Amendment 80

Proposal for a regulation

Article 2914 – paragraph 4 – ~~subparagraph~~point 1a

Text proposed by the Commission

Amendment

4. Users shall monitor the operation of the high-risk AI system on the basis of the instructions of use. When they have reasons to consider that the use in accordance with the instructions of use may result in the AI system presenting a risk within the meaning of Article 65(1) they shall inform the provider or distributor and suspend the use of the system. They shall also inform the provider or distributor when they have identified any serious incident or any malfunctioning within the meaning of Article 62 and interrupt the use of the AI system. In case the user is not able to reach the provider, Article 62 shall apply mutatis mutandis.

4. Users shall monitor the operation of the high-risk AI system on the basis of the instructions of use and, when relevant, inform the provider in accordance with Article 61. To the extent the user exercises control over the high-risk AI system, it shall also perform a risk assessment in accordance with Article 9 but limited to the potential adverse effects of using the high-risk AI system as well as the respective mitigation measures. When they have reasons to consider that the use in accordance with the instructions of use may result in the AI system presenting a risk within the meaning of Article 65(1) they shall inform the provider or distributor and suspend the use of the system. They shall also inform the provider or distributor and competent supervisory authority when they have identified any serious incident or malfunctioning and interrupt the use of the AI system. In case the user is not able to reach the provider importer or distributer, Article 62 shall apply mutatis mutandis.

(a) have an appropriate understanding of the capacities and limitations of the high-risk AI system and be able to duly monitor its operation, so that signs of anomalies, dysfunctions and unexpected performance can be detected and addressed as soon as possible;

Amendment 81

Proposal for a regulation

Article 2914 – paragraph 54 – ~~subparagraph~~point 1d

Text proposed by the Commission

Amendment

5. Users of high-risk AI systems shall keep the logs automatically generated by that high-risk AI system, to the extent such logs are under their control. The logs shall be kept for a period that is appropriate in the light of the intended purpose of the high-risk AI system and applicable legal obligations under Union or national law.

(d) be able to decide, in any particular situation, not to use the high-risk AI system or otherwise disregard, override or reverse the output of the high-risk AI system;

5. Users of high-risk AI systems shall keep the logs automatically generated by that high-risk AI system, to the extent such logs are under their control and is feasible from a technical point of view. They shall keep them for a period of at least six months, unless provided otherwise in applicable Union or national law.

(d) be able to decide, in any particular situation, not to use the high-risk AI system or otherwise disregard, override or reverse the output of the high-risk AI system, unless there is clear evidence that such human intervention is deemed to increase risks or otherwise negatively impact the system’s performance.

Amendment 82

Proposal for a regulation

Article 2914 – paragraph 64 – point e

Text proposed by the Commission

Amendment

6. Users of high-risk AI systems shall use the information provided under Article 13 to comply with their obligation to carry out a data protection impact assessment under Article 35 of Regulation (EU) 2016/679 or Article 27 of Directive (EU) 2016/680, where applicable.

(e) be able to intervene on the operation of the high-risk AI system or interrupt the system through a “stop” button or a similar procedure.

(e) be able to intervene on the operation of the high-risk AI system put the system into fail-safe mode, put the system into manual control mode or stop the system through a “stop” button or a similar procedure unless there is clear evidence that such human intervention is deemed to increase risks or otherwise negatively impact the system’s performance.

Amendment 83

Proposal for a regulation

Article 2914 – paragraph 64 – point e a (new)

Text proposed by the Commission

Amendment

6a. The provider shall be obliged to cooperate closely with the user and in particular provide the user with the necessary and appropriate information to allow the fulfilment of the obligations set out in this Article.

(ea) be able to comprehend when a high risk AI system decision is preferable to human oversight.

Amendment 84

Proposal for a regulation

Article 2914 – paragraph ~~6 b (new)~~5

Text proposed by the Commission

Amendment

~~6b. Users shall cooperate with national competent authorities on any action those authorities take in relation to an AI system.~~

5. For high-risk AI systems referred to in point 1(a) of Annex III, the measures referred to in paragraph 3 shall be such as to ensure that, in addition, no action or decision is taken by the user on the basis of the identification resulting from the system unless this has been separately verified and confirmed by at least two natural persons.

Amendment 85

Proposal for a regulation

Article 4015 – paragraph 1

Text proposed by the Commission

Amendment

High-risk AI systems which are in conformity with harmonised standards or parts thereof the references of which have been published in the Official Journal of the European Union shall be presumed to be in conformity with the requirements set out in Chapter 2 of this Title, to the extent those standards cover those requirements.

1. High-risk AI systems shall be designed and developed in such a way that they achieve, in the light of their intended purpose, an appropriate level of accuracy, robustness and cybersecurity, and perform consistently in those respects throughout their lifecycle.

1. High-risk AI systems which are in conformity with harmonised standards or parts thereof the references of which have been published in the Official Journal of the European Union shall be presumed to be in conformity with the requirements set out in Chapter 2 of this Title, to the extent those standards cover those requirements.

1. High-risk AI systems shall be designed and developed in such a way that they achieve, in the light of their intended purpose, the highest level of accuracy, robustness and cybersecurity possible, and perform consistently in those respects throughout their lifecycle.

Amendment 86

Proposal for a regulation

Article 4015 – paragraph 13 a– ~~(new)~~subparagraph 1

Text proposed by the Commission

Amendment

1a. When issuing a standardisation request to European standardisation organisations in accordance with Article 10 of Regulation (EU)1025/2012, the Commission shall specify that standards are coherent, easy to implement and drafted in such a way that they aim to fulfil in particular the following objectives:

High-risk AI systems shall be resilient as regards errors, faults or inconsistencies that may occur within the system or the environment in which the system operates, in particular due to their interaction with natural persons or other systems.

~~a) ensure that AI systems placed on the market or put into service in the Union are safe, trustworthy and respect Union values and strengthen the Union's digital sovereignty;~~

Providers should take all appropriate and feasible measures to ensure that high-risk AI systems are resilient as regards errors, faults or inconsistencies that may occur within the system or the environment in which the system operates, in particular due to their interaction with natural persons or other systems.

~~b) take into account the general principles for trustworthy AI set out in Article 4a;~~

~~c) promote investment and innovation in AI, as well as competitiveness and growth of the Union market;~~

~~d) enhance multistakeholder governance, representative of all relevant European stakeholders (e.g. industry, SMEs, civil society, social partners, researchers);~~

~~e) contribute to strengthening global cooperation on standardisation in the field of AI that is consistent with Union values, fundamental rights and interests.~~

~~The Commission shall request the European standardisation organisations to provide evidence of their best efforts to fulfil the above objectives.~~

1b. The Commission shall issue standardisation requests covering all requirements of this Regulation in accordance with Article 10 of Regulation (EU) No1025/2012 before the date of entry into force of this Regulation.

Amendment 87

Proposal for a regulation

Article 5215 – ~~title~~paragraph 3 – subparagraph 3

Text proposed by the Commission

Amendment

~~Transparency obligations for certain AI systems~~

High-risk AI systems that continue to learn after being placed on the market or put into service shall be developed in such a way to ensure that possibly biased outputs due to outputs used as an input for future operations (‘feedback loops’) are duly addressed with appropriate mitigation measures.

~~Transparency obligations~~

High-risk AI systems that continue to learn after being placed on the market or put into service shall be developed in such a way to ensure that possibly biased outputs influencing an input for future operations (‘feedback loops’) are duly addressed with appropriate mitigation measures.

Amendment 88

Proposal for a regulation

Article 5228 – paragraph 1 – point c a (new)

Text proposed by the Commission

Amendment

1. Providers shall ensure that AI systems intended to interact with natural persons are designed and developed in such a way that natural persons are informed that they are interacting with an AI system, unless this is obvious from the circumstances and the context of use. This obligation shall not apply to AI systems authorised by law to detect, prevent, investigate and prosecute criminal offences, unless those systems are available for the public to report a criminal offence.

(ca) they modify the intended purpose of an AI system which is not high-risk and is already placed on the market or put into service, in a way which makes the modified system a high-risk AI system.

1. Providers shall ensure that AI systems intended to interact with natural persons are designed and developed in such a way that the AI system, the provider itself or the user informs the natural person exposed to an AI system that they are interacting with an AI system in a timely, clear and intelligible manner, unless this is obvious from the circumstances and the context of use.

Where appropriate and relevant, this information shall also include which functions are AI enabled, if there is human oversight, and who is responsible for the decision-making process, as well as the existing rights and processes that, according to Union and national law, allow natural persons or their representatives to object against the application of such systems to them and to seek judicial redress against decisions taken by or harm caused by AI systems, including their right to seek an explanation. This obligation shall not apply to AI systems authorised by law to detect, prevent, investigate and prosecute criminal offences, unless those systems are available for the public to report a criminal offence.

Amendment 89

Proposal for a regulation

Article 5229 – paragraph 26 a (new)

Text proposed by the Commission

Amendment

2. Users of an emotion recognition system or a biometric categorisation system shall inform of the operation of the system the natural persons exposed thereto. This obligation shall not apply to AI systems used for biometric categorisation, which are permitted by law to detect, prevent and investigate criminal offences.

6a. This Article only applies to users acting in their professional capacity and not to those using AI in the course of a personal non-professional activity.

2. Users of an emotion recognition system or a biometric categorisation system which is not prohibited pursuant to Article 5 shall inform in a timely, clear and intelligible manner of the operation of the system the natural persons exposed thereto and obtain their consent prior to the processing of their biometric and other personal data in accordance with Regulation (EU) 2016/679, Regulation (EU) 2016/1725 and Directive (EU) 2016/280, as applicable. This obligation shall not apply to AI systems used for biometric categorisation, which are permitted by law to detect, prevent and investigate criminal offences.

Amendment 90

Proposal for a regulation

Article 5229 – paragraph ~~3 –~~6 ~~subparagraph~~b 1(new)

Text proposed by the Commission

Amendment

3. Users of an AI system that generates or manipulates image, audio or video content that appreciably resembles existing persons, objects, places or other entities or events and would falsely appear to a person to be authentic or truthful (‘deep fake’), shall disclose that the content has been artificially generated or manipulated.

6b. Users of high risk AI systems, who modify or extend the purpose for which the conformity of the AI system was originally assessed, shall establish and document a post-market monitoring system(Art. 61) and must undergo a new conformity assessment (Art. 43) involved by a notified body.

3. Users of an AI system that generates or manipulates text, audio or visual content that would falsely appear to be authentic or truthful and which features depictions of people appearing to say or do things they did not say or do, without their consent (‘deep fake’), shall disclose in an appropriate, timely, clear and visible manner that the content has been artificially generated or manipulated, as well as, whenever possible, the name of the natural or legal person that generated or manipulated it. Disclosure shall mean labelling the content in a way that informs that the content is inauthentic and that is clearly visible for the recipient of that content. To label the content, users shall take into account the generally acknowledged state of the art and relevant harmonised standards and specifications.

Amendment 91

Proposal for a regulation

Article 5230 – paragraph ~~3 – subparagraph 2~~8

Text proposed by the Commission

Amendment

However, the first subparagraph shall not apply where the use is authorised by law to detect, prevent, investigate and prosecute criminal offences or it is necessary for the exercise of the right to freedom of expression and the right to freedom of the arts and sciences guaranteed in the Charter of Fundamental Rights of the EU, and subject to appropriate safeguards for the rights and freedoms of third parties.

8. Notifying authorities shall make sure that conformity assessments are carried out in a proportionate manner, avoiding unnecessary burdens for providers and that notified bodies perform their activities taking due account of the size of an undertaking, the sector in which it operates, its structure and the degree of complexity of the AI system in question.

3a. Paragraph 3 shall not apply where the use of an AI system that generates or manipulates text, audio or visual content is authorized by law or if it is necessary for the exercise of the right to freedom of expression and the right to freedom of the arts and sciences guaranteed in the Charter of Fundamental Rights of the EU, and subject to appropriate safeguards for the rights and freedoms of third parties. Where the content forms part of an evidently creative, satirical, artistic or fictional cinematographic, video games visuals and analogous work or programme, transparency obligations set out in paragraph 3 are limited to disclosing of the existence of such generated or manipulated content in an appropriate clear and visible manner that does not hamper the display of the work and disclosing the applicable copyrights, where relevant. It shall also not prevent law enforcement authorities from using AI systems intended to detect deep fakes and prevent, investigate and prosecute criminal offences linked with their use

Amendment 92

Proposal for a regulation

Article 5233 – paragraph ~~3 b (new)~~6

Text proposed by the Commission

Amendment

3b. The information referred to in paragraphs 1 to 3 shall be provided to the natural persons at the latest at the time of the first interaction or exposure. It shall be accessible to vulnerable persons, such as persons with disabilities or children, complete, where relevant and appropriate, with intervention or flagging procedures for the exposed natural person taking into account the generally acknowledged state of the art and relevant harmonised standards and common specifications.

6. Notified bodies shall have documented procedures in place ensuring that their personnel, committees, subsidiaries, subcontractors and any associated body or personnel of external bodies respect the confidentiality of the information which comes into their possession during the performance of conformity assessment activities, except when disclosure is required by law. The staff of notified bodies shall be bound to observe professional secrecy with regard to all information obtained in carrying out their tasks under this Regulation, except in relation to the notifying authorities of the Member State in which their activities are carried out.

Amendment 93

Proposal for a regulation

~~Article 56 – paragraph 1~~

Article 39 a (new)

Text proposed by the Commission

Amendment

~~1. A ‘European Artificial Intelligence Board’ (the ‘Board’) is established.~~

Article 39a

1. A ‘European Artificial Intelligence Board’ (the ‘Board’) is established as an independent body with its own legal personality to promote a trustworthy, effective and competitive internal market for artificial intelligence. The Board shall be organised in a way that guarantees the independence, objectivity and impartiality of its activities and shall have a secretariat, a strong mandate as well as sufficient resources and skilled personnel at its disposal for assistance in the proper performance of its tasks laid down in Article 58.

Exchange of knowhow and best practices

The Commission shall facilitate regular consultative meetings for the exchange of knowhow and best practices between the Member States' national authorities responsible for notification policy.

Amendment 94

Proposal for a regulation

Article 5640 – paragraph ~~2 –~~1 ~~introductory~~a ~~part~~(new)

Text proposed by the Commission

Amendment

~~2. The Board shall provide advice and assistance to the Commission in order to:~~

When issuing a standardisation request to European standardisation organisations in accordance with Article 10 of Regulation 1025/2012, the Commission shall specify that standards are coherent, easy to implement and drafted in such a way that they aim to fulfil in particular the following objectives:

2. The Board shall provide advice and assistance to the Commission and the Member States, when implementing Union law related to artificial intelligence as well as cooperate with the providers and users of AI systems in order to:

a) ensure that AI systems placed on the market or put into service in the Union are safe and respect Union values and public interests, and strengthen the Union's digital leadership;

b) promote investment and innovation in AI, as well as competitiveness and growth of the Union market;

c) enhance multi-stakeholder governance, by ensuring it is inclusive and representative of all relevant European stakeholders (e.g. civil society, researchers industry, SMEs).

d) contribute to strengthening global cooperation on standardisation in the field of AI that is consistent with Union values and interests.

The Commission shall request the European standardisation organisations to regularly report on their progress with regard to the above objectives.

Amendment 95

Proposal for a regulation

Article 5641 – paragraph ~~2 – point a~~1

Text proposed by the Commission

Amendment

~~(a) contribute to the effective cooperation of the national supervisory authorities and the Commission with regard to matters covered by this Regulation;~~

1. Where harmonised standards referred to in Article 40 do not exist or where the Commission considers that the relevant harmonised standards are insufficient or that there is a need to address specific safety or fundamental right concerns, the Commission may, by means of implementing acts, adopt common specifications in respect of the requirements set out in Chapter 2 of this Title. Those implementing acts shall be adopted in accordance with the examination procedure referred to in Article 74(2).

~~(a) promote and support the effective cooperation of the national supervisory authorities and the Commission;~~

1. Where harmonised standards referred to in Article 40 do not exist or where the Commission considers that the relevant harmonised standards are insufficient or that there is a need to address specific safety or fundamental right concerns, the Commission may, after consulting the AI Board referred to in Article 56 and the responsible authorities and organizations for a given sector, by means of implementing acts, adopt common specifications in respect of the requirements set out in Chapter 2 of this Title. Those implementing acts shall be adopted in accordance with the examination procedure referred to in Article 74(2).

Amendment 96

Proposal for a regulation

Article 5641 – paragraph 2 ~~– point c~~

Text proposed by the Commission

Amendment

~~(c) assist the national supervisory authorities and the Commission in ensuring the consistent application of this Regulation.~~

2. The Commission, when preparing the common specifications referred to in paragraph 1, shall gather the views of relevant bodies or expert groups established under relevant sectorial Union law.

(c) assist the Commission, national supervisory authorities and other national competent authorities in ensuring the consistent application of this Regulation, in particular in line with the consistency mechanism referred to in Article 59a (3).

2. When preparing the common specifications referred to in paragraph 1, the Commission shall fulfil the objectives referred of Article 40(2) and gather the views of relevant bodies or expert groups established under relevant sectorial Union law as well as relevant sector-specific stakeholders.

Amendment 97

Proposal for a regulation

Article 5643 – paragraph ~~2 – point c a (new)~~6

Text proposed by the Commission

Amendment

~~(ca) assist providers and users of AI systems to meet the requirements of this Regulation, as well as those set out in present and future Union legislation, in particular SMEs and start-ups.~~

6. The Commission is empowered to adopt delegated acts to amend paragraphs 1 and 2 in order to subject high-risk AI systems referred to in points 2 to 8 of Annex III to the conformity assessment procedure referred to in Annex VII or parts thereof. The Commission shall adopt such delegated acts taking into account the effectiveness of the conformity assessment procedure based on internal control referred to in Annex VI in preventing or minimizing the risks to health and safety and protection of fundamental rights posed by such systems as well as the availability of adequate capacities and resources among notified bodies.

6. The Commission is empowered to adopt delegated acts to amend paragraphs 1 and 2 in order to subject high-risk AI systems referred to in points 2 to 8 of Annex III to the conformity assessment procedure referred to in Annex VII or parts thereof. The Commission shall adopt such delegated acts taking into account the effectiveness of the conformity assessment procedure based on internal control referred to in Annex VI in preventing or minimizing the risks to health, safety, the environment and protection of fundamental rights posed by such systems as well as the availability of adequate capacities and resources among notified bodies.

Amendment 98

Proposal for a regulation

~~Article 56 – paragraph 2 – point c b (new)~~

Article 52 – title

Text proposed by the Commission

Amendment

(cb) provide particular oversight, monitoring and regular dialogue with the providers of general purpose AI systems about their compliance with this Regulation. Any such meeting shall be open to national supervisory authorities, notified bodies and market surveillance authorities to attend and contribute;

Transparency obligations for certain AI systems

Transparency obligations for AI systems

Amendment 99

Proposal for a regulation

Article 5652 – paragraph ~~2 – point c~~3 ca (new)

Text proposed by the Commission

Amendment

~~(cc) propose amendments to Annex I and III.~~

3a. Providers of any AI system should document and make available upon request the parameters regarding the environmental impact, including but not limited to resource consumption, resulting from the design, data management and training, the underlying infrastructures of the AI system, and of the methods to reduce such impact.

Amendment 100

Proposal for a regulation

Article 5653 – paragraph ~~2 a (new)~~1

Text proposed by the Commission

Amendment

2a. The Board shall act as a reference point for advice and expertise for Union institutions, bodies, offices and agencies as well as for other relevant stakeholders on matters related to artificial intelligence.

1. AI regulatory sandboxes established by one or more Member States competent authorities or the European Data Protection Supervisor shall provide a controlled environment that facilitates the development, testing and validation of innovative AI systems for a limited time before their placement on the market or putting into service pursuant to a specific plan. This shall take place under the direct supervision and guidance by the competent authorities with a view to ensuring compliance with the requirements of this Regulation and, where relevant, other Union and Member States legislation supervised within the sandbox.

1. AI regulatory sandboxes established by one or more Member States competent authorities or the European Data Protection Supervisor shall provide a controlled environment that facilitates the development, testing and validation of innovative AI systems and secure processing of personal data for a limited time before their placement on the market or putting into service pursuant to a specific plan. This shall take place under the direct supervision and guidance by the competent authorities with a view to ensuring compliance with the requirements of this Regulation and, where relevant, other Union and Member States legislation supervised within the sandbox.

Amendment 101

Proposal for a regulation

~~Article 57 – title~~

Article 53 – paragraph 1 a (new)

Text proposed by the Commission

Amendment

~~Structure of the Board~~

1a. The organisers of AI regulatory sandboxes shall ensure an easy access for SMEs and start-ups by facilitating and supporting their participation.

~~Mandate and structure of the Board~~

Amendment 102

Proposal for a regulation

Article 5753 – paragraph 1 b (new)

Text proposed by the Commission

Amendment

1. The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority, and the European Data Protection Supervisor. Other national authorities may be invited to the meetings, where the issues discussed are of relevance for them.

1b. The controllers of personal data referred to in Article 4 (7) of the Regulation (EU) 2016/679 may further process personal data in an AI regulatory sandbox to the extent that it is necessary for the purposes of development, testing and validation of AI systems. Right of processing is subject to appropriate safeguards for the fundamental rights and freedoms of natural persons. This processing shall not be considered incompatible with the initial purposes.

1. The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority. Other national authorities may be invited to the meetings, where the issues discussed are of relevance for them. The Board composition shall be gender balanced.

The European Data Protection Supervisor, the Chairperson of the EU Agency for Fundamental Rights, the Executive director of the EU Agency for Cybersecurity, the Chair of the High Level Expert Group on AI, the Director-General of the Joint Research Centre, and the presidents of the European Committee for Standardization, the European Committee for Electrotechnical Standardization, and the European Telecommunications Standards Institute shall be invited as permanent observers with the right to speak but without voting rights.

Amendment 103

Proposal for a regulation

Article 5753 – paragraph 23

Text proposed by the Commission

Amendment

2. The Board shall adopt its rules of procedure by a simple majority of its members, following the consent of the Commission. The rules of procedure shall also contain the operational aspects related to the execution of the Board’s tasks as listed in Article 58. The Board may establish sub-groups as appropriate for the purpose of examining specific questions.

3. The AI regulatory sandboxes shall not affect the supervisory and corrective powers of the competent authorities. Any significant risks to health and safety and fundamental rights identified during the development and testing of such systems shall result in immediate mitigation and, failing that, in the suspension of the development and testing process until such mitigation takes place.

2. The Board shall adopt its rules of procedure by a simple majority of its members with the assistance of its secretariat. The rules of procedure shall also contain the operational aspects related to the execution of the Board’s tasks as listed in Article 58. The Board may establish standing or temporary sub-groups as appropriate for the purpose of examining specific questions.

3. The AI regulatory sandboxes shall not affect the supervisory and corrective powers of the competent authorities. Any significant risks to health, safety, the environment or fundamental rights identified during the development and testing of such systems shall result in immediate mitigation and, failing that, in the suspension of the development and testing process until such mitigation takes place.

Amendment 104

Proposal for a regulation

Article 5753 – paragraph 35

Text proposed by the Commission

Amendment

3. The Board shall be chaired by the Commission. The Commission shall convene the meetings and prepare the agenda in accordance with the tasks of the Board pursuant to this Regulation and with its rules of procedure. The Commission shall provide administrative and analytical support for the activities of the Board pursuant to this Regulation.

5. Member States’ competent authorities that have established AI regulatory sandboxes shall coordinate their activities and cooperate within the framework of the European Artificial Intelligence Board. They shall submit annual reports to the Board and the Commission on the results from the implementation of those scheme, including good practices, lessons learnt and recommendations on their setup and, where relevant, on the application of this Regulation and other Union legislation supervised within the sandbox.

3. The Board shall be co-chaired by the Commission and a representative chosen from among the delegates of the Member States. The Board’s secretariat shall convene the meetings and prepare the agenda in accordance with the tasks of the Board pursuant to this Regulation and with its rules of procedure. The Board’s secretariat shall also provide administrative and analytical support for the activities of the Board pursuant to this Regulation.

5. Member States’ competent authorities shall coordinate their activities with regards to AI regulatory sandboxes and cooperate within the framework of the European Artificial Intelligence Board. They shall submit annual reports to the Board and the Commission on the results from the implementation of those scheme, including good practices, lessons learnt and recommendations on their setup and, where relevant, on the application of this Regulation and other Union legislation supervised within the sandbox.

Amendment 105

Proposal for a regulation

Article 5754 – paragraph 41 – point a – point iii a (new)

Text proposed by the Commission

Amendment

(iiia) safety and resilience of transport systems, infrastructure and networks.

4. The Board shall regularly invite external experts, in particular from organisations representing the interests of the providers and users of AI systems, SMEs and start-ups, civil society organisations, trade unions, representatives of affected persons, academia and researchers, testing and experimentation facilities and standardisation organisations, to attend its meetings in order to ensure accountability and appropriate participation of external actors. The agenda and the minutes of its meetings shall be published online. The Commission may facilitate exchanges between the Board and other Union bodies, offices, agencies and advisory groups.

Amendment 106

Proposal for a regulation

~~Article 57 – paragraph 4 a (new)~~

Article 55 – title

Text proposed by the Commission

Amendment

4a. Without prejudice to paragraph 4, the Board’s Secretariat shall organise four additional meetings between the Board and the High Level Expert Group on Trustworthy AI to allow them to share their practical and technical expertise every quarter of the year.

Measures for small-scale providers and users

Measures for SMEs, start-ups and users

Amendment 107

Proposal for a regulation

Article 5855 – paragraph 1 – ~~introductory~~point ~~part~~a

Text proposed by the Commission

Amendment

~~When providing advice and assistance to the Commission in the context of Article 56(2), the Board shall in particular:~~

(a) provide small-scale providers and start-ups with priority access to the AI regulatory sandboxes to the extent that they fulfil the eligibility conditions;

~~When providing advice and assistance to the Commission and the Member States in the context of Article 56(2), the Board shall in particular:~~

(a) provide SMEs and start-ups with priority access to the AI regulatory sandboxes to the extent that they fulfil the eligibility conditions;

Amendment 108

Proposal for a regulation

Article 5855 – paragraph 1 – point ab

Text proposed by the Commission

Amendment

~~(a) collect and share expertise and best practices among Member States;~~

(b) organise specific awareness raising activities about the application of this Regulation tailored to the needs of the small-scale providers and users;

~~(a) collect and share expertise and best practices among Member States, including on the promotion of AI literacy and awareness raising initiatives on Artificial Intelligence and this Regulation;~~

(b) organise specific awareness raising activities about the application of this Regulation tailored to the needs of SMEs, start-ups and users;

Amendment 109

Proposal for a regulation

Article 5855 – paragraph 1 – point ~~a a (new)~~c

Text proposed by the Commission

Amendment

~~(aa) promote and support the cooperation among national supervisory authorities and the Commission;~~

(c) where appropriate, establish a dedicated channel for communication with small-scale providers and user and other innovators to provide guidance and respond to queries about the implementation of this Regulation.

(c) where appropriate, establish a dedicated channel for communication with SMEs and user, start-ups and other innovators to provide guidance and respond to queries about the implementation of this Regulation.

Amendment 110

Proposal for a regulation

Article 5855 – paragraph ~~1 –~~2 ~~point~~a b(new)

Text proposed by the Commission

Amendment

~~(b) contribute to uniform administrative practices in the Member States, including for the functioning of regulatory sandboxes referred to in Article 53;~~

2a. Where appropriate, Member States shall find synergies and cooperate with relevant instruments funded by Union programmes, such as the European Digital Innovation Hubs.

(b) contribute to uniform administrative practices in the Member States, including for the assessment, establishing, managing with the meaning of fostering cooperation and guaranteeing consistency among regulatory sandboxes, and functioning of regulatory sandboxes referred to in Article 53;

Amendment 111

Proposal for a regulation

Article 5857 – paragraph 1 ~~– point c – introductory part~~

Text proposed by the Commission

Amendment

~~(c) issue opinions, recommendations or written contributions on matters related to the implementation of this Regulation, in particular~~

~~(c) issue guidelines, recommendations or written contributions on matters related to the implementation of this Regulation, in particular~~

1. The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority, the European Data Protection Supervisor, AI ethics experts and industry representatives. Other national, regional and local authorities may be invited to the meetings, where the issues discussed are of relevance for them.

Amendment 112

Proposal for a regulation

Article 5857 – paragraph ~~1 – point c – point ii a (new)~~3

Text proposed by the Commission

Amendment

~~(iia) on the provisions related to post market monitoring as referred to in Article 61,~~

3. The Board shall be co-chaired by the Commission and representative chosen from among the delegates of the Member States. The Commission shall convene the meetings and prepare the agenda in accordance with the tasks of the Board pursuant to this Regulation and with its rules of procedure. The Commission shall provide administrative and analytical support for the activities of the Board pursuant to this Regulation.

Amendment 113

Proposal for a regulation

Article 5857 – paragraph ~~1 – point c – point iii~~3 a (new)

Text proposed by the Commission

Amendment

(iiia) on the need for the amendment of each of the Annexes as referred to in Article 73, as well as all other provisions in this Regulation that the Commission can amend, in light of the available evidence.

3a. The Board shall organise consultations with stakeholders at least twice a year. Such stakeholders shall include representatives from industry, SMEs and start-ups, civil society organisations such as NGOs, consumer associations, the social partners and academia, to assess the evolution of trends in technology, issues related to the implementation and the effectiveness of this Regulation, regulatory gaps or loopholes observed in practice.

Amendment 114

Proposal for a regulation

Article 5857 – paragraph ~~1 – point c – point iii b (new)~~4

Text proposed by the Commission

Amendment

~~(iiib) on activities and decisions of Member States regarding post-market monitoring, information sharing, market surveillance referred to in Title VIII;~~

4. The Board may invite external experts and observers to attend its meetings and may hold exchanges with interested third parties to inform its activities to an appropriate extent. To that end, the Commission may facilitate exchanges between the Board and other Union bodies, offices, agencies and advisory groups. The Board shall actively reach out to and hear representatives from groups, which are more vulnerable to discriminatory effects posed by AI, such as people with disabilities.

Amendment 115

Proposal for a regulation

Article 5859 – paragraph ~~1 – point c – point iii c (new)~~4

Text proposed by the Commission

Amendment

(iiic) on common criteria for market operators and competent authorities having the same understanding of concepts such as the 'generally acknowledged state of the art' referred to in Articles 9(3), 13(1), 14(4), 23a(3) or 52(3a), 'foreseeable risks' referred to in Articles 9(2), point (a), and 'foreseeable misuse' referred to in Article 3(13), Article 9(2), point (b), Article 9(4), Article 13(3), point (b)(iii), Article 14(2) and Article 23a(3c);

4. Member States shall ensure that national competent authorities are provided with adequate financial and human resources to fulfil their tasks under this Regulation. In particular, national competent authorities shall have a sufficient number of personnel permanently available whose competences and expertise shall include an in-depth understanding of artificial intelligence technologies, data and data computing, fundamental rights, health and safety risks and knowledge of existing standards and legal requirements.

4. Member States shall ensure that national competent authorities are provided with adequate financial and human resources to fulfil their tasks under this Regulation. In particular, national competent authorities shall have a sufficient number of personnel permanently available whose competences and expertise shall include an in-depth understanding of artificial intelligence technologies, data and data computing, fundamental rights, health, safety and environmental risks and knowledge of existing standards and legal requirements.

Amendment 116

Proposal for a regulation

Article 5859 – paragraph ~~1 – point c – point iii~~4 da (new)

Text proposed by the Commission

Amendment

~~(iiid) on the verification of the alignment with the legal acts listed in Annex II, including with the implementation matters related to those acts.~~

4a. Any information and documentation obtained by the national competent authorities pursuant to the provisions of this Article shall be treated in compliance with the confidentiality obligations set out in Article 70.

Amendment 117

Proposal for a regulation

Article 5860 – paragraph ~~1 – point c – point iii e (new)~~3

Text proposed by the Commission

Amendment

~~(iiie) on the respect of the general principles applicable to all AI systems referred to in Article 4a;~~

3. Information contained in the EU database shall be accessible to the public.

3. Information contained in the EU database shall be accessible to the public, user-friendly, easily navigable and machine-readable.

Amendment 118

Proposal for a regulation

Article 5860 – paragraph ~~1 – point c~~5 a (new)

Text proposed by the Commission

Amendment

(ca) carry out annual reviews and analyses of the complaints sent to and findings made by national supervisory authorities, of the serious incidents and malfunctioning reports referred to in Article 62, and of the new registration in the EU Database referred to in Article 60 to identify trends and potential emerging issues threatening the future health and safety and fundamental rights of citizens that are not adequately addressed by this Regulation;

5a. Any information and documentation obtained by the Commission and Member States pursuant to this Article shall be treated in compliance with the confidentiality obligations set out in Article 70.

Amendment 119

Proposal for a regulation

Article 5861 – paragraph ~~1 – point c b (new)~~2

Text proposed by the Commission

Amendment

~~(cb) carry out biannual horizontal scanning and foresight exercises to extrapolate the impact that scientific developments, trends and emerging issues can have on the Union;~~

2. The post-market monitoring system shall actively and systematically collect, document and analyse relevant data provided by users or collected through other sources on the performance of high-risk AI systems throughout their lifetime, and allow the provider to evaluate the continuous compliance of AI systems with the requirements set out in Title III, Chapter 2. Post-market monitoring must include continuous analysis of the AI environment, including other devices, software, and other AI systems that will interact with the AI system.

Amendment 120

Proposal for a regulation

Article 5865 – paragraph 1 ~~– point c c (new)~~

Text proposed by the Commission

Amendment

(cc) annually publish recommendations to the Commission, in particular on the categorisation of prohibited practices, high-risk systems, and codes of conduct for AI systems that are not classified as high-risk;

1. AI systems presenting a risk shall be understood as a product presenting a risk defined in Article 3, point 19 of Regulation (EU) 2019/1020 insofar as risks to the health or safety or to the protection of fundamental rights of persons are concerned.

1. AI systems presenting a risk shall be understood as a product presenting a risk defined in Article 3, point 19 of Regulation(EU) 2019/1020 insofar as risks to health, safety or the environment, or to the protection of fundamental rights of persons are concerned.

Amendment 121

Proposal for a regulation

Article 5867 – paragraph 1 ~~– point c d (new)~~

Text proposed by the Commission

Amendment

~~(cd) encourage and facilitate the drawing up of codes of conduct as referred to in Article 69;~~

1. Where, having performed an evaluation under Article 65, the market surveillance authority of a Member State finds that although an AI system is in compliance with this Regulation, it presents a risk to the health or safety of persons, to the compliance with obligations under Union or national law intended to protect fundamental rights or to other aspects of public interest protection, it shall require the relevant operator to take all appropriate measures to ensure that the AI system concerned, when placed on the market or put into service, no longer presents that risk, to withdraw the AI system from the market or to recall it within a reasonable period, commensurate with the nature of the risk, as it may prescribe.

1. Where, having performed an evaluation under Article 65, the market surveillance authority of a Member State finds that although an AI system is in compliance with this Regulation, it presents a risk to the health or safety of persons, to the environment, to the compliance with obligations under Union or national law intended to protect fundamental rights or to other aspects of public interest protection, it shall require the relevant operator to take all appropriate measures to ensure that the AI system concerned, when placed on the market or put into service, no longer presents that risk, to withdraw the AI system from the market or to recall it within a reasonable period, commensurate with the nature of the risk, as it may prescribe.

Amendment 122

Proposal for a regulation

Article 5869 – paragraph ~~1 – point c e (new)~~3

Text proposed by the Commission

Amendment

~~(ce) coordinate among national competent authorities and make sure that the consistency mechanism in Article 59a(3) is observed, in particular for all major cross-border cases;~~

3. Codes of conduct may be drawn up by individual providers of AI systems or by organisations representing them or by both, including with the involvement of users and any interested stakeholders and their representative organisations. Codes of conduct may cover one or more AI systems taking into account the similarity of the intended purpose of the relevant systems.

3. Codes of conduct may be drawn up by national, regional or local authorities, by individual providers of AI systems or by organisations representing them or by both, including with the involvement of users and any interested stakeholders and their representative organisations. Codes of conduct may cover one or more AI systems taking into account the similarity of the intended purpose of the relevant systems.

Amendment 123

Proposal for a regulation

Article 5872 – paragraph 1 – point ~~c f (new)~~a

Text proposed by the Commission

Amendment

(cf) adopt binding decisions for national supervisory authorities in case the consistency mechanism is not able to solve the conflict among national supervisory authorities as it is clarified in Article 59a (6).

(a) the nature, gravity and duration of the infringement and of its consequences;

(a) the nature, gravity and duration of the infringement and of its consequences; taking into account the number of subjects affected and the level of damage suffered by them, the intentional or negligent character of the infringement and any relevant previous infringement;

Amendment 124

Proposal for a regulation

Article 5872 – paragraph 1 – point cb ga (new)

Text proposed by the Commission

Amendment

~~(cg) provide guidance material to providers and users regarding the compliance with the requirements set out in this Regulation. In particular, it shall issue guidelines:~~

(ba) the degree of cooperation with the supervisory authority, in order to remedy the infringement and mitigate the possible adverse effects of the infringement;

~~i) for the trustworthy AI technical assessment referred to in Article 4a,~~

~~ii) for the methods for performing the conformity assessment based on internal control referred to Article 43;~~

~~iii) to facilitate compliance with the reporting of serious incidents or malfunctioning referred to in Article 62;~~

iv) on any other concrete procedures to be performed by providers and users when complying with this Regulation, in particular those regarding the documentation to be delivered to notified bodies and methods to provide authorities with other relevant information.

Amendment 125

Proposal for a regulation

Article 5872 – paragraph 1 – point cb hb (new)

Text proposed by the Commission

Amendment

~~(ch) provide specific guidance to support SMEs and start-ups in complying with the obligations set out in this Regulation;~~

(bb) any action taken by the provider to mitigate the damage suffered by subjects;

Amendment 126

Proposal for a regulation

Article 5872 – paragraph 1 – point c ia (new)

Text proposed by the Commission

Amendment

(ci) raise awareness and provide guidance material to providers and users regarding the compliance with the requirement to put in place tools and measures to ensure a sufficient level of AI literacy in line with Article 4b;

(ca) any other aggravating or mitigating factor applicable to the circumstances of the case, such as financial benefits gained, or losses avoided, directly or indirectly, from the infringement.

Amendment 127

Proposal for a regulation

Article 5875 – paragraph 1 ~~– point c j (new)~~

Regulation (EC) No 300/2008

Article 4 – paragraph 3 – subparagraph 3

Text proposed by the Commission

Amendment

~~(cj) contribute to the Union efforts to cooperate with third countries and international organisations in view of promoting a common global approach towards trustworthy AI;~~

When adopting detailed measures related to technical specifications and procedures for approval and use of security equipment concerning Artificial Intelligence systems in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Chapter 2, Title III of that Regulation shall be taken into account.

When adopting detailed measures related to technical specifications and procedures for approval and use of security equipment concerning Artificial Intelligence systems in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, without interfering with existing governance, the requirements set out in Chapter 2, Title III of that Regulation shall be taken into account.

_______

________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).”

Amendment 128

Proposal for a regulation

Article 5876 – paragraph 1 ~~– point c k (new)~~

Regulation (EU) No 167/2013

Article 17 – paragraph 5 – subparagraph 3

Text proposed by the Commission

Amendment

~~(ck) issue yearly reports on the implementation of this Regulation, including an assessment of its impact on economic operators;~~

When adopting delegated acts pursuant to the first subparagraph concerning artificial intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

When adopting delegated acts pursuant to the first subparagraph concerning artificial intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, without interfering with existing governance, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

_______

________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).”

Amendment 129

Proposal for a regulation

Article 5878 – paragraph 1 ~~– point c l (new)~~

Directive 2014/90/EU

Article 8 – paragraph 4

Text proposed by the Commission

Amendment

~~(cl) provide guidance on the governance of research and development.~~

4. “For Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, when carrying out its activities pursuant to paragraph 1 and when adopting technical specifications and testing standards in accordance with paragraphs 2 and 3, the Commission shall take into account the requirements set out in Title III, Chapter 2 of that Regulation.

4. “For Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, when carrying out its activities pursuant to paragraph 1 and when adopting technical specifications and testing standards in accordance with paragraphs 2 and 3, and without interfering with existing governance, the Commission shall take into account the requirements set out in Title III, Chapter 2 of that Regulation.

_______

________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).”

Amendment 130

Proposal for a regulation

Article 5979 – ~~title~~paragraph 1

Directive (EU) 2016/797

Article 5 – paragraph 12

Text proposed by the Commission

Amendment

~~Designation of national competent authorities~~

12. “When adopting delegated acts pursuant to paragraph 1 and implementing acts pursuant to paragraph 11 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

~~Designation of national supervisory authorities~~

12. “When adopting delegated acts pursuant to paragraph 1 and implementing acts pursuant to paragraph 11 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, without interfering with existing governance, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

_______

________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).”

Amendment 131

Proposal for a regulation

Article 5980 – paragraph 1

Regulation (EU) 2018/858

Article 5 – paragraph 4

Text proposed by the Commission

Amendment

1. National competent authorities shall be established or designated by each Member State for the purpose of ensuring the application and implementation of this Regulation. National competent authorities shall be organised so as to safeguard the objectivity and impartiality of their activities and tasks.

4. “When adopting delegated acts pursuant to paragraph 3 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council *, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

~~1. Each Member State shall establish or designate one national supervisory authority, which shall be organised so as to safeguard the objectivity and impartiality of its activities and tasks.~~

4. “When adopting delegated acts pursuant to paragraph 3 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council *, without interfering with existing governance, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

_______

________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).”

Amendment 132

Proposal for a regulation

Article 5981 – paragraph 21 – point 1

Regulation (EU) 2018/1139

Article 17 – paragraph 3

Text proposed by the Commission

Amendment

2. Each Member State shall designate a national supervisory authority among the national competent authorities. The national supervisory authority shall act as notifying authority and market surveillance authority unless a Member State has organisational and administrative reasons to designate more than one authority.

3. “Without prejudice to paragraph 2, when adopting implementing acts pursuant to paragraph 1 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

2. The national supervisory authority shall be in charge to ensure the application and implementation of this Regulation. With regard to high-risk AI systems, related to products to which legal acts listed in Annex II apply, the competent authorities designated under those legal acts shall continue to lead the administrative procedures. However, to the extent a case involves aspects covered by this Regulation, the competent authorities shall be bound by measures issued by the national supervisory authority designated under this Regulation. The national supervisory authority shall also act as notifying authority and market surveillance authority.

3. “Without prejudice to paragraph 2, and to the certification, oversight and enforcement system referred to in Article 62 of this Regulation, when adopting implementing acts pursuant to paragraph 1 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

_______

________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).”

Amendment 133

Proposal for a regulation

Article 5981 – paragraph 31 – point 2

Regulation (EU) 2018/1139

Article 19 – paragraph 4

Text proposed by the Commission

Amendment

~~3. Member States shall inform the Commission of their designation or designations and, where applicable, the reasons for designating more than one authority.~~

4. When adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

3. The national competent authority in each Member State shall be the lead authority, ensure adequate coordination and act as single point of contact for this Regulation. Member States shall inform the Commission of their designations. In addition, the central contact point of each Member State should be contactable through electronic communications means.

4. Without prejudice to the certification, oversight and enforcement system referred to in Article 62 of this Regulation, when adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation(EU) YYY/XX [on Artificial Intelligence], only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Amendment 134

Proposal for a regulation

Article 5981 – paragraph 41 – point 3

Regulation (EU) 2018/1139

Article 43 – paragraph 4

Text proposed by the Commission

Amendment

4. When adopting implementing acts pursuant to paragraph 1 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

4. Member States shall ensure that national supervisory authority is provided with adequate financial and human resources to fulfil its tasks under this Regulation. In particular, national supervisory authorities shall have a sufficient number of permanently available personnel, whose competences and expertise shall include an in-depth understanding of artificial intelligence technologies, data, data protection and data computing, cybersecurity, competition law, fundamental rights, health and safety risks as well as knowledge of existing standards and legal requirements.

4. Without prejudice to the certification, oversight and enforcement system referred to in Article 62 of this Regulation, when adopting implementing acts pursuant to paragraph 1 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX[on Artificial Intelligence], only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Amendment 135

Proposal for a regulation

Article 5981 – paragraph 41 a– ~~(new)~~point 4

Regulation (EU) 2018/1139

Article 47 – paragraph 3

Text proposed by the Commission

Amendment

4a. The national competent authority shall satisfy the minimum cybersecurity requirements set out for public administration entities identified as operators of essential services pursuant to Directive (…) on measures for a high common level of cybersecurity across the Union, repealing Directive (EU) 2016/1148.

3. When adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

3. Without prejudice to the certification, oversight and enforcement system referred to in Article 62 of this Regulation, when adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation(EU) YYY/XX [on Artificial Intelligence], only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Amendment 136

Proposal for a regulation

Article 5981 – paragraph 41 b– ~~(new)~~point 5

Regulation (EU) 2018/1139

Article 57 – paragraph 3

Text proposed by the Commission

Amendment

4b. Any information and documentation obtained by the national supervisory authority pursuant to the provisions of this Article shall be treated in compliance with the confidentiality obligations set out in Article 70.

When adopting those implementing acts concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Without prejudice to the certification, oversight and enforcement system referred to in Article 62of this Regulation, when adopting those implementing acts concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account."

Amendment 137

Proposal for a regulation

Article 5981 – paragraph 51 – point 6

Regulation (EU) 2018/1139

Article 58 – paragraph 3

Text proposed by the Commission

Amendment

5. Member States shall report to the Commission on an annual basis on the status of the financial and human resources of the national competent authorities with an assessment of their adequacy. The Commission shall transmit that information to the Board for discussion and possible recommendations.

3. When adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] , the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account..

5. Member States shall report to the Commission on an annual basis on the status of the financial and human resources of the national supervisory authority with an assessment of their adequacy. The Commission shall transmit that information to the Board for discussion and possible recommendations.

Amendment 138

Proposal for a regulation

Article 5982 – paragraph 61

Regulation (EU) 2019/2144

Article 11

Text proposed by the Commission

Amendment

~~6. The Commission shall facilitate the exchange of experience between national competent authorities.~~

3. “When adopting the implementing acts pursuant to paragraph 2, concerning artificial intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

~~6. The Commission and the Board shall facilitate the exchange of experience between national supervisory authorities.~~

3. “When adopting the implementing acts pursuant to paragraph 2, concerning artificial intelligence systems which are safety components in the meaning of Regulation(EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, without interfering with existing governance, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

_______

________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).”

Amendment 139

Proposal for a regulation

Article 5984 – paragraph 73 a (new)

Text proposed by the Commission

Amendment

7. National competent authorities may provide guidance and advice on the implementation of this Regulation, including to small-scale providers. Whenever national competent authorities intend to provide guidance and advice with regard to an AI system in areas covered by other Union legislation, the competent national authorities under that Union legislation shall be consulted, as appropriate. Member States may also establish one central contact point for communication with operators.

3a. Within [two years after the date of application of this Regulation referred to in Article 85(2)] and every two years thereafter, the Commission shall evaluate the impact and effectiveness of the Regulation with regards to the energy use and other environmental impact of AI systems and evaluate bringing legislation to regulate the energy efficiency of ICT systems in order for the sector to contribute to Union climate strategy and targets.

7. National supervisory authorities may provide guidance and advice on the implementation of this Regulation, including to SMEs and start-ups, as long as it is not in contradiction with the Board’s or the Commission’s guidance and advice. Whenever national supervisory authorities intend to provide guidance and advice with regard to an AI system in areas covered by other Union legislation, the competent national authorities under that Union legislation shall be consulted as appropriate.

Amendment 140

Proposal for a regulation

Article 5984 – paragraph 87 a (new)

Text proposed by the Commission

Amendment

~~8. When Union institutions, agencies and bodies fall within the scope of this Regulation, the European Data Protection Supervisor shall act as the competent authority for their supervision.~~

7a. Any relevant future delegated or implementing acts to Regulations listed in Annex II, section B, introducing mandatory requirements for High-Risk AI systems laid down in this Regulation, shall take into account the regulatory specificities of each sector and shall not overlap with existing governance, conformity assessment, and enforcement mechanisms and authorities established therein.

8. When Union institutions, agencies and bodies fall within the scope of this Regulation, the European Data Protection Supervisor shall act as the competent authority for their supervision and coordination.

Amendment 141

Proposal for a regulation

~~Article 59 a (new)~~

Annex I – title

Text proposed by the Commission

Amendment

~~Article 59a~~

ARTIFICIAL INTELLIGENCE TECHNIQUES AND APPROACHESreferred to in Article 3, point 1

~~Consistency mechanism for cross-border cases~~

ARTIFICIAL INTELLIGENCE TECHNIQUES AND APPROACHES referred to in Article 3, point 1

~~1. Each national supervisory authority shall perform the tasks assigned to and the exercise of the powers conferred on it in accordance with this Regulation on the territory of its own Member State.~~

2. In the event of a cross-border case involving two or more national supervisory authorities, the national supervisory authority of the Member State where the provider’s or user’s place of central administration in the Union is established or where the authorised representative is appointed, shall be competent to act as lead national supervisory authority for a cross-border case that involves an AI-system.

3. In the case referred to in paragraph 2, the national supervisory authorities shall cooperate, exchange all relevant information with each other in due time, provide mutual assistance and execute joint operations. National supervisory authorities shall cooperate in order to reach a consensus.

4. In case of a serious disagreement between two or more national supervisory authorities, the lead national supervisory authority shall notify the Board and communicate without delay all relevant information related to the case to the Board.

~~5. The Board shall within three months of the notification referred to in paragraph 4, issue a binding decision to the national supervisory authorities.~~

Amendment 142

Proposal for a regulation

~~Article~~Annex 62III – paragraph 1 – ~~subparagraph~~point 12 – point a

~~Text proposed by the Commission~~

~~Amendment~~

Such notification shall be made immediately after the provider has established a causal link between the AI system and the incident or malfunctioning or the reasonable likelihood of such a link, and, in any event, not later than 15 days after the providers becomes aware of the serious incident or of the malfunctioning.

Such notification shall be made without undue delay after the provider has established a causal link between the AI system and the serious incident or malfunctioning or the reasonable likelihood of such a link, and, in any event, not later than 72 hours after the provider becomes aware of the serious incident or of the malfunctioning.

~~Amendment 143~~

~~Proposal for a regulation~~

~~Article 62 – paragraph 1 – subparagraph 1 a (new)~~

~~Text proposed by the Commission~~

~~Amendment~~

No report under this Article is required if the serious incident or malfunctioning is also to be reported by providers to comply with obligations laid down by other acts of Union law. In that case, the authorities competent under those acts of Union law shall forward the received report to the national supervisory authority designated under this Regulation.

~~Amendment 144~~

~~Proposal for a regulation~~

~~Article 69 – paragraph 1~~

~~Text proposed by the Commission~~

~~Amendment~~

1. The Commission and the Member States shall encourage and facilitate the drawing up of codes of conduct intended to foster the voluntary application to AI systems other than high-risk AI systems of the requirements set out in Title III, Chapter 2 on the basis of technical specifications and solutions that are appropriate means of ensuring compliance with such requirements in light of the intended purpose of the systems.

1. The Commission, the Board and the Member States shall encourage and facilitate the drawing up of codes of conduct intended, including where they are drawn up in order to demonstrate how AI systems respect the principles set out in Article 4a and can thereby be considered trustworthy, to foster the voluntary application to AI systems other than high-risk AI systems of the requirements set out in Title III, Chapter 2 on the basis of technical specifications and solutions that are appropriate means of ensuring compliance with such requirements in light of the intended purpose of the systems.

~~Amendment 145~~

~~Proposal for a regulation~~

~~Article 69 – paragraph 2~~

~~Text proposed by the Commission~~

~~Amendment~~

2. The Commission and the Board shall encourage and facilitate the drawing up of codes of conduct intended to foster the voluntary application to AI systems of requirements related for example to environmental sustainability, accessibility for persons with a disability, stakeholders participation in the design and development of the AI systems and diversity of development teams on the basis of clear objectives and key performance indicators to measure the achievement of those objectives.

~~2. Codes of conduct intended to foster the voluntary compliance with the principles underpinning trustworthy AI systems, shall, in particular:~~

~~(a) aim for a sufficient level of AI literacy among their staff and other persons dealing with the operation and use of AI systems in order to observe such principles;~~

(b) assess to what extent their AI systems may affect vulnerable persons or groups of persons, including children, the elderly, migrants and persons with disabilities or whether measures could be put in place in order to increase accessibility, or otherwise support such persons or groups of persons;

~~(c) consider the way in which the use of their AI systems may have an impact or can increase diversity, gender balance and equality;~~

~~(d) have regard to whether their AI systems can be used in a way that, directly or indirectly, may residually or significantly reinforce existing biases or inequalities;~~

~~(e) reflect on the need and relevance of having in place diverse development teams in view of securing an inclusive design of their systems;~~

~~(f) give careful consideration to whether their systems can have a negative societal impact, notably concerning political institutions and democratic processes;~~

(g) evaluate how AI systems can contribute to environmental sustainability and in particular to the Union’s commitments under the European Green Deal and the European Declaration on Digital Rights and Principles.

~~Amendment 146~~

~~Proposal for a regulation~~

~~Article 69 – paragraph 3~~

~~Text proposed by the Commission~~

~~Amendment~~

3. Codes of conduct may be drawn up by individual providers of AI systems or by organisations representing them or by both, including with the involvement of users and any interested stakeholders and their representative organisations. Codes of conduct may cover one or more AI systems taking into account the similarity of the intended purpose of the relevant systems.

3. Codes of conduct may be drawn up by individual providers of AI systems or by organisations representing them or by both, including with the involvement of users and any interested stakeholders, including scientific researchers, and their representative organisations, in particular trade unions, and consumer organisations. Codes of conduct may cover one or more AI systems taking into account the similarity of the intended purpose of the relevant systems. Providers adopting codes of conduct will designate at least one natural person responsible for internal monitoring.

~~Amendment 147~~

~~Proposal for a regulation~~

~~Article 69 – paragraph 4~~

~~Text proposed by the Commission~~

~~Amendment~~

4. The Commission and the Board shall take into account the specific interests and needs of the small-scale providers and start-ups when encouraging and facilitating the drawing up of codes of conduct.

~~4. The Commission and the Board shall take into account the specific interests and needs of SMEs and start-ups when encouraging and facilitating the drawing up of codes of conduct.~~

~~Amendment 148~~

~~Proposal for a regulation~~

~~Article 69 a (new)~~

~~Text proposed by the Commission~~

~~Amendment~~

~~Article 69a~~

~~Right to lodge a complaint before a supervisory authority~~

1. Without prejudice to any other administrative or judicial remedy, every natural or legal person shall have the right to lodge a complaint with a supervisory authority, in particular in the Member State of his or her habitual residence, place of work or place of the alleged infringement if the natural or legal person considers that their health, safety, fundamental rights, their right to an explanation or any other of their rights deriving from the obligations laid down in this Regulation have been breached by the provider or the user of an AI system falling within the scope of this Regulation. Such complaint may be lodged through a representative action for the protection of the collective interests of consumers as provided under Directive (EU) 2020/1828.

2. Natural or legal persons shall have a right to be heard in the complaint handling procedure and in the context of any investigations conducted by the national supervisory authority as a result of their complaint.

3. The national supervisory authority with which the complaint has been lodged shall inform the complainants about the progress and outcome of their complaint. In particular, the national supervisory authority shall take all the necessary actions to follow up on the complaints it receives and, within three months of the reception of a complaint, give the complainant a preliminary response indicating the measures it intends to take and the next steps in the procedure, if any.

4. The national supervisory authority shall take a decision on the complaint and inform the complainant on the progress and the outcome of the complaint, including the possibility of a judicial remedy pursuant to Article 68b, without delay and no later than six months after the date on which the complaint was lodged.

~~Amendment 149~~

~~Proposal for a regulation~~

~~Article 69 b (new)~~

~~Text proposed by the Commission~~

~~Amendment~~

~~Article 69b~~

~~Right to an effective judicial remedy against a national supervisory authority~~

1. Without prejudice to any other administrative or non-judicial remedy, each natural or legal person shall have the right to an effective judicial or non-judicial remedy, including repair, replacement, price reduction, contract termination, reimbursement of the price paid or compensation for material and immaterial damages, against a legally binding decision of a national supervisory authority concerning them that infringes their rights.

2. Without prejudice to any other administrative or non-judicial remedy, each affected person shall have the right to a an effective judicial remedy where the national supervisory authority does not handle a complaint, does not inform the complainant on the progress or preliminary outcome of the complaint lodged within three months pursuant to Article 68a(3) or does not comply with its obligation to reach a final decision on the complaint within six months pursuant to Article 68a(4) or its obligations under Article 65.

~~3. Proceedings against a supervisory authority shall be brought before the courts of the Member State where the national supervisory authority is established.~~

~~Amendment 150~~

~~Proposal for a regulation~~

~~Article 69 c (new)~~

~~Text proposed by the Commission~~

~~Amendment~~

~~Article 69c~~

~~Right to an explanation~~

1. Any affected persons subject to a decision taken by a provider or an user, on the basis of an output from an AI system falling within the scope of this Regulation, which produces legal effects that they consider to adversely impact their health, safety, fundamental rights, socio-economic well-being or any other of their rights deriving from the obligations laid down in this Regulation, shall receive from the provider or the user, at the time when the decision is communicated, a clear and meaningful explanation pursuant to Article 13(1) on the role of the AI system in the decision-making procedure, the main parameters of the decision taken and on the related input data.

~~2. Paragraph 1 shall not apply to the use of AI systems:~~

(a) for which exceptions from, or restrictions to, the obligation under paragraph 1 follow from Union or national law, which lays down other appropriate safeguards for the affected persons’ rights, freedoms and legitimate interests; or

(b) where the affected person has given free, explicit, specific and informed consent not to receive an explanation. The affected person shall have the right to withdraw his or her consent not to receive an explanation at any time. Prior to giving consent, the affected person shall be informed thereof. It shall be as easy to withdraw as to give consent.

~~Amendment 151~~

~~Proposal for a regulation~~

~~Article 69 d (new)~~

~~Text proposed by the Commission~~

~~Amendment~~

~~Article 69d~~

~~Representative actions~~

~~1. The following is added to Annex I of Directive (EU) 2020/1828 on Representative actions for the protection of the collective interests of consumers:~~

“Regulation xxxx/xxxx of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending certain Union legislative acts”.

~~Amendment 152~~

~~Proposal for a regulation~~

~~TITLE X – title~~

~~Text proposed by the Commission~~

~~Amendment~~

~~CONFIDENTIALITY AND PENALTIES~~

~~CONFIDENTIALITY, REMEDIES AND PENALTIES~~

~~Amendment 153~~

~~Proposal for a regulation~~

~~Article 84 – paragraph 4 a (new)~~

~~Text proposed by the Commission~~

~~Amendment~~

4a. Within [three years after the date of application of this Regulation referred to in Article 85(2)], the Commission shall evaluate the effectiveness of the Board to carry out its tasks and assess whether an EU Agency would be best placed to ensure an effective and harmonised implementation of this Regulation.

~~Amendment 154~~

~~Proposal for a regulation~~

~~Annex VIII – point 5 a (new)~~

Text proposed by the Commission

Amendment

~~5a. The outcome of the trustworthy technology assessment;~~

(a) AI systems intended to be used as safety components in the management and operation of road traffic and the supply of water, gas, heating and electricity.

(a) AI systems intended to be used as safety components in the management and operation of road traffic and the supply of water, gas, heating and electricity, whose failure or malfunctioning would directly cause significant harm to the health, natural environment or safety of natural persons, unless these systems are regulated in harmonisation legislation or sectorial regulation.

ANNEX: LIST OF ENTITIES OR PERSONS FROM WHOM THE RAPPORTEUR FOR THE OPINION HAS RECEIVED INPUT

 BEUC

 ETF

 Google

 Amazon

 Airbus

 Hitachi

 DG MOVE

 DG Connect

 EASA

 AMCHAM

 ACEA

 CLEPA

 Ericsson

PROCEDURE – COMMITTEE ASKED FOR OPINION

Date announced in plenary

~~JURI~~

TRAN

7.6.2021

~~Associated committees - date announced in plenary~~

~~16.12.2021~~

Rapporteur for the opinion

Date appointed

~~Axel Voss~~

Josianne Cutajar

~~10.1.2022~~

4.11.2021

Rule 58 – Joint committee procedure

Discussed in committee

~~26.1.2022~~

20.4.2022

~~15.3.2022~~

~~28.3.2022~~

Date adopted

~~5.9.2022~~

12.7.2022

Result of final vote

Members present for the final vote

Magdalena Adamowicz, Andris Ameriks, Izaskun Bilbao Barandica, Karolin Braunsberger-Reinhold, Marco Campomenosi, Ciarán Cuffe, Karima Delli, Anna Deparnay-Grunenberg, Ismail Ertug, Gheorghe Falcă, Carlo Fidanza, Søren Gade, Isabel García Muñoz, Jens Gieseke, Elsi Katainen, Kateřina Konečná, Bogusław Liberadzki, Peter Lundgren, Benoît Lutgen, Elżbieta Katarzyna Łukacijewska, Marian-Jean Marinescu, Tilly Metz, Cláudia Monteiro de Aguiar, Jan-Christoph Oetjen, Rovana Plumb, Dominique Riquet, Massimiliano Salini, Barbara Thaler, István Ujhelyi, Petar Vitanov, Roberts Zīle, Kosma Złotowski

Substitutes present for the final vote

Josianne Cutajar, Nicola Danti, Vlad Gheorghe, Roman Haider, Pär Holmgren, Guido Reil, Marianne Vind, Jörgen Warborn

Substitutes under Rule 209(7) present for the final vote

Susanna Ceccardi, Salvatore De Meo

FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION