Sittings · Document

OPINION (COM(2021)0206 – C90146/2021 – 2021/0106(COD)) 2022-07-12

On the proposal for a regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending certain union legislative acts

Committee on Transport and Tourism · Rapporteur: Josianne Cutajar

PA_Legam

SHORT JUSTIFICATION

The Rapporteur supports the development of an EU legislative framework on Artificial Intelligence, a necessary step to ensure the Union becomes leader in the field, all while upholding our European values. The horizontal risk-based approach of the Artificial Intelligence Act will create a system in which rules will apply exclusively to applications of AI systems where threats may occur. This selective approach will allow the Union to continue developing its AI sector without hindering technological progress and digital transformation.

Transport is a sector posed to benefit greatly from the integration of AI systems in its every day operations and logistics. The application of AI systems can help achieve the EU’s safety, environmental and, in some instances, societal objectives for the sector. The Union must therefore aim at removing those existing barriers that hinder development and investment, first among all, legal fragmentation and uncertainty, which hurts business and customers.

The key elements of the Rapporteur’s report are the following:

 Ensuring the AI Act does not overlap with sectoral legislation by imposing double/conflicting obligations on transport actors;

 Promoting the development of, and upholding, international standards, that are particularly important for the transport sector;

 Fostering Research and Innovation to ensure the EU’s transport sector develops its own know-how in the implementation of AI, while upholding the highest ethical standards.

The general approach to the opinion by the Rapporteur has been to give a voice to the transport sector on the AI Act. For this reason, amendments had to be carefully drafted within the scope of the TRAN Committee, a complicated task when dealing with harmonized legislation. Still, the Rapporteur believes that in instances where horizontal provisions clearly affect transport, the TRAN Committee should have a say. In addition, some amendments have been included to clarify the key messages of the Commission’s report, such as in the case of the definition of AI systems and High-Risk, which are both essential for transport.

Following the consultation and research phase for this work, three clear issues emerged from a TRAN perspective. First, each segment of transport already presents, in different degrees and forms, sectoral legislation, provisions or initiatives to ensure the highest level of safety when it comes to AI system integration. Aviation, road, rail and maritime, all require measures specifically tailored to the sector to ensure the successful management of operations and services, while upholding the highest level of security. Harmonized legislation might fall short of the required measures to guarantee the aforementioned safety in the sector. For this reason, it was paramount to stress in the AI Act the need for sectoral legislation to be respected and prevail, in some instances, over harmonized rules. This was indeed the intention of the Commission as clear from Annex II, Part B. Yet, the text required further detail.

Secondly, due to the international nature of transport, a second key element that emerged from assessment is that international procedures are already in motion to develop global norms and guidelines for the safety of AI systems for each sector. It would be preferable therefore, that such global standards are respected and integrated in EU law and standards. Generally speaking, EU standards are developed through formal agreements between the European Standardization Organizations (ESOs) and International Standardization Organizations, which allows them in practice to work together. Nonetheless, in the interest of preserving the competitiveness of the EU’s transport sector vis-a-vis other regions, the language and provisions in the AI Act had to be strengthened to ensure provisions on AI for transport are respectful of the international norms and standards.

Lastly, Research and Development is the motor for the digitalization of each sector, let alone transport. New harmonized rules on AI must not impede research where such research is limited to controlled environments and the studied AI System is not placed onto the market. For this reason, language is added to clarify the scope in this regard. It must be noted, that the Commission proposal presents measures in support of innovation through specific articles on sandboxes and small-scale providers. Here, however, the Rapporteur amends to ensure Small and Medium Enterprises (SMEs) are able to benefit from such provisions.

Further minor amendments by the Rapporteur cover transparency of algorithms for transport work (simply reflecting Commission’s language in Annex III), an understanding of Human Oversight in the context of transport, and clarification on errors in data sets.

AMENDMENTS

The Committee on Transport and Tourism calls on the Committee on the Internal Market and Consumer Protection and the Committee on Civil Liberties, Justice and Home Affairs, as the committees responsible, to take into account the following amendments:

Amendment 1

Proposal for a regulation

Recital 1

Text proposed by the Commission

Amendment

(1) The purpose of this Regulation is to improve the functioning of the internal market by laying down a uniform legal framework in particular for the development, marketing and use of artificial intelligence in conformity with Union values. This Regulation pursues a number of overriding reasons of public interest, such as a high level of protection of health, safety and fundamental rights, and it ensures the free movement of AI-based goods and services cross-border, thus preventing Member States from imposing restrictions on the development, marketing and use of AI systems, unless explicitly authorised by this Regulation.

(1) The purpose of this Regulation is to improve the functioning of the internal market by laying down a uniform legal framework in particular for the development, marketing and use of artificial intelligence in conformity with Union values. This Regulation pursues a number of overriding reasons of public interest, such as a high level of protection of health, safety, fundamental rights and the environment, and it ensures the free movement of AI-based goods and services cross-border, thus preventing Member States from imposing restrictions on the development, marketing and use of AI systems, unless explicitly authorised by this Regulation.

Amendment 2

Proposal for a regulation

Recital 5

Text proposed by the Commission

Amendment

(5) A Union legal framework laying down harmonised rules on artificial intelligence is therefore needed to foster the development, use and uptake of artificial intelligence in the internal market that at the same time meets a high level of protection of public interests, such as health and safety and the protection of fundamental rights, as recognised and protected by Union law. To achieve that objective, rules regulating the placing on the market and putting into service of certain AI systems should be laid down, thus ensuring the smooth functioning of the internal market and allowing those systems to benefit from the principle of free movement of goods and services. By laying down those rules, this Regulation supports the objective of the Union of being a global leader in the development of secure, trustworthy and ethical artificial intelligence, as stated by the European Council33 , and it ensures the protection of ethical principles, as specifically requested by the European Parliament34 .

(5) A Union legal framework laying down harmonised rules on artificial intelligence is therefore needed to foster the development, use and uptake of artificial intelligence in the internal market that at the same time meets a high level of protection of public interests, such as health and safety, the environment, and the protection of fundamental rights, as recognised and protected by Union law. To achieve that objective, rules regulating the placing on the market and putting into service of certain AI systems should be laid down, thus ensuring the smooth functioning of the internal market and allowing those systems to benefit from the principle of free movement of goods and services. By laying down those rules, this Regulation supports the objective of the Union of being a global leader in the development of secure, trustworthy and ethical artificial intelligence, as stated by the European Council33 , and it ensures the protection of ethical principles, as specifically requested by the European Parliament34.

__________________

33 European Council, Special meeting of the European Council (1 and 2 October 2020) – Conclusions, EUCO 13/20, 2020, p. 6.

34 European Parliament resolution of 20 October 2020 with recommendations to the Commission on a framework of ethical aspects of artificial intelligence, robotics and related technologies, 2020/2012(INL).

Amendment 3

Proposal for a regulation

Recital 5 a (new)

Text proposed by the Commission

Amendment

(5a) Union legislation on artificial intelligence should contribute to the dual green and digital transition. The artificial intelligence can contribute positively to the green transition but also has significant environmental impacts due to the critical raw material required to design and build its infrastructure and microprocessors and the energy used for its development, training, tuning and use. Development and use of AI should therefore be compatible with sustainable environmental resources at all stages of the lifecycle of AI systems. Also, unnecessary data acquisition and processing should be avoided. Moreover, Union legislation on artificial intelligence should be accompanied by actions aimed at addressing the main barriers hindering the digital transformation of the economy. Such measures should focus on education, upskilling and reskilling of workers, fostering investment in research and innovation, and boosting security in the digital sphere in line with initiatives aimed at achieving the targets of the Digital Decade. Digital transformation should occur in a harmonized manner across regions, paying particular attention to less digitally developed areas of the Union.

Amendment 4

Proposal for a regulation

Recital 5 b (new)

Text proposed by the Commission

Amendment

(5b) Harmonised Union legislation on artificial intelligence can contribute to create legal certainty and coherence across the Union. However, due to risks associated with passenger and goods transport, the sector has been carefully monitored and regulated to avoid incidents and loss of life. The Union legal framework for transport presents sectoral legislation for the aviation, road, rail and maritime transport. With the progressive integration of AI systems in the sector, new challenges could emerge in risk management. This Regulation should only apply to high risk applications in the transport sector in so far as that they are not already covered by sectoral legislation and where they could have a harmful impact on the environment or health, safety and fundamental rights of persons. Double regulation should therefore be avoided.

Amendment 5

Proposal for a regulation

Recital 5 c (new)

Text proposed by the Commission

Amendment

(5c) The Union aviation sector, for example, through the work of the European Aviation Safety Agency (EASA) and its stakeholders, is gradually developing its own guidance material and rules on the application and security management of AI systems in aviation. In the EASA’s roadmap for AI, AI systems with application to aviation are categorised in three distinct levels, from assistance to human, to human-machine cooperation, to full machine automation. A sector-specific oversight on AI systems laying out rules for the highest-level of safety for aviation while preserving the global competitiveness of Union businesses is needed.

Amendment 6

Proposal for a regulation

Recital 6

Text proposed by the Commission

Amendment

(6) The notion of AI system should be clearly defined to ensure legal certainty, while providing the flexibility to accommodate future technological developments. The definition should be based on the key functional characteristics of the software, in particular the ability, for a given set of human-defined objectives, to generate outputs such as content, predictions, recommendations, or decisions which influence the environment with which the system interacts, be it in a physical or digital dimension. AI systems can be designed to operate with varying levels of autonomy and be used on a stand-alone basis or as a component of a product, irrespective of whether the system is physically integrated into the product (embedded) or serve the functionality of the product without being integrated therein (non-embedded). The definition of AI system should be complemented by a list of specific techniques and approaches used for its development, which should be kept up-to–date in the light of market and technological developments through the adoption of delegated acts by the Commission to amend that list.

(6) The notion of AI system should be clearly defined to ensure legal certainty, while providing the flexibility to accommodate future technological developments. The definition should be based on the key functional characteristics of the software, and possibly also the hardware.1a In particular, for the purpose of this Regulation, AI systems should be intended as having the ability, on the basis of machine- and/or human-based data and inputs, to infer the way to achieve a given set of human-defined objectives through learning, reasoning or modelling and generate specific outputs in the form of content for generative AI systems, as well as predictions, recommendations, or decisions which influence the environment with which the system interacts, be it in a physical or digital dimension. AI systems can be designed to operate with varying levels of autonomy and be used on a stand-alone basis or as a component of a product, irrespective of whether the system is physically integrated into the product (embedded) or serve the functionality of the product without being integrated therein (non-embedded). The definition of AI system should be complemented by a list of specific techniques and approaches used for its development, which should be kept up-to–date in the light of market and technological developments through the adoption of delegated acts by the Commission to amend that list.

__________________

1a https://digital-strategy.ec.europa.eu/en/library/definition-artificial-intelligence-main-capabilities-and-scientific-disciplines

Amendment 7

Proposal for a regulation

Recital 8 a (new)

Text proposed by the Commission

Amendment

(8a) The use of biometrics and high technologies in transport and tourism may vastly benefit user experience and overall safety and security. This Regulation should accompany these developments by setting the highest level of protection, in particular when use of biometrics data is involved, in line with the data protection framework of the Union, while fostering research and investment for the development and deployment of AI systems that can positively contribute to society.

Amendment 8

Proposal for a regulation

Recital 12 a (new)

Text proposed by the Commission

Amendment

(12a) This Regulation should support research and innovation for the application of AI systems in the transport and tourism sectors while ensuring a high level of protection of public interests, such as health, safety, fundamental rights, the environment and democracy. For this reason, this Regulation should exclude from its scope applications of AI systems developed, applied and assessed in a controlled testing environment, for the sole purpose of evaluating their use and functionality. As regards product oriented research activity by providers, the provisions of this Regulation should apply insofar as such research leads to or entails placing an AI system on the market or putting it into service. All forms of research and development should be conducted in compliance with the highest ethical standards for scientific research.

Amendment 9

Proposal for a regulation

Recital 13

Text proposed by the Commission

Amendment

(13) In order to ensure a consistent and high level of protection of public interests as regards health, safety and fundamental rights, common normative standards for all high-risk AI systems should be established. Those standards should be consistent with the Charter of fundamental rights of the European Union (the Charter) and should be non-discriminatory and in line with the Union’s international trade commitments.

(13) In order to ensure a consistent and high level of protection of public interests as regards health, safety and fundamental rights and the environment, common normative standards for all high-risk AI systems should be established. Those standards should be consistent with the Charter of fundamental rights of the European Union (the Charter) and should be non-discriminatory and in line with the Union’s international trade commitments. This is of particular importance in the transport sector in order to ensure the highest level of interoperability among transport vehicles, infrastructure and intelligent systems and to guarantee safety and security. The Union and its standards organisations should participate actively in the development of global standards for the different transport modes with a view to align them as much as possible with any applicable European standards and to ensure that they are in compliance with Union law. Regular reviews of this Regulation should take into account updated standards for the transport sector.

Amendment 10

Proposal for a regulation

Recital 17 a (new)

Text proposed by the Commission

Amendment

(17a) The use of AI in work can be beneficial to both the management and operations of an enterprise, supporting workers in their tasks and improving safety at the workplace. Still, AI systems applied to the management of workers, in particular by digital labour platforms, including in the field of transport, can entail a number of risks such as unjust/unnecessary social scoring, rooted in biased data sets or intrusive surveillance practice which can lead to violation of workers’ and fundamental rights. This Regulation should therefore aim at protecting the rights of transport workers managed with the assistance of AI systems, including those working via digital labour platforms and promote transparency, fairness and accountability in algorithmic management, to ensure that workers have a broad understanding of how algorithms work, which personal data is issued and how their behaviour affects decisions taken by the automated system.

Amendment 11

Proposal for a regulation

Recital 17 b (new)

Text proposed by the Commission

Amendment

(17b) In addition, users and individuals should have the right to object to a decision taken solely by an AI system, or relying to a significant degree on the output of an AI system, which produces legal effects concerning them, or similarly significantly affects them.

Amendment 12

Proposal for a regulation

Recital 27

Text proposed by the Commission

Amendment

(27) High-risk AI systems should only be placed on the Union market or put into service if they comply with certain mandatory requirements. Those requirements should ensure that high-risk AI systems available in the Union or whose output is otherwise used in the Union do not pose unacceptable risks to important Union public interests as recognised and protected by Union law. AI systems identified as high-risk should be limited to those that have a significant harmful impact on the health, safety and fundamental rights of persons in the Union and such limitation minimises any potential restriction to international trade, if any.

(27) High-risk AI systems should only be placed on the Union market or put into service if they comply with certain mandatory requirements. Those requirements should ensure that high-risk AI systems available in the Union or whose output is otherwise used in the Union do not pose unacceptable risks to important Union public interests as recognised and protected by Union law. AI systems identified as high-risk should be limited to those that have a significant harmful impact on the health, safety and fundamental rights of persons in the Union or the environment and such limitation minimises any potential restriction to international trade, if any.

Amendment 13

Proposal for a regulation

Recital 29

Text proposed by the Commission

Amendment

(29) As regards high-risk AI systems that are safety components of products or systems, or which are themselves products or systems falling within the scope of Regulation (EC) No 300/2008 of the European Parliament and of the Council39 , Regulation (EU) No 167/2013 of the European Parliament and of the Council40 , Regulation (EU) No 168/2013 of the European Parliament and of the Council41 , Directive 2014/90/EU of the European Parliament and of the Council42 , Directive (EU) 2016/797 of the European Parliament and of the Council43 , Regulation (EU) 2018/858 of the European Parliament and of the Council44 , Regulation (EU) 2018/1139 of the European Parliament and of the Council45 , and Regulation (EU) 2019/2144 of the European Parliament and of the Council46 , it is appropriate, if required, to amend those acts to ensure that the Commission takes into account, on the basis of the technical and regulatory specificities of each sector, and without overlapping with existing governance, conformity assessment and enforcement mechanisms and authorities established therein, the mandatory requirements for high-risk AI systems laid down in this Regulation when adopting any relevant future delegated or implementing acts on the basis of those acts. Transport sectoral legislation should prevail over this Regulation and it should be ensured that no conflicting overlap exists between this Regulation and other current and upcoming legal acts (i.e. Data Act, ITS Review) to avoid duplication of obligations on providers and manufacturers, which would cause legal uncertainty for business and slow down the uptake of new technologies in the market. This Regulation should also provide for an efficient review mechanism in order to take into account future technological developments and to ensure fair, proportionate and targeted implementation. In order to avoid substantial legal uncertainty, and to ensure that this Regulation applies to all sectors concerned by it without undue delays, those acts should be amended to integrate the provisions of this Regulation no later than 24 months after its entry into force.

__________________

39 Regulation (EC) No 300/2008 of the European Parliament and of the Council of 11 March 2008 on common rules in the field of civil aviation security and repealing Regulation (EC) No 2320/2002 (OJ L 97, 9.4.2008, p. 72).

40 Regulation (EU) No 167/2013 of the European Parliament and of the Council of 5 February 2013 on the approval and market surveillance of agricultural and forestry vehicles (OJ L 60, 2.3.2013, p. 1).

41 Regulation (EU) No 168/2013 of the European Parliament and of the Council of 15 January 2013 on the approval and market surveillance of two- or three-wheel vehicles and quadricycles (OJ L 60, 2.3.2013, p. 52).

42 Directive 2014/90/EU of the European Parliament and of the Council of 23 July 2014 on marine equipment and repealing Council Directive 96/98/EC (OJ L 257, 28.8.2014, p. 146).

43 Directive (EU) 2016/797 of the European Parliament and of the Council of 11 May 2016 on the interoperability of the rail system within the European Union (OJ L 138, 26.5.2016, p. 44).

44 Regulation (EU) 2018/858 of the European Parliament and of the Council of 30 May 2018 on the approval and market surveillance of motor vehicles and their trailers, and of systems, components and separate technical units intended for such vehicles, amending Regulations (EC) No 715/2007 and (EC) No 595/2009 and repealing Directive 2007/46/EC (OJ L 151, 14.6.2018, p. 1).

45 Regulation (EU) 2018/1139 of the European Parliament and of the Council of 4 July 2018 on common rules in the field of civil aviation and establishing a European Union Aviation Safety Agency, and amending Regulations (EC) No 2111/2005, (EC) No 1008/2008, (EU) No 996/2010, (EU) No 376/2014 and Directives 2014/30/EU and 2014/53/EU of the European Parliament and of the Council, and repealing Regulations (EC) No 552/2004 and (EC) No 216/2008 of the European Parliament and of the Council and Council Regulation (EEC) No 3922/91 (OJ L 212, 22.8.2018, p. 1).

46 Regulation (EU) 2019/2144 of the European Parliament and of the Council of 27 November 2019 on type-approval requirements for motor vehicles and their trailers, and systems, components and separate technical units intended for such vehicles, as regards their general safety and the protection of vehicle occupants and vulnerable road users, amending Regulation (EU) 2018/858 of the European Parliament and of the Council and repealing Regulations (EC) No 78/2009, (EC) No 79/2009 and (EC) No 661/2009 of the European Parliament and of the Council and Commission Regulations (EC) No 631/2009, (EU) No 406/2010, (EU) No 672/2010, (EU) No 1003/2010, (EU) No 1005/2010, (EU) No 1008/2010, (EU) No 1009/2010, (EU) No 19/2011, (EU) No 109/2011, (EU) No 458/2011, (EU) No 65/2012, (EU) No 130/2012, (EU) No 347/2012, (EU) No 351/2012, (EU) No 1230/2012 and (EU) 2015/166 (OJ L 325, 16.12.2019, p. 1).

Amendment 14

Proposal for a regulation

Recital 32

Text proposed by the Commission

Amendment

(32) As regards stand-alone AI systems, meaning high-risk AI systems other than those that are safety components of products, or which are themselves products, it is appropriate to classify them as high-risk if, in the light of their intended purpose, they pose a high risk of harm to the health and safety or the fundamental rights of persons, taking into account both the severity of the possible harm and its probability of occurrence and they are used in a number of specifically pre-defined areas specified in the Regulation. The identification of those systems is based on the same methodology and criteria envisaged also for any future amendments of the list of high-risk AI systems.

(32) As regards stand-alone AI systems, meaning high-risk AI systems other than those that are safety components of products, or which are themselves products, it is appropriate to classify them as high-risk if, in the light of their intended purpose, they pose a high risk of harm to the health and safety or the fundamental rights of persons or the environment, taking into account both the severity of the possible harm and its probability of occurrence and they are used in a number of specifically pre-defined areas specified in the Regulation. The identification of those systems is based on the same methodology and criteria envisaged also for any future amendments of the list of high-risk AI systems.

Amendment 15

Proposal for a regulation

Recital 34

Text proposed by the Commission

Amendment

(34) As regards the management and operation of critical infrastructure, it is appropriate to classify as high-risk the AI systems intended to be used as safety components in the management and operation of road traffic and the supply of water, gas, heating and electricity, since their failure or malfunctioning may put at risk the life and health of persons at large scale and lead to appreciable disruptions in the ordinary conduct of social and economic activities. Some examples of critical infrastructure management systems for road covered by Annex III should include traffic management control systems, intelligent transport systems and ICT infrastructure connected transport.

Amendment 16

Proposal for a regulation

Recital 37

Text proposed by the Commission

Amendment

(37) Another area in which the use of AI systems deserves special consideration is the access to and enjoyment of certain essential private and public services and benefits necessary for people to fully participate in society or to improve one’s standard of living. In particular, AI systems used to evaluate the credit score or creditworthiness of natural persons should be classified as high-risk AI systems, since they determine those persons’ access to financial resources or essential services such as housing, electricity, and telecommunication services. AI systems used for this purpose may lead to discrimination of persons or groups and perpetuate historical patterns of discrimination, for example based on racial or ethnic origins, disabilities, age, sexual orientation, or create new forms of discriminatory impacts. Considering the very limited scale of the impact and the available alternatives on the market, it is appropriate to exempt AI systems for the purpose of creditworthiness assessment and credit scoring when put into service by small-scale providers for their own use. Natural persons applying for or receiving public assistance benefits and services from public authorities are typically dependent on those benefits and services and in a vulnerable position in relation to the responsible authorities. If AI systems are used for determining whether such benefits and services should be denied, reduced, revoked or reclaimed by authorities, they may have a significant impact on persons’ livelihood and may infringe their fundamental rights, such as the right to social protection, non-discrimination, human dignity or an effective remedy. Those systems should therefore be classified as high-risk. Nonetheless, this Regulation should not hamper the development and use of innovative approaches in the public administration, which would stand to benefit from a wider use of compliant and safe AI systems, provided that those systems do not entail a high risk to legal and natural persons. Finally, AI systems used to dispatch or establish priority in the dispatching of emergency first response services should also be classified as high-risk since they make decisions in very critical situations for the life and health of persons and their property.

(37) Another area in which the use of AI systems deserves special consideration is the access to and enjoyment of certain essential private and public services and benefits necessary for people to fully participate in society or to improve one’s standard of living. In particular, AI systems used to evaluate the credit score or creditworthiness of natural persons should be classified as high-risk AI systems, since they determine those persons’ access to financial resources or essential services such as housing, electricity, and telecommunication services. AI systems used for this purpose may lead to discrimination of persons or groups and perpetuate historical patterns of discrimination, for example based on racial or ethnic origins, gender, disabilities, age, sexual orientation, or create new forms of discriminatory impacts. Considering the very limited scale of the impact and the available alternatives on the market, it is appropriate to exempt AI systems for the purpose of creditworthiness assessment and credit scoring when put into service by small-scale providers for their own use. Natural persons applying for or receiving public assistance benefits and services from public authorities are typically dependent on those benefits and services and in a vulnerable position in relation to the responsible authorities. If AI systems are used for determining whether such benefits and services should be denied, reduced, revoked or reclaimed by authorities, they may have a significant impact on persons’ livelihood and may infringe their fundamental rights, such as the right to social protection, non-discrimination, human dignity or an effective remedy. Those systems should therefore be classified as high-risk. Nonetheless, this Regulation should not hamper the development and use of innovative approaches in the public administration, which would stand to benefit from a wider use of compliant and safe AI systems, provided that those systems do not entail a high risk to legal and natural persons. Finally, AI systems used to dispatch or establish priority in the dispatching of emergency first response services should also be classified as high-risk since they make decisions in very critical situations for the life and health of persons and their property.

Amendment 17

Proposal for a regulation

Recital 43

Text proposed by the Commission

Amendment

(43) Requirements should apply to high-risk AI systems as regards the quality of data sets used, technical documentation and record-keeping, transparency and the provision of information to users, human oversight, and robustness, accuracy and cybersecurity. Those requirements are necessary to effectively mitigate the risks for health, safety and fundamental rights, as applicable in the light of the intended purpose of the system, and no other less trade restrictive measures are reasonably available, thus avoiding unjustified restrictions to trade.

(43) Requirements should apply to high-risk AI systems as regards the quality of data sets used, technical documentation and record-keeping, transparency and the provision of information to users, human oversight, and robustness, accuracy and cybersecurity. Those requirements are necessary to effectively mitigate the risks for health, safety, fundamental rights and the environment, as applicable in the light of the intended purpose of the system, and no other less trade restrictive measures are reasonably available, thus avoiding unjustified restrictions to trade.

Amendment 18

Proposal for a regulation

Recital 44

Text proposed by the Commission

Amendment

(44) High data quality is essential for the performance of many AI systems, especially when techniques involving the training of models are used, with a view to ensure that the high-risk AI system performs as intended and safely and it does not become the source of discrimination prohibited by Union law. High quality training, validation and testing data sets require the implementation of appropriate data governance and management practices. Training, validation and testing data sets should be sufficiently relevant, representative and free of errors and complete in view of the intended purpose of the system. They should also have the appropriate statistical properties, including as regards the persons or groups of persons on which the high-risk AI system is intended to be used. In particular, training, validation and testing data sets should take into account, to the extent required in the light of their intended purpose, the features, characteristics or elements that are particular to the specific geographical, behavioural or functional setting or context within which the AI system is intended to be used. In order to protect the right of others from the discrimination that might result from the bias in AI systems, the providers shouldbe able to process also special categories of personal data, as a matter of substantial public interest, in order to ensure the bias monitoring, detection and correction in relation to high-risk AI systems.

(44) High data quality is essential for the performance of many AI systems, especially when techniques involving the training of models are used, with a view to ensure that the high-risk AI system performs as intended and safely and it does not become the source of discrimination prohibited by Union law. High quality training, validation and testing data sets require the implementation of appropriate data governance and management practices. Training, validation and testing data sets should be sufficiently relevant, representative, up to date and, to the best extent possible free of errors and as complete as possible in view of the intended purpose of the system and to ensure the highest level of security. They should also have the appropriate statistical properties, including as regards the persons or groups of persons on which the high-risk AI system is intended to be used. In particular, training, validation and testing data sets should take into account, to the extent required in the light of their intended purpose, the features, characteristics or elements that are particular to the specific geographical, behavioural or functional setting or context within which the AI system is intended to be used. In order to protect the right of others from the discrimination that might result from the bias in AI systems, the providers should ensure that databases contain adequate data on groups which are more vulnerable to discriminatory effects posed by AI, such as people with disabilities, and be able to process also special categories of personal data, as a matter of substantial public interest, in order to ensure the bias monitoring, detection, update, and correction in relation to high-risk AI systems.

Amendment 19

Proposal for a regulation

Recital 47

Text proposed by the Commission

Amendment

(47) To address the opacity that may make certain AI systems incomprehensible to or too complex for natural persons, a certain degree of transparency should be required for high-risk AI systems. Users should be able to interpret the system output and use it appropriately. High-risk AI systems should therefore be accompanied by relevant documentation and instructions of use and include concise and clear information, including in relation to possible risks to fundamental rights and discrimination, where appropriate.

(47) To address the opacity that may make certain AI systems incomprehensible to or too complex for natural persons, a certain degree of transparency should be required for high-risk AI systems, in particular when applied to digital labour platforms managing the activities of transport workers. Users should be able to interpret the system output and use it appropriately. Transparency, fairness, accountability and explanability of AI systems can also be a beneficial factor for their uptake by consumers in the market. High-risk AI systems should therefore be accompanied by relevant documentation and instructions of use and include concise and clear information, including in relation to possible risks to fundamental rights and discrimination, where appropriate.

Amendment 20

Proposal for a regulation

Recital 47 a (new)

Text proposed by the Commission

Amendment

(47a) Based on previous experience, it is particularly important to ensure clear requirements and guidelines for interoperability between AI systems both within and amongst different economic sectors, contributing to foster innovation and providing favourable conditions for small and medium enterprises (SMEs).

Amendment 21

Proposal for a regulation

Recital 48

Text proposed by the Commission

Amendment

(48) High-risk AI systems should be designed and developed in such a way that natural persons can oversee their functioning. For this purpose, appropriate human oversight measures should be identified by the provider of the system before its placing on the market or putting into service. In particular, where appropriate, such measures should guarantee that the system is subject to in-built operational constraints that cannot be overridden by the system itself and is responsive to the human operator, and that the natural persons to whom human oversight has been assigned have the necessary competence, training and authority to carry out that role.

(48) High-risk AI systems should be designed and developed in such a way that natural persons can oversee their functioning, unless there is clear evidence that it doesn't add value and could even be detrimental to the protection of health, safety and fundamental rights. For this purpose, appropriate human oversight measures should be identified by the provider of the system before its placing on the market or putting into service. In particular, where appropriate, such measures should guarantee that the system is subject to in-built operational constraints that cannot be overridden by the system itself and is responsive to the human operator, and that the natural persons to whom human oversight has been assigned have the necessary competence, training and authority to carry out that role. With regards to the transport sector, the AI system applications should respect the sector-specific legislation in place. When physical security is at stake, Union standards, and where applicable international standards, should determine in which case the possibility for a human operator to take back control should take prevalence over AI system’s decision.

Amendment 22

Proposal for a regulation

Recital 51

Text proposed by the Commission

Amendment

(51) Cybersecurity plays a crucial role in ensuring that AI systems are resilient against attempts to alter their use, behaviour, performance or compromise their security properties by malicious third parties exploiting the system’s vulnerabilities. Cyberattacks against AI systems can leverage AI specific assets, such as training data sets (e.g. data poisoning) or trained models (e.g. adversarial attacks), or exploit vulnerabilities in the AI system’s digital assets or the underlying ICT infrastructure. To ensure a level of cybersecurity appropriate to the risks, suitable measures should therefore be taken by the providers of high-risk AI systems, also taking into account as appropriate the underlying ICT infrastructure.

(51) Cybersecurity plays a crucial role in ensuring that AI systems are resilient against attempts to alter their use, behaviour, performance or compromise their security properties by malicious third parties exploiting the system’s vulnerabilities. Cyberattacks against AI systems can leverage AI specific assets, such as training data sets (e.g. data poisoning) or trained models (e.g. adversarial attacks), or exploit vulnerabilities in the AI system’s digital assets or the underlying ICT infrastructure. To ensure a level of cybersecurity appropriate to the risks, suitable measures should therefore be taken by the providers of high-risk AI systems, as well as the notified bodies, competent national authorities and market surveillance authorities accessing the data of providers of high risk AI systems, also taking into account as appropriate the underlying ICT infrastructure.

Amendment 23

Proposal for a regulation

Recital 54

Text proposed by the Commission

Amendment

(54) The provider should establish a sound quality management system, ensure the accomplishment of the required conformity assessment procedure, draw up the relevant documentation and establish a robust post-market monitoring system. Public authorities which put into service high-risk AI systems for their own use may adopt and implement the rules for the quality management system as part of the quality management system adopted at a national or regional level, as appropriate, taking into account the specificities of the sector and the competences and organisation of the public authority in question.

(54) The provider should establish a sound quality management system, ensure the accomplishment of the required conformity assessment procedure, draw up the relevant documentation in the language of the Member State concerned and establish a robust post-market monitoring system. All elements, from design to future development, should be made transparent for the user. Public authorities which put into service high-risk AI systems for their own use may adopt and implement the rules for the quality management system as part of the quality management system adopted at a national or regional level, as appropriate, taking into account the specificities of the sector and the competences and organisation of the public authority in question.

Amendment 24

Proposal for a regulation

Recital 59

Text proposed by the Commission

Amendment

(59) It is appropriate to envisage that the user of the AI system should be the natural or legal person, public authority, agency or other body under whose authority the AI system is operated except where the use is made in the course of a personal non-professional activity.

(59) It is appropriate to envisage that the user of the AI system should be the natural or legal person, public authority, agency or other body under whose authority the AI system is operated.

Amendment 25

Proposal for a regulation

Recital 71

Text proposed by the Commission

Amendment

(71) Artificial intelligence is a rapidly developing family of technologies that requires novel forms of regulatory oversight and a safe space for experimentation, while ensuring responsible innovation and integration of appropriate safeguards and risk mitigation measures. To ensure a legal framework that is innovation-friendly, future-proof and resilient to disruption, national competent authorities from one or more Member States should be encouraged to establish artificial intelligence regulatory sandboxes and make such regulatory sandboxes widely available throughout the Union, in order to facilitate the development and testing of innovative AI systems under strict regulatory oversight before these systems are placed on the market or otherwise put into service. It is especially important to ensure that SMEs and start-ups can easily access these sandboxes, are actively involved and participate in the development and testing of innovative AI systems, in order to be able to contribute with their knowhow and experience. Their participation should be supported and facilitated.

Amendment 26

Proposal for a regulation

Recital 72

Text proposed by the Commission

Amendment

(72) The objectives of the regulatory sandboxes should be to foster AI innovation by establishing a controlled experimentation and testing environment in the development and pre-marketing phase with a view to ensuring compliance of the innovative AI systems with this Regulation and other relevant Union and Member States legislation; to enhance legal certainty for innovators and the competent authorities’ oversight and understanding of the opportunities, emerging risks and the impacts of AI use, and to accelerate access to markets, including by removing barriers for small and medium enterprises (SMEs) and start-ups. To ensure uniform implementation across the Union and economies of scale, it is appropriate to establish common rules for the regulatory sandboxes’ implementation and a framework for cooperation between the relevant authorities involved in the supervision of the sandboxes. This Regulation should provide the legal basis for the use of personal data collected for other purposes for developing certain AI systems in the public interest within the AI regulatory sandbox, in line with Article 6(4) of Regulation (EU) 2016/679, and Article 6 of Regulation (EU) 2018/1725, and without prejudice to Article 4(2) of Directive (EU) 2016/680. Participants in the sandbox should ensure appropriate safeguards and cooperate with the competent authorities, including by following their guidance and acting expeditiously and in good faith to mitigate any high-risks to safety and fundamental rights that may arise during the development and experimentation in the sandbox. The conduct of the participants in the sandbox should be taken into account when competent authorities decide whether to impose an administrative fine under Article 83(2) of Regulation 2016/679 and Article 57 of Directive 2016/680.

(72) The objectives of the regulatory sandboxes should be to foster AI innovation by establishing a controlled experimentation and testing environment in the development and pre-marketing phase with a view to ensuring compliance of the innovative AI systems with this Regulation and other relevant Union and Member States legislation; to enhance legal certainty for innovators and the competent authorities’ oversight and understanding of the opportunities, emerging risks and the impacts of AI use, and to accelerate access to markets, including by removing barriers for small and medium enterprises and start-ups, as well as to contribute to achieving the targets on AI as set in the Policy Programme “Path to the Digital Decade". To ensure uniform implementation across the Union and economies of scale, it is appropriate to establish common rules for the regulatory sandboxes’ implementation and a framework for cooperation between the relevant authorities involved in the supervision of the sandboxes. This Regulation should provide the legal basis for the use of personal data collected for other purposes for developing certain AI systems in the public interest within the AI regulatory sandbox, in line with Article 6(4) of Regulation (EU) 2016/679, and Article 6 of Regulation (EU) 2018/1725, and without prejudice to Article 4(2) of Directive (EU) 2016/680. Participants in the sandbox should ensure appropriate safeguards and cooperate with the competent authorities, including by following their guidance and acting expeditiously and in good faith to mitigate any high-risks to safety and fundamental rights that may arise during the development and experimentation in the sandbox. The conduct of the participants in the sandbox should be taken into account when competent authorities decide whether to impose an administrative fine under Article 83(2) of Regulation 2016/679 and Article 57 of Directive 2016/680.

Amendment 27

Proposal for a regulation

Recital 73

Text proposed by the Commission

Amendment

(73) In order to promote and protect innovation, it is important that the interests of small-scale providers and users of AI systems are taken into particular account. To this objective, Member States should develop initiatives, which are targeted at those operators, including on awareness raising and information communication. Moreover, the specific interests and needs of small-scale providers shall be taken into account when Notified Bodies set conformity assessment fees. Translation costs related to mandatory documentation and communication with authorities may constitute a significant cost for providers and other operators, notably those of a smaller scale. Member States should possibly ensure that one of the languages determined and accepted by them for relevant providers’ documentation and for communication with operators is one which is broadly understood by the largest possible number of cross-border users.

(73) In order to promote and protect innovation, it is important that the interests of SMEs and users of AI systems are taken into particular account. To this objective, Member States should develop initiatives, which are targeted at those operators, including on awareness raising and information communication. Moreover, the specific interests and needs of SMEs should be taken into account when Notified Bodies set conformity assessment fees. Translation costs related to mandatory documentation and communication with authorities may constitute a significant cost for providers and other operators, notably those of a smaller scale. Member States should possibly ensure that one of the languages determined and accepted by them for relevant providers’ documentation and for communication with operators is one which is broadly understood by the largest possible number of cross-border users.

Amendment 28

Proposal for a regulation

Recital 76

Text proposed by the Commission

Amendment

(76) In order to facilitate a smooth, effective and harmonised implementation of this Regulation a European Artificial Intelligence Board should be established. The Board should be responsible for a number of advisory tasks, including issuing opinions, recommendations, advice or guidance on matters related to the implementation of this Regulation, including on technical specifications or existing standards regarding the requirements established in this Regulation and providing advice to and assisting the Commission on specific questions related to artificial intelligence.

Amendment 29

Proposal for a regulation

Recital 77 a (new)

Text proposed by the Commission

Amendment

(77a) To encourage knowledge sharing from best practices, the Commission should organise regular consultative meetings for knowhow exchange between different Member States' national authorities responsible for notification policy.

Amendment 30

Proposal for a regulation

Article 1 – paragraph 1 – point e

Text proposed by the Commission

Amendment

(e) rules on market monitoring and surveillance.

(e) rules on market monitoring, market surveillance and governance.

Amendment 31

Proposal for a regulation

Article 1 – paragraph 1 – point e a (new)

Text proposed by the Commission

Amendment

(ea) provision to foster and support research and development for innovation.

Amendment 32

Proposal for a regulation

Article 2 – paragraph 2 – introductory part

Text proposed by the Commission

Amendment

2. For high-risk AI systems that are safety components of products or systems, or which are themselves products or systems, falling within the scope of the following acts, only Article 84 of this Regulation shall apply:

2. For AI systems classified as high-risk AI in accordance with Article 6 related to products covered by Union harmonisation legislation listed in Annex II, section B, only Article 84 of this Regulation shall apply:

Amendment 33

Proposal for a regulation

Article 2 – paragraph 2 – point a

Text proposed by the Commission

Amendment

(a) Regulation (EC) 300/2008;

deleted

Amendment 34

Proposal for a regulation

Article 2 – paragraph 2 – point b

Text proposed by the Commission

Amendment

(b) Regulation (EU) No 167/2013;

deleted

Amendment 35

Proposal for a regulation

Article 2 – paragraph 2 – point c

Text proposed by the Commission

Amendment

deleted

Amendment 36

Proposal for a regulation

Article 2 – paragraph 2 – point d

Text proposed by the Commission

Amendment

(d) Directive 2014/90/EU;

deleted

Amendment 37

Proposal for a regulation

Article 2 – paragraph 2 – point e

Text proposed by the Commission

Amendment

(e) Directive (EU) 2016/797;

deleted

Amendment 38

Proposal for a regulation

Article 2 – paragraph 2 – point f

Text proposed by the Commission

Amendment

(f) Regulation (EU) 2018/858;

deleted

Amendment 39

Proposal for a regulation

Article 2 – paragraph 2 – point g

Text proposed by the Commission

Amendment

(g) Regulation (EU) 2018/1139;

deleted

Amendment 40

Proposal for a regulation

Article 2 – paragraph 2 – point h

Text proposed by the Commission

Amendment

(h) Regulation (EU) 2019/2144.

deleted

Amendment 41

Proposal for a regulation

Article 2 – paragraph 5 a (new)

Text proposed by the Commission

Amendment

5a. This Regulation shall not apply to AI systems, including their output, developed and put into service for the sole purpose of research and development.

Amendment 42

Proposal for a regulation

Article 2 – paragraph 5 b (new)

Text proposed by the Commission

Amendment

5b. This Regulation shall not apply to any research and development activity regarding AI systems in so far as such activity does not lead to or require placing an AI system on the market or putting it into service and is in full respect of approved scientific ethical standards.

Amendment 43

Proposal for a regulation

Article 2 – paragraph 5 c (new)

Text proposed by the Commission

Amendment

5c. This Regulation is without prejudice to the rules laid down by other Union legal acts regulating the protection of personal data, in particular Regulation (EU) 2016/679, Regulation (EU) 2018/1725, Directive 2002/57/EC and Directive (EU) 2016/680.

Amendment 44

Proposal for a regulation

Article 3 – paragraph 1 – point 1

Text proposed by the Commission

Amendment

(1) ‘artificial intelligence system’ (AI system) means software that is developed with one or more of the techniques and approaches listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with;

(1) ‘artificial intelligence system’ (AI system) means a system that:

i) receives machine and/or human-based data and inputs,

ii) infers how to achieve a given set of human-defined objectives using learning, reasoning or modelling implemented with the techniques and approaches listed in Annex I, and

iii) generates outputs in the form of content (generative AI systems), predictions, recommendations or decisions, which influence the environments it interacts with;

Amendment 45

Proposal for a regulation

Article 3 – paragraph 1 – point 4

Text proposed by the Commission

Amendment

(4) ‘user’ means any natural or legal person, public authority, agency or other body using an AI system under its authority, except where the AI system is used in the course of a personal non-professional activity;

(4) ‘user’ means any natural or legal person, public authority, agency or other body using an AI system under its authority;

Amendment 46

Proposal for a regulation

Article 3 – paragraph 1 – point 5 a (new)

Text proposed by the Commission

Amendment

(5 a) ‘product manufacturer’ means a manufacturer within the meaning of any of the Union legislation listed in Annex II;

Amendment 47

Proposal for a regulation

Article 3 – paragraph 1 – point 13

Text proposed by the Commission

Amendment

(13) ‘reasonably foreseeable misuse’ means the use of an AI system in a way that is not in accordance with its intended purpose, but which may result from reasonably foreseeable human behaviour or interaction with other systems;

(13) ‘reasonably foreseeable misuse’ means the use of an AI system in a way that is not in accordance with its purpose as indicated in instruction for use or technical specification, but which may result from reasonably foreseeable human behaviour or interaction with other systems;

Amendment 48

Proposal for a regulation

Article 3 – paragraph 1 – point 14

Text proposed by the Commission

Amendment

(14) ‘safety component of a product or system’ means a component of a product or of a system which fulfils a safety function for that product or system or the failure or malfunctioning of which endangers the health and safety of persons or property;

(14) ‘safety component of a product or system’ means a component of a product or of a system the failure or malfunctioning of which endangers the health and safety of persons or property;

Amendment 49

Proposal for a regulation

Article 3 – paragraph 1 – point 35

Text proposed by the Commission

Amendment

(35) ‘biometric categorisation system’ means an AI system for the purpose of assigning natural persons to specific categories, such as sex, age, hair colour, eye colour, tattoos, ethnic origin or sexual or political orientation, on the basis of their biometric data;

(35) ‘biometric categorisation system’ means an AI system for the purpose of assigning natural persons to specific categories, such as sex, age, disability, hair colour, eye colour, tattoos, ethnic origin or sexual or political orientation, on the basis of their biometric data;

Amendment 50

Proposal for a regulation

Article 3 – paragraph 1 – point 44 – introductory part

Text proposed by the Commission

Amendment

(44) ‘serious incident’ means any incident that directly or indirectly leads, might have led or might lead to any of the following:

(44) ‘serious incident’ means any incident or malfunctioning of an AI system that directly or indirectly leads, might have led or might lead to any of the following:

Amendment 51

Proposal for a regulation

Article 3 – paragraph 1 – point 44 a (new)

Text proposed by the Commission

Amendment

(44a) 'personal data' means data as defined in point (1) of Article 4 of Regulation (EU)2016/679;

Amendment 52

Proposal for a regulation

Article 3 – paragraph 1 – point 44 b (new)

Text proposed by the Commission

Amendment

(44b) ‘non-personal data’ means data other than personal data as defined in point (1) of Article 4 of Regulation (EU) 2016/679;

Amendment 53

Proposal for a regulation

Article 4 – paragraph 1

Text proposed by the Commission

Amendment

The Commission is empowered to adopt delegated acts in accordance with Article 73 to amend the list of techniques and approaches listed in Annex I, in order to update that list to market and technological developments on the basis of characteristics that are similar to the techniques and approaches listed therein.

The Commission is empowered to adopt delegated acts in accordance with Article 73 to amend the list of techniques and approaches listed in Annex I within the scope of the definition of an AI system as provided for in Article 3(1), in order to update that list to market and technological developments on the basis of characteristics that are similar to the techniques and approaches listed therein

Amendment 54

Proposal for a regulation

Article 6 – paragraph 1 – introductory part

Text proposed by the Commission

Amendment

1. Irrespective of whether an AI system is placed on the market or put into service independently from the products referred to in points (a) and (b), that AI system shall be considered high-risk where both of the following conditions are fulfilled:

1. An AI system that is itself a product covered by the Union harmonisation legislation listed in Annex II shall be considered as high risk if it is required to undergo a third-party conformity assessment with a view to the placing on the market or putting into service of that product pursuant to the above mentioned legislation.

Amendment 55

Proposal for a regulation

Article 6 – paragraph 1 – point a

Text proposed by the Commission

Amendment

(a) the AI system is intended to be used as a safety component of a product, or is itself a product, covered by the Union harmonisation legislation listed in Annex II;

2. An AI system intended to be used as a safety component of a product covered by the legislation referred to in paragraph 1 shall be considered as high risk if it is required to undergo a third-party conformity assessment with a view to the placing on the market or putting into service of that product pursuant to abovementioned legislation. This provision shall apply irrespective of whether the AI system is placed on the market or put into service independently from the product.

Amendment 56

Proposal for a regulation

Article 6 – paragraph 2

Text proposed by the Commission

Amendment

2. In addition to the high-risk AI systems referred to in paragraph 1, AI systems referred to in Annex III shall also be considered high-risk.

3. AI systems referred to in Annex III shall be considered high-risk.

Amendment 57

Proposal for a regulation

Article 6 – paragraph 2 a (new)

Text proposed by the Commission

Amendment

2a. The classification as high-risk as a consequence of Article 6(1) 6(2) and 6(3) shall be disregarded for AI systems whose intended purpose demonstrates that the generated output is a recommendation requiring a human intervention to convert this recommendation into a decision and for AI systems, which do not lead to autonomous decisions or actions of the overall system.

Amendment 58

Proposal for a regulation

Article 7 – paragraph 1 – point b

Text proposed by the Commission

Amendment

(b) the AI systems pose a risk of harm to the health and safety, or a risk of adverse impact on fundamental rights, that is, in respect of its severity and probability of occurrence, equivalent to or greater than the risk of harm or of adverse impact posed by the high-risk AI systems already referred to in Annex III.

(b) the AI systems pose a risk of harm to the health or safety, or a risk of adverse impact on fundamental rights or the environment, that is, in respect of its severity and probability of occurrence, equivalent to or greater than the risk of harm or of adverse impact posed by the high-risk AI systems already referred to in Annex III.

Amendment 59

Proposal for a regulation

Article 7 – paragraph 2 – introductory part

Text proposed by the Commission

Amendment

2. When assessing for the purposes of paragraph 1 whether an AI system poses a risk of harm to the health and safety or a risk of adverse impact on fundamental rights that is equivalent to or greater than the risk of harm posed by the high-risk AI systems already referred to in Annex III, the Commission shall take into account the following criteria:

2. When assessing for the purposes of paragraph 1 whether an AI system poses a risk of harm to the health and safety or a risk of adverse impact on fundamental rights or on the environment that is equivalent to or greater than the risk of harm posed by the high-risk AI systems already referred to in Annex III, the Commission shall take into account the following criteria:

Amendment 60

Proposal for a regulation

Article 7 – paragraph 2 – point c

Text proposed by the Commission

Amendment

(c) the extent to which the use of an AI system has already caused harm to the health and safety or adverse impact on the fundamental rights or has given rise to significant concerns in relation to the materialisation of such harm or adverse impact, as demonstrated by reports or documented allegations submitted to national competent authorities;

(c) the extent to which the use of an AI system has already caused harm to the health and safety or adverse impact on the fundamental rights or on the environment or has given rise to significant concerns in relation to the materialisation of such harm or adverse impact, as demonstrated by reports or documented allegations submitted to national competent authorities;

Amendment 61

Proposal for a regulation

Article 7 – paragraph 2 – point d

Text proposed by the Commission

Amendment

(d) the potential extent of such harm or such adverse impact, in particular in terms of its intensity and its ability to affect a plurality of persons;

(d) the potential extent of such harm or such adverse impact, in particular in terms of its intensity and its ability to affect a plurality of persons or the environment;

Amendment 62

Proposal for a regulation

Article 7 – paragraph 2 – point g

Text proposed by the Commission

Amendment

(g) the extent to which the outcome produced with an AI system is easily reversible, whereby outcomes having an impact on the health or safety of persons shall not be considered as easily reversible;

(g) the extent to which the outcome produced with an AI system is easily reversible, whereby outcomes having an adverse impact on the health or safety of persons, or on the environment shall not be considered as easily reversible;

Amendment 63

Proposal for a regulation

Article 8 – paragraph 1

Text proposed by the Commission

Amendment

1. High-risk AI systems shall comply with the requirements established in this Chapter.

1. High-risk AI systems shall comply with the requirements established in this Chapter, taking into account sectoral legislation where applicable, harmonised standards and common specifications.

Amendment 64

Proposal for a regulation

Article 9 – paragraph 2 – point a

Text proposed by the Commission

Amendment

(a) identification and analysis of the known and foreseeable risks associated with each high-risk AI system;

(a) identification and analysis of the known and foreseeable risks associated with each high-risk AI system that might cause harm or damage to the environment or to the health, safety and fundamental rights of persons in view of the intended purpose of or misuse of the high-risk AI system.

Amendment 65

Proposal for a regulation

Article 9 – paragraph 2 – point c

Text proposed by the Commission

Amendment

(c) evaluation of other possibly arising risks based on the analysis of data gathered from the post-market monitoring system referred to in Article 61;

(c) evaluation of other possibly arising risks based on the analysis of data gathered from the post-market monitoring system;

Amendment 66

Proposal for a regulation

Article 9 – paragraph 4 – subparagraph 1

Text proposed by the Commission

Amendment

The risk management measures referred to in paragraph 2, point (d) shall be such that any residual risk associated with each hazard as well as the overall residual risk of the high-risk AI systems is judged acceptable, provided that the high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse. Those residual risks shall be communicated to the user.

The risk management measures referred to in paragraph 2, point (d) shall be such that any residual risk associated with each hazard as well as the overall residual risk of the high-risk AI systems is judged acceptable, provided that the high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, subject to terms, conditions as made available by the provider, and contractual and license restrictions. Those residual risks shall be communicated to the user.

Amendment 67

Proposal for a regulation

Article 10 – paragraph 1

Text proposed by the Commission

Amendment

1. High-risk AI systems which make use of techniques involving the training of models with data shall be developed on the basis of training, validation and testing data sets that meet the quality criteria referred to in paragraphs 2 to 5.

Amendment 68

Proposal for a regulation

Article 10 – paragraph 2 – point g

Text proposed by the Commission

Amendment

(g) the identification of any possible data gaps or shortcomings, and how those gaps and shortcomings can be addressed.

(g) the identification of any other data gaps or shortcomings that materially increase the risks of harm to the health, environment and safety or the fundamental rights of persons, and how those gaps and shortcomings can be addressed.

Amendment 69

Proposal for a regulation

Article 10 – paragraph 3

Text proposed by the Commission

Amendment

3. Training, validation and testing data sets shall be relevant, representative, free of errors and complete. They shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons on which the high-risk AI system is intended to be used. These characteristics of the data sets may be met at the level of individual data sets or a combination thereof.

3. Training, validation and testing data sets shall be relevant, representative, free of errors and to the best extent possible and as complete as possible. They shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons on which the high-risk AI system is intended to be used. These characteristics of the data sets may be met at the level of individual data sets or a combination thereof.

Amendment 70

Proposal for a regulation

Article 10 – paragraph 4

Text proposed by the Commission

Amendment

4. Training, validation and testing data sets shall take into account, to the extent required by the intended purpose, the characteristics or elements that are particular to the specific geographical, behavioural or functional setting within which the high-risk AI system is intended to be used.

4. Training, validation and testing data sets shall be sufficiently diverse to accurately capture, to the extent required by the intended purpose, the characteristics or elements that are particular to the specific geographical, behavioural or functional setting within which the high-risk AI system is intended to be used.

Amendment 71

Proposal for a regulation

Article 11 – paragraph 1 – subparagraph 2

Text proposed by the Commission

Amendment

The technical documentation shall be drawn up in such a way to demonstrate that the high-risk AI system complies with the requirements set out in this Chapter and provide national competent authorities and notified bodies with all the necessary information to assess the compliance of the AI system with those requirements. It shall contain, at a minimum, the elements set out in Annex IV or, in the case of SMEs and start-ups, any equivalent documentation meeting the same objectives, subject to approval of the competent authority. Documentation shall be kept up to date throughout its entire lifecycle.

Amendment 72

Proposal for a regulation

Article 12 – paragraph 2

Text proposed by the Commission

Amendment

2. The logging capabilities shall ensure a level of traceability of the AI system’s functioning throughout its lifecycle that is appropriate to the intended purpose of the system.

2. The logging capabilities shall ensure a level of traceability of the AI system’s functioning while the AI system is used within its lifecycle that is appropriate to the intended purpose of the system.

Amendment 73

Proposal for a regulation

Article 13 – paragraph 2

Text proposed by the Commission

Amendment

2. High-risk AI systems shall be accompanied by instructions for use in an appropriate digital format or otherwise that include concise, complete, correct and clear information that is relevant, accessible and comprehensible to users.

2. High-risk AI systems shall be accompanied by instructions for use in an appropriate digital format or made otherwise available, that include concise, complete, correct and clear information that is relevant, accessible and comprehensible to users to assist them in operating and maintaining the AI system, taking into consideration the system’s intended purpose and the expected audience for the instructions.

Amendment 74

Proposal for a regulation

Article 13 – paragraph 3 – point b – point ii

Text proposed by the Commission

Amendment

(ii) the level of accuracy, robustness and cybersecurity referred to in Article 15 against which the high-risk AI system has been tested and validated and which can be expected, and any known and foreseeable circumstances that may have an impact on that expected level of accuracy, robustness and cybersecurity;

(ii) the level of accuracy, robustness and cybersecurity referred to in Article 15 against which the high-risk AI system has been tested and validated and which can be expected, and any known and reasonably foreseeable circumstances that could materially impact that expected level of accuracy, robustness and cybersecurity;

Amendment 75

Proposal for a regulation

Article 13 – paragraph 3 – point b – point iii

Text proposed by the Commission

Amendment

(iii) any known or foreseeable circumstance, related to the use of the high-risk AI system in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, which may lead to risks to the health and safety or fundamental rights;

Amendment 76

Proposal for a regulation

Article 13 – paragraph 3 – point e

Text proposed by the Commission

Amendment

(e) the expected lifetime of the high-risk AI system and any necessary maintenance and care measures to ensure the proper functioning of that AI system, including as regards software updates.

(e) the expected lifetime of the high-risk AI system, the description of the procedure of withdrawing it from use and any necessary maintenance and care measures to ensure the proper functioning of that AI system, including as regards software updates.

Amendment 77

Proposal for a regulation

Article 14 – paragraph 1

Text proposed by the Commission

Amendment

1. High-risk AI systems shall be designed and developed in such a way, including with appropriate human-machine interface tools, that they can be effectively overseen by natural persons during the period in which the AI system is in use, unless there is clear evidence that human intervention compromises the safety of the high risk AI system concerned.

Amendment 78

Proposal for a regulation

Article 14 – paragraph 2

Text proposed by the Commission

Amendment

2. Human oversight shall aim at preventing or minimising the risks to health, safety or fundamental rights that may emerge when a high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, in particular when such risks persist notwithstanding the application of other requirements set out in this Chapter.

2. Human oversight shall aim at preventing or minimising the risks to health, safety or fundamental rights or the environment that may emerge when a high-risk AI system is used in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, in particular when such risks persist notwithstanding the application of other requirements set out in this Chapter.

Amendment 79

Proposal for a regulation

Article 14 – paragraph 4 – introductory part

Text proposed by the Commission

Amendment

4. The measures referred to in paragraph 3 shall enable the individuals to whom human oversight is assigned to do the following, as appropriate to the circumstances:

4. The measures referred to in paragraph 3 shall enable the individuals to whom human oversight is assigned to do the following, as appropriate and proportionate to the circumstances:

Amendment 80

Proposal for a regulation

Article 14 – paragraph 4 – point a

Text proposed by the Commission

Amendment

(a) fully understand the capacities and limitations of the high-risk AI system and be able to duly monitor its operation, so that signs of anomalies, dysfunctions and unexpected performance can be detected and addressed as soon as possible;

(a) have an appropriate understanding of the capacities and limitations of the high-risk AI system and be able to duly monitor its operation, so that signs of anomalies, dysfunctions and unexpected performance can be detected and addressed as soon as possible;

Amendment 81

Proposal for a regulation

Article 14 – paragraph 4 – point d

Text proposed by the Commission

Amendment

(d) be able to decide, in any particular situation, not to use the high-risk AI system or otherwise disregard, override or reverse the output of the high-risk AI system;

(d) be able to decide, in any particular situation, not to use the high-risk AI system or otherwise disregard, override or reverse the output of the high-risk AI system, unless there is clear evidence that such human intervention is deemed to increase risks or otherwise negatively impact the system’s performance.

Amendment 82

Proposal for a regulation

Article 14 – paragraph 4 – point e

Text proposed by the Commission

Amendment

(e) be able to intervene on the operation of the high-risk AI system or interrupt the system through a “stop” button or a similar procedure.

(e) be able to intervene on the operation of the high-risk AI system put the system into fail-safe mode, put the system into manual control mode or stop the system through a “stop” button or a similar procedure unless there is clear evidence that such human intervention is deemed to increase risks or otherwise negatively impact the system’s performance.

Amendment 83

Proposal for a regulation

Article 14 – paragraph 4 – point e a (new)

Text proposed by the Commission

Amendment

(ea) be able to comprehend when a high risk AI system decision is preferable to human oversight.

Amendment 84

Proposal for a regulation

Article 14 – paragraph 5

Text proposed by the Commission

Amendment

5. For high-risk AI systems referred to in point 1(a) of Annex III, the measures referred to in paragraph 3 shall be such as to ensure that, in addition, no action or decision is taken by the user on the basis of the identification resulting from the system unless this has been verified and confirmed by at least two natural persons.

5. For high-risk AI systems referred to in point 1(a) of Annex III, the measures referred to in paragraph 3 shall be such as to ensure that, in addition, no action or decision is taken by the user on the basis of the identification resulting from the system unless this has been separately verified and confirmed by at least two natural persons.

Amendment 85

Proposal for a regulation

Article 15 – paragraph 1

Text proposed by the Commission

Amendment

1. High-risk AI systems shall be designed and developed in such a way that they achieve, in the light of their intended purpose, an appropriate level of accuracy, robustness and cybersecurity, and perform consistently in those respects throughout their lifecycle.

1. High-risk AI systems shall be designed and developed in such a way that they achieve, in the light of their intended purpose, the highest level of accuracy, robustness and cybersecurity possible, and perform consistently in those respects throughout their lifecycle.

Amendment 86

Proposal for a regulation

Article 15 – paragraph 3 – subparagraph 1

Text proposed by the Commission

Amendment

High-risk AI systems shall be resilient as regards errors, faults or inconsistencies that may occur within the system or the environment in which the system operates, in particular due to their interaction with natural persons or other systems.

Providers should take all appropriate and feasible measures to ensure that high-risk AI systems are resilient as regards errors, faults or inconsistencies that may occur within the system or the environment in which the system operates, in particular due to their interaction with natural persons or other systems.

Amendment 87

Proposal for a regulation

Article 15 – paragraph 3 – subparagraph 3

Text proposed by the Commission

Amendment

High-risk AI systems that continue to learn after being placed on the market or put into service shall be developed in such a way to ensure that possibly biased outputs due to outputs used as an input for future operations (‘feedback loops’) are duly addressed with appropriate mitigation measures.

High-risk AI systems that continue to learn after being placed on the market or put into service shall be developed in such a way to ensure that possibly biased outputs influencing an input for future operations (‘feedback loops’) are duly addressed with appropriate mitigation measures.

Amendment 88

Proposal for a regulation

Article 28 – paragraph 1 – point c a (new)

Text proposed by the Commission

Amendment

(ca) they modify the intended purpose of an AI system which is not high-risk and is already placed on the market or put into service, in a way which makes the modified system a high-risk AI system.

Amendment 89

Proposal for a regulation

Article 29 – paragraph 6 a (new)

Text proposed by the Commission

Amendment

6a. This Article only applies to users acting in their professional capacity and not to those using AI in the course of a personal non-professional activity.

Amendment 90

Proposal for a regulation

Article 29 – paragraph 6 b (new)

Text proposed by the Commission

Amendment

6b. Users of high risk AI systems, who modify or extend the purpose for which the conformity of the AI system was originally assessed, shall establish and document a post-market monitoring system(Art. 61) and must undergo a new conformity assessment (Art. 43) involved by a notified body.

Amendment 91

Proposal for a regulation

Article 30 – paragraph 8

Text proposed by the Commission

Amendment

8. Notifying authorities shall make sure that conformity assessments are carried out in a proportionate manner, avoiding unnecessary burdens for providers and that notified bodies perform their activities taking due account of the size of an undertaking, the sector in which it operates, its structure and the degree of complexity of the AI system in question.

Amendment 92

Proposal for a regulation

Article 33 – paragraph 6

Text proposed by the Commission

Amendment

6. Notified bodies shall have documented procedures in place ensuring that their personnel, committees, subsidiaries, subcontractors and any associated body or personnel of external bodies respect the confidentiality of the information which comes into their possession during the performance of conformity assessment activities, except when disclosure is required by law. The staff of notified bodies shall be bound to observe professional secrecy with regard to all information obtained in carrying out their tasks under this Regulation, except in relation to the notifying authorities of the Member State in which their activities are carried out.

Amendment 93

Proposal for a regulation

Article 39 a (new)

Text proposed by the Commission

Amendment

Article 39a

Exchange of knowhow and best practices

The Commission shall facilitate regular consultative meetings for the exchange of knowhow and best practices between the Member States' national authorities responsible for notification policy.

Amendment 94

Proposal for a regulation

Article 40 – paragraph 1 a (new)

Text proposed by the Commission

Amendment

When issuing a standardisation request to European standardisation organisations in accordance with Article 10 of Regulation 1025/2012, the Commission shall specify that standards are coherent, easy to implement and drafted in such a way that they aim to fulfil in particular the following objectives:

a) ensure that AI systems placed on the market or put into service in the Union are safe and respect Union values and public interests, and strengthen the Union's digital leadership;

b) promote investment and innovation in AI, as well as competitiveness and growth of the Union market;

c) enhance multi-stakeholder governance, by ensuring it is inclusive and representative of all relevant European stakeholders (e.g. civil society, researchers industry, SMEs).

d) contribute to strengthening global cooperation on standardisation in the field of AI that is consistent with Union values and interests.

The Commission shall request the European standardisation organisations to regularly report on their progress with regard to the above objectives.

Amendment 95

Proposal for a regulation

Article 41 – paragraph 1

Text proposed by the Commission

Amendment

1. Where harmonised standards referred to in Article 40 do not exist or where the Commission considers that the relevant harmonised standards are insufficient or that there is a need to address specific safety or fundamental right concerns, the Commission may, by means of implementing acts, adopt common specifications in respect of the requirements set out in Chapter 2 of this Title. Those implementing acts shall be adopted in accordance with the examination procedure referred to in Article 74(2).

1. Where harmonised standards referred to in Article 40 do not exist or where the Commission considers that the relevant harmonised standards are insufficient or that there is a need to address specific safety or fundamental right concerns, the Commission may, after consulting the AI Board referred to in Article 56 and the responsible authorities and organizations for a given sector, by means of implementing acts, adopt common specifications in respect of the requirements set out in Chapter 2 of this Title. Those implementing acts shall be adopted in accordance with the examination procedure referred to in Article 74(2).

Amendment 96

Proposal for a regulation

Article 41 – paragraph 2

Text proposed by the Commission

Amendment

2. The Commission, when preparing the common specifications referred to in paragraph 1, shall gather the views of relevant bodies or expert groups established under relevant sectorial Union law.

2. When preparing the common specifications referred to in paragraph 1, the Commission shall fulfil the objectives referred of Article 40(2) and gather the views of relevant bodies or expert groups established under relevant sectorial Union law as well as relevant sector-specific stakeholders.

Amendment 97

Proposal for a regulation

Article 43 – paragraph 6

Text proposed by the Commission

Amendment

6. The Commission is empowered to adopt delegated acts to amend paragraphs 1 and 2 in order to subject high-risk AI systems referred to in points 2 to 8 of Annex III to the conformity assessment procedure referred to in Annex VII or parts thereof. The Commission shall adopt such delegated acts taking into account the effectiveness of the conformity assessment procedure based on internal control referred to in Annex VI in preventing or minimizing the risks to health and safety and protection of fundamental rights posed by such systems as well as the availability of adequate capacities and resources among notified bodies.

6. The Commission is empowered to adopt delegated acts to amend paragraphs 1 and 2 in order to subject high-risk AI systems referred to in points 2 to 8 of Annex III to the conformity assessment procedure referred to in Annex VII or parts thereof. The Commission shall adopt such delegated acts taking into account the effectiveness of the conformity assessment procedure based on internal control referred to in Annex VI in preventing or minimizing the risks to health, safety, the environment and protection of fundamental rights posed by such systems as well as the availability of adequate capacities and resources among notified bodies.

Amendment 98

Proposal for a regulation

Article 52 – title

Text proposed by the Commission

Amendment

Transparency obligations for certain AI systems

Transparency obligations for AI systems

Amendment 99

Proposal for a regulation

Article 52 – paragraph 3 a (new)

Text proposed by the Commission

Amendment

3a. Providers of any AI system should document and make available upon request the parameters regarding the environmental impact, including but not limited to resource consumption, resulting from the design, data management and training, the underlying infrastructures of the AI system, and of the methods to reduce such impact.

Amendment 100

Proposal for a regulation

Article 53 – paragraph 1

Text proposed by the Commission

Amendment

1. AI regulatory sandboxes established by one or more Member States competent authorities or the European Data Protection Supervisor shall provide a controlled environment that facilitates the development, testing and validation of innovative AI systems for a limited time before their placement on the market or putting into service pursuant to a specific plan. This shall take place under the direct supervision and guidance by the competent authorities with a view to ensuring compliance with the requirements of this Regulation and, where relevant, other Union and Member States legislation supervised within the sandbox.

1. AI regulatory sandboxes established by one or more Member States competent authorities or the European Data Protection Supervisor shall provide a controlled environment that facilitates the development, testing and validation of innovative AI systems and secure processing of personal data for a limited time before their placement on the market or putting into service pursuant to a specific plan. This shall take place under the direct supervision and guidance by the competent authorities with a view to ensuring compliance with the requirements of this Regulation and, where relevant, other Union and Member States legislation supervised within the sandbox.

Amendment 101

Proposal for a regulation

Article 53 – paragraph 1 a (new)

Text proposed by the Commission

Amendment

1a. The organisers of AI regulatory sandboxes shall ensure an easy access for SMEs and start-ups by facilitating and supporting their participation.

Amendment 102

Proposal for a regulation

Article 53 – paragraph 1 b (new)

Text proposed by the Commission

Amendment

1b. The controllers of personal data referred to in Article 4 (7) of the Regulation (EU) 2016/679 may further process personal data in an AI regulatory sandbox to the extent that it is necessary for the purposes of development, testing and validation of AI systems. Right of processing is subject to appropriate safeguards for the fundamental rights and freedoms of natural persons. This processing shall not be considered incompatible with the initial purposes.

Amendment 103

Proposal for a regulation

Article 53 – paragraph 3

Text proposed by the Commission

Amendment

3. The AI regulatory sandboxes shall not affect the supervisory and corrective powers of the competent authorities. Any significant risks to health and safety and fundamental rights identified during the development and testing of such systems shall result in immediate mitigation and, failing that, in the suspension of the development and testing process until such mitigation takes place.

3. The AI regulatory sandboxes shall not affect the supervisory and corrective powers of the competent authorities. Any significant risks to health, safety, the environment or fundamental rights identified during the development and testing of such systems shall result in immediate mitigation and, failing that, in the suspension of the development and testing process until such mitigation takes place.

Amendment 104

Proposal for a regulation

Article 53 – paragraph 5

Text proposed by the Commission

Amendment

5. Member States’ competent authorities that have established AI regulatory sandboxes shall coordinate their activities and cooperate within the framework of the European Artificial Intelligence Board. They shall submit annual reports to the Board and the Commission on the results from the implementation of those scheme, including good practices, lessons learnt and recommendations on their setup and, where relevant, on the application of this Regulation and other Union legislation supervised within the sandbox.

5. Member States’ competent authorities shall coordinate their activities with regards to AI regulatory sandboxes and cooperate within the framework of the European Artificial Intelligence Board. They shall submit annual reports to the Board and the Commission on the results from the implementation of those scheme, including good practices, lessons learnt and recommendations on their setup and, where relevant, on the application of this Regulation and other Union legislation supervised within the sandbox.

Amendment 105

Proposal for a regulation

Article 54 – paragraph 1 – point a – point iii a (new)

Text proposed by the Commission

Amendment

(iiia) safety and resilience of transport systems, infrastructure and networks.

Amendment 106

Proposal for a regulation

Article 55 – title

Text proposed by the Commission

Amendment

Measures for small-scale providers and users

Measures for SMEs, start-ups and users

Amendment 107

Proposal for a regulation

Article 55 – paragraph 1 – point a

Text proposed by the Commission

Amendment

(a) provide small-scale providers and start-ups with priority access to the AI regulatory sandboxes to the extent that they fulfil the eligibility conditions;

(a) provide SMEs and start-ups with priority access to the AI regulatory sandboxes to the extent that they fulfil the eligibility conditions;

Amendment 108

Proposal for a regulation

Article 55 – paragraph 1 – point b

Text proposed by the Commission

Amendment

(b) organise specific awareness raising activities about the application of this Regulation tailored to the needs of the small-scale providers and users;

(b) organise specific awareness raising activities about the application of this Regulation tailored to the needs of SMEs, start-ups and users;

Amendment 109

Proposal for a regulation

Article 55 – paragraph 1 – point c

Text proposed by the Commission

Amendment

(c) where appropriate, establish a dedicated channel for communication with small-scale providers and user and other innovators to provide guidance and respond to queries about the implementation of this Regulation.

(c) where appropriate, establish a dedicated channel for communication with SMEs and user, start-ups and other innovators to provide guidance and respond to queries about the implementation of this Regulation.

Amendment 110

Proposal for a regulation

Article 55 – paragraph 2 a (new)

Text proposed by the Commission

Amendment

2a. Where appropriate, Member States shall find synergies and cooperate with relevant instruments funded by Union programmes, such as the European Digital Innovation Hubs.

Amendment 111

Proposal for a regulation

Article 57 – paragraph 1

Text proposed by the Commission

Amendment

1. The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority, and the European Data Protection Supervisor. Other national authorities may be invited to the meetings, where the issues discussed are of relevance for them.

1. The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority, the European Data Protection Supervisor, AI ethics experts and industry representatives. Other national, regional and local authorities may be invited to the meetings, where the issues discussed are of relevance for them.

Amendment 112

Proposal for a regulation

Article 57 – paragraph 3

Text proposed by the Commission

Amendment

3. The Board shall be chaired by the Commission. The Commission shall convene the meetings and prepare the agenda in accordance with the tasks of the Board pursuant to this Regulation and with its rules of procedure. The Commission shall provide administrative and analytical support for the activities of the Board pursuant to this Regulation.

3. The Board shall be co-chaired by the Commission and representative chosen from among the delegates of the Member States. The Commission shall convene the meetings and prepare the agenda in accordance with the tasks of the Board pursuant to this Regulation and with its rules of procedure. The Commission shall provide administrative and analytical support for the activities of the Board pursuant to this Regulation.

Amendment 113

Proposal for a regulation

Article 57 – paragraph 3 a (new)

Text proposed by the Commission

Amendment

3a. The Board shall organise consultations with stakeholders at least twice a year. Such stakeholders shall include representatives from industry, SMEs and start-ups, civil society organisations such as NGOs, consumer associations, the social partners and academia, to assess the evolution of trends in technology, issues related to the implementation and the effectiveness of this Regulation, regulatory gaps or loopholes observed in practice.

Amendment 114

Proposal for a regulation

Article 57 – paragraph 4

Text proposed by the Commission

Amendment

4. The Board may invite external experts and observers to attend its meetings and may hold exchanges with interested third parties to inform its activities to an appropriate extent. To that end, the Commission may facilitate exchanges between the Board and other Union bodies, offices, agencies and advisory groups. The Board shall actively reach out to and hear representatives from groups, which are more vulnerable to discriminatory effects posed by AI, such as people with disabilities.

Amendment 115

Proposal for a regulation

Article 59 – paragraph 4

Text proposed by the Commission

Amendment

4. Member States shall ensure that national competent authorities are provided with adequate financial and human resources to fulfil their tasks under this Regulation. In particular, national competent authorities shall have a sufficient number of personnel permanently available whose competences and expertise shall include an in-depth understanding of artificial intelligence technologies, data and data computing, fundamental rights, health and safety risks and knowledge of existing standards and legal requirements.

4. Member States shall ensure that national competent authorities are provided with adequate financial and human resources to fulfil their tasks under this Regulation. In particular, national competent authorities shall have a sufficient number of personnel permanently available whose competences and expertise shall include an in-depth understanding of artificial intelligence technologies, data and data computing, fundamental rights, health, safety and environmental risks and knowledge of existing standards and legal requirements.

Amendment 116

Proposal for a regulation

Article 59 – paragraph 4 a (new)

Text proposed by the Commission

Amendment

4a. Any information and documentation obtained by the national competent authorities pursuant to the provisions of this Article shall be treated in compliance with the confidentiality obligations set out in Article 70.

Amendment 117

Proposal for a regulation

Article 60 – paragraph 3

Text proposed by the Commission

Amendment

3. Information contained in the EU database shall be accessible to the public.

3. Information contained in the EU database shall be accessible to the public, user-friendly, easily navigable and machine-readable.

Amendment 118

Proposal for a regulation

Article 60 – paragraph 5 a (new)

Text proposed by the Commission

Amendment

5a. Any information and documentation obtained by the Commission and Member States pursuant to this Article shall be treated in compliance with the confidentiality obligations set out in Article 70.

Amendment 119

Proposal for a regulation

Article 61 – paragraph 2

Text proposed by the Commission

Amendment

2. The post-market monitoring system shall actively and systematically collect, document and analyse relevant data provided by users or collected through other sources on the performance of high-risk AI systems throughout their lifetime, and allow the provider to evaluate the continuous compliance of AI systems with the requirements set out in Title III, Chapter 2. Post-market monitoring must include continuous analysis of the AI environment, including other devices, software, and other AI systems that will interact with the AI system.

Amendment 120

Proposal for a regulation

Article 65 – paragraph 1

Text proposed by the Commission

Amendment

1. AI systems presenting a risk shall be understood as a product presenting a risk defined in Article 3, point 19 of Regulation (EU) 2019/1020 insofar as risks to the health or safety or to the protection of fundamental rights of persons are concerned.

1. AI systems presenting a risk shall be understood as a product presenting a risk defined in Article 3, point 19 of Regulation(EU) 2019/1020 insofar as risks to health, safety or the environment, or to the protection of fundamental rights of persons are concerned.

Amendment 121

Proposal for a regulation

Article 67 – paragraph 1

Text proposed by the Commission

Amendment

1. Where, having performed an evaluation under Article 65, the market surveillance authority of a Member State finds that although an AI system is in compliance with this Regulation, it presents a risk to the health or safety of persons, to the compliance with obligations under Union or national law intended to protect fundamental rights or to other aspects of public interest protection, it shall require the relevant operator to take all appropriate measures to ensure that the AI system concerned, when placed on the market or put into service, no longer presents that risk, to withdraw the AI system from the market or to recall it within a reasonable period, commensurate with the nature of the risk, as it may prescribe.

1. Where, having performed an evaluation under Article 65, the market surveillance authority of a Member State finds that although an AI system is in compliance with this Regulation, it presents a risk to the health or safety of persons, to the environment, to the compliance with obligations under Union or national law intended to protect fundamental rights or to other aspects of public interest protection, it shall require the relevant operator to take all appropriate measures to ensure that the AI system concerned, when placed on the market or put into service, no longer presents that risk, to withdraw the AI system from the market or to recall it within a reasonable period, commensurate with the nature of the risk, as it may prescribe.

Amendment 122

Proposal for a regulation

Article 69 – paragraph 3

Text proposed by the Commission

Amendment

3. Codes of conduct may be drawn up by individual providers of AI systems or by organisations representing them or by both, including with the involvement of users and any interested stakeholders and their representative organisations. Codes of conduct may cover one or more AI systems taking into account the similarity of the intended purpose of the relevant systems.

3. Codes of conduct may be drawn up by national, regional or local authorities, by individual providers of AI systems or by organisations representing them or by both, including with the involvement of users and any interested stakeholders and their representative organisations. Codes of conduct may cover one or more AI systems taking into account the similarity of the intended purpose of the relevant systems.

Amendment 123

Proposal for a regulation

Article 72 – paragraph 1 – point a

Text proposed by the Commission

Amendment

(a) the nature, gravity and duration of the infringement and of its consequences;

(a) the nature, gravity and duration of the infringement and of its consequences; taking into account the number of subjects affected and the level of damage suffered by them, the intentional or negligent character of the infringement and any relevant previous infringement;

Amendment 124

Proposal for a regulation

Article 72 – paragraph 1 – point b a (new)

Text proposed by the Commission

Amendment

(ba) the degree of cooperation with the supervisory authority, in order to remedy the infringement and mitigate the possible adverse effects of the infringement;

Amendment 125

Proposal for a regulation

Article 72 – paragraph 1 – point b b (new)

Text proposed by the Commission

Amendment

(bb) any action taken by the provider to mitigate the damage suffered by subjects;

Amendment 126

Proposal for a regulation

Article 72 – paragraph 1 – point c a (new)

Text proposed by the Commission

Amendment

(ca) any other aggravating or mitigating factor applicable to the circumstances of the case, such as financial benefits gained, or losses avoided, directly or indirectly, from the infringement.

Amendment 127

Proposal for a regulation

Article 75 – paragraph 1

Regulation (EC) No 300/2008

Article 4 – paragraph 3 – subparagraph 3

Text proposed by the Commission

Amendment

When adopting detailed measures related to technical specifications and procedures for approval and use of security equipment concerning Artificial Intelligence systems in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Chapter 2, Title III of that Regulation shall be taken into account.

When adopting detailed measures related to technical specifications and procedures for approval and use of security equipment concerning Artificial Intelligence systems in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, without interfering with existing governance, the requirements set out in Chapter 2, Title III of that Regulation shall be taken into account.

_______

________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).”

Amendment 128

Proposal for a regulation

Article 76 – paragraph 1

Regulation (EU) No 167/2013

Article 17 – paragraph 5 – subparagraph 3

Text proposed by the Commission

Amendment

When adopting delegated acts pursuant to the first subparagraph concerning artificial intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

When adopting delegated acts pursuant to the first subparagraph concerning artificial intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, without interfering with existing governance, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

_______

________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).”

Amendment 129

Proposal for a regulation

Article 78 – paragraph 1

Directive 2014/90/EU

Article 8 – paragraph 4

Text proposed by the Commission

Amendment

4. “For Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, when carrying out its activities pursuant to paragraph 1 and when adopting technical specifications and testing standards in accordance with paragraphs 2 and 3, the Commission shall take into account the requirements set out in Title III, Chapter 2 of that Regulation.

4. “For Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, when carrying out its activities pursuant to paragraph 1 and when adopting technical specifications and testing standards in accordance with paragraphs 2 and 3, and without interfering with existing governance, the Commission shall take into account the requirements set out in Title III, Chapter 2 of that Regulation.

_______

________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).”

Amendment 130

Proposal for a regulation

Article 79 – paragraph 1

Directive (EU) 2016/797

Article 5 – paragraph 12

Text proposed by the Commission

Amendment

12. “When adopting delegated acts pursuant to paragraph 1 and implementing acts pursuant to paragraph 11 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

12. “When adopting delegated acts pursuant to paragraph 1 and implementing acts pursuant to paragraph 11 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, without interfering with existing governance, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

_______

________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).”

Amendment 131

Proposal for a regulation

Article 80 – paragraph 1

Regulation (EU) 2018/858

Article 5 – paragraph 4

Text proposed by the Commission

Amendment

4. “When adopting delegated acts pursuant to paragraph 3 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council *, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

4. “When adopting delegated acts pursuant to paragraph 3 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council *, without interfering with existing governance, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

_______

________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).”

Amendment 132

Proposal for a regulation

Article 81 – paragraph 1 – point 1

Regulation (EU) 2018/1139

Article 17 – paragraph 3

Text proposed by the Commission

Amendment

3. “Without prejudice to paragraph 2, when adopting implementing acts pursuant to paragraph 1 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

3. “Without prejudice to paragraph 2, and to the certification, oversight and enforcement system referred to in Article 62 of this Regulation, when adopting implementing acts pursuant to paragraph 1 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

_______

________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).”

Amendment 133

Proposal for a regulation

Article 81 – paragraph 1 – point 2

Regulation (EU) 2018/1139

Article 19 – paragraph 4

Text proposed by the Commission

Amendment

4. When adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

4. Without prejudice to the certification, oversight and enforcement system referred to in Article 62 of this Regulation, when adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation(EU) YYY/XX [on Artificial Intelligence], only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Amendment 134

Proposal for a regulation

Article 81 – paragraph 1 – point 3

Regulation (EU) 2018/1139

Article 43 – paragraph 4

Text proposed by the Commission

Amendment

4. When adopting implementing acts pursuant to paragraph 1 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

4. Without prejudice to the certification, oversight and enforcement system referred to in Article 62 of this Regulation, when adopting implementing acts pursuant to paragraph 1 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX[on Artificial Intelligence], only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Amendment 135

Proposal for a regulation

Article 81 – paragraph 1 – point 4

Regulation (EU) 2018/1139

Article 47 – paragraph 3

Text proposed by the Commission

Amendment

3. When adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

3. Without prejudice to the certification, oversight and enforcement system referred to in Article 62 of this Regulation, when adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation(EU) YYY/XX [on Artificial Intelligence], only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Amendment 136

Proposal for a regulation

Article 81 – paragraph 1 – point 5

Regulation (EU) 2018/1139

Article 57 – paragraph 3

Text proposed by the Commission

Amendment

When adopting those implementing acts concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

Without prejudice to the certification, oversight and enforcement system referred to in Article 62of this Regulation, when adopting those implementing acts concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence], only the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account."

Amendment 137

Proposal for a regulation

Article 81 – paragraph 1 – point 6

Regulation (EU) 2018/1139

Article 58 – paragraph 3

Text proposed by the Commission

Amendment

3. When adopting delegated acts pursuant to paragraphs 1 and 2 concerning Artificial Intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] , the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account..

Amendment 138

Proposal for a regulation

Article 82 – paragraph 1

Regulation (EU) 2019/2144

Article 11

Text proposed by the Commission

Amendment

3. “When adopting the implementing acts pursuant to paragraph 2, concerning artificial intelligence systems which are safety components in the meaning of Regulation (EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

3. “When adopting the implementing acts pursuant to paragraph 2, concerning artificial intelligence systems which are safety components in the meaning of Regulation(EU) YYY/XX [on Artificial Intelligence] of the European Parliament and of the Council*, without interfering with existing governance, the requirements set out in Title III, Chapter 2 of that Regulation shall be taken into account.

_______

________

* Regulation (EU) YYY/XX [on Artificial Intelligence] (OJ …).”

Amendment 139

Proposal for a regulation

Article 84 – paragraph 3 a (new)

Text proposed by the Commission

Amendment

3a. Within [two years after the date of application of this Regulation referred to in Article 85(2)] and every two years thereafter, the Commission shall evaluate the impact and effectiveness of the Regulation with regards to the energy use and other environmental impact of AI systems and evaluate bringing legislation to regulate the energy efficiency of ICT systems in order for the sector to contribute to Union climate strategy and targets.

Amendment 140

Proposal for a regulation

Article 84 – paragraph 7 a (new)

Text proposed by the Commission

Amendment

7a. Any relevant future delegated or implementing acts to Regulations listed in Annex II, section B, introducing mandatory requirements for High-Risk AI systems laid down in this Regulation, shall take into account the regulatory specificities of each sector and shall not overlap with existing governance, conformity assessment, and enforcement mechanisms and authorities established therein.

Amendment 141

Proposal for a regulation

Annex I – title

Text proposed by the Commission

Amendment

ARTIFICIAL INTELLIGENCE TECHNIQUES AND APPROACHESreferred to in Article 3, point 1

ARTIFICIAL INTELLIGENCE TECHNIQUES AND APPROACHES referred to in Article 3, point 1

Amendment 142

Proposal for a regulation

Annex III – paragraph 1 – point 2 – point a

Text proposed by the Commission

Amendment

(a) AI systems intended to be used as safety components in the management and operation of road traffic and the supply of water, gas, heating and electricity.

(a) AI systems intended to be used as safety components in the management and operation of road traffic and the supply of water, gas, heating and electricity, whose failure or malfunctioning would directly cause significant harm to the health, natural environment or safety of natural persons, unless these systems are regulated in harmonisation legislation or sectorial regulation.

ANNEX: LIST OF ENTITIES OR PERSONS FROM WHOM THE RAPPORTEUR FOR THE OPINION HAS RECEIVED INPUT

 BEUC

 ETF

 Google

 Amazon

 Airbus

 Hitachi

 DG MOVE

 DG Connect

 EASA

 AMCHAM

 ACEA

 CLEPA

 Ericsson

PROCEDURE – COMMITTEE ASKED FOR OPINION

Title

Harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union Legislative Acts

References

COM(2021)0206 – C9-0146/2021 – 2021/0106(COD)

Committees responsible

Date announced in plenary

IMCO

7.6.2021

LIBE

7.6.2021

Opinion by

Date announced in plenary

TRAN

7.6.2021

Rapporteur for the opinion

Date appointed

Josianne Cutajar

4.11.2021

Rule 58 – Joint committee procedure

Date announced in plenary

16.12.2021

Discussed in committee

20.4.2022

Date adopted

12.7.2022

Result of final vote

–:

Members present for the final vote

Magdalena Adamowicz, Andris Ameriks, Izaskun Bilbao Barandica, Karolin Braunsberger-Reinhold, Marco Campomenosi, Ciarán Cuffe, Karima Delli, Anna Deparnay-Grunenberg, Ismail Ertug, Gheorghe Falcă, Carlo Fidanza, Søren Gade, Isabel García Muñoz, Jens Gieseke, Elsi Katainen, Kateřina Konečná, Bogusław Liberadzki, Peter Lundgren, Benoît Lutgen, Elżbieta Katarzyna Łukacijewska, Marian-Jean Marinescu, Tilly Metz, Cláudia Monteiro de Aguiar, Jan-Christoph Oetjen, Rovana Plumb, Dominique Riquet, Massimiliano Salini, Barbara Thaler, István Ujhelyi, Petar Vitanov, Roberts Zīle, Kosma Złotowski

Substitutes present for the final vote

Josianne Cutajar, Nicola Danti, Vlad Gheorghe, Roman Haider, Pär Holmgren, Guido Reil, Marianne Vind, Jörgen Warborn

Substitutes under Rule 209(7) present for the final vote

Susanna Ceccardi, Salvatore De Meo

FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION

Key to symbols: