Sittings · Document

OPINION (COM(2021)0206 – C90146/2021 – 2021/0106(COD)) 2022-06-14

On the proposal for a regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain Union legislative acts

Committee on Industry, Research and Energy · Rapporteur: Eva Maydell

PA_Legam

SHORT JUSTIFICATION

Introduction

The Rapporteur welcomes the Commission’s proposal on an Artificial Intelligence Act and especially the horizontal risk-based approach that it puts forward. This approach will allow for the development of AI systems in line with European values and for the fostering of social trust in these new technologies, so that the EU can fulfil the full economic and social benefits of AI.

The Rapporteur is of the opinion that through the AI Act, we need to create an environment with the right balance between freedom and supervision. The Rapporteur proposes that further provisions are made in order for companies, especially start-ups and SMEs, to remain competitive and creative in the face of new obligations required of them. The Rapporteur believes this will increase both the legitimacy and relevance of the AI Act. We need to provide companies with clearer guidelines, simpler tools and more efficient resources to cope with regulation. This would allow us to support AI innovation, development and market uptake.

Therefore, the Rapporteur’s draft pursues four main objectives in this direction:

1. Enhancing measures to support innovation, such as the ones foreseen for regulatory sandboxes, with a particular focus on start-ups and SMEs

2. Providing a concise and internationally recognised definition of Artificial Intelligence System and setting high but realistic standards for accuracy, robustness, cybersecurity and data

3. Encouraging the uptake of AI systems by industry by placing an emphasis on social trust and value chain responsibility

4. Future-proofing the Act through better linkages to the green transition and possible changes in the industry, technology and power of AI

This draft opinion focuses mainly on issues related to ITRE’s competences but also broader issues related to innovation, competitiveness, research, sustainability and future changes in industry.

Supporting innovation, focus on start-ups and SMEs, enhancing regulatory sandboxes

The Rapporteur welcomes the introduction of Article 55 on measures for small-scale providers, but believes SMEs and start-ups should be more involved throughout the AI Act in a holistic approach. More specifically, in the development of Codes of Conduct, standardisation, and representation in the European Artificial Intelligence Board. By far, one of the biggest focuses for the Rapporteur is the provision of opportunities to SMEs and start-ups to participate in the AI regulatory sandboxes. This is why the Rapporteur proposes to strengthen the existing provisions by giving the regulatory sandboxes a more European dimension, preserving the unity of the Single Market and calling for the development of an EU AI Regulatory Sandboxing Programme whose modalities are set out in a new Annex.

Clear definition and realistic standards

The Rapporteur calls for the use of an internationally recognised definition of Artificial Intelligence System, which would be in line with the EU’s broader goals of setting global standards, working closely with transatlantic partners and likeminded allies and providing legal certainty for businesses, citizens and civil society. The Rapporteur believes that high standards for accuracy, robustness and cybersecurity as well as data and data governance are key to developing safe AI systems that protect fundamental rights. The key here is to balance this aim with the practical and pragmatic approach needed for achieving it. The Rapporteur calls for a common European authority on benchmarking that brings together national metrology and benchmarking authorities to set a unified approach to measurement of accuracy, robustness, and other relevant criteria.

Encouraging uptake of AI systems, fostering social trust, value chain responsibility

To encourage uptake and deployment of AI systems, the Rapporteur believes we need to foster social trust of both businesses and citizens. The Rapporteur seeks to address the challenge of social trust by encouraging a collaborative relationship between developers and users of AI that is better aligned to their responsibilities along the value chain, strengthening the Codes of Conduct and enhancing the measures on regulatory sandboxes to enable compliance-by-design. This in turn creates a healthy and integrated ecosystem, which will help reduce legal uncertainty and implementation gaps, all of which in turn will increase social trust.

Future-proofing, sustainability and changes in the industry and power of AI

AI is a mature and ready-to-use technology that can be used to process the ever growing amount of data created along industrial processes. To facilitate investments to AI-based analysis and optimisation solutions, this regulation should provide a predictable environment for low-risk industrial solutions. Furthermore, this Regulation should take into account future changes in the industry and power of AI. This is why the Rapporteur proposes great involvement of the High Level Expert Group on AI with both the Commission and the European Artificial Intelligence Board as well as the monitoring of market trends and foresight by the European AI Board.

AMENDMENTS

The Committee on Industry, Research and Energy calls on the Committee on the Internal Market and Consumer Protection and the Committee on Civil Liberties, Justice and Home Affairs, as the committees responsible, to take into account the following amendments:

Amendment 1

Proposal for a regulation

Recital 3 a (new)

Text proposed by the Commission

Amendment

(3a) Furthermore, in order for the Member States to reach their climate targets and to meet the United Nation’s Sustainable Development Goals (SDGs), Union companies should be encouraged to utilise available technological advancements in realising this goal. AI is a well-developed and ready-to-use technology that can be used to process ever-growing amount of data created along industrial processes. To facilitate investments in AI-based analysis and optimisation solutions that can help to achieve the climate goals, this Regulation should provide a predictable and proportionate environment for low- risk industrial solutions. To ensure coherence, this requires that AI systems themselves need to be designed sustainably to reduce resource usage and energy consumption, thereby limiting the damage to the environment.

Amendment 2

Proposal for a regulation

Recital 3 b (new)

Text proposed by the Commission

Amendment

(3b) Furthermore, in order to foster the development of artificial intelligence in line with Union values, the Union needs to address the main gaps and barriers blocking the potential of the digital transformation including the shortage of digitally skilled workers, cybersecurity concerns, lack of investment and access to investment, and existing and potential gaps between large companies and SMEs. Special attention should be paid to ensuring that the benefits of artificial intelligence and innovation in new technologies are felt across all regions of the Union and that sufficient investment and resources are provided especially to those regions that may be lagging behind in some digital indicators.

Amendment 3

Proposal for a regulation

Recital 6

Text proposed by the Commission

Amendment

(6) The notion of AI system should be clearly defined to ensure legal certainty, while providing the flexibility to accommodate future technological developments. The definition should be based on the key functional characteristics of the software, in particular the ability, for a given set of human-defined objectives, to generate outputs such as content, predictions, recommendations, or decisions which influence the environment with which the system interacts, be it in a physical or digital dimension. AI systems can be designed to operate with varying levels of autonomy and be used on a stand-alone basis or as a component of a product, irrespective of whether the system is physically integrated into the product (embedded) or serve the functionality of the product without being integrated therein (non-embedded). The definition of AI system should be complemented by a list of specific techniques and approaches used for its development, which should be kept up-to–date in the light of market and technological developments through the adoption of delegated acts by the Commission to amend that list.

(6) The notion of AI system should be clearly defined to ensure legal certainty, while providing the flexibility to accommodate future technological developments. This definition should be in line with definitions that have been accepted internationally. The definition should be based on the key functional characteristics of the AI system, in particular the ability, for a given set of human-defined objectives, to make predictions, recommendations, or decisions influencing real or virtual environments. More specifically, the definition of AI system should take into account key features such as the ability to perceive real and/or virtual environments, to abstract such perceptions into models through analysis in an automated manner and to use model inference to formulate options for information or action. AI systems are designed to operate with varying levels of autonomy and be used on a stand-alone basis or as a component of a product, irrespective of whether the system is physically integrated into the product (embedded) or serve the functionality of the product without being integrated therein (non-embedded). The definition of AI system should be complemented by a list of specific techniques and approaches used for its development, which should be kept up-to–date in the light of market and technological developments through the adoption of delegated acts by the Commission to amend that list. While drafting these delegated acts, the Commission should insure the input of all relevant stakeholders including the technical experts and developers of AI systems. This consultation could take place through existing bodies such as the High Level Expert Group on AI or a newly established similar advisory body that is closely included in the work of the European Artificial Intelligence Board. Furthermore, the Commission should engage in dialogue with key international organisations such as the Organisation for Economic Cooperation and Development and other key organisations working on the definition of AI systems to ensure alignment between definitions of AI, while keeping the prerogative of the Union to set its own definition and standards through enacting legislation.

Amendment 4

Proposal for a regulation

Recital 12 a (new)

Text proposed by the Commission

Amendment

(12a) This Regulation should not undermine research and development activity and should respect freedom of science. It is therefore necessary to ensure that this Regulation does not otherwise affect scientific research and development activity on AI systems. As regards product oriented research activity by providers, this Regulation should apply insofar as such research leads to or entails placing an AI system on the market or putting it into service. Under all circumstances, any research and development activity should be carried out in accordance with recognised ethical standards for scientific research.

Amendment 5

Proposal for a regulation

Recital 29

Text proposed by the Commission

Amendment

(29) As regards high-risk AI systems that are safety components of products or systems, or which are themselves products or systems falling within the scope of Regulation (EC) No 300/2008 of the European Parliament and of the Council39 , Regulation (EU) No 167/2013 of the European Parliament and of the Council40 , Regulation (EU) No 168/2013 of the European Parliament and of the Council41 , Directive 2014/90/EU of the European Parliament and of the Council42 , Directive (EU) 2016/797 of the European Parliament and of the Council43 , Regulation (EU) 2018/858 of the European Parliament and of the Council44 , Regulation (EU) 2018/1139 of the European Parliament and of the Council45 , and Regulation (EU) 2019/2144 of the European Parliament and of the Council46 , it is appropriate to amend those acts to ensure that the Commission takes into account, on the basis of the technical and regulatory specificities of each sector, and without interfering with existing governance, conformity assessment and enforcement mechanisms and authorities established therein, the mandatory requirements for high-risk AI systems laid down in this Regulation when adopting any relevant future delegated or implementing acts on the basis of those acts.

__________________

39 Regulation (EC) No 300/2008 of the European Parliament and of the Council of 11 March 2008 on common rules in the field of civil aviation security and repealing Regulation (EC) No 2320/2002 (OJ L 97, 9.4.2008, p. 72).

40 Regulation (EU) No 167/2013 of the European Parliament and of the Council of 5 February 2013 on the approval and market surveillance of agricultural and forestry vehicles (OJ L 60, 2.3.2013, p. 1).

41 Regulation (EU) No 168/2013 of the European Parliament and of the Council of 15 January 2013 on the approval and market surveillance of two- or three-wheel vehicles and quadricycles (OJ L 60, 2.3.2013, p. 52).

42 Directive 2014/90/EU of the European Parliament and of the Council of 23 July 2014 on marine equipment and repealing Council Directive 96/98/EC (OJ L 257, 28.8.2014, p. 146).

43 Directive (EU) 2016/797 of the European Parliament and of the Council of 11 May 2016 on the interoperability of the rail system within the European Union (OJ L 138, 26.5.2016, p. 44).

44 Regulation (EU) 2018/858 of the European Parliament and of the Council of 30 May 2018 on the approval and market surveillance of motor vehicles and their trailers, and of systems, components and separate technical units intended for such vehicles, amending Regulations (EC) No 715/2007 and (EC) No 595/2009 and repealing Directive 2007/46/EC (OJ L 151, 14.6.2018, p. 1).

45 Regulation (EU) 2018/1139 of the European Parliament and of the Council of 4 July 2018 on common rules in the field of civil aviation and establishing a European Union Aviation Safety Agency, and amending Regulations (EC) No 2111/2005, (EC) No 1008/2008, (EU) No 996/2010, (EU) No 376/2014 and Directives 2014/30/EU and 2014/53/EU of the European Parliament and of the Council, and repealing Regulations (EC) No 552/2004 and (EC) No 216/2008 of the European Parliament and of the Council and Council Regulation (EEC) No 3922/91 (OJ L 212, 22.8.2018, p. 1).

46 Regulation (EU) 2019/2144 of the European Parliament and of the Council of 27 November 2019 on type-approval requirements for motor vehicles and their trailers, and systems, components and separate technical units intended for such vehicles, as regards their general safety and the protection of vehicle occupants and vulnerable road users, amending Regulation (EU) 2018/858 of the European Parliament and of the Council and repealing Regulations (EC) No 78/2009, (EC) No 79/2009 and (EC) No 661/2009 of the European Parliament and of the Council and Commission Regulations (EC) No 631/2009, (EU) No 406/2010, (EU) No 672/2010, (EU) No 1003/2010, (EU) No 1005/2010, (EU) No 1008/2010, (EU) No 1009/2010, (EU) No 19/2011, (EU) No 109/2011, (EU) No 458/2011, (EU) No 65/2012, (EU) No 130/2012, (EU) No 347/2012, (EU) No 351/2012, (EU) No 1230/2012 and (EU) 2015/166 (OJ L 325, 16.12.2019, p. 1).

Amendment 6

Proposal for a regulation

Recital 44

Text proposed by the Commission

Amendment

(44) High data quality is essential for the performance of many AI systems, especially when techniques involving the training of models are used, with a view to ensure that the high-risk AI system performs as intended and safely and it does not become the source of discrimination prohibited by Union law. High quality training, validation and testing data sets require the implementation of appropriate data governance and management practices. Training, validation and testing data sets should be sufficiently relevant, representative and free of errors and complete in view of the intended purpose of the system. They should also have the appropriate statistical properties, including as regards the persons or groups of persons on which the high-risk AI system is intended to be used. In particular, training, validation and testing data sets should take into account, to the extent required in the light of their intended purpose, the features, characteristics or elements that are particular to the specific geographical, behavioural or functional setting or context within which the AI system is intended to be used. In order to protect the right of others from the discrimination that might result from the bias in AI systems, the providers shouldbe able to process also special categories of personal data, as a matter of substantial public interest, in order to ensure the bias monitoring, detection and correction in relation to high-risk AI systems.

(44) High data quality is essential for the performance of many AI systems, especially when techniques involving the training of models are used, with a view to ensure that the high-risk AI system performs as intended and safely and it does not become the source of discrimination prohibited by Union law. High quality training, validation and testing data sets require the implementation of appropriate data governance and management practices. Training, validation and testing data sets are designed with the best possible efforts to ensure that they are relevant, representative, free of errors and appropriately vetted for errors in view of the intended purpose of the system. They should also have the appropriate statistical properties, including as regards the persons or groups of persons on which the high-risk AI system is intended to be used, with specific attention to the mitigation of possible biases in the datasets, that might lead to risks to fundamental rights or discriminatory outcomes for the persons affected by the high-risk AI system. In particular, training, validation and testing data sets should take into account, to the extent required in the light of their intended purpose, the features, characteristics or elements that are particular to the specific geographical, behavioural, contextual or functional setting or context within which the AI system is intended to be used, with specific attention to women, vulnerable groups and children. In order to protect the right of others from the discrimination that might result from the bias in AI systems, the providers should be able to process also special categories of personal data, as a matter of substantial public interest, in order to ensure the bias monitoring, detection and correction in relation to high-risk AI systems.

Amendment 7

Proposal for a regulation

Recital 46

Text proposed by the Commission

Amendment

(46) Having information on how high-risk AI systems have been developed and how they perform throughout their lifecycle is essential to verify compliance with the requirements under this Regulation. This requires keeping records and the availability of a technical documentation, containing information which is necessary to assess the compliance of the AI system with the relevant requirements. Such information should include the general characteristics, capabilities and limitations of the system, algorithms, data, training, testing and validation processes used as well as documentation on the relevant risk management system. The technical documentation should be kept up to date.

(46) Having comprehensible information on how high-risk AI systems have been developed and how they perform throughout their lifecycle is essential to verify compliance with the requirements under this Regulation and to allow users to make informed and autonomous decisions about their use. This requires keeping records and the availability of a technical documentation, containing information which is necessary to assess the compliance of the AI system with the relevant requirements. Such information should include the general characteristics, capabilities and limitations of the system, algorithms, data, training, testing and validation processes used as well as documentation on the relevant risk management system. The technical documentation should be kept up to date.

Amendment 8

Proposal for a regulation

Recital 49

Text proposed by the Commission

Amendment

(49) High-risk AI systems should perform consistently throughout their lifecycle and meet an appropriate level of accuracy, robustness and cybersecurity in accordance with the generally acknowledged state of the art. Accuracy metrics and their expected level should be defined with the primary objective to mitigate risks and negative impact of the AI system to individuals and the society as a whole. The expected level of accuracy and accuracy metrics should be communicated in a clear, transparent, easily understandable and intelligible way to the users. The declaration of accuracy metrics cannot however be considered proof of future levels but relevant methods need to be applied to ensure sustainable levels during use. While standardisation organisations exist to establish standards, coordination on benchmarking is needed to establish how these standards should be met and measured. The European Artificial Intelligence Board should bring together national metrology and benchmarking authorities and provide non-binding guidance to address the technical aspects of to how to measure the appropriate levels of accuracy and robustness.

Amendment 9

Proposal for a regulation

Recital 50

Text proposed by the Commission

Amendment

(50) The technical robustness is a key requirement for high-risk AI systems. They should be resilient against risks connected to the limitations of the system (e.g. errors, faults, inconsistencies, unexpected situations) as well as against malicious actions that may compromise the security of the AI system and result in harmful or otherwise undesirable behaviour. Failure to protect against these risks could lead to safety impacts or negatively affect the fundamental rights, for example due to erroneous decisions or wrong or biased outputs generated by the AI system.

Amendment 10

Proposal for a regulation

Recital 51

Text proposed by the Commission

Amendment

(51) Cybersecurity plays a crucial role in ensuring that AI systems are resilient against attempts to alter their use, behaviour, performance or compromise their security properties by malicious third parties exploiting the system’s vulnerabilities. Cyberattacks against AI systems can leverage AI specific assets, such as training data sets (e.g. data poisoning) or trained models (e.g. adversarial attacks or confidentiality attacks), or exploit vulnerabilities in the AI system’s digital assets or the underlying ICT infrastructure. To ensure a level of cybersecurity appropriate to the risks, suitable measures should therefore be taken by the providers of high-risk AI systems, as well as the notified bodies, competent national authorities and market surveillance authorities, also taking into account as appropriate the underlying ICT infrastructure. High-risk AI should be accompanied by security solutions and patches for the lifetime of the product, or in case of the absence of dependence on a specific product, for a time that needs to be stated by the manufacturer.

Amendment 11

Proposal for a regulation

Recital 61

Text proposed by the Commission

Amendment

(61) Standardisation should play a key role to provide technical solutions to providers to ensure compliance with this Regulation. Compliance with harmonised standards as defined in Regulation (EU) No 1025/2012 of the European Parliament and of the Council54 should be a means for providers to demonstrate conformity with the requirements of this Regulation. In addition to technical details, the standardisation process should also take into account risks to fundamental rights, the environment, and society as a whole and other democratic and sociotechnical aspects of the AI system, and should ensure that the relevant subject-matter experts are included and consulted in the standardisation process. The standardisation process should be transparent in terms of legal and natural persons participating in the standardisation activities. However, the Commission could adopt common technical specifications in areas where no harmonised standards exist or where they are insufficient. In developing these common specifications Commission should involve views of relevant stakeholders, in particular when the common specifications address specific fundamental rights concerns. In particular, the Commission should adopt common specifications setting out how risk management systems give specific consideration to impact on children.

__________________

54 Regulation (EU) No 1025/2012 of the European Parliament and of the Council of 25 October 2012 on European standardisation, amending Council Directives 89/686/EEC and 93/15/EEC and Directives 94/9/EC, 94/25/EC, 95/16/EC, 97/23/EC, 98/34/EC, 2004/22/EC, 2007/23/EC, 2009/23/EC and 2009/105/EC of the European Parliament and of the Council and repealing Council Decision 87/95/EEC and Decision No 1673/2006/EC of the European Parliament and of the Council (OJ L 316, 14.11.2012, p. 12).

Amendment 12

Proposal for a regulation

Recital 71

Text proposed by the Commission

Amendment

(71) Artificial intelligence is a rapidly developing family of technologies that requires novel forms of regulatory oversight and a safe space for experimentation, while ensuring responsible innovation and integration of appropriate and ethically justified safeguards and risk mitigation measures. To ensure a legal framework that is innovation-friendly, future-proof and resilient to disruption, national competent authorities from one or more Member States should be encouraged to establish artificial intelligence regulatory sandboxes and make such regulatory sandboxes widely available throughout the Union, in order to facilitate the development and testing of innovative AI systems under strict regulatory oversight before these systems are placed on the market or otherwise put into service. Any significant risks identified during the development and testing of AI systems shall result in immediate mitigation and, failing that, in the suspension of the development and testing process until such mitigation takes place.

Amendment 13

Proposal for a regulation

Recital 72

Text proposed by the Commission

Amendment

(72) The objectives of the regulatory sandboxes should be to foster AI innovation by establishing a controlled experimentation and testing environment in the development and pre-marketing phase with a view to ensuring compliance of the innovative AI systems with this Regulation and other relevant Union and Member States legislation; to enhance legal certainty for innovators and the competent authorities’ oversight and understanding of the opportunities, emerging risks and the impacts of AI use, and to accelerate access to markets, including by removing barriers for small and medium enterprises (SMEs) and start-ups. To ensure uniform implementation across the Union and economies of scale, it is appropriate to establish common rules for the regulatory sandboxes’ implementation and a framework for cooperation between the relevant authorities involved in the supervision of the sandboxes. This Regulation should provide the legal basis for the use of personal data collected for other purposes for developing certain AI systems in the public interest within the AI regulatory sandbox, in line with Article 6(4) of Regulation (EU) 2016/679, and Article 6 of Regulation (EU) 2018/1725, and without prejudice to Article 4(2) of Directive (EU) 2016/680. Participants in the sandbox should ensure appropriate safeguards and cooperate with the competent authorities, including by following their guidance and acting expeditiously and in good faith to mitigate any high-risks to safety and fundamental rights that may arise during the development and experimentation in the sandbox. The conduct of the participants in the sandbox should be taken into account when competent authorities decide whether to impose an administrative fine under Article 83(2) of Regulation 2016/679 and Article 57 of Directive 2016/680.

(72) The objectives of the regulatory sandboxes should be to foster AI innovation by establishing a controlled experimentation and testing environment in the development and pre-marketing phase with a view to ensuring compliance of the innovative AI systems with this Regulation and other relevant Union and Member States legislation, as well as with the Charter of fundamental rights of the European Union and the General Data Protection Regulation; to enhance legal certainty for innovators and the competent authorities’ oversight and understanding of the opportunities, emerging risks and the impacts of AI use, to provide safeguards needed to build trust and reliance on AI systems and to accelerate access to markets, including by removing barriers for small and medium enterprises (SMEs) and start-ups; to contribute to achieving the targets on AI as set in the Policy Programme “Path to the Digital Decade"; to contribute to the development of ethical, socially responsible and environmentally sustainable AI systems; to permit effective participation of SMEs and start-ups in regulatory sandboxes, compliance costs should be kept to a reasonable level to ensure the development of trustworthy European artificial intelligence solutions; it is appropriate to establish common rules for the regulatory sandboxes’ implementation and a framework for cooperation between the relevant authorities involved in the supervision of the sandboxes, while encouraging innovation. This Regulation should provide the legal basis for the use of personal data collected for other purposes for developing certain AI systems in the public interest within the AI regulatory sandbox, in line with Article 6(4) of Regulation (EU) 2016/679, and Article 6 of Regulation (EU) 2018/1725, and without prejudice to Article 4(2) of Directive (EU) 2016/680. Participants in the sandbox should ensure appropriate safeguards and cooperate with the competent authorities, including by following their guidance and acting expeditiously and in good faith to mitigate any high-risks to safety and fundamental rights that may arise during the development and experimentation in the sandbox. The conduct of the participants in the sandbox should be taken into account when competent authorities decide whether to impose an administrative fine under Article 83(2) of Regulation 2016/679 and Article 57 of Directive 2016/680.

Amendment 14

Proposal for a regulation

Recital 72 a (new)

Text proposed by the Commission

Amendment

(72a) It is desirable for the establishment of regulatory sandboxes, which is at present left to the discretion of Member States, as a next step to be made obligatory, with properly established criteria, to ensure both the effectiveness of the AI system and easier access for enterprises, in particular SMEs. Research enterprises and institutions should be involved in developing the conditions for the creation of regulatory sandboxes.

Amendment 15

Proposal for a regulation

Recital 73

Text proposed by the Commission

Amendment

(73) In order to promote and protect innovation, it is important that the interests of small-scale providers and users of AI systems are taken into particular account. To this objective, Member States should develop initiatives, which are targeted at those operators, including on AI literacy, awareness raising and information communication. Member States should utilise existing channels and where appropriate, establish new dedicated channels for communication with SMEs, start-ups, users and other innovators to provide guidance and respond to queries about the implementation of this Regulation. Such existing channels could include, inter alia, ENISA’s Computer Security Incident Response Teams, National Data Protection Agencies, the AI-on demand platform, the European Digital Innovation Hubs and other relevant instruments funded by EU programmes as well as the Testing and Experimentation Facilities established by the Commission and the Member States at national or Union level. Where appropriate, these channels should work together to create synergies and ensure homogeneity in their guidance to start-ups, SMEs and users. Moreover, the specific interests and needs of small-scale providers shall be taken into account when Notified Bodies set conformity assessment fees. The Commission should regularly assess the certification and compliance costs for SMEs and start-ups, including through transparent consultations with SMEs, start-ups and users and work with Member States to lower such costs. For example, translation costs related to mandatory documentation and communication with authorities may constitute a significant cost for providers and other operators, notably those of a smaller scale. Member States should possibly ensure that one of the languages determined and accepted by them for relevant providers’ documentation and for communication with operators is one which is broadly understood by the largest possible number of cross-border users. Medium-sized enterprises which recently changed from the small to medium-size category within the meaning of Recommendation 2003/361/EC should have access to these initiatives and guidance for a period of time deemed appropriate by the Member States, as these new medium-sized enterprises may sometimes lack the legal resources and training necessary to ensure proper understanding and compliance with provisions.

Amendment 16

Proposal for a regulation

Recital 76 a (new)

Text proposed by the Commission

Amendment

(76a) An AI advisory council (‘the Advisory Council’) should be established as a sub-group of the Board consisting of relevant representatives from industry, research, academia, civil society, standardisation organisations, social partners, SMEs, fundamental rights experts and other relevant stakeholders representing all Member States to maintain geographical balance. The Advisory Council should support the work of the Board by providing advice relating to the tasks of the Board. The Advisory Council should nominate a representative to attend meetings of the Board and to participate in its work.

Amendment 17

Proposal for a regulation

Recital 81

Text proposed by the Commission

Amendment

(81) The development of AI systems other than high-risk AI systems in accordance with the requirements of this Regulation may lead to a larger uptake of trustworthy artificial intelligence in the Union. Providers of non-high-risk AI systems should be encouraged to create codes of conduct intended to foster the voluntary application of the mandatory requirements applicable to high-risk AI systems. Providers should also be encouraged to apply on a voluntary basis additional requirements related, for example, to environmental sustainability, accessibility to persons with disability, stakeholders’ participation in the design and development of AI systems, and diversity of the development teams. The Commission may develop initiatives, including of a sectorial nature, to facilitate the lowering of technical barriers hindering cross-border exchange of data for AI development, including on data access infrastructure, semantic and technical interoperability of different types of data.

(81) The development of AI systems other than high-risk AI systems in accordance with the requirements of this Regulation may lead to a larger uptake of trustworthy, socially responsible and environmentally sustainable artificial intelligence in the Union. Providers of non-high-risk AI systems should be encouraged to create codes of conduct intended to foster the voluntary application of the mandatory requirements applicable to high-risk AI systems. Providers should also be encouraged to apply on a voluntary basis additional requirements related, for example, to environmental sustainability, accessibility to persons with disability, stakeholders’ participation in the design and development of AI systems, and diversity of the development teams. The Commission may develop initiatives, including of a sectorial nature, to facilitate the lowering of technical barriers hindering cross-border exchange of data for AI development, including on data access infrastructure, semantic and technical interoperability of different types of data.

Amendment 18

Proposal for a regulation

Article 2 – paragraph 5 a (new)

Text proposed by the Commission

Amendment

5a. This Regulation shall not affect research activities regarding AI systems insofar as such activities do not lead to or entail placing an AI system on the market or putting it into service. These research activities shall not violate the fundamental rights of the affected persons.

Amendment 19

Proposal for a regulation

Article 2 – paragraph 5 b (new)

Text proposed by the Commission

Amendment

5b. This Regulation shall not apply to AI systems, including their output, specifically developed and put into service for the sole purpose of scientific research in the general interest of the Union.

Amendment 20

Proposal for a regulation

Article 3 – paragraph 1 – point 1

Text proposed by the Commission

Amendment

(1) ‘artificial intelligence system’ (AI system) means software that is developed with one or more of the techniques and approaches listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with;

(1) ‘artificial intelligence system’ (AI system) means a machine-based system that can, with varying levels of autonomy, for a given set of human-defined objectives, make predictions, content, recommendations, or decisions influencing real or virtual environments they interact with;

Amendment 21

Proposal for a regulation

Article 3 – paragraph 1 – point 1 a (new)

Text proposed by the Commission

Amendment

(1a) ‘autonomy’ means that an AI system operates by interpreting certain input and by using a set of pre-determined objectives, without being limited to such instructions, despite the system’s behaviour being constrained by, and targeted at, fulfilling the goal it was given and other relevant design choices made by its developer;

Amendment 22

Proposal for a regulation

Article 3 – paragraph 1 – point 2

Text proposed by the Commission

Amendment

(1) ‘provider’ means a natural or legal person, public authority, agency or other body that develops an AI system or that has an AI system developed with a view to placing it on the market or putting it into service under its own name or trademark, whether for payment or free of charge;

(2) ‘provider’ means a natural or legal person, public authority, agency or other body that develops an AI system or that has an AI system developed and places that system on the market or puts it into service under its own name or trademark, whether for payment or free of charge;

Amendment 23

Proposal for a regulation

Article 3 – paragraph 1 – point 14

Text proposed by the Commission

Amendment

(14) ‘safety component of a product or system’ means a component of a product or of a system which fulfils a safety function for that product or system or the failure or malfunctioning of which endangers the health and safety of persons or property;

(14) ‘safety component of a product or system’ means a component of a product or of a system which fulfils a safety function for that product or system and the failure or malfunctioning of which endangers the health and safety of persons or property;

Amendment 24

Proposal for a regulation

Article 3 – paragraph 1 – point 44 – introductory part

Text proposed by the Commission

Amendment

(44) ‘serious incident’ means any incident that directly or indirectly leads, might have led or might lead to any of the following:

(44) ‘serious incident’ means any incident or malfunctioning of an AI system that directly or indirectly leads, might have led or might lead to any of the following:

Amendment 25

Proposal for a regulation

Article 3 – paragraph 1 – point 44 – point a

Text proposed by the Commission

Amendment

(a) the death of a person or serious damage to a person’s health, to property or the environment,

(a) the death of a person or serious damage to a person’s fundamental rights, health, safety, property or the environment,

Amendment 26

Proposal for a regulation

Article 3 – paragraph 1 – point 44 – point b a (new)

Text proposed by the Commission

Amendment

(ba) breach of obligations under Union law intended to protect fundamental rights.

Amendment 27

Proposal for a regulation

Article 3 – paragraph 1 – point 44 a (new)

Text proposed by the Commission

Amendment

(44a) ‘regulatory sandbox’ means a facility established by one or more Member States’ competent authorities in collaboration with the Commission or by the European Data Protection Supervisor, that provides an appropriate controlled and flexible environment to facilitate the safe development, testing and validation of innovative AI systems for a limited time before their placement on the market or putting into service pursuant to a specific plan;

Amendment 28

Proposal for a regulation

Article 3 – paragraph 1 – point 44 b (new)

Text proposed by the Commission

Amendment

(44b) ‘AI literacy’ means the skills, knowledge and understanding regarding AI systems that are necessary for compliance with and enforcement of this Regulation;

Amendment 29

Proposal for a regulation

Article 3 – paragraph 1 – point 44 c (new)

Text proposed by the Commission

Amendment

(44c) ‘deep fake’ means manipulated or synthetic audio and/or visual material that gives an authentic impression, in which events appear to be taking place, which never happened, and which has been produced using techniques in the field of artificial intelligence, including machine learning and deep learning, without the user, or end-user being aware that the audio and/or visual material has been produced using artificial intelligence;

Amendment 30

Proposal for a regulation

Article 3 – paragraph 1 – point 44 d (new)

Text proposed by the Commission

Amendment

(44d) ‘critical infrastructure’ means an asset, system or part thereof which is necessary for the delivery of a service that is essential for the maintenance of vital societal functions or economic activities within the meaning of Article 2(4) and (5) of Directive ____ on the resilience of critical entities (2020/0365(COD));

Amendment 31

Proposal for a regulation

Article 3 – paragraph 1 – point 44 e (new)

Text proposed by the Commission

Amendment

(44e) ‘personal data’ means personal data as defined in Article 4, point (1), of Regulation (EU) 2016/679;

Amendment 32

Proposal for a regulation

Article 3 – paragraph 1 – point 44 f (new)

Text proposed by the Commission

Amendment

(44f) ‘non personal data’ means data other than personal data as defined in point (1) of Article 4 of Regulation (EU) 2016/679.

Amendment 33

Proposal for a regulation

Article 4 – paragraph 1

Text proposed by the Commission

Amendment

The Commission is empowered to adopt delegated acts in accordance with Article 73 to amend the list of techniques and approaches listed in Annex I, in order to update that list to market and technological developments on the basis of characteristics that are similar to the techniques and approaches listed therein.

The Commission is empowered to adopt delegated acts in accordance with Article 73 to amend the list of techniques and approaches listed in Annex I within the scope of the AI system as defined in Article 3, point (1), in order to update that list to market and technological developments on the basis of characteristics that are similar to the techniques and approaches listed therein.

Amendment 34

Proposal for a regulation

Article 4 – paragraph 1 a (new)

Text proposed by the Commission

Amendment

When drafting these delegated acts, the Commission shall ensure the input of all relevant stakeholders such as technical experts and developers of AI systems.

Amendment 35

Proposal for a regulation

Article 10 – paragraph 1

Text proposed by the Commission

Amendment

1. High-risk AI systems which make use of techniques involving the training of models with data shall be developed on the basis of training, validation and testing data sets that meet the quality criteria referred to in paragraphs 2 to 5.

1. High-risk AI systems which make use of techniques involving the training of models with data shall be developed on the basis of training, assessment, validation and testing data sets that meet the quality criteria referred to in paragraphs 2 to 5 as far as this is feasible from a technical point of view while taking into account the latest state-of-the-art measures, according to the specific market segment or scope of application.

Amendment 36

Proposal for a regulation

Article 10 – paragraph 1 a (new)

Text proposed by the Commission

Amendment

1a. Techniques such as unsupervised learning and reinforcement learning, that do not use validation and testing data sets, shall be developed on the basis of training data sets that meet the quality criteria referred to in paragraphs 2 to 5.

Amendment 37

Proposal for a regulation

Article 10 – paragraph 1 b (new)

Text proposed by the Commission

Amendment

1b. Providers of high-risk AI systems that utilise data collected and/or managed by third parties may rely on representations from those third parties with regard to quality criteria referred to in paragraph 2, points (a), (b) and (c)

Amendment 38

Proposal for a regulation

Article 10 – paragraph 2 – introductory part

Text proposed by the Commission

Amendment

2. Training, validation and testing data sets shall be subject to appropriate data governance and management practices. Those practices shall concern in particular,

2. Training, assessment, validation and testing data sets shall be subject to appropriate data governance and management practices for the entire lifecycle of data processing. Those practices shall concern in particular, the following elements:

Amendment 39

Proposal for a regulation

Article 10 – paragraph 2 – point a a (new)

Text proposed by the Commission

Amendment

(aa) transparency as regards the original purpose of data collection;

Amendment 40

Proposal for a regulation

Article 10 – paragraph 2 – point b

Text proposed by the Commission

Amendment

(b) data collection;

(b) data collection processes;

Amendment 41

Proposal for a regulation

Article 10 – paragraph 2 – point f

Text proposed by the Commission

Amendment

(f) examination in view of possible biases;

(f) examination in view of possible biases that are likely to affect health and safety of persons, negatively impact fundamental rights or lead to discrimination prohibited by Union law; including the cases where data outputs are used as an input for future operations (‘feedback loops’);

Amendment 42

Proposal for a regulation

Article 10 – paragraph 2 – point g

Text proposed by the Commission

Amendment

(g) the identification of any possible data gaps or shortcomings, and how those gaps and shortcomings can be addressed.

(g) the identification of possible data gaps or shortcomings, and how those gaps and shortcomings can be addressed;

Amendment 43

Proposal for a regulation

Article 10 – paragraph 2 – point g a (new)

Text proposed by the Commission

Amendment

(ga) the purpose and the environment in which the system is to be used.

Amendment 44

Proposal for a regulation

Article 10 – paragraph 3

Text proposed by the Commission

Amendment

3. Training, validation and testing data sets shall be relevant, representative, free of errors and complete. They shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons on which the high-risk AI system is intended to be used. These characteristics of the data sets may be met at the level of individual data sets or a combination thereof.

3. Training, validation and testing datasets are designed with the best possible efforts to ensure that they are relevant, representative and appropriately vetted for errors in view of the intended purpose of the AI system. In particular, they shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons on which the high-risk AI system is intended to be used. These characteristics of the data sets may be met at the level of individual data sets or a combination thereof.

Amendment 45

Proposal for a regulation

Article 10 – paragraph 4

Text proposed by the Commission

Amendment

4. Training, validation and testing data sets shall take into account, to the extent required by the intended purpose, the characteristics or elements that are particular to the specific geographical, behavioural or functional setting within which the high-risk AI system is intended to be used.

4. Training, validation and testing data sets shall take into account, to the extent required by the intended purpose, the characteristics or elements that are particular to the specific geographical, behavioural, contextual or functional setting within which the high-risk AI system is intended to be used.

Amendment 46

Proposal for a regulation

Article 15 – paragraph 1

Text proposed by the Commission

Amendment

1. High-risk AI systems shall be designed and developed in such a way that they achieve, in the light of their intended purpose, an appropriate level of accuracy, robustness and cybersecurity, and perform consistently in those respects throughout their lifecycle.

1. High-risk AI systems shall be designed and developed following the principle of security by design and by default. In the light of their intended purpose, they should achieve an appropriate level of accuracy, robustness, safety, and cybersecurity, and perform consistently in those respects throughout their lifecycle. Compliance with these requirements shall include implementation of state-of-the-art measures, according to the specific market segment or scope of application.

Amendment 47

Proposal for a regulation

Article 15 – paragraph 1 a (new)

Text proposed by the Commission

Amendment

1a. To address the technical aspects of to how to measure the appropriate levels of accuracy and robustness set out in paragraph 1 of this Article, the European Artificial Intelligence Board shall bring together national metrology and benchmarking authorities and provide non-binding guidance on the matter as set out in Article 56, paragraph 2, point (a).

Amendment 48

Proposal for a regulation

Article 15 – paragraph 1 b (new)

Text proposed by the Commission

Amendment

1b. To address any emerging issues across the internal market with regard to cybersecurity, the European Union Agency for Cybersecurity (ENISA) shall be involved alongside the European Artificial Intelligence Board as set out Article 56, paragraph 2, point (b).

Amendment 49

Proposal for a regulation

Article 15 – paragraph 2

Text proposed by the Commission

Amendment

2. The levels of accuracy and the relevant accuracy metrics of high-risk AI systems shall be declared in the accompanying instructions of use.

2. The levels of accuracy and the relevant accuracy metrics of high-risk AI systems shall be declared in the accompanying instructions of use. The language used shall be clear, free of misunderstandings or misleading statements.

Amendment 50

Proposal for a regulation

Article 15 – paragraph 3 – subparagraph 1

Text proposed by the Commission

Amendment

High-risk AI systems shall be resilient as regards errors, faults or inconsistencies that may occur within the system or the environment in which the system operates, in particular due to their interaction with natural persons or other systems.

Technical and organisational measures shall be taken to ensure that high-risk AI systems shall be as resilient as possible regarding errors, faults or inconsistencies that may occur within the system or the environment in which the system operates, in particular due to their interaction with natural persons or other systems.

Amendment 51

Proposal for a regulation

Article 15 – paragraph 3 – subparagraph 2

Text proposed by the Commission

Amendment

The robustness of high-risk AI systems may be achieved through technical redundancy solutions, which may include backup or fail-safe plans.

The robustness of high-risk AI systems may be achieved by the appropriate provider with input from the user, where necessary, through technical redundancy solutions, which may include backup or fail-safe plans.

Amendment 52

Proposal for a regulation

Article 15 – paragraph 3 – subparagraph 3

Text proposed by the Commission

Amendment

High-risk AI systems that continue to learn after being placed on the market or put into service shall be developed in such a way to ensure that possibly biased outputs due to outputs used as an input for future operations (‘feedback loops’) are duly addressed with appropriate mitigation measures.

High-risk AI systems that continue to learn after being placed on the market or put into service shall be developed in such a way to ensure that possibly biased outputs influencing input for future operations (‘feedback loops’) and malicious manipulation of inputs used in learning during operation are duly addressed with appropriate mitigation measures.

Amendment 53

Proposal for a regulation

Article 15 – paragraph 4 – subparagraph 1

Text proposed by the Commission

Amendment

High-risk AI systems shall be resilient as regards attempts by unauthorised third parties to alter their use or performance by exploiting the system vulnerabilities.

High-risk AI systems shall be resilient as regards to attempts by unauthorised third parties to alter their use, behaviour, outputs or performance by exploiting the system vulnerabilities.

Amendment 54

Proposal for a regulation

Article 15 – paragraph 4 – subparagraph 3

Text proposed by the Commission

Amendment

The technical solutions to address AI specific vulnerabilities shall include, where appropriate, measures to prevent and control for attacks trying to manipulate the training dataset (‘data poisoning’), inputs designed to cause the model to make a mistake (‘adversarial examples’), or model flaws.

The technical solutions to address AI specific vulnerabilities shall include, where appropriate, measures to prevent, detect, respond to, resolve and control for attacks trying to manipulate the training dataset (‘data poisoning’), or pre-trained components used in training (‘model poisoning’) , inputs designed to cause the model to make a mistake (‘adversarial examples’ or ‘model evasion’), confidentiality attacks or model flaws, which could lead to harmful decision-making.

Amendment 55

Proposal for a regulation

Article 40 – paragraph 1 a (new)

Text proposed by the Commission

Amendment

The Commission shall ensure that the process of developing harmonised standards takes into account risks to fundamental rights, environment and society as a whole.

Amendment 56

Proposal for a regulation

Article 40 – paragraph 1 b (new)

Text proposed by the Commission

Amendment

The Commission shall ensure that the process of developing harmonised standards on AI systems is open to stakeholders, including SMEs in accordance with Articles 5 and 6 of Regulation (EU) No 1025/2012.

Amendment 57

Proposal for a regulation

Article 40 – paragraph 1 c (new)

Text proposed by the Commission

Amendment

To this end the Commission shall direct funds in accordance with Article 17 of Regulation (EU) No 1025/2012 to facilitate their effective participation.

Amendment 58

Proposal for a regulation

Article 40 – paragraph 1 d (new)

Text proposed by the Commission

Amendment

The Commission shall review the harmonised standards before their publication in the Official Journal and prepare a report outlining their adequacy with paragraphs 1a and 1b of this Article.

Amendment 59

Proposal for a regulation

Article 41 – paragraph 1

Text proposed by the Commission

Amendment

1. Where harmonised standards referred to in Article 40 do not exist or where the Commission considers that the relevant harmonised standards are insufficient or that there is a need to address specific safety or fundamental right concerns, the Commission may, by means of implementing acts, adopt common specifications in respect of the requirements set out in Chapter 2 of this Title. Those implementing acts shall be adopted in accordance with the examination procedure referred to in Article 74(2).

1. Where harmonised standards referred to in Article 40 do not exist or where the Commission considers that the relevant harmonised standards are insufficient or that there is a need to address specific and pressing safety or fundamental right concern that cannot be sufficiently settled by development of harmonised standards, the Commission may, by means of implementing acts, adopt common specifications in respect of the requirements set out in Chapter 2 of this Title. Those implementing acts shall be adopted in accordance with the examination procedure referred to in Article 74(2).

Amendment 60

Proposal for a regulation

Article 41 – paragraph 2

Text proposed by the Commission

Amendment

2. The Commission, when preparing the common specifications referred to in paragraph 1, shall gather the views of relevant bodies or expert groups established under relevant sectorial Union law.

2. The Commission, when preparing the common specifications referred to in paragraph 1, shall gather the views of developers and providers of High-risk AI systems as well as relevant stakeholders, such as SME's and start-ups, civil society and social partners or expert groups established under relevant sectorial Union law.

Amendment 61

Proposal for a regulation

Article 42 – paragraph 1

Text proposed by the Commission

Amendment

1. Taking into account their intended purpose, high-risk AI systems that have been trained and tested on data concerning the specific geographical, behavioural and functional setting within which they are intended to be used shall be presumed to be in compliance with the requirement set out in Article 10(4).

1. Taking into account their intended purpose, high-risk AI systems that have been trained and tested on data concerning the specific geographical, behavioural, contextual and functional setting within which they are intended to be used shall be presumed to be in compliance with the requirement set out in Article 10(4).

Amendment 62

Proposal for a regulation

Article 42 – paragraph 2

Text proposed by the Commission

Amendment

2. High-risk AI systems that have been certified or for which a statement of conformity has been issued under a cybersecurity scheme pursuant to Regulation (EU) 2019/881 of the European Parliament and of the Council63 and the references of which have been published in the Official Journal of the European Union shall be presumed to be in compliance with the cybersecurity requirements set out in Article 15 of this Regulation in so far as the cybersecurity certificate or statement of conformity or parts thereof cover those requirements.

2. High-risk AI systems that have been certified or for which a statement of conformity has been issued under a cybersecurity scheme pursuant to Regulation (EU) 2019/881 of the European Parliament and of the Council63 and the references of which have been published in the Official Journal of the European Union shall be presumed to be in compliance with the cybersecurity requirements set out in Article 15 of this Regulation, where applicable, in so far as the cybersecurity certificate or statement of conformity or parts thereof cover those requirements.

__________________

63 Regulation (EU) 2019/881 of the European Parliament and of the Council of 17 April 2019 on ENISA (the European Union Agency for Cybersecurity) and on information and communications technology cybersecurity certification and repealing Regulation (EU) No 526/2013 (Cybersecurity Act) (OJ L 151, 7.6.2019, p. 1).

Amendment 63

Proposal for a regulation

Article 43 – paragraph 1 – subparagraph 1 a (new)

Text proposed by the Commission

Amendment

Should the provider already have established internal organisation and structures for existing conformity assessments or requirements under other existing rules, the provider may utilise those, or parts of those, existing compliance structures, so long as they also have the capacity and competence needed to fulfil the requirements for the product set out in this Regulation.

Amendment 64

Proposal for a regulation

Article 43 – paragraph 5

Text proposed by the Commission

Amendment

5. The Commission is empowered to adopt delegated acts in accordance with Article 73 for the purpose of updating Annexes VI and Annex VII in order to introduce elements of the conformity assessment procedures that become necessary in light of technical progress. The Commission shall consult the European Artificial Intelligence Board established in Article 56 as well as all relevant stakeholders.

Amendment 65

Proposal for a regulation

Article 43 – paragraph 6

Text proposed by the Commission

Amendment

6. The Commission is empowered to adopt delegated acts to amend paragraphs 1 and 2 in order to subject high-risk AI systems referred to in points 2 to 8 of Annex III to the conformity assessment procedure referred to in Annex VII or parts thereof. The Commission shall adopt such delegated acts taking into account the effectiveness of the conformity assessment procedure based on internal control referred to in Annex VI in preventing or minimizing the risks to health and safety and protection of fundamental rights posed by such systems as well as the availability of adequate capacities and resources among notified bodies.

Amendment 66

Proposal for a regulation

Article 44 – paragraph 1

Text proposed by the Commission

Amendment

1. Certificates issued by notified bodies in accordance with Annex VII shall be drawn-up in an official Union language determined by the Member State in which the notified body is established or in an official Union language otherwise acceptable to the notified body.

1. Certificates issued by notified bodies in accordance with Annex VII shall be drawn-up in one or several official languages determined by the Member State in which the notified body is established or in one or several official languages otherwise acceptable to the notified body.

Amendment 67

Proposal for a regulation

Article 48 – paragraph 1

Text proposed by the Commission

Amendment

1. The provider shall draw up a written EU declaration of conformity for each AI system and keep it at the disposal of the national competent authorities for 10 years after the AI system has been placed on the market or put into service. The EU declaration of conformity shall identify the AI system for which it has been drawn up. A copy of the EU declaration of conformity shall be given to the relevant national competent authorities upon request.

Amendment 68

Proposal for a regulation

Article 51 – paragraph 1 a (new)

Text proposed by the Commission

Amendment

A high-risk AI system designed, developed, trained, validate, tested or approved to be placed on the market or put into service, outside the Union, can be registered in the EU database referred to in Article 60 and placed on the market or put into service in the Union only if it is proven that at all the stages of its design, development, training, validation, testing or approval, all the obligations required from such AI systems in the Union have been met.

Amendment 69

Proposal for a regulation

Article 51 – paragraph 1 b (new)

Text proposed by the Commission

Amendment

Before using a high-risk AI system referred to in Article 6(2) the user or where applicable the authorised representative shall register the uses of that system in the EU database referred to in the Article 60. A new registration entry shall be complemented by the user for each high risk use of the AI system.

Amendment 70

Proposal for a regulation

Article 53 – paragraph 1

Text proposed by the Commission

Amendment

1. AI regulatory sandboxes established by one or more Member States competent authorities or the European Data Protection Supervisor shall provide a controlled environment that facilitates the development, testing and validation of innovative AI systems for a limited time before their placement on the market or putting into service pursuant to a specific plan. This shall take place under the direct supervision and guidance by the competent authorities with a view to ensuring compliance with the requirements of this Regulation and, where relevant, other Union and Member States legislation supervised within the sandbox.

1. AI regulatory sandboxes established by one or more Member States competent authorities in collaboration with the Commission, or the European Data Protection Supervisor shall provide a controlled environment that facilitates the safe development, testing and validation of innovative AI systems for a limited time before their placement on the market or putting into service pursuant to a specific plan, SMEs, start-ups, enterprises, innovators or other relevant actors could be included as partners in the regulatory sandboxes. This shall take place under the direct supervision and guidance of the Commission in collaboration with the competent authorities with a view to identifying risks, in particular to health, safety, and fundamental rights, and ensuring compliance with the requirements of this Regulation and, where relevant, other Union and Member States legislation supervised within the sandbox. The Commission shall play a complementary role, allowing those Member States with demonstrated experience with sandboxing to build on their expertise and, on the other hand, assisting and providing technical understanding and resources to those Member States that seek guidance on the set-up and running of these regulatory sandboxes.

Amendment 71

Proposal for a regulation

Article 53 – paragraph 2

Text proposed by the Commission

Amendment

2. Member States shall ensure that to the extent the innovative AI systems involve the processing of personal data or otherwise fall under the supervisory remit of other national authorities or competent authorities providing or supporting access to data, the national data protection authorities and those other national authorities are associated to the operation of the AI regulatory sandbox.

2. Member States, in collaboration with the Commission, shall ensure that to the extent the innovative AI systems involve the processing of personal data or otherwise fall under the supervisory remit of other national authorities or competent authorities providing or supporting access to data, the national data protection authorities and those other national authorities are associated to the operation of the AI regulatory sandbox.

Amendment 72

Proposal for a regulation

Article 53 – paragraph 3

Text proposed by the Commission

Amendment

3. The AI regulatory sandboxes shall not affect the supervisory and corrective powers of the competent authorities. Any significant risks to health and safety and fundamental rights identified during the development and testing of such systems shall result in immediate mitigation and, failing that, in the suspension of the development and testing process until such mitigation takes place.

3. The AI regulatory sandboxes shall not affect the supervisory and corrective powers of the competent authorities, including at regional or local level. Any significant risks to health and safety, fundamental rights, democracy or the environment, identified during the development and testing of AI systems, shall result in immediate mitigation and, failing that, in the suspension of the development and testing process until such mitigation takes place.

Amendment 73

Proposal for a regulation

Article 53 – paragraph 5

Text proposed by the Commission

Amendment

5. Member States’ competent authorities that have established AI regulatory sandboxes shall coordinate their activities and cooperate within the framework of the European Artificial Intelligence Board. They shall submit annual reports to the Board and the Commission on the results from the implementation of those scheme, including good practices, lessons learnt and recommendations on their setup and, where relevant, on the application of this Regulation and other Union legislation supervised within the sandbox.

5. Member States’ competent authorities and the Commission shall coordinate their activities with regard to AI regulatory sandboxes and cooperate within the framework of the European Artificial Intelligence Board. The Commission shall submit annual reports to the European Artificial Intelligence Board on the results from the implementation of those schemes, including best practices, computational energy use and efficiency, lessons learnt and recommendations on their setup and, where relevant, on the application of this Regulation and other Union legislation supervised within the sandbox. SMEs, start-ups, enterprises and other innovators shall be invited to share their good practices, lessons learnt and recommendations on their AI sandboxes with Member States’ competent authorities.

Amendment 74

Proposal for a regulation

Article 53 – paragraph 6 a (new)

Text proposed by the Commission

Amendment

6a. The Commission shall establish an EU AI Regulatory Sandboxing Work Programme whose modalities referred to in Article 53(6) shall cover the elements set out in Annex IXa. The Commission shall proactively coordinate with national and local authorities, where relevant.

Amendment 75

Proposal for a regulation

Article 55 – title

Text proposed by the Commission

Amendment

Measures for small-scale providers and users

Measures for SMEs, start-ups and users

Amendment 76

Proposal for a regulation

Article 55 – paragraph 1 – point a

Text proposed by the Commission

Amendment

(a) provide small-scale providers and start-ups with priority access to the AI regulatory sandboxes to the extent that they fulfil the eligibility conditions;

(a) provide SMEs and start-ups, established in the Union, with priority access to the AI regulatory sandboxes, to the extent that they fulfil the eligibility conditions;

Amendment 77

Proposal for a regulation

Article 55 – paragraph 1 – point b

Text proposed by the Commission

Amendment

(b) organise specific awareness raising activities about the application of this Regulation tailored to the needs of the small-scale providers and users;

(b) organise specific awareness raising and enhanced digital skills development activities on the application of this Regulation tailored to the needs of SMEs, start-ups and users;

Amendment 78

Proposal for a regulation

Article 55 – paragraph 1 – point c

Text proposed by the Commission

Amendment

(c) where appropriate, establish a dedicated channel for communication with small-scale providers and user and other innovators to provide guidance and respond to queries about the implementation of this Regulation.

(c) utilise existing dedicated channels and where appropriate, establish new dedicated channels for communication with SMEs, start-ups, users and other innovators to provide guidance and respond to queries about the implementation of this Regulation;

Amendment 79

Proposal for a regulation

Article 55 – paragraph 1 – point c a (new)

Text proposed by the Commission

Amendment

(ca) foster the participation of SMEs and other relevant stakeholders in the standardisation development process.

Amendment 80

Proposal for a regulation

Article 55 – paragraph 2

Text proposed by the Commission

Amendment

2. The specific interests and needs of the small-scale providers shall be taken into account when setting the fees for conformity assessment under Article 43, reducing those fees proportionately to their size and market size.

2. The specific interests and needs of the SMEs, start-ups and users shall be taken into account when setting the fees for conformity assessment under Article 43, reducing those fees proportionately to development stage, their size, market size and market demand. The Commission shall regularly assess the certification and compliance costs for SMEs and start-ups, including through transparent consultations with SMEs, start-ups and users and shall work with Member States to lower such costs where possible. The Commission shall report on these findings to the European Parliament and to the Council as part of the report on the evaluation and review of this Regulation provided for in Article 84(2).

Amendment 81

Proposal for a regulation

Article 57 – paragraph 3 a (new)

Text proposed by the Commission

Amendment

3a. The Board shall establish an AI Advisory Council (Advisory Council). The Advisory Council shall be composed of relevant representatives from industry, research, academia, civil society, standardisation organisations, and other relevant stakeholders or third parties appointed by the Board, representing all Member States to maintain geographical balance. The Advisory Council shall support the work of the Board by providing advice relating to the tasks of the Board. The Advisory Council shall nominate a relevant representative, depending on the configuration in which the Board meets, to attend meetings of the Board and to participate in its work. The composition of the Advisory Council and its recommendations to the Board shall be made public.

Amendment 82

Proposal for a regulation

Annex I – point c

Text proposed by the Commission

Amendment

Amendment 83

Proposal for a regulation

Annex IX a (new)

Text proposed by the Commission

Amendment

ANNEX IXa

Modalities for an EU AI regulatory sandboxing work programme

1. The AI Regulatory Sandboxes shall be part of the EU AI Regulatory Sandboxing Work Programme (‘sandboxing programme’) to be established by the Commission in collaboration with Member States.

2. The Commission shall play a complementary role, allowing those Member States with demonstrated experience with sandboxing to build on their expertise and the expertise of relevant stakeholders from industry, SMEs, academia and civil society and, on the other hand, assisting and providing technical understanding and resources to those Member States that seek guidance on the set-up of these regulatory sandboxes.

3. The criteria for the access to the regulatory sandbox shall be transparent and competitive.

4. Participants in the sandboxing programme, in particular small-scale providers, are granted access to pre-deployment services, such as preliminary registration of their AI system, compliance R&D support services, and to all the other relevant elements of the Union’s AI ecosystem and other Digital Single Market initiatives such as Testing & Experimentation Facilities, Digital Hubs, Centres of Excellence, and EU benchmarking capabilities; and to other value-adding services such as standardisation documents and certification, consultation and support to conduct impact assessments of the AI systems to fundamental rights, environment or the society at large, an online social platform for the community, contact databases, existing portal for tenders and grant making and lists of EU investors.

5. The sandboxing programme shall, in a later development phase, aim at assisting Member States in developing and managing two types of regulatory sandboxes: Physical Regulatory Sandboxes for AI systems embedded in physical products or services and Cyber Regulatory Sandboxes for AI systems operated and used on a stand-alone basis, not embedded in physical products or services.

6. The sandboxing programme shall work with the already established Digital Innovation Hubs in Member States to provide a dedicated point of contact for entrepreneurs to raise enquiries with competent authorities and to seek non-binding guidance on the conformity of innovative products, services or business models embedding AI technologies.

7. One of the objectives of the sandboxing programme is to enable firms’ compliance with this Regulation at the design stage of the AI system (‘compliance-by-design’). To do so, the programme shall facilitate the development of software tools and infrastructure for testing, benchmarking, assessing and explaining dimensions of AI systems relevant to sandboxes, such as accuracy, robustness and cybersecurity as well as minimisation of risks to fundamental rights, environment and the society at large.

8. The sandboxing programme shall be rolled out in a phased fashion, with the various phases launched by the Commission upon success of the previous phase.

9. The sandboxing programme will have a built-in impact assessment procedure to facilitate the review of cost-effectiveness against the agreed-upon objectives. This assessment shall be drafted with input from Member States based on their experiences and shall be included as part of the Annual Report submitted by the Commission to the European Artificial Intelligence Board.

PROCEDURE – COMMITTEE ASKED FOR OPINION

Title

Harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union Legislative Acts

References

COM(2021)0206 – C9-0146/2021 – 2021/0106(COD)

Committees responsible

Date announced in plenary

IMCO

7.6.2021

LIBE

7.6.2021

Opinion by

Date announced in plenary

ITRE

7.6.2021

Associated committees - date announced in plenary

16.12.2021

Rapporteur for the opinion

Date appointed

Eva Maydell

11.1.2022

Rule 58 – Joint committee procedure

Date announced in plenary

16.12.2021

Discussed in committee

21.3.2022

Members present for the final vote

François-Xavier Bellamy, Hildegard Bentele, Tom Berendsen, Vasile Blaga, Michael Bloss, Marc Botenga, Cristian-Silviu Buşoi, Jerzy Buzek, Maria da Graça Carvalho, Ignazio Corrao, Ciarán Cuffe, Nicola Danti, Pilar del Castillo Vera, Martina Dlabajová, Christian Ehler, Niels Fuglsang, Lina Gálvez Muñoz, Jens Geier, Nicolás González Casares, Christophe Grudler, Henrike Hahn, Robert Hajšel, Ivars Ijabs, Romana Jerković, Seán Kelly, Łukasz Kohut, Zdzisław Krasnodębski, Andrius Kubilius, Thierry Mariani, Marisa Matias, Eva Maydell, Iskra Mihaylova, Dan Nica, Angelika Niebler, Niklas Nienaß, Mauri Pekkarinen, Mikuláš Peksa, Tsvetelina Penkova, Morten Petersen, Pina Picierno, Clara Ponsatí Obiols, Manuela Ripa, Robert Roos, Sara Skyttedal, Maria Spyraki, Patrizia Toia, Pernille Weiss, Carlos Zorrinho

Substitutes present for the final vote

Franc Bogovič, Andrea Caroppo, Jakop G. Dalunde, Jens Gieseke, Klemen Grošelj, Elena Kountoura, Marian-Jean Marinescu, Dace Melbārde, Dominique Riquet, Rob Rooken, Susana Solís Pérez

Substitutes under Rule 209(7) present for the final vote

Alessandra Basso, Bas Eickhout, Carlo Fidanza, Rob Rooken

FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION

Key to symbols: