Sittings · Document

DRAFT OPINION (COM(2021)0206 – C90146/2021 – 2021/0106(COD)) 2022-03-03

On the proposal for a regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain Union legislative acts

Committee on Industry, Research and Energy · Rapporteur: Eva Maydell

PA_Legam

SHORT JUSTIFICATION

Introduction

The Rapporteur welcomes the Commission’s proposal on an Artificial Intelligence Act and especially the horizontal risk-based approach that it puts forward. This approach will allow for the development of AI systems in line with European values and for the fostering of social trust in these new technologies, so that the EU can fulfil the full economic and social benefits of AI.

The Rapporteur is of the opinion that through the AI Act, we need to create an environment with the right balance between freedom and supervision. The Rapporteur proposes that further provisions are made in order for companies, especially start-ups and SMEs, to remain competitive and creative in the face of new obligations required of them. The Rapporteur believes this will increase both the legitimacy and relevance of the AI Act. We need to provide companies with clearer guidelines, simpler tools and more efficient resources to cope with regulation. This would allow us to support AI innovation, development and market uptake.

Therefore, the Rapporteur’s draft pursues four main objectives in this direction:

1. Enhancing measures to support innovation, such as the ones foreseen for regulatory sandboxes, with a particular focus on start-ups and SMEs

2. Providing a concise and internationally recognised definition of Artificial Intelligence System and setting high but realistic standards for accuracy, robustness, cybersecurity and data

3. Encouraging the uptake of AI systems by industry by placing an emphasis on social trust and value chain responsibility

4. Future-proofing the Act through better linkages to the green transition and possible changes in the industry, technology and power of AI

This draft opinion focuses mainly on issues related to ITRE’s competences but also broader issues related to innovation, competitiveness, research, sustainability and future changes in industry.

Supporting innovation, focus on start-ups and SMEs, enhancing regulatory sandboxes

The Rapporteur welcomes the introduction of Article 55 on measures for small-scale providers, but believes SMEs and start-ups should be more involved throughout the AI Act in a holistic approach. More specifically, in the development of Codes of Conduct, standardisation, and representation in the European Artificial Intelligence Board. By far, one of the biggest focuses for the Rapporteur is the provision of opportunities to SMEs and start-ups to participate in the AI regulatory sandboxes. This is why the Rapporteur proposes to strengthen the existing provisions by giving the regulatory sandboxes a more European dimension, preserving the unity of the Single Market and calling for the development of an EU AI Regulatory Sandboxing Programme whose modalities are set out in a new Annex.

Clear definition and realistic standards

The Rapporteur calls for the use of an internationally recognised definition of Artificial Intelligence System, which would be in line with the EU’s broader goals of setting global standards, working closely with transatlantic partners and likeminded allies and providing legal certainty for businesses, citizens and civil society. The Rapporteur believes that high standards for accuracy, robustness and cybersecurity as well as data and data governance are key to developing safe AI systems that protect fundamental rights. The key here is to balance this aim with the practical and pragmatic approach needed for achieving it. The Rapporteur calls for a common European authority on benchmarking that brings together national metrology and benchmarking authorities to set a unified approach to measurement of accuracy, robustness, and other relevant criteria.

Encouraging uptake of AI systems, fostering social trust, value chain responsibility

To encourage uptake and deployment of AI systems, the Rapporteur believes we need to foster social trust of both businesses and citizens. The Rapporteur seeks to address the challenge of social trust by encouraging a collaborative relationship between developers and users of AI that is better aligned to their responsibilities along the value chain, strengthening the Codes of Conduct and enhancing the measures on regulatory sandboxes to enable compliance-by-design. This in turn creates a healthy and integrated ecosystem, which will help reduce legal uncertainty and implementation gaps, all of which in turn will increase social trust.

Future-proofing, sustainability and changes in the industry and power of AI

AI is a mature and ready-to-use technology that can be used to process the ever growing amount of data created along industrial processes. To facilitate investments to AI-based analysis and optimisation solutions, this regulation should provide a predictable environment for low-risk industrial solutions. Furthermore, this Regulation should take into account future changes in the industry and power of AI. This is why the Rapporteur proposes great involvement of the High Level Expert Group on AI with both the Commission and the European Artificial Intelligence Board as well as the monitoring of market trends and foresight by the European AI Board.

AMENDMENTS

The Committee on Industry, Research and Energy calls on the Committee on the Internal Market and Consumer Protection and the Committee on Civil Liberties, Justice and Home Affairs, as the committees responsible, to take into account the following amendments:

Amendment 1

Proposal for a regulation

Title 1

Text proposed by the Commission

Amendment

Proposal for a

REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL

LAYING DOWN HARMONISED RULES ON ARTIFICIAL INTELLIGENCE (ARTIFICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION LEGISLATIVE ACTS

(Text with EEA relevance)

Or. en

Amendment 2

Proposal for a regulation

Recital 3 a (new)

Text proposed by the Commission

Amendment

(3a) Furthermore, in order for Member States to reach the carbon neutrality targets, European companies should seek to utilise all available technological advancements that can assist in realising this goal. AI is a well-developed and ready-to-use technology that can be used to process the ever growing amount of data created along industrial processes. To facilitate investments in AI-based analysis and optimisation solutions, this regulation should provide a predictable and proportionate environment for low-risk industrial solutions.

Or. en

Amendment 3

Proposal for a regulation

Recital 5 a (new)

Text proposed by the Commission

Amendment

(5a) Furthermore, in order to foster the development of artificial intelligence in line with Union values, the Union needs to address the main gaps and barriers blocking the potential of the digital transformation including the shortage of digitally skilled workers, cybersecurity concerns, lack of investment and access to investment, and existing and potential gaps between large companies and SMEs. Special attention should be paid to ensuring that the benefits of artificial intelligence and innovation in new technologies are felt across all regions of the Union and that sufficient investment and resources are provided especially to those regions that may be lagging behind in some digital indicators.

Or. en

Amendment 4

Proposal for a regulation

Recital 6

Text proposed by the Commission

Amendment

(6) The notion of AI system should be clearly defined to ensure legal certainty, while providing the flexibility to accommodate future technological developments. The definition should be based on the key functional characteristics of the software, in particular the ability, for a given set of human-defined objectives, to generate outputs such as content, predictions, recommendations, or decisions which influence the environment with which the system interacts, be it in a physical or digital dimension. AI systems can be designed to operate with varying levels of autonomy and be used on a stand-alone basis or as a component of a product, irrespective of whether the system is physically integrated into the product (embedded) or serve the functionality of the product without being integrated therein (non-embedded). The definition of AI system should be complemented by a list of specific techniques and approaches used for its development, which should be kept up-to–date in the light of market and technological developments through the adoption of delegated acts by the Commission to amend that list.

(6) The notion of AI system should be clearly defined to ensure legal certainty, while providing the flexibility to accommodate future technological developments. This definition should be in line with definitions that have found international acceptance. The definition should be based on the key functional characteristics of the AI system, in particular the ability, for a given set of human-defined objectives, to make predictions, recommendations, or decisions influencing real or virtual environments. AI systems are designed to operate with varying levels of autonomy and be used on a stand-alone basis or as a component of a product, irrespective of whether the system is physically integrated into the product (embedded) or serve the functionality of the product without being integrated therein (non-embedded). The definition of AI system should be complemented by a list of specific techniques and approaches used for its development, which should be kept up-to–date in the light of market and technological developments through the adoption of delegated acts by the Commission to amend that list. The Commission should engage in dialogue with key international organisations such as the Organisation for Economic Cooperation and Development to ensure alignment between AI definitions.

Or. en

Justification

If we want the AI Act to be a global standard, we cannot alienate our allies and other countries with whom we worked together through other platforms to develop a common definition on AI. The definition developed by the Organisation for Economic Co-operation and Development is the result of a lengthy multi-stakeholder process that was inclusive of the international community. While the Rapporteur welcomes that the Commission's definition is largely based on the OECD definition, using the same definition as the OECD would provide more certainty to industry, businesses, start-ups, and SMEs - one of the main goals of this committee. Aligning our approach with international partners and building upon the existing framework is key to the future development of common international standards. This is why the Rapporteur also proposes to have continued dialogue between the European Union and the OECD as our understanding of these systems evolves. Businesses, citizens and stakeholders in Europe should not be dealing with one definition of AI system within the Union and another beyond its borders - alignment would benefit all involved in this multistakeholder model.

Amendment 5

Proposal for a regulation

Recital 12 a (new)

Text proposed by the Commission

Amendment

(12a) This Regulation should not undermine research and development activity and should respect freedom of science. It is therefore necessary to ensure that this Regulation does not otherwise affect scientific research and development activity on AI systems. As regards product oriented research activity by providers, the provisions of this Regulation should apply insofar as such research leads to or entails placing an AI system on the market or putting it into service. Under all circumstances, any research and development activity should be carried out in accordance with recognised ethical standards for scientific research.

Or. en

Amendment 6

Proposal for a regulation

Recital 36

Text proposed by the Commission

Amendment

(36) AI systems used in employment, workers management and access to self-employment, notably for the recruitment and selection of persons, for making decisions on promotion and termination and for task allocation, monitoring or evaluation of persons in work-related contractual relationships, should also be classified as high-risk, since those systems may appreciably impact future career prospects and livelihoods of these persons. Relevant work-related contractual relationships should involve employees and persons providing services through platforms as referred to in the Commission Work Programme 2021. Such persons should in principle not be considered users within the meaning of this Regulation. Throughout the recruitment process and in the evaluation, promotion, or retention of persons in work-related contractual relationships, such systems may perpetuate historical patterns of discrimination, for example against women, certain age groups, persons with disabilities, or persons of certain racial or ethnic origins or sexual orientation. AI systems used to monitor the performance and behaviour of these persons may also impact their rights to data protection and privacy.

(36) AI systems used in employment, workers management and access to self-employment, notably for the recruitment and selection of persons, for making decisions on promotion and termination, monitoring or evaluation of persons in work-related contractual relationships, should also be classified as high-risk, since those systems may appreciably impact future career prospects and livelihoods of these persons. Relevant work-related contractual relationships should involve employees and persons providing services through platforms as referred to in the Commission Work Programme 2021. Such persons should in principle not be considered users within the meaning of this Regulation. Throughout the recruitment process and in the evaluation, promotion, or retention of persons in work-related contractual relationships, such systems may perpetuate historical patterns of discrimination, for example against women, certain age groups, persons with disabilities, or persons of certain racial or ethnic origins or sexual orientation. AI systems used to monitor the performance and behaviour of these persons may also impact their rights to data protection and privacy.

Or. en

Amendment 7

Proposal for a regulation

Recital 42

Text proposed by the Commission

Amendment

(42) To mitigate the risks from high-risk AI systems placed or otherwise put into service on the Union market for users and affected persons, certain mandatory requirements should apply, taking into account the intended purpose of the use of the system and according to the risk management system to be established by the provider.

(42) To mitigate the risks from high-risk AI systems placed or otherwise put into service on the Union market for users and affected persons, certain mandatory requirements should apply, taking into account the intended purpose of the use of the system, the level of reliance of the user on the output of the AI system for the final outcome and according to the risk management system to be established by the provider.

Or. en

Amendment 8

Proposal for a regulation

Recital 49

Text proposed by the Commission

Amendment

(49) High-risk AI systems should perform consistently throughout their lifecycle and meet an appropriate level of accuracy, robustness and cybersecurity in accordance with the generally acknowledged state of the art. The level of accuracy and accuracy metrics should be communicated to the users. The European Artificial Intelligence Board shall work to set up a common European authority on benchmarking that brings together national metrology and benchmarking authorities. This would address the current problem of not having relevant metrics on a European level to guide developers and providers of AI. While standardisation organisations exist to establish what the standard should be, benchmarking organisations are needed to establish how these standards should be met and measured. The creation of a common European authority - such as a European Benchmarking Institute or as a subgroup of the European AI Board - would allow for a cohesive European approach to benchmarking and metrics.

Or. en

Amendment 9

Proposal for a regulation

Recital 66

Text proposed by the Commission

Amendment

(66) In line with the commonly established notion of substantial modification for products regulated by Union harmonisation legislation, it is appropriate that an AI system undergoes a new conformity assessment whenever a change occurs which may affect the compliance of the system with this Regulation or when the intended purpose of the system changes. In addition, as regards AI systems which continue to ‘learn’ after being placed on the market or put into service (i.e. they automatically adapt how functions are carried out), it is necessary to provide rules establishing that changes to the algorithm and its performance that have been pre-determined by the provider and assessed at the moment of the conformity assessment should not constitute a substantial modification.

(66) In line with the commonly established notion of substantial modification for products regulated by Union harmonisation legislation, it is appropriate that an AI system undergoes a new conformity assessment whenever a change occurs which may affect the compliance of the system with this Regulation or when the intended purpose of the system changes. In addition, as regards AI systems which continue to ‘learn’ after being placed on the market or put into service (i.e. they automatically adapt how functions are carried out), it is necessary to provide rules establishing that changes to the algorithm and its performance that have been considered by the provider and assessed at the moment of the conformity assessment should not constitute a substantial modification.

Or. en

Amendment 10

Proposal for a regulation

Recital 70 a (new)

Text proposed by the Commission

Amendment

(70a) There is fast-paced innovation in the field of AI which reduces the relevance of static “blanket” governance mechanisms in favour of adaptive, flexible controls tailored to the model. Considering the complexity of the value chain for AI systems, it is essential to have clear value chain responsibility. It is necessary to clarify that general purpose AI systems should not be considered as having an intended purpose in the context of this Regulation. If a person places on the market or puts into service under its own name or trademark or uses a general purpose AI system made available on the market for an intended purpose within the meaning of this Regulation, that person should be considered the provider of the AI system. Similarly, if a person integrates a general purpose AI system made available on the market, with or without modifying it, into an AI system that is subject to the provisions of this Regulation, that person should also be considered the provider of the latter AI system. To ensure that the regulatory burden associated with becoming a provider does not fall solely on SMEs and start-ups, providers of general purpose AI systems should work with users of their products to aid them in fulfilling some of the requirements set out in this Regulation by providing technical expertise. Such an exchange should be in full respect of trade secrets and current market indicators and should be only on those obligations, relating to the technical design and development of the AI system before an intended purpose is attributed to it. Providers of general purpose AI systems should register in the EU database referred to in Article 60.

Or. en

Amendment 11

Proposal for a regulation

Recital 73

Text proposed by the Commission

Amendment

(73) In order to promote and protect innovation, it is important that the interests of small-scale providers and users of AI systems are taken into particular account. To this objective, Member States should develop initiatives, which are targeted at those operators, including on awareness raising and information communication. Moreover, the specific interests and needs of small-scale providers shall be taken into account when Notified Bodies set conformity assessment fees. Translation costs related to mandatory documentation and communication with authorities may constitute a significant cost for providers and other operators, notably those of a smaller scale. Member States should possibly ensure that one of the languages determined and accepted by them for relevant providers’ documentation and for communication with operators is one which is broadly understood by the largest possible number of cross-border users.

Or. en

Amendment 12

Proposal for a regulation

Recital 74

Text proposed by the Commission

Amendment

(74) In order to minimise the risks to implementation resulting from lack of knowledge and expertise in the market as well as to facilitate compliance of providers and notified bodies with their obligations under this Regulation, the AI-on demand platform, the European Digital Innovation Hubs and the Testing and Experimentation Facilities established by the Commission and the Member States at national or EU level should possibly contribute to the implementation of this Regulation. Within their respective mission and fields of competence, they may provide in particular technical and scientific support to providers and notified bodies.

(74) In order to minimise the risks to implementation resulting from lack of knowledge and expertise in the market as well as to facilitate compliance of providers and notified bodies with their obligations under this Regulation, Member States should utilise existing dedicated channels for communication with SMEs and start-ups. Such existing channels could include but are not limited to ENISA’s Computer Security Incident Response Teams, National Data Protection Agencies, the AI-on demand platform, the European Digital Innovation Hubs and the Testing and Experimentation Facilities established by the Commission and the Member States at national or EU level. Within their respective mission and fields of competence, they may provide in particular technical and scientific support to providers and notified bodies.

Or. en

Amendment 13

Proposal for a regulation

Recital 81 a (new)

Text proposed by the Commission

Amendment

(81a) The Commission and the European Artificial Intelligence Board should in the process of drawing up of Codes of Conduct include provisions for general purpose AI systems taking into account the latest technological developments, impacts on start ups and SMEs and the effectiveness of existing regulations surrounding general purpose AI systems.

Or. en

Amendment 14

Proposal for a regulation

Article 1 – paragraph 1 – point d a (new)

Text proposed by the Commission

Amendment

(da) measures in support of innovation, including the setting up of regulatory sandboxes, and measures to reduce the regulatory burden on SMEs and start-ups.

Or. en

Amendment 15

Proposal for a regulation

Article 2 – paragraph 2 – introductory part

Text proposed by the Commission

Amendment

2. For high-risk AI systems that are safety components of products or systems, or which are themselves products or systems, falling within the scope of the following acts, only Article 84 of this Regulation shall apply:

2. For AI systems classified as high-risk AI related to products covered by Union harmonisation legislation listed in Annex II, Section B, to this Regulation that are safety components of products or systems, or which are themselves products or systems, falling within the scope of the following acts, only Article 84 of this Regulation shall apply.

Or. en

Justification

The belowmentioned regulations are already included in Annex II Section B so there is no need to list them in the Article itself. The following change provides legal clarity to businesses and industry operating under these Regulation and is more in line with the approach undertaken in other Annexes.

Amendment 16

Proposal for a regulation

Article 2 – paragraph 2 – point a

Text proposed by the Commission

Amendment

(a) Regulation (EC) 300/2008;

deleted

Or. en

Amendment 17

Proposal for a regulation

Article 2 – paragraph 2 – point b

Text proposed by the Commission

Amendment

(b) Regulation (EU) No 167/2013;

deleted

Or. en

Amendment 18

Proposal for a regulation

Article 2 – paragraph 2 – point c

Text proposed by the Commission

Amendment

deleted

Or. en

Amendment 19

Proposal for a regulation

Article 2 – paragraph 2 – point d

Text proposed by the Commission

Amendment

(d) Directive 2014/90/EU;

deleted

Or. en

Amendment 20

Proposal for a regulation

Article 2 – paragraph 2 – point e

Text proposed by the Commission

Amendment

(e) Directive (EU) 2016/797;

deleted

Or. en

Amendment 21

Proposal for a regulation

Article 2 – paragraph 2 – point f

Text proposed by the Commission

Amendment

(f) Regulation (EU) 2018/858;

deleted

Or. en

Amendment 22

Proposal for a regulation

Article 2 – paragraph 2 – point g

Text proposed by the Commission

Amendment

(g) Regulation (EU) 2018/1139;

deleted

Or. en

Amendment 23

Proposal for a regulation

Article 2 – paragraph 2 – point h

Text proposed by the Commission

Amendment

(h) Regulation (EU) 2019/2144.

deleted

Or. en

Amendment 24

Proposal for a regulation

Article 2 – paragraph 5 a (new)

Text proposed by the Commission

Amendment

5a. This Regulation shall not affect any research and development activity regarding AI systems in so far as such activity does not lead to or entail placing an AI system on the market or putting it into service.

Or. en

Justification

The Commission proposal only applies to products once they are placed on the market as defined in Article 3.9 but this amendment provides more legal clarity. It defends academic freedom and encourages research, which is a fundamental backbone to both developing AI systems for good and understanding cases where AI could be used in a dangerous way.

Amendment 25

Proposal for a regulation

Article 3 – paragraph 1 – point 1

Text proposed by the Commission

Amendment

(1) ‘artificial intelligence system’ (AI system) means software that is developed with one or more of the techniques and approaches listed in Annex I and can, for a given set of human-defined objectives, generate outputs such as content, predictions, recommendations, or decisions influencing the environments they interact with;

(1) ‘artificial intelligence system’ (AI system) means a machine-based system that can for a given set of human-defined objectives, make predictions, recommendations, or decisions influencing real or virtual environments; AI systems can be designed to operate with varying levels of autonomy and can be developed with one or more of the techniques and approaches listed in Annex I;

Or. en

Justification

The definition in Article 3.1 - a shared competence of the ITRE committee - should be in line with the broadly recognised definition from the recommendation of the Organisation for Economic Co-Operation and Development (OECD/LEGAL/0449). As the Commission itself points out in the introduction of the AI Act, the proposed AI framework should be compatible with the EU’s international trade partners. This would be in line with the Union’s broader goals of setting global standards, would allow the EU to work better with transatlantic partners and likeminded allies, and would provide legal certainty for businesses, citizens and other stakeholders. This definition is also an improvement on the Commission’s proposal as it is more narrow and does not risk including software tools which are not actually AI systems. The Rapporteur believes that including the word “generate” in the Commission’s proposal was a step in the right direction, but including the full definition from the OECD does a better job of highlighting the principle of various degrees of autonomy, a defining feature of AI. Should the definition of ‘artificial intelligence system’ from international organisations such as the Organisation for Economic Co-operation and Development be adjusted in the years to come, the Commission should engage in dialogue with the OECD to ensure alignment between the two definitions. Should the AI Act still be undergoing legislative procedure, the co-legislators should consider these latest developments during the legislative process, so as to ensure alignment, legal clarity and broad international acceptance of the AI Act Definition of ‘AI Systems’.

Amendment 26

Proposal for a regulation

Article 3 – paragraph 1 – point 1 a (new)

Text proposed by the Commission

Amendment

(1a) 'autonomy' means that to some degree an AIsystem operates by interpreting certain input and by using a set of pre-determined objectives, without being limited to such instructions, despite the system’s behaviour being constrained by, and targeted at, fulfilling the goal it was given and other relevant design choices made by its developer;

Or. en

Amendment 27

Proposal for a regulation

Article 3 – paragraph 1 – point 12 a (new)

Text proposed by the Commission

Amendment

(12a) 'general purpose AI application' means AI applications that are able to perform generally applicable functions such as image or speech recognition, audio or video generation, pattern detection, question answering, and translation, and are largely customizable and therefore shall not be considered as having an intended purpose within the meaning of this Regulation;

Or. en

Amendment 28

Proposal for a regulation

Article 3 – paragraph 1 – point 44 a (new)

Text proposed by the Commission

Amendment

(44a) ‘regulatory sandbox’ means a facility established by the Commission in collaboration with one or more Member States competent authorities or the European Data Protection Supervisor that provides a controlled environment that facilitates the safe development, testing and validation of innovative AI systems for a limited time before their placement on the market or putting into service pursuant to a specific plan;

Or. en

Amendment 29

Proposal for a regulation

Article 3 – paragraph 1 – point 44 b (new)

Text proposed by the Commission

Amendment

(44b) ‘deep fake’ means manipulated or synthetic audio or visual media that seem authentic, and which feature people that appear to say or do something they have never said or done, produced using artificial intelligence techniques, including machine learning and deep learning;

Or. en

Amendment 30

Proposal for a regulation

Article 3 – paragraph 1 – point 44 c (new)

Text proposed by the Commission

Amendment

(44c) 'critical infrastructure' means an asset, system or part thereof which is necessary for the delivery of a service that is essential for the maintenance of vital societal functions or economic activities within the meaning of Article 2(4) and (5) of Directive …../….. of the European Parliament and of the Council on the resilience of critical entities (2020/0365(COD));

Or. en

Amendment 31

Proposal for a regulation

Article 4 – paragraph 1

Text proposed by the Commission

Amendment

The Commission is empowered to adopt delegated acts in accordance with Article 73 to amend the list of techniques and approaches listed in Annex I, in order to update that list to market and technological developments on the basis of characteristics that are similar to the techniques and approaches listed therein.

The Commission is empowered to adopt delegated acts in accordance with Article 73 to amend the list of techniques and approaches listed in Annex I within the scope of the definition of an AI system as provided for in Article 3(1), in order to update that list to market and technological developments on the basis of characteristics that are similar to the techniques and approaches listed therein.

Or. en

Amendment 32

Proposal for a regulation

Article 4 – paragraph 1 a (new)

Text proposed by the Commission

Amendment

In the drafting process of these delegated acts, the Commission shall ensure the input of all relevant stakeholders, such as the High Level Expert Group on AI, including technical experts and developers of Artificial Intelligence Systems. The Commission shall also take account the annual recommendations on market trends, as provided for in Article 56 paragraph 2b, submitted to it by the European Artificial Intelligence Board.

Or. en

Amendment 33

Proposal for a regulation

Article 6 – paragraph 2

Text proposed by the Commission

Amendment

2. In addition to the high-risk AI systems referred to in paragraph 1, AI systems referred to in Annex III shall also be considered high-risk.

2. In addition to the high-risk AI systems referred to in paragraph 1 of this Article, AI systems referred to in Annex III shall also be considered high-risk. In case there is uncertainty over the AI system's classification, the provider shall deem the AI system high-risk if its use or application poses a risk of harm to the health and safety or a risk of adverse impact on fundamental rights of users, as outlined in Article 7(2).

Or. en

Amendment 34

Proposal for a regulation

Article 7 – paragraph 1 a (new)

Text proposed by the Commission

Amendment

1a. When adopting the delegated act, the Commission shall consider the input of all relevant stakeholders, including the High Level Expert Group on AI, as well as the technical experts and developers of Artificial Intelligence Systems. The Commission shall also take into account the annual recommendations on market trends, provided for in Article 56 paragraph 2c, submitted to it by the European Artificial Intelligence Board.

Or. en

Amendment 35

Proposal for a regulation

Article 7 – paragraph 2 – point e

Text proposed by the Commission

Amendment

(e) the extent to which potentially harmed or adversely impacted persons are dependent on the outcome produced with an AI system, in particular because for practical or legal reasons it is not reasonably possible to opt-out from that outcome;

(e) the extent to which potentially harmed or adversely impacted persons are dependent on the outcome produced with an AI system, with a distinction to be made between an AI system used in an advisory capacity or one used directly to inform decision-making, in particular because for practical or legal reasons it is not reasonably possible to opt-out from that outcome;

Or. en

Justification

The extent to which a user relies on the AI system is an important consideration for the Commission to consider. Where an AI system is one of a number of inputs, it would be playing an advisory role. Where an AI system’s output is the sole determinant factor in a decision, it would be a system that is heavily relied upon by the user. There are a range of points in between but for the sake of legal clarity and simplicity, the Rapporteur has chosen to make the distinction in the following way.

Amendment 36

Proposal for a regulation

Article 7 – paragraph 2 – point g

Text proposed by the Commission

Amendment

(g) the extent to which the outcome produced with an AI system is easily reversible, whereby outcomes having an impact on the health or safety of persons shall not be considered as easily reversible;

(g) the extent to which the outcome produced with an AI system is not easily reversible or remedied, whereby outcomes having an impact on the health or safety of persons shall not be considered as easily reversible;

Or. en

Amendment 37

Proposal for a regulation

Article 7 – paragraph 2 – point g a (new)

Text proposed by the Commission

Amendment

(ga) the size and nature of data processed;

Or. en

Justification

The proposed approach should not disregard the role of the availability of data and the access to it when it comes to determining societal risks. There is a lot to be done before huge amount of data of appropriate to high quality is unleashed in the Union. The proposed criteria should take the market reality into account in order to avoid excessive regulation for EU companies that currently can not take advantage of large amounts of data nor of its good quality.

Amendment 38

Proposal for a regulation

Article 7 – paragraph 2 – point g b (new)

Text proposed by the Commission

Amendment

(gb) the general capabilities and functionalities of the AI system independent of its intended purpose;

Or. en

Amendment 39

Proposal for a regulation

Article 7 – paragraph 2 – point g c (new)

Text proposed by the Commission

Amendment

(gc) the extent to which the AI system acts autonomously;

Or. en

Amendment 40

Proposal for a regulation

Article 7 – paragraph 2 – point g d (new)

Text proposed by the Commission

Amendment

(gd) magnitude and likelihood of benefit of the AI use for individuals, groups, or society at large;

Or. en

Amendment 41

Proposal for a regulation

Article 9 – paragraph 1 – subparagraph 1 a (new)

Text proposed by the Commission

Amendment

A risk management system can be integrated into already existing risk management procedures.

Or. en

Amendment 42

Proposal for a regulation

Article 9 – paragraph 2 – point a

Text proposed by the Commission

Amendment

(a) identification and analysis of the known and foreseeable risks associated with each high-risk AI system;

(a) identification and analysis of the known and foreseeable risks associated with each high-risk AI system with respect to health, safety and fundamental rights in view of the intended purpose of the high-risk AI system;

Or. en

Justification

The Article doesn’t offer guidance on what specific risks need to be considered by the risk management system. This wording was taken from Recitals 27 and 43 which state that the goal of this Regulation is to mitigate risks related to “health, safety and fundamental rights.”

Amendment 43

Proposal for a regulation

Article 9 – paragraph 2 a (new)

Text proposed by the Commission

Amendment

2a. The risks referred to in paragraph 2 shall concern only those which may be reasonably mitigated or eliminated through the development or design of the high-risk AI system, or the provision of adequate technical information.

Or. en

Amendment 44

Proposal for a regulation

Article 10 – paragraph 1

Text proposed by the Commission

Amendment

1. High-risk AI systems which make use of techniques involving the training of models with data shall be developed on the basis of training, validation and testing data sets that meet the quality criteria referred to in paragraphs 2 to 5.

Or. en

Amendment 45

Proposal for a regulation

Article 10 – paragraph 2 – introductory part

Text proposed by the Commission

Amendment

2. Training, validation and testing data sets shall be subject to appropriate data governance and management practices. Those practices shall concern in particular,

2. Training, validation and testing of the AI applications shall be subject to appropriate data governance and management practices. Those practices shall concern in particular the following elements:

Or. en

Amendment 46

Proposal for a regulation

Article 10 – paragraph 2 – point b

Text proposed by the Commission

Amendment

(b) data collection;

(b) data collection processes;

Or. en

Amendment 47

Proposal for a regulation

Article 10 – paragraph 2 – point f

Text proposed by the Commission

Amendment

(f) examination in view of possible biases;

(f) examination in view of possible biases that are likely to affect health and safety of persons or lead to discrimination prohibited by Union law;

Or. en

Amendment 48

Proposal for a regulation

Article 10 – paragraph 2 – point g

Text proposed by the Commission

Amendment

(g) the identification of any possible data gaps or shortcomings, and how those gaps and shortcomings can be addressed.

(g) the identification of possible data gaps or shortcomings, and how those gaps and shortcomings can be addressed.

Or. en

Amendment 49

Proposal for a regulation

Article 10 – paragraph 3

Text proposed by the Commission

Amendment

3. Training, validation and testing data sets shall be relevant, representative, free of errors and complete. They shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons on which the high-risk AI system is intended to be used. These characteristics of the data sets may be met at the level of individual data sets or a combination thereof.

3. High-risk AI systems shall be designed and developed with the best-efforts to ensure that training, validation and testing data sets shall be relevant, representative and appropriately vetted for errors and completeness in accordance with industry standards. They shall have the appropriate statistical properties, including, where applicable, as regards the persons or groups of persons on which the high-risk AI system is intended to be used. These characteristics of the data sets may be met at the level of individual data sets or a combination thereof.

Or. en

Justification

While the Rapporteur understands the Commission’s intent here, there is a need for further clarification and certainty. To have a data set that is 100% accurate and free of errors is not feasible. It also risks presenting users with a false sense of reliability of a particular AI system. Furthermore, data can be missing or uncollected for a variety of reasons but still be valuable.

Amendment 50

Proposal for a regulation

Article 11 – paragraph 1 – subparagraph 2

Text proposed by the Commission

Amendment

The technical documentation shall be drawn up in such a way to demonstrate that the high-risk AI system complies with the requirements set out in this Chapter and provide national competent authorities and notified bodies with all the necessary information to assess the compliance of the AI system with those requirements. It shall contain, at a minimum, the elements set out in Annex IV.

Or. en

Justification

It is important to provide the necessary flexibility for providers having in mind their various sizes, resources, capacities as the proposed minimum requirements in Annex IV could be too cumbersome. We should be also mindful of the resources of national competent bodies and ensure they have the resources and technical expertise required to check such technical documentation.

Amendment 51

Proposal for a regulation

Article 11 – paragraph 2

Text proposed by the Commission

Amendment

2. Where a high-risk AI system related to a product, to which the legal acts listed in Annex II, section A apply, is placed on the market or put into service one single technical documentation shall be drawn up containing all the information set out in Annex IV as well as the information required under those legal acts.

2. Where a high-risk AI system related to a product, to which the legal acts listed in Annex II, section A apply, is placed on the market or put into service appropriate technical documentation shall be drawn up containing all the information set out in Annex IV as well as the information required under those legal acts.

Or. en

Amendment 52

Proposal for a regulation

Article 12 – paragraph 1

Text proposed by the Commission

Amendment

1. High-risk AI systems shall be designed and developed with capabilities enabling the automatic recording of events (‘logs’) while the high-risk AI systems is operating. Those logging capabilities shall conform to recognised standards or common specifications.

1. High-risk AI systems shall be designed and developed with capabilities enabling the technical possibility for recording of events (‘logs’) while the high-risk AI systems is operating. Those logging capabilities shall conform to recognised standards or common specifications.

Or. en

Amendment 53

Proposal for a regulation

Article 12 – paragraph 2

Text proposed by the Commission

Amendment

2. The logging capabilities shall ensure a level of traceability of the AI system’s functioning throughout its lifecycle that is appropriate to the intended purpose of the system.

2. The logging capabilities shall ensure a level of traceability of the AI system’s functioning for a period of time that is appropriate to the intended purpose of the system.

Or. en

Amendment 54

Proposal for a regulation

Article 14 – paragraph 3 – introductory part

Text proposed by the Commission

Amendment

3. Human oversight shall be ensured through either one or all of the following measures:

3. The degree of human oversight shall be adapted to the specific risks, the level of automation, and context of the AI system and shall be ensured through either one or all of the following measures:

Or. en

Amendment 55

Proposal for a regulation

Article 14 – paragraph 4 – point a

Text proposed by the Commission

Amendment

(a) fully understand the capacities and limitations of the high-risk AI system and be able to duly monitor its operation, so that signs of anomalies, dysfunctions and unexpected performance can be detected and addressed as soon as possible;

(a) sufficiently understand the capacities and limitations of the high-risk AI system and be able to duly monitor its operation, so that signs of anomalies, dysfunctions and unexpected performance can be detected and addressed as soon as possible;

Or. en

Justification

To provide legal certainty, to sufficiently understand a system and be able to duly monitor its operation should be enough. Understanding the capacities and limitations of an AI system well enough to be able to monitor for signs of anomalies, dysfunctions and unexpected performance is necessary - whereas fully understanding every detail of the complex machine learning process is not necessarily needed and would not always be possible for companies with limited resources and employees.

Amendment 56

Proposal for a regulation

Article 14 – paragraph 4 a (new)

Text proposed by the Commission

Amendment

4a. As part of its reports, the European Artificial Intelligence Board shall monitor the development of digital skills related to AI technologies across the Union, with a particular focus to the characteristics set out in paragraph 4.

Or. en

Amendment 57

Proposal for a regulation

Article 15 – paragraph 1

Text proposed by the Commission

Amendment

1. High-risk AI systems shall be designed and developed in such a way that they achieve, in the light of their intended purpose, an appropriate level of accuracy, robustness and cybersecurity, and perform consistently in those respects throughout their lifecycle.

1. High-risk AI systems shall be designed and developed in such a way that they achieve, in the light of their intended purpose, an appropriate reasonably expected level of accuracy, robustness and cybersecurity, and perform consistently in those respects throughout their lifecycle.

Or. en

Justification

This addition makes the paragraph in line with existing product safety legislation as market expectations differ regarding software as opposed to hardware products.

Amendment 58

Proposal for a regulation

Article 15 – paragraph 1 a (new)

Text proposed by the Commission

Amendment

1a. To address the technical questions as to how to measure the appropriate levels of accuracy, robustness, or cybersecurity, the European Artificial Intelligence Board shall work to set up a common European authority on benchmarking that brings together national metrology and benchmarking authorities.

Or. en

Justification

While Article 15 states the need for “appropriate levels” of “accuracy, robustness and cybersecurity”, one big gap exists. There are no relevant accuracy metrics on a European level to guide developers and providers of AI. While standardisation organisations exist to establish what the standard should be, benchmarking organisations are needed to establish how these standards should be met and measured. The creation of a common European authority - such as a European Benchmarking Institute or as a subgroup of the European AI Board - would allow for a cohesive European approach to benchmarking and metrics. It should draw inspiration from existing structures such as the “Laboratoire national de métrologie et d'essais (LNE)” in France or the National Institute for Science and Technology’s Software Quality Group on Metrics and Measures in the United States.

Amendment 59

Proposal for a regulation

Article 15 – paragraph 3 – subparagraph 1

Text proposed by the Commission

Amendment

3. High-risk AI systems shall be resilient as regards errors, faults or inconsistencies that may occur within the system or the environment in which the system operates, in particular due to their interaction with natural persons or other systems.

3. Appropriate measures shall be taken to ensure that high-risk AI systems shall be as resilient as possible regarding errors, faults or inconsistencies that may occur within the system or the environment in which the system operates, in particular due to their interaction with natural persons or other systems.

Or. en

Amendment 60

Proposal for a regulation

Article 15 – paragraph 3 – subparagraph 2

Text proposed by the Commission

Amendment

The robustness of high-risk AI systems may be achieved through technical redundancy solutions, which may include backup or fail-safe plans.

The robustness of high-risk AI systems may be achieved through technical redundancy solutions, which may include backup or fail-safe plans by the appropriate provider with input from the user, where necessary.

Or. en

Justification

In certain instances, the user of the AI system will have better knowledge of the reliability and resiliency including back up or fail safe plans. This would give flexibility to the market players to choose how to fulfill this obligation depending on the individual circumstnaces of the AI system, while still keeping the main obligations with the provider.

Amendment 61

Proposal for a regulation

Article 15 – paragraph 3 – subparagraph 3

Text proposed by the Commission

Amendment

High-risk AI systems that continue to learn after being placed on the market or put into service shall be developed in such a way to ensure that possibly biased outputs due to outputs used as an input for future operations (‘feedback loops’) are duly addressed with appropriate mitigation measures.

High-risk AI systems that continue to learn after being placed on the market or put into service shall be developed in such a way to ensure that possibly biased outputs influencing input for future operations (‘feedback loops’) are duly addressed with appropriate mitigation measures.

Or. en

Amendment 62

Proposal for a regulation

Article 15 – paragraph 4 – subparagraph 3

Text proposed by the Commission

Amendment

The technical solutions to address AI specific vulnerabilities shall include, where appropriate, measures to prevent and control for attacks trying to manipulate the training dataset (‘data poisoning’), inputs designed to cause the model to make a mistake (‘adversarial examples’), or model flaws.

The technical solutions to address AI specific vulnerabilities may include, where appropriate, measures to prevent and control for attacks trying to manipulate the training dataset (‘data poisoning’), inputs designed to cause the model to make a mistake (‘adversarial examples’), or model flaws.

Or. en

Amendment 63

Proposal for a regulation

Article 18 – paragraph 1

Text proposed by the Commission

Amendment

1. Providers of high-risk AI systems shall draw up the technical documentation referred to in Article 11 in accordance with Annex IV.

1. Providers of high-risk AI systems shall draw up the technical documentation referred to in Article 11 and the instructions of use referred to in Article 15 in accordance with Annex IV.

Or. en

Amendment 64

Proposal for a regulation

Article 20 – paragraph 1

Text proposed by the Commission

Amendment

1. Providers of high-risk AI systems shall keep the logs automatically generated by their high-risk AI systems, to the extent such logs are under their control by virtue of a contractual arrangement with the user or otherwise by law. The logs shall be kept for a period that is appropriate in the light of the intended purpose of high-risk AI system and applicable legal obligations under Union or national law.

1. Providers of high-risk AI systems shall keep the logs automatically generated by their high-risk AI systems, to the extent such logs are under their control by virtue of a contractual arrangement with the user or otherwise by law. They shall keep them for a period of at least six months, unless provided otherwise in applicable Union or national law.

Or. en

Justification

Introducing data logging requirements for the entire lifecycle of an AI system would lead to extensive amounts of data storage which requires reliable electricity supply (which comes with considerable storage and maintenance costs). Furthermore, the storage of such data in data centres for periods of time longer than needed for the intended purpose would have a detrimental impact on the environment.

Amendment 65

Proposal for a regulation

Article 23 – paragraph 1

Text proposed by the Commission

Amendment

Providers of high-risk AI systems shall, upon request by a national competent authority, provide that authority with all the information and documentation necessary to demonstrate the conformity of the high-risk AI system with the requirements set out in Chapter 2 of this Title, in an official Union language determined by the Member State concerned. Upon a reasoned request from a national competent authority, providers shall also give that authority access to the logs automatically generated by the high-risk AI system, to the extent such logs are under their control by virtue of a contractual arrangement with the user or otherwise by law.

Or. en

Amendment 66

Proposal for a regulation

Article 26 – paragraph 1 – point c a (new)

Text proposed by the Commission

Amendment

(ca) the authorised representative referred to in Article 25 has been appointed by the provider.

Or. en

Amendment 67

Proposal for a regulation

Article 28 – paragraph 1 – point c a (new)

Text proposed by the Commission

Amendment

(ca) they decide on the intended purpose of an AI system, or train or modify an AI system already placed on the market or put into service, in such a way that it becomes a high-risk AI system.

Or. en

Justification

Justification: See proposed changes and justification in New Article 28a.

Amendment 68

Proposal for a regulation

Article 28 – paragraph 2

Text proposed by the Commission

Amendment

2. Where the circumstances referred to in paragraph 1, point (b) or (c), occur, the provider that initially placed the high-risk AI system on the market or put it into service shall no longer be considered a provider for the purposes of this Regulation.

2. Where the circumstances referred to in paragraph 1, point (b), (c) or (ca), occur, the provider, the importer or the user that initially placed the high-risk AI system on the market or put it into service shall no longer be considered a provider for the purposes of this Regulation.

Or. en

Amendment 69

Proposal for a regulation

Article 28 a (new)

Text proposed by the Commission

Amendment

Article 28a

General purpose AI systems

1. The placing on the market, putting into service or use of general purpose AI systems shall not, on its own, make those systems subject to this Regulation.

Providers of general purpose AI systems shall work with users of their products to aid them in fulfilling certain requirements set out in this Regulation by providing technical expertise. The shift from user to provider in Article 28 paragraph1, point (ca), still applies. Such an exchange shall be in full respect of trade secrets and current market indicators and shall have within its scope only those obligations, relating to the technical design and development of the system before an intended purpose is attributed to it. The provider of the general purpose AI system shall register the system in the Union database as referred to in Article 60.

2. Any person who places on the market or puts into service under its own name or trademark or uses a general purpose AI system made available on the market or put into service for an intended purpose that makes it subject to this Regulation shall be considered the provider of the AI system in accordance with this Regulation.

3. Paragraph 2 shall apply, mutatis mutandis, to any person who integrates a general purpose AI system made available on the market, with or without modifying it, into an AI system whose intended purpose makes it subject to this Regulation.

4. This Article shall apply irrespective of whether the general purpose AI system is open source software or not.

5. The Commission and the European Artificial Intelligence Board shall in the process of drawing up of Codes of Conduct, include provisions for general purpose AI systems taking into account the latest technological developments, impacts on start ups and SMEs and the effectiveness of existing regulations surrounding general purpose AI systems. Confidentiality and protection of trade secrets are crucial to this approach.

Or. en

Justification

A provider that puts a general purpose AI system - without an intended purpose - on the market cannot fulfil all of the obligations under Articles 9-15 given the centrality of “intended purpose” to fulfilling those criteria. Therefore, those systems shall not be subject to the provisions of this Regulation. This clarification is essential to allowing European businesses to compete and innovate - rather than stifling off an industry with regulation that has not even fully matured yet. To ensure that consumer-facing users (who become providers in accordance with Article 28 paragraph1(d)), which may be smaller players on the market, have the technical understanding and help needed in fulfilling the criteria set out in this Regulation, the Rapporteur aims to strike a balanced approach. Such an approach encourages the general purpose AI system provider to aid in the fulfilment of some minimal provisions, which can be controlled during the development phase, even without an intended purpose. Keeping in mind concerns over confidentiality, which could exist on both sides of such a contractual relationship, the Rapporteur does not go as far as to mandate such an exchange, but rather allows for an exchange based on market principles and protection of trade secrets. Combined with the other measures for general purpose systems, the Rapporteur believes it is the way to strike the right balance in the value chain responsibility. Finally, keeping a database of general purpose AI systems - with minimal information given that they do not have an intended purpose and therefore cannot fulfil many of these criteria - would allow for public oversight and monitoring of the systems in use on the EU market.

Amendment 70

Proposal for a regulation

Article 29 – paragraph 1 a (new)

Text proposed by the Commission

Amendment

1a. Users shall assign human oversight to natural persons who have the necessary competence, training and authority.

Or. en

Amendment 71

Proposal for a regulation

Article 29 – paragraph 2

Text proposed by the Commission

Amendment

2. The obligations in paragraph 1 are without prejudice to other user obligations under Union or national law and to the user’s discretion in organising its own resources and activities for the purpose of implementing the human oversight measures indicated by the provider.

2. The obligations in paragraphs 1 and 1a are without prejudice to other user obligations under Union or national law and to the user’s discretion in organising its own resources and activities for the purpose of implementing the human oversight measures indicated by the provider.

Or. en

Amendment 72

Proposal for a regulation

Article 29 – paragraph 5 – subparagraph 1

Text proposed by the Commission

Amendment

5. Users of high-risk AI systems shall keep the logs automatically generated by that high-risk AI system, to the extent such logs are under their control. The logs shall be kept for a period that is appropriate in the light of the intended purpose of the high-risk AI system and applicable legal obligations under Union or national law.

5. Users of high-risk AI systems shall keep the logs automatically generated by that high-risk AI system, to the extent such logs are under their control. They shall keep them for a period of at least six months, unless otherwise provided in applicable Union or national law.

Or. en

Amendment 73

Proposal for a regulation

Article 40 – paragraph 1

Text proposed by the Commission

Amendment

High-risk AI systems which are in conformity with harmonised standards or parts thereof the references of which have been published in the Official Journal of the European Union shall be presumed to be in conformity with the requirements set out in Chapter 2 of this Title, to the extent those standards cover those requirements.

For the sake of ensuring a higher level of SMEs representation in the preparation of harmonised standards, the participation of SMEs in the standards development processes shall be facilitated in accordance with Article 6 of Regulation (EU) No 1025/2012.

Or. en

Justification

Research shows that SME participation in standardisation bodies is generally not representative of their percentage in the economy. This increases the likelihood that the standards will be written in a way that would not be feasible and workable for SMEs. Therefore, it is very important to further foster already existing mechanisms to ensure the active participation of SMEs in standards development processes. On top of the already existing measures for SMEs in Article 55, the Rapporteur’s aim is to ensure the proactive involvement of SMEs in the standardisation process as a further supporting measure for smaller players.

Amendment 74

Proposal for a regulation

Article 41 – paragraph 2

Text proposed by the Commission

Amendment

2. The Commission, when preparing the common specifications referred to in paragraph 1, shall gather the views of relevant bodies or expert groups established under relevant sectorial Union law.

2. The Commission, when preparing the common specifications referred to in paragraph 1, shall gather the views of relevant stakeholders, including SMEs and start ups, relevant bodies or expert groups established under relevant sectorial Union law.

Or. en

Amendment 75

Proposal for a regulation

Article 42 – paragraph 2

Text proposed by the Commission

Amendment

2. High-risk AI systems that have been certified or for which a statement of conformity has been issued under a cybersecurity scheme pursuant to Regulation (EU) 2019/881 of the European Parliament and of the Council63 and the references of which have been published in the Official Journal of the European Union shall be presumed to be in compliance with the cybersecurity requirements set out in Article 15 of this Regulation in so far as the cybersecurity certificate or statement of conformity or parts thereof cover those requirements.

2. High-risk AI systems that have been certified or for which a statement of conformity has been issued under a cybersecurity scheme pursuant to Regulation (EU) 2019/881 of the European Parliament and of the Council63 and the references of which have been published in the Official Journal of the European Union shall be presumed to be in compliance with the cybersecurity requirements set out in Article 15 of this Regulation, where applicable in so far as the cybersecurity certificate or statement of conformity or parts thereof cover those requirements.

__________________

63 Regulation (EU) 2019/881 of the European Parliament and of the Council of 17 April 2019 on ENISA (the European Union Agency for Cybersecurity) and on information and communications technology cybersecurity certification and repealing Regulation (EU) No 526/2013 (Cybersecurity Act) (OJ L 151, 7.6.2019, p. 1).

Or. en

Amendment 76

Proposal for a regulation

Article 43 – paragraph 1 – subparagraph 2

Text proposed by the Commission

Amendment

Where, in demonstrating the compliance of a high-risk AI system with the requirements set out in Chapter 2 of this Title, the provider has not applied or has applied only in part harmonised standards referred to in Article 40, or where such harmonised standards do not exist and common specifications referred to in Article 41 are not available, the provider shall follow the conformity assessment procedure set out in Annex VII. Should the provider already have established internal organisation and structures for existing conformity assessments or requirements under other existing rules, the provider may utilise those, or parts of those, existing compliance structures, so long as they also have the capacity and competence needed to fulfil the requirements for the product set out in this Regulation.

Or. en

Justification

If a provider already has established structures for compliance with requirements and assessments from existing regulation (such as GDPR, cybersecurity, or other), the provider should be able to utilise those already existing structures as long as they can also fulfill the requirements for the product set out in this regulation. This would be especially beneficial for SMEs and start-ups, which may lack the resources to keep up with the obligation and compliance costs.

Amendment 77

Proposal for a regulation

Article 43 – paragraph 5

Text proposed by the Commission

Amendment

5. The Commission is empowered to adopt delegated acts in accordance with Article 73 for the purpose of updating Annexes VI and Annex VII in order to introduce elements of the conformity assessment procedures that become necessary in light of technical progress. The Commission shall consult the European Artificial Intelligence Board established in Article 56 as well as all relevant stakeholders.

Or. en

Amendment 78

Proposal for a regulation

Article 43 – paragraph 6

Text proposed by the Commission

Amendment

6. The Commission is empowered to adopt delegated acts to amend paragraphs 1 and 2 in order to subject high-risk AI systems referred to in points 2 to 8 of Annex III to the conformity assessment procedure referred to in Annex VII or parts thereof. The Commission shall adopt such delegated acts taking into account the effectiveness of the conformity assessment procedure based on internal control referred to in Annex VI in preventing or minimizing the risks to health and safety and protection of fundamental rights posed by such systems as well as the availability of adequate capacities and resources among notified bodies.

Or. en

Amendment 79

Proposal for a regulation

Article 52 – paragraph 3 – subparagraph 1

Text proposed by the Commission

Amendment

3. Users of an AI system that generates or manipulates image, audio or video content that appreciably resembles existing persons, objects, places or other entities or events and would falsely appear to a person to be authentic or truthful (‘deep fake’), shall disclose that the content has been artificially generated or manipulated.

3. Users of an AI system that generates or manipulates image, audio or video content that appreciably resembles existing persons, objects, places or other entities or events and would falsely appear to a person to be authentic or truthful (‘deep fake’), shall clearly and explicitly disclose that the content has been artificially generated or manipulated.

Or. en

Justification

Considering the potential harmful effects of deep fakes - such as the rapid spread of disinformation - and the impacts on social trust, the Rapporteur recommends that such disclosures be clear and explicit (not hidden in general terms of business, for example). Deep fakes pose a tremendous danger to democracy and fundamental rights. Social trust is a key ingredient to successful uptake and deployment of AI in different sectors of society.

Amendment 80

Proposal for a regulation

Article 53 – paragraph 1

Text proposed by the Commission

Amendment

1. AI regulatory sandboxes established by one or more Member States competent authorities or the European Data Protection Supervisor shall provide a controlled environment that facilitates the development, testing and validation of innovative AI systems for a limited time before their placement on the market or putting into service pursuant to a specific plan. This shall take place under the direct supervision and guidance by the competent authorities with a view to ensuring compliance with the requirements of this Regulation and, where relevant, other Union and Member States legislation supervised within the sandbox.

1. AI regulatory sandboxes established by the Commission in collaboration with one or more Member States competent authorities or the European Data Protection Supervisor shall provide a controlled environment that facilitates the safe development, testing and validation of innovative AI systems for a limited time before their placement on the market or putting into service pursuant to a specific plan. This shall take place under the direct supervision and guidance of the Commission in collaboration with the competent authorities with a view to ensuring compliance with the requirements of this Regulation and, where relevant, other Union and Member States legislation supervised within the sandbox. The Commission shall play a complementary role, allowing those Member States with demonstrated experience with sandboxing to build on their expertise and, on the other hand, assisting and providing technical understanding and resources to those Member States that seek guidance on the set-up and running of these regulatory sandboxes.

Or. en

Justification

The AI Act does not go far enough in supporting innovation, especially for SMEs and start-ups, in return for all of the obligations that are expected of them. That is why the Rapporteur proposes to strengthen the existing provisions for AI Regulatory Sandboxes. The Rapporteur very much welcomes the inclusion of this Article and takes it a step further by allowing more involvement of the Commission in their set-up and laying out more precisely in a new Annex the guiding principles for the functioning of these sandboxes. It is worth noting that the involvement of the Commission is done in a flexible manner, which would accommodate the extent to which different Member States may seek the technical guidance or expertise of the Commission and to what extent they would wish to include the Commission in the process. However, the principles laid out in the Annex should be the minimal one adopted for sandboxes across the Union, so as not to allow fragmentation of the Digital Single Market. The Rapporteur believes that such an ambitious proposal would allow European companies to continue to innovate - to continue to grow in areas where European companies are strong and to catch up to competitors in areas where needed, while still keeping open markets and competitiveness rather than being protectionist.

Amendment 81

Proposal for a regulation

Article 53 – paragraph 2

Text proposed by the Commission

Amendment

2. Member States shall ensure that to the extent the innovative AI systems involve the processing of personal data or otherwise fall under the supervisory remit of other national authorities or competent authorities providing or supporting access to data, the national data protection authorities and those other national authorities are associated to the operation of the AI regulatory sandbox.

2. The Commission in collaboration with Member States shall ensure that to the extent the innovative AI systems involve the processing of personal data or otherwise fall under the supervisory remit of other national authorities or competent authorities providing or supporting access to data, the national data protection authorities and those other national authorities are associated to the operation of the AI regulatory sandbox.

Or. en

Amendment 82

Proposal for a regulation

Article 53 – paragraph 3

Text proposed by the Commission

Amendment

3. The AI regulatory sandboxes shall not affect the supervisory and corrective powers of the competent authorities. Any significant risks to health and safety and fundamental rights identified during the development and testing of such systems shall result in immediate mitigation and, failing that, in the suspension of the development and testing process until such mitigation takes place.

3. The AI regulatory sandboxes shall not affect the supervisory and corrective powers of the competent authorities. Any significant risks to health and safety and fundamental rights identified during the development and testing of AI systems shall result in immediate mitigation and, failing that, in the suspension of the development and testing process until such mitigation takes place.

Or. en

Amendment 83

Proposal for a regulation

Article 53 – paragraph 5

Text proposed by the Commission

Amendment

5. Member States’ competent authorities that have established AI regulatory sandboxes shall coordinate their activities and cooperate within the framework of the European Artificial Intelligence Board. They shall submit annual reports to the Board and the Commission on the results from the implementation of those scheme, including good practices, lessons learnt and recommendations on their setup and, where relevant, on the application of this Regulation and other Union legislation supervised within the sandbox.

5. The Commission, Member States’ competent authorities that have established AI regulatory sandboxes shall coordinate their activities and cooperate within the framework of the Commission's AI Regulatory Sandboxing programme. The Commission shall submit annual reports to the European Artificial Intelligence Board on the results from the implementation of those schemes, including good practices, lessons learnt and recommendations on their setup and, where relevant, on the application of this Regulation and other Union legislation supervised within the sandbox.

Or. en

Amendment 84

Proposal for a regulation

Article 53 – paragraph 6 a (new)

Text proposed by the Commission

Amendment

6a. The Commission shall establish an EU AI Regulatory Sandboxing Programme whose modalities referred to in Article 53(6) shall cover the elements set out in Annex IXa. The Commission shall proactively coordinate with national and also local authorities, as relevant.

Or. en

Justification

To avoid fragmenting of the Digital Single Market and avoid “cross-border” arbitrage across Member States, the Rapporteur proposes that the regulatory sandboxing programme shall be centrally run by the Commission, while still allowing for the specifities of different Member States regarding liability laws or insurance schemes.

Amendment 85

Proposal for a regulation

Article 55 – title

Text proposed by the Commission

Amendment

Measures for small-scale providers and users

Measures for SME providers, start-ups and users

Or. en

Amendment 86

Proposal for a regulation

Article 55 – paragraph 1 – point a

Text proposed by the Commission

Amendment

(a) provide small-scale providers and start-ups with priority access to the AI regulatory sandboxes to the extent that they fulfil the eligibility conditions;

(a) provide SMEs and start-ups with priority access to the AI regulatory sandboxes to the extent that they fulfil the eligibility conditions;

Or. en

Amendment 87

Proposal for a regulation

Article 55 – paragraph 1 – point b

Text proposed by the Commission

Amendment

(b) organise specific awareness raising activities about the application of this Regulation tailored to the needs of the small-scale providers and users;

(b) organise specific awareness raising and enhanced digital skills development activities about the application of this Regulation tailored to the needs of SME providers, start-ups and users;

Or. en

Amendment 88

Proposal for a regulation

Article 55 – paragraph 1 – point c

Text proposed by the Commission

Amendment

(c) where appropriate, establish a dedicated channel for communication with small-scale providers and user and other innovators to provide guidance and respond to queries about the implementation of this Regulation.

(c) where appropriate, establish a dedicated channel for communication with SME providers, including start-ups, and user and other innovators to provide guidance and respond to queries about the implementation of this Regulation.

Or. en

Amendment 89

Proposal for a regulation

Article 55 – paragraph 1 – point c a (new)

Text proposed by the Commission

Amendment

(ca) foster the participation of SMEs in the standardisation development process;

Or. en

Justification

The Rapporteur believes it is worth addressing the issue of SME involvement in the standardisation development process, so that smaller players can also have a say in the industry standards that will later be expected of them.

Amendment 90

Proposal for a regulation

Article 55 – paragraph 1 – point c b (new)

Text proposed by the Commission

Amendment

(cb) work with the Commission to channel existing funding towards lowering costs of compliance for SME providers and start-ups.

Or. en

Amendment 91

Proposal for a regulation

Article 55 – paragraph 2

Text proposed by the Commission

Amendment

2. The specific interests and needs of the small-scale providers shall be taken into account when setting the fees for conformity assessment under Article 43, reducing those fees proportionately to their size and market size.

2. The specific interests and needs of the SME providers and start-ups shall be taken into account when setting the fees for conformity assessment under Article 43, reducing those fees proportionately to their size, market size and market demand.

The Commission shall regularly assess the certification and compliance costs for SMEs and start-ups, including through consultations with the SME providers, start-ups and users.

Or. en

Justification

According to the Commission’s Impact Assessment, the estimated compliance costs for SMEs that develop or deploy high-risk AI applications are estimated at around 6 000 - 7 0000 EUR. The conformity assessment (for the notified body to monitor compliance with the documentation requirements) is estimated to cost between 3 5000 - 7 500 EUR, which makes for a total of 9 500 - 14 5000 EUR. Costs could be even higher when one takes into account external consultancy, internal costs and auditing costs. Given that we need to hold even smaller players accountable for high-risk AI, the Rapporteur proposes for the EC to continue monitoring and assessing these costs. This would provide policymakers at both the European and national level with the information needed to better inform policymaking towards SMEs.

Amendment 92

Proposal for a regulation

Article 55 – paragraph 2 a (new)

Text proposed by the Commission

Amendment

2a. Where appropriate, Member States shall utilise existing dedicated channels for communication with SMEs and start-ups. Such existing channels could include but are not limited to ENISA’s Computer Security Incident Response Teams, National Data Protection Agencies, the AI-on demand platform, the European Digital Innovation Hubs and the Testing and Experimentation Facilities established by the Commission and the Member States at national or Union level.

Or. en

Amendment 93

Proposal for a regulation

Article 56 – paragraph 1

Text proposed by the Commission

Amendment

1. A ‘European Artificial Intelligence Board’ (the ‘Board’) is established.

1. A ‘European Artificial Intelligence Board’ (the ‘Board’) is established. The Board shall have a strong mandate as well as sufficient resources and skilled personnel at its disposal for assistance in the proper performance of its tasks laid down in Article 58.

Or. en

Amendment 94

Proposal for a regulation

Article 56 – paragraph 2 – point c a (new)

Text proposed by the Commission

Amendment

(ca) consider how the Union can take advantage of the benefits of AI, while appropriately mitigating risks.

Or. en

Amendment 95

Proposal for a regulation

Article 57 – paragraph 1

Text proposed by the Commission

Amendment

1. The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority, and the European Data Protection Supervisor. Other national authorities may be invited to the meetings, where the issues discussed are of relevance for them. Where appropriate, AI developers, data scientists, SMEs and start-ups, and other relevant stakeholders may be invited to meetings of the Board. These AI developers and data scientists shall be selected as outlined in the rules of procedure of the Board.

Or. en

Amendment 96

Proposal for a regulation

Article 57 – paragraph 4

Text proposed by the Commission

Amendment

4. The Board may invite external experts and observers to attend its meetings and may hold exchanges with interested third parties to inform its activities to an appropriate extent. To that end the Commission may facilitate exchanges between the Board and other Union bodies, offices, agencies and advisory groups.

4. The Board shall institutionalise a structured dialogue with the High Level Expert Group on AI. It shall regularly invite external experts and observers to attend its meetings and may hold exchanges with interested third parties to inform its activities to an appropriate extent. To that end the Commission may facilitate exchanges between the Board and other Union bodies, offices, agencies and advisory groups.

Or. en

Amendment 97

Proposal for a regulation

Article 58 – paragraph 1 – point b

Text proposed by the Commission

Amendment

(b) contribute to uniform administrative practices in the Member States, including for the functioning of regulatory sandboxes referred to in Article 53;

(b) contribute to uniform administrative practices in the Member States, including for the functioning of regulatory sandboxes referred to in Article 53, Article 54 and Annex IXa;

Or. en

Amendment 98

Proposal for a regulation

Article 58 – paragraph 1 – point c a (new)

Text proposed by the Commission

Amendment

(ca) carry out annual horizon scanning and foresight exercises to extrapolate the impact trends and emerging issues in respect of this Regulation including a particular focus on digital skills development;

Or. en

Amendment 99

Proposal for a regulation

Article 58 – paragraph 1 – point c b (new)

Text proposed by the Commission

Amendment

(cb) annually publish recommendations to the Commission, including but not limited to opinions on the need for amendment of Annex I and Annex III in light of available evidence;

Or. en

Amendment 100

Proposal for a regulation

Article 58 – paragraph 1 – point c c (new)

Text proposed by the Commission

Amendment

(cc) work to set up a common European authority on benchmarking that brings together national metrology and benchmarking authorities, such as a European Benchmarking Institute or as a subgroup of the European AI Board.

Or. en

Justification

Amendment 101

Proposal for a regulation

Article 60 – paragraph 1

Text proposed by the Commission

Amendment

1. The Commission shall, in collaboration with the Member States, set up and maintain a EU database containing information referred to in paragraph 2 concerning high-risk AI systems referred to in Article 6(2) which are registered in accordance with Article 51.

1. The Commission shall, in collaboration with Member States, set up and maintain an EU database for general purpose AI systems in accordance with Article 28a.

Or. en

Amendment 102

Proposal for a regulation

Article 61 – paragraph 2

Text proposed by the Commission

Amendment

2. The post-market monitoring system shall actively and systematically collect, document and analyse relevant data provided by users or collected through other sources on the performance of high-risk AI systems throughout their lifetime, and allow the provider to evaluate the continuous compliance of AI systems with the requirements set out in Title III, Chapter 2.

2. The post-market monitoring system shall include activities to receive, collect, document and analyse relevant data provided by users under Article 29 or collected through other sources on the performance of high-risk AI systems throughout their lifetime, and allow the provider to evaluate the continuous compliance of AI systems with the requirements set out in Title III, Chapter 2.

Or. en

Amendment 103

Proposal for a regulation

Article 64 – paragraph 1

Text proposed by the Commission

Amendment

1. Access to data and documentation in the context of their activities, the market surveillance authorities shall be granted full access to the training, validation and testing datasets used by the provider, including through application programming interfaces (‘API’) or other appropriate technical means and tools enabling remote access.

1. When appropriately motivated and proportionate, market surveillance authorities may request access to data and documentation in the context of their activities. The market surveillance authorities shall be granted full access to the training, validation and testing datasets used by the provider, including through application programming interfaces (‘API’) or other appropriate technical means and tools enabling remote access, that are strictly necessary for the purpose of its request.

Or. en

Amendment 104

Proposal for a regulation

Article 64 – paragraph 2

Text proposed by the Commission

Amendment

2. Where necessary to assess the conformity of the high-risk AI system with the requirements set out in Title III, Chapter 2 and upon a reasoned request, the market surveillance authorities shall be granted access to the source code of the AI system.

2. Where necessary to assess the conformity of the high-risk AI system, such as in a force majeure situation, with the requirements set out in Title III, Chapter 2 and upon a reasoned request and after all other ways for access are exhausted or shown to be insufficient, the market surveillance authorities shall be granted access to the source code of the AI system. Such access shall be subject to existing Union and national law on intellectual property and trade secrets.

Or. en

Amendment 105

Proposal for a regulation

Article 69 – paragraph 2

Text proposed by the Commission

Amendment

2. The Commission and the Board shall encourage and facilitate the drawing up of codes of conduct intended to foster the voluntary application to AI systems of requirements related for example to environmental sustainability, accessibility for persons with a disability, stakeholders participation in the design and development of the AI systems and diversity of development teams on the basis of clear objectives and key performance indicators to measure the achievement of those objectives.

2. The Codes of Conduct intended to foster the voluntary application to AI systems of requirements shall in particular include factors related to environmental sustainability, accessibility for persons with a disability, stakeholders participation in the design and development of the AI systems and diversity of development teams on the basis of clear objectives and key performance indicators to measure the achievement of those objectives.

Or. en

Amendment 106

Proposal for a regulation

Article 69 – paragraph 2 – subparagraph 1 a (new)

Text proposed by the Commission

Amendment

The Codes of Conduct shall include provisions for general purpose AI systems, as set out in Article 28a, paragraph 5.

Or. en

Amendment 107

Proposal for a regulation

Article 69 – paragraph 4

Text proposed by the Commission

Amendment

4. The Commission and the Board shall take into account the specific interests and needs of the small-scale providers and start-ups when encouraging and facilitating the drawing up of codes of conduct.

4. The Commission and Member States shall take into account the specific interests and needs of SMEs and start-ups when encouraging and facilitating the drawing up of codes of conduct.

Or. en

Amendment 108

Proposal for a regulation

Article 70 – paragraph 1 – point a

Text proposed by the Commission

Amendment

(a) intellectual property rights, and confidential business information or trade secrets of a natural or legal person, including source code, except the cases referred to in Article 5 of Directive 2016/943 on the protection of undisclosed know-how and business information (trade secrets) against their unlawful acquisition, use and disclosure apply.

(a) intellectual property rights, and confidential business information or trade secrets of a natural or legal person in line with the 2016 EU Trade Secrets Directive (Directive (EU) 2016/943) as well as the 2004 Directive on the enforcement of intellectual property rights (Directive 2004/48/EC), including source code, except the cases referred to in Article 5 of Directive (EU) 2016/943 on the protection of undisclosed know-how and business information (trade secrets) against their unlawful acquisition, use and disclosure apply.

Or. en

Amendment 109

Proposal for a regulation

Article 71 – paragraph 1

Text proposed by the Commission

Amendment

1. In compliance with the terms and conditions laid down in this Regulation, Member States shall lay down the rules on penalties, including administrative fines, applicable to infringements of this Regulation and shall take all measures necessary to ensure that they are properly and effectively implemented. The penalties provided for shall be effective, proportionate, and dissuasive. They shall take into particular account the interests of small-scale providers and start-up and their economic viability.

1. In compliance with the terms and conditions laid down in this Regulation, Member States shall lay down the rules on penalties, including administrative fines, applicable to infringements of this Regulation and shall take all measures necessary to ensure that they are properly and effectively implemented. The penalties provided for shall be effective, proportionate, and dissuasive. They shall take into particular account the size and the interests of SME providers, including start-ups and their economic viability.

Or. en

Justification

Penalties need to be proportionate so as not to hamper economic competitiveness and innovation of smaller players.

Amendment 110

Proposal for a regulation

Article 71 – paragraph 3 – point b

Text proposed by the Commission

Amendment

(b) non-compliance of the AI system with the requirements laid down in Article 10.

deleted

Or. en

Justification

There is a difference in the severity of non-compliance between Articles 5 and 10. Therefore, Article 10 should be included in the provisions for penalties in paragraph 4.

Amendment 111

Proposal for a regulation

Article 71 – paragraph 4

Text proposed by the Commission

Amendment

4. The non-compliance of the AI system with any requirements or obligations under this Regulation, other than those laid down in Articles 5 and 10, shall be subject to administrative fines of up to 20 000 000 EUR or, if the offender is a company, up to 4 % of its total worldwide annual turnover for the preceding financial year, whichever is higher.

4. The non-compliance of the AI system with any requirements or obligations under this Regulation, other than those laid down in Article 5, shall be subject to administrative fines of up to 20 000 000 EUR or, if the offender is a company, up to 4 % of its total worldwide annual turnover for the preceding financial year, whichever is higher.

Or. en

Amendment 112

Proposal for a regulation

Article 71 – paragraph 5 – subparagraph 1 a (new)

Text proposed by the Commission

Amendment

In the case of supply of incomplete information by an SME provider or start-up, the Member State shall allow the SME provider or start-up a period of 1 month, unless another period of time is stipulated by national law, to supply the complete information before imposing the fines set out in this paragraph.

Or. en

Amendment 113

Proposal for a regulation

Article 71 – paragraph 6 – point c

Text proposed by the Commission

Amendment

Or. en

Amendment 114

Proposal for a regulation

Article 73 – paragraph 2

Text proposed by the Commission

Amendment

2. The delegation of power referred to in Article 4, Article 7(1), Article 11(3), Article 43(5) and (6) and Article 48(5) shall be conferred on the Commission for an indeterminate period of time from [entering into force of the Regulation].

2. The power to adopt delegated acts referred to in Article 4, Article 7(1), Article 11(3), Article 43(5) and (6) and Article 48(5) shall be conferred on the Commission for an indeterminate period of time from [date of entry into force of this Regulation].

Or. en

Amendment 115

Proposal for a regulation

Article 84 – paragraph 7

Text proposed by the Commission

Amendment

7. The Commission shall, if necessary, submit appropriate proposals to amend this Regulation, in particular taking into account developments in technology and in the light of the state of progress in the information society.

Or. en

Amendment 116

Proposal for a regulation

Annex I – point c

Text proposed by the Commission

Amendment

Or. en

Amendment 117

Proposal for a regulation

Annex III – paragraph 1 – introductory part

Text proposed by the Commission

Amendment

High-risk AI systems pursuant to Article 6(2) are the AI systems listed in any of the following areas:

High-risk AI systems pursuant to Article 6(2) are the AI systems listed in any of the following areas whose use cases or application poses a risk of harm to the health and safety or a risk of adverse impact on fundamental rights of users:

Or. en

Justification

High risks for society are better regulated by the application than the technology. This Annex could be even further specified to applications which have an impact on fundamental rights rather than broad sectoral designations.

Amendment 118

Proposal for a regulation

Annex III – paragraph 1 – point 2 – introductory part

Text proposed by the Commission

Amendment

2. Management and operation of critical infrastructure:

2. Critical infrastructure:

Or. en

Amendment 119

Proposal for a regulation

Annex III – paragraph 1 – point 2 – point a

Text proposed by the Commission

Amendment

(a) AI systems intended to be used as safety components in the management and operation of road traffic and the supply of water, gas, heating and electricity.

(a) AI systems intended to be used as safety components in the management and operation of road traffic and the supply of water, gas, heating and electricity provided the failure of the AI system might lead to an imminent threat to such supply.

Or. en

Amendment 120

Proposal for a regulation

Annex III – paragraph 1 – point 4 – point a

Text proposed by the Commission

Amendment

(a) AI systems intended to be used for recruitment or selection of natural persons, notably for advertising vacancies, screening or filtering applications, evaluating candidates in the course of interviews or tests;

(a) AI systems intended for the sole purpose of recruitment or selection of natural persons, notably for screening or filtering applications, evaluating candidates in the course of interviews or tests;

Or. en

Justification

Classifying all HR applications of AI as high-risk does not recognise the need to differentiate between applications in the area of HR (some of which pose a much higher risk to one’s individual rights). Some businesses use AI systems to help train staff or screen CVs based on very generic job requirements. This is a different risk level than AI systems making final decisions on hiring or firing a person (in which case, of course, the process needs clear safeguards). We need to distinguish based on the application.

Amendment 121

Proposal for a regulation

Annex III – paragraph 1 – point 4 – point b

Text proposed by the Commission

Amendment

(b) AI intended to be used for making decisions on promotion and termination of work-related contractual relationships, for task allocation and for monitoring and evaluating performance and behavior of persons in such relationships.

(b) AI intended to be used for making decisions on promotion and termination of work-related contractual relationships and for monitoring and evaluating performance and behavior of persons in such relationships.

Or. en

Justification

Same justification as above.

Amendment 122

Proposal for a regulation

Annex IV – paragraph 1 – point 2 – point b

Text proposed by the Commission

Amendment

(b) the design specifications of the system, namely the general logic of the AI system and of the algorithms; the key design choices including the rationale and assumptions made, also with regard to persons or groups of persons on which the system is intended to be used; the main classification choices; what the system is designed to optimise for and the relevance of the different parameters; the decisions about any possible trade-off made regarding the technical solutions adopted to comply with the requirements set out in Title III, Chapter 2;

(b) where relevant, proportionate and as long as trade secrets are not affected, the design specifications of the system, namely the general logic of the AI system and of the algorithms; the key design choices including the rationale and assumptions made, also with regard to persons or groups of persons on which the system is intended to be used; the main classification choices; what the system is designed to optimise for and the relevance of the different parameters; the decisions about any possible trade-off made regarding the technical solutions adopted to comply with the requirements set out in Title III, Chapter 2;

Or. en

Amendment 123

Proposal for a regulation

Annex IV – paragraph 1 – point 2 – point d

Text proposed by the Commission

Amendment

(d) where relevant, the data requirements in terms of datasheets describing the training methodologies and techniques and the training data sets used, including information about the provenance of those data sets, their scope and main characteristics; how the data was obtained and selected; labelling procedures (e.g. for supervised learning), data cleaning methodologies (e.g. outliers detection);

(d) where relevant, the data requirements in terms of datasheets describing the training methodologies and techniques and the training data sets used, including a general description of those data sets, information about their provenance, their scope and main characteristics; how the data was obtained and selected; labelling procedures (e.g. for supervised learning), data cleaning methodologies (e.g. outliers detection);

Or. en

Amendment 124

Proposal for a regulation

Annex IV – paragraph 1 – point 5

Text proposed by the Commission

Amendment

5. A description of any change made to the system through its lifecycle;

5. A description of relevant changes made by the providers to the system through its lifecycle;

Or. en

Amendment 125

Proposal for a regulation

Annex VII – point 4 – point 4.5

Text proposed by the Commission

Amendment

4.5. Where necessary to assess the conformity of the high-risk AI system with the requirements set out in Title III, Chapter 2 and upon a reasoned request, the notified body shall also be granted access to the source code of the AI system.

4.5. Where necessary to assess the conformity of the high-risk AI system with the requirements set out in Title III and after all other ways for access are exhausted or shown to be insufficient, Chapter 2 and upon a reasoned request, the notified body shall also be granted access to the source code of the AI system. Such access shall be subject to existing Union and national law on intellectual property and trade secrets.

Or. en

Amendment 126

Proposal for a regulation

Annex IX a (new)

Text proposed by the Commission

Amendment

ANNEX IXa:

MODALITIES FOR AN EU AI REGULATORY SANDBOXING PROGRAMME

1. The AI Regulatory Sandboxes shall be part of the EU AI Regulatory Sandboxing Programme (‘sandboxing programme’) to be established by the Commission in collaboration with Member States.

2. The Commission shall play a complementary role, allowing those Member States with demonstrated experience with sandboxing to build on their expertise and, on the other hand, assisting and providing technical understanding and resources to those Member States that seek guidance on the set-up of these regulatory sandboxes.

3. Participants in the sandboxing programme, in particular small-scale providers, are granted access to pre-deployment services, such as preliminary registration of their AI system, compliance R&D support services, and to all the other relevant elements of the Union’s AI ecosystem and other Digital Single Market initiatives such as Testing & Experimentation Facilities, Digital Hubs, Centres of Excellence, and EU benchmarking capabilities; and to other value-adding services such as standardisation documents and certification, an online social platform for the community, contact databases, existing portal for tenders and grant making and lists of EU investors.

4. The sandboxing programme shall, in a later development phase, develop and manage two types of regulatory sandboxes: Physical Regulatory Sandboxes for AI systems embedded in physical products or services and Cyber Regulatory Sandboxes for AI systems operated and used on a stand-alone basis, not embedded in physical products or services.

5. The sandboxing programme shall work with the already established Digital Innovation Hubs in Member States to provide a dedicated point of contact for entrepreneurs to raise enquiries with competent authorities and to seek non-binding guidance on the conformity of innovative products, services or business models embedding AI technologies.

6. One of the objectives of the sandboxing programme is to enable firms’ compliance with this Regulation at the design stage of the AI system (‘compliance-by-design’). To do so, the programme shall facilitate the development of software tools and infrastructure for testing, benchmarking, assessing and explaining dimensions of AI systems relevant to sandboxes, such as accuracy, robustness and cybersecurity.

7. The sandboxing programme shall be rolled out in a phased fashion, with the various phases launched by the Commission upon success of the previous phase.

8. The sandboxing programme will have a built-in impact assessment procedure to facilitate the review of cost-effectiveness against the agreed-upon objectives. This assessment shall be drafted with input from Member States based on their experiences and shall be included as part of the Annual Report submitted by the Commission to the European Artificial Intelligence Board.

Or. en

Justification

To avoid fragmenting of the Digital Single Market and avoid “cross-border” arbitrage across Member States, the Rapporteur lays out in a new Annex modalities for the EU AI regulatory sandboxing programme, which would be established by the Commission in collaboration with Member States. By laying out more precisely such modalities and guiding principles in a new Annex, the Rapporteur calls for a bold and ambitious EU AI Regulatory Sandboxing programme that rises to the occasion of supporting innovation across the continent. It is worth noting that the involvement of the Commission is done in a flexible manner, which would accommodate the extent to which different Member States may seek the technical guidance or expertise of the Commission and to what extent they would wish to include the Commission in the process. However, the principles laid out in the Annex should be the minimal one adopted for sandboxes across the Union, so as not to allow fragmentation of the Digital Single Market. This measure ensures that participants are incentivized to participate in these regulatory sandboxes. These services, especially on standardisation and certification, would benefit mainly SMEs and start-ups, who do not typically have the ability yet to attribute vast resources to compliance as larger companies do.Developing two types of sandboxes - one for software tools, shared virtual environments and online communications and others requiring physical sites or on-site demonstrations, would allow for a more efficient use of resources and expertise.To promote a successful rollout of these regulatory sandboxes, the authorities should re-evaluate and assess their progress so far before moving to the next phase. Similar to the justification above, conducting assessments is key to finding out whether these sandboxes truly work for the SMEs and start-ups that are using them.