Sittings · Document

OPINION (COM(2021)0206 – C90146/2021 – 2021/0106(COD)) 2022-06-16

On the proposal for a regulation of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and amending certain Union Legislative Acts

Committee on Culture and Education · Rapporteur: Marcel Kolaja

PA_Legam

SHORT JUSTIFICATION

On 24 April 2021, the European Commission published its legislative proposal laying down harmonised rules on artificial intelligence (AI Act), which introduces a regulatory framework with the objective of ensuring that AI systems placed on the European Union market are safe to use and respect fundamental rights and European Union values.

Furthermore, the proposal facilitates development of a single market for lawful, safe and trustworthy AI applications, enhances governance and effective enforcement of existing law on fundamental rights and safety requirements applicable to AI systems and ensures legal certainty to facilitate investment and innovation in AI.

Overall, the Rapporteur welcomes the European Commission’s proposal; however, would like to suggest a few amendments mainly to extend the list of high-risk AI applications in areas of education, media and culture under Annex III and to modify certain provisions related to banned practices under Article 5.

More specifically, the Rapporteur reflects on the increased deployment of AI technologies in education and training facilities. Therefore, he proposes listing, among high-risk technologies, also those AI technologies used for monitoring of students during tests and technologies used to determine an area or a programme a student should study. Regarding media and culture, the Rapporteur suggests listing high-risk AI technologies used to create or disseminate machine-generated news articles used by news media outlets and AI technologies used to recommend or rank audiovisual content.

In addition, the Rapporteur proposes to extend the ban on deployment of social scoring systems to usage by public and private entities given the inherent threat of discrimination and exclusion of certain groups or individuals.

Finally, in light of the danger that deployment of remote biometric identification systems in publicly accessible places poses to citizens’ fundamental rights, freedom of assembly, work of investigative journalists, activists and political representatives, the Rapporteur proposes to ban deployment of such technologies in publicly accessible places following up on the Parliament resolution of 6 October 2021 on “Artificial intelligence in criminal law and its use by the police and judicial authorities in criminal matters”.

AMENDMENTS

The Committee on Culture and Education calls on the Committee on the Internal Market and Consumer Protection and the Committee on Civil Liberties, Justice and Home Affairs, as the committees responsible, to take into account the following amendments:

Amendment 1

Proposal for a regulation

Recital 1

Text proposed by the Commission

Amendment

(1) The purpose of this Regulation is to improve the functioning of the internal market by laying down a uniform legal framework in particular for the development, marketing and use of artificial intelligence in conformity with Union values. This Regulation pursues a number of overriding reasons of public interest, such as a high level of protection of health, safety and fundamental rights, and it ensures the free movement of AI-based goods and services cross-border, thus preventing Member States from imposing restrictions on the development, marketing and use of AI systems, unless explicitly authorised by this Regulation.

(1) The purpose of this Regulation is to improve the functioning of the internal market by laying down a uniform legal framework, based on ethical principles in particular for the development, marketing and use of artificial intelligence in conformity with Union values, minimising any risk of adverse and discriminatory impact on people and without hindering innovation. This Regulation pursues a number of overriding reasons of public interest, such as a high level of protection of health, safety and fundamental rights, democracy, the rule of law and the environment, and it ensures the free movement of AI-based goods and services cross-border, thus preventing Member States from imposing restrictions on the development, marketing and use of AI systems, unless explicitly authorised by this Regulation.

Amendment 2

Proposal for a regulation

Recital 2

Text proposed by the Commission

Amendment

(2) Artificial intelligence systems (AI systems) can be easily deployed in multiple sectors of the economy and society, including cross border, and circulate throughout the Union. Certain Member States have already explored the adoption of national rules to ensure that artificial intelligence is safe and is developed and used in compliance with fundamental rights obligations. Differing national rules may lead to fragmentation of the internal market and decrease legal certainty for operators that develop or use AI systems. A consistent and high level of protection throughout the Union should therefore be ensured, while divergences hampering the free circulation of AI systems and related products and services within the internal market should be prevented, by laying down uniform obligations for operators and guaranteeing the uniform protection of overriding reasons of public interest and of rights of persons throughout the internal market based on Article 114 of the Treaty on the Functioning of the European Union (TFEU). To the extent that this Regulation contains specific rules on the protection of individuals with regard to the processing of personal data concerning restrictions of the use of AI systems for ‘real-time’ remote biometric identification in publicly accessible spaces for the purpose of law enforcement, it is appropriate to base this Regulation, in as far as those specific rules are concerned, on Article 16 of the TFEU. In light of those specific rules and the recourse to Article 16 TFEU, it is appropriate to consult the European Data Protection Board.

(2) Artificial intelligence systems (AI systems) can be easily deployed in multiple sectors of the economy and society, including cross border, and circulate throughout the Union. Certain Member States have already explored the adoption of national rules to ensure that artificial intelligence is trustworthy and safe and is developed and used in compliance with fundamental rights obligations. Differing national rules may lead to fragmentation of the internal market and decrease legal certainty for operators that develop or use AI systems. A consistent and high level of protection throughout the Union should therefore be ensured in order to achieve trustworthy AI, while divergences hampering the free circulation, innovation, deployment and uptake of AI systems and related products and services within the internal market should be prevented, by laying down uniform obligations for operators and guaranteeing the uniform protection of overriding reasons of public interest and of rights of persons throughout the internal market based on Article 114 of the Treaty on the Functioning of the European Union (TFEU). To the extent that this Regulation contains specific rules on the protection of individuals with regard to the processing of personal data concerning restrictions of the use of AI systems for ‘real-time’ remote biometric identification in publicly accessible spaces for the purpose of law enforcement, it is appropriate to base this Regulation, in as far as those specific rules are concerned, on Article 16 of the TFEU. In light of those specific rules and the recourse to Article 16 TFEU, it is appropriate to consult the European Data Protection Board.

Amendment 3

Proposal for a regulation

Recital 3

Text proposed by the Commission

Amendment

(3) Artificial intelligence is a fast evolving family of technologies that can contribute to a wide array of economic and societal benefits across the entire spectrum of industries and social activities. By improving prediction, optimising operations and resource allocation, and personalising digital solutions available for individuals and organisations, the use of artificial intelligence can provide key competitive advantages to companies and support socially and environmentally beneficial outcomes, for example in healthcare, farming, education and training, infrastructure management, energy, transport and logistics, public services, security, justice, resource and energy efficiency, and climate change mitigation and adaptation.

(3) Artificial intelligence is a fast evolving family of technologies that can contribute and is already contributing to a wide array of economic and societal benefits across the entire spectrum of industries and social activities, if developed in accordance with ethical principles. By improving prediction, optimising operations and resource allocation, and personalising digital solutions available for individuals and organisations, the use of artificial intelligence can provide key competitive advantages to companies and support socially and environmentally beneficial outcomes, for example in healthcare, farming, education and training, media, sports, culture infrastructure management, energy, transport and logistics, public services, security, justice, resource and energy efficiency, and climate change mitigation and adaptation.

Amendment 4

Proposal for a regulation

Recital 4

Text proposed by the Commission

Amendment

(4) At the same time, depending on the circumstances regarding its specific application and use, artificial intelligence may generate risks and cause harm to public interests and rights, that are protected by Union law including fundamental rights of workers, people in learning processes and socially engaged people, privacy, data protection and informational self-determination, societal or environmental rights. Such harm might be material or immaterial.

Amendment 5

Proposal for a regulation

Recital 5

Text proposed by the Commission

Amendment

(5) A Union legal framework laying down harmonised rules on artificial intelligence is therefore needed to foster the development, use and uptake of artificial intelligence in the internal market that at the same time meets a high level of protection of public interests, such as health and safety and the protection of fundamental rights, as recognised and protected by Union law. To achieve that objective, rules regulating the placing on the market and putting into service of certain AI systems should be laid down, thus ensuring the smooth functioning of the internal market and allowing those systems to benefit from the principle of free movement of goods and services. By laying down those rules, this Regulation supports the objective of the Union of being a global leader in the development of secure, trustworthy and ethical artificial intelligence, as stated by the European Council33 , and it ensures the protection of ethical principles, as specifically requested by the European Parliament34 .

(5) A Union legal framework laying down harmonised rules on artificial intelligence is therefore needed to foster the development, use and uptake of artificial intelligence in the internal market that at the same time meets a high level of protection of public interests, such as health and safety and the protection of fundamental rights, as recognised and protected by Union law, democracy, the rule of law and of the environment. To achieve that objective, rules regulating the placing on the market and putting into service of certain AI systems should be laid down, thus ensuring the smooth functioning of the internal market and allowing those systems to benefit from the principle of free movement of goods and services. By laying down those rules, this Regulation supports the objective of the Union of being a global leader in the development of secure, trustworthy and ethical artificial intelligence based on fundamental rights, as stated by the European Council33, and it ensures the protection of ethical principles, as specifically requested by the European Parliament34, with a human-centric approach and in compliance with freedom of expression, freedom of speech, media freedom, pluralism and diversity.

__________________

33 European Council, Special meeting of the European Council (1 and 2 October 2020) – Conclusions, EUCO 13/20, 2020, p. 6.

34 European Parliament resolution of 20 October 2020 with recommendations to the Commission on a framework of ethical aspects of artificial intelligence, robotics and related technologies, 2020/2012(INL).

Amendment 6

Proposal for a regulation

Recital 5 a (new)

Text proposed by the Commission

Amendment

(5a) In order to help promote the development, uptake and understanding of AI, the Union needs to put further effort into education and training, thus, inter alia, addressing the shortage of ICT professionals and AI undergraduate courses, digitally skilled workers as well as lack of even basic digital skills amongst a significant share of the population of the Union.

Amendment 7

Proposal for a regulation

Recital 5 b (new)

Text proposed by the Commission

Amendment

(5b) Lack of both public and private investment is currently undermining development and use of AI systems across the Union, especially when compared to other major industrial economies. Special attention, incentives and support should be devised to promote AI uptake amongst SMEs, including those in education and cultural and creative sectors and industries.

Amendment 8

Proposal for a regulation

Recital 9

Text proposed by the Commission

Amendment

(9) For the purposes of this Regulation the notion of publicly accessible space should be understood as referring to any physical place that is accessible to the public, irrespective of whether the place in question is privately or publicly owned. Therefore, the notion does not cover places that are private in nature and normally not freely accessible for third parties, including law enforcement authorities, unless those parties have been specifically invited or authorised, such as homes, private clubs, offices, warehouses and factories. Online spaces are not covered either, as they are not physical spaces. However, the mere fact that certain conditions for accessing a particular space may apply, such as admission tickets or age restrictions, does not mean that the space is not publicly accessible within the meaning of this Regulation. Consequently, in addition to public spaces such as streets, relevant parts of government buildings and most transport infrastructure, spaces such as cinemas, theatres, shops and shopping centres are normally also publicly accessible. Whether a given space is accessible to the public should however be determined on a case-by-case basis, having regard to the specificities of the individual situation at hand.

(9) For the purposes of this Regulation the notion of publicly accessible space should be understood as referring to any physical or virtual place that is accessible to the public, irrespective of whether the place in question is privately or publicly owned. Therefore, the notion does not cover places that are private in nature and normally not freely accessible for third parties, including law enforcement authorities, unless those parties have been specifically invited or authorised, such as homes, private clubs, offices, warehouses and factories, and other private spaces. The same principle should apply to protected virtual publicly accessible spaces. However, the mere fact that certain conditions for accessing a particular space may apply, such as admission tickets or age restrictions, does not mean that the space is not publicly accessible within the meaning of this Regulation. Consequently, in addition to public spaces such as streets, parks, sport complexes relevant parts of government buildings and most transport infrastructure, spaces such as cinemas, theatres, shops, museums, libraries monuments, cultural sites, cultural institutions and shopping centres are normally also publicly accessible. Whether a given space is accessible to the public should however be determined on a case-by-case basis, having regard to the specificities of the individual situation at hand.

Amendment 9

Proposal for a regulation

Recital 13

Text proposed by the Commission

Amendment

(13) In order to ensure a consistent and high level of protection of public interests as regards health, safety and fundamental rights, common normative standards for all high-risk AI systems should be established. Those standards should be consistent with the Charter of fundamental rights of the European Union (the Charter) and should be non-discriminatory and in line with the Union’s international trade commitments.

(13) In order to ensure a consistent and high level of protection of public interests as regards health, safety, fundamental rights, democracy, the rule of law, as well as the environment, a set of ethical principles and common normative standards for all high-risk AI systems should be established. Those standards should be consistent with the Charter of fundamental rights of the European Union (the Charter), the communication of the Commission of 11 December 2019 entitled ‘The European Green Deal’ and the European Declaration of 26 January 2022 on Digital Rights and Principles for the Digital Decade and should be non-discriminatory and in line with the Union’s international trade commitments.

Amendment 10

Proposal for a regulation

Recital 14 b (new)

Text proposed by the Commission

Amendment

(14b) AI literacy refers to skills, knowledge and understanding that allows both citizens and operators in the context of the obligations set out in this Regulation to make an informed deployment and use of AI systems, as well as to gain awareness about the opportunities and risks of AI and thereby promote its democratic control. AI literacy should not be limited to learning about tools and technologies, but should also aim to equip citizens and operators in the context of the obligations set out in this Regulation with the critical thinking skills required to identify harmful or manipulative uses, as well as to improve their agency and their ability to fully comply with and benefit from trustworthy AI. It is therefore necessary that the Commission, the Member States and operators of AI systems, in cooperation with all relevant stakeholders, promote the development of AI literacy, in all sectors of society, for citizens of all ages, including women and girls, and that progress in that regard is closely followed.

Amendment 11

Proposal for a regulation

Recital 15

Text proposed by the Commission

Amendment

(15) Aside from the many beneficial uses of artificial intelligence, that technology can also be misused and provide novel and powerful tools for manipulative, exploitative and social control practices. Such practices are particularly harmful and should be prohibited because they contradict Union values of respect for human dignity, freedom, equality, democracy and the rule of law and Union fundamental rights, including the right to non-discrimination, data protection and privacy and the rights of the child.

(15) Aside from the many beneficial uses of artificial intelligence, that technology can also be misused and provide novel and powerful tools for manipulative, exploitative and social control practices. Such practices are particularly harmful and should be prohibited because they contradict Union values of respect for human dignity, freedom, equality, democracy and the rule of law and Union fundamental rights, including the right to non-discrimination, the protection of employees and workers, data protection and privacy and gender equality and the rights of the child.

Amendment 12Proposal for a regulation

Recital 27

Text proposed by the Commission

Amendment

(27) High-risk AI systems should only be placed on the Union market or put into service if they comply with certain mandatory requirements. Those requirements should ensure that high-risk AI systems available in the Union or whose output is otherwise used in the Union do not pose unacceptable risks to important Union public interests as recognised and protected by Union law. AI systems identified as high-risk should be limited to those that have a significant harmful impact on the health, safety and fundamental rights of persons in the Union and such limitation minimises any potential restriction to international trade, if any.

(27) High-risk AI systems should only be placed on the Union market or put into service if they comply with certain mandatory requirements. Those requirements should ensure that high-risk AI systems available in the Union or whose output is otherwise used in the Union do not pose unacceptable risks to important Union public interests as recognised and protected by Union law. AI systems identified as high-risk should be limited to those that have a harmful impact on the health, safety and fundamental rights of persons in the Union, as well as on society and on the environment, and such limitation minimises any potential restriction to international trade, if any.

Amendment 13

Proposal for a regulation

Recital 28

Text proposed by the Commission

Amendment

(28) AI systems could produce adverse outcomes to health and safety of persons, in particular when such systems operate as components of products. Consistently with the objectives of Union harmonisation legislation to facilitate the free movement of products in the internal market and to ensure that only safe and otherwise compliant products find their way into the market, it is important that the safety risks that may be generated by a product as a whole due to its digital components, including AI systems, are duly prevented and mitigated. For instance, increasingly autonomous robots, whether in the context of manufacturing or personal assistance and care should be able to safely operate and performs their functions in complex environments. Similarly, in the health sector where the stakes for life and health are particularly high, increasingly sophisticated diagnostics systems and systems supporting human decisions should be reliable and accurate. The extent of the adverse impact caused by the AI system on the fundamental rights protected by the Charter is of particular relevance when classifying an AI system as high-risk. Those rights include the right to human dignity, respect for private and family life, protection of personal data, freedom of expression and information, freedom of assembly and of association, and non-discrimination, consumer protection, workers’ rights, rights of persons with disabilities, right to an effective remedy and to a fair trial, right of defence and the presumption of innocence, right to good administration. In addition to those rights, it is important to highlight that children have specific rights as enshrined in Article 24 of the EU Charter and in the United Nations Convention on the Rights of the Child (further elaborated in the UNCRC General Comment No. 25 as regards the digital environment), both of which require consideration of the children’s vulnerabilities and provision of such protection and care as necessary for their well-being. The fundamental right to a high level of environmental protection enshrined in the Charter and implemented in Union policies should also be considered when assessing the severity of the harm that an AI system can cause, including in relation to the health and safety of persons.

(28) AI systems could produce adverse outcomes to health and safety of persons, in particular when such systems operate as components of products. Consistently with the objectives of Union harmonisation legislation to facilitate the free movement of products in the internal market and to ensure that only safe and otherwise compliant products find their way into the market, it is important that the safety risks that may be generated by a product as a whole due to its digital components, including AI systems, are duly prevented and mitigated. For instance, increasingly autonomous robots, whether in the context of manufacturing or personal assistance and care should be able to safely operate and performs their functions in complex environments. Similarly, in the health sector where the stakes for life and health are particularly high, increasingly sophisticated diagnostics systems and systems supporting human decisions should be reliable and accurate. The extent of the adverse impact caused by the AI system on the fundamental rights protected by the Charter is of particular relevance when classifying an AI system as high-risk. Those rights include the right to human dignity, respect for private and family life, protection of personal data, freedom of expression and information, freedom of assembly and of association, and non-discrimination, right to education, consumer protection, workers’ rights. Special attention should be paid to gender equality, rights of persons with disabilities, right to an effective remedy and to a fair trial, right of defence and the presumption of innocence, right to good administration, protection of intellectual property rights and ensuring cultural diversity. In addition to those rights, it is important to highlight that children have specific rights as enshrined in Article 24 of the EU Charter and in the United Nations Convention on the Rights of the Child (further elaborated in the UNCRC General Comment No. 25 as regards the digital environment), both of which require consideration of the children’s vulnerabilities and provision of such protection and care as necessary for their well-being. The fundamental right to a high level of environmental protection enshrined in the Charter and implemented in Union policies should also be considered when assessing the harm that an AI system can cause, including in relation to the health and safety of persons or to the environment, taking into account the extraction and consumption of natural resources, waste and the carbon footprint of those AI systems.

Amendment 14

Proposal for a regulation

Recital 33

Text proposed by the Commission

Amendment

(33) Technical inaccuracies of AI systems intended for the remote biometric identification of natural persons can lead to biased results and entail discriminatory effects. This is particularly relevant when it comes to age, ethnicity, sex or disabilities. Therefore, ‘real-time’ and ‘post’ remote biometric identification systems should be classified as high-risk. In view of the risks that they pose, both types of remote biometric identification systems should be subject to specific requirements on logging capabilities and human oversight.

(33) Technical inaccuracies of AI systems intended for the biometric identification of natural persons can lead to biased results and entail discriminatory effects. This is particularly relevant when it comes to age, ethnicity, sex or disabilities. Therefore, ‘real-time’ and ‘post’ remote biometric identification systems should be classified as high-risk. In view of the risks that they pose, both types of biometric identification systems should be subject to specific requirements on logging capabilities and human oversight. Non-remote biometric identification systems intended to be used in publicly accessible spaces, workplaces and education and training institutions can also present a high risk. The high risk of non-remote biometric identification systems intended to be used in publicly accessible spaces, workplaces and education and training institutions should be determined on a case-by-case basis.

Amendment 15

Proposal for a regulation

Recital 34 a (new)

Text proposed by the Commission

Amendment

(34a) Deployment of AI systems in education is crucial in order to help modernise entire education systems, to increase educational quality, both offline and online, and to accelerate digital education, thus also making it available to a broader audience. AI-aided digital education, whilst not a replacement for an in-person learning, is increasingly necessary to promote societal and economic growth, promote inclusiveness and increase educational attainment and accessibility to individuals.

Amendment 16

Proposal for a regulation

Recital 35

Text proposed by the Commission

Amendment

(35) AI systems used in education or vocational training, notably for determining access or assigning persons to educational and vocational training institutions or to evaluate persons on tests as part of or as a precondition for their education should be considered high-risk, since they may determine the educational and professional course of a person’s life and therefore affect their ability to secure their livelihood. When improperly designed and used, such systems may violate the right to education and training as well as the right not to be discriminated against and perpetuate historical patterns of discrimination.

(35) AI systems used in education or training, notably for determining access or assigning persons to educational and training institutions to evaluate persons on tests as part of or as a precondition for their education or for determining the areas of study a student should follow should be considered high-risk, since they may determine the educational and professional course of a person’s life and therefore affect their ability to secure their livelihood. When improperly designed, developed and used, such systems may violate the right to education and training as well as the right not to be discriminated against and perpetuate historical patterns of discrimination, for example against women, certain age groups, persons with disabilities, or persons of certain racial or ethnic origins or sexual orientation. AI systems used to monitor students’ behaviour and emotion during tests at education and training institutions should be considered high-risk, since they are also interfering with students’ rights to privacy and data protection. The use of AI to check assessments, such as exam papers for plagiarism, should not be considered high-risk.

Amendment 17

Proposal for a regulation

Recital 36

Text proposed by the Commission

Amendment

(36) AI systems used in employment, workers management and access to self-employment, notably for the recruitment and selection of persons, for making decisions on promotion and termination and for task allocation, monitoring or evaluation of persons in work-related contractual relationships, should also be classified as high-risk, since those systems may appreciably impact future career prospects and livelihoods of these persons. Relevant work-related contractual relationships should involve employees and persons providing services through platforms as referred to in the Commission Work Programme 2021. Such persons should in principle not be considered users within the meaning of this Regulation. Throughout the recruitment process and in the evaluation, promotion, or retention of persons in work-related contractual relationships, such systems may perpetuate historical patterns of discrimination, for example against women, certain age groups, persons with disabilities, or persons of certain racial or ethnic origins or sexual orientation. AI systems used to monitor the performance and behaviour of these persons may also impact their rights to data protection and privacy.

(36) AI systems used in employment, employment support workers management and access to self-employment, notably for the recruitment and selection of persons, for making decisions on promotion and termination and for task allocation, for monitoring compliance with workplace rules and for monitoring or evaluation of persons in work-related relationships, should also be classified as high-risk, since those systems may appreciably impact the health, safety and security rules applicable in their work and at their workplaces and future career prospects and livelihoods of these persons. Relevant work-related relationships should involve employees and persons providing services through platforms as referred to in the Commission Work Programme 2021. Such persons should in principle not be considered users within the meaning of this Regulation. Throughout the recruitment process and in the evaluation, promotion, or retention of persons in work-related relationships, such systems may perpetuate historical patterns of discrimination, for example against women, certain age groups, persons with disabilities, or persons of certain racial or ethnic origins or sexual orientation. AI systems used to monitor the performance and behaviour of these persons may also impact their rights to data protection and privacy. In this regard, specific requirements on transparency, information and human oversight should apply. Trade unions and workers’ representatives should be informed and they should have access to any relevant documentation created under this Regulation for high-risk AI systems deployed or used in their work or at their workplace.

Amendment 18

Proposal for a regulation

Recital 70

Text proposed by the Commission

Amendment

(70) Certain AI systems intended to interact with natural persons or to generate content may pose specific risks of impersonation or deception irrespective of whether they qualify as high-risk or not. In certain circumstances, the use of these systems should therefore be subject to specific transparency obligations without prejudice to the requirements and obligations for high-risk AI systems. In particular, natural persons should be notified that they are interacting with an AI system, unless this is obvious from the circumstances and the context of use. Moreover, natural persons should be notified when they are exposed to an emotion recognition system or a biometric categorisation system. Such information and notifications should be provided in accessible formats for persons with disabilities. Further, users, who use an AI system to generate or manipulate image, audio or video content that appreciably resembles existing persons, places or events and would falsely appear to a person to be authentic, should disclose that the content has been artificially created or manipulated by labelling the artificial intelligence output accordingly and disclosing its artificial origin.

(70) Certain AI systems used to interact with natural persons or to generate content may pose specific risks of impersonation or deception irrespective of whether they qualify as high-risk or not. In certain circumstances, the use of these systems should therefore be subject to specific transparency obligations without prejudice to the requirements and obligations for high-risk AI systems. In particular, natural persons should be notified that they are interacting with an AI system, unless this is obvious from the circumstances and the context of use or where the content is evidently used to form part of a creative, artistic or fictional cinematographic work. Moreover, natural persons should be notified when they are exposed to an emotion recognition system or a biometric categorisation system. Such information and notifications should include a disclaimer and should be provided inaccessible formats for children, the elderly, migrants and persons with disabilities or other vulnerabilities. Further, users, who use an AI system to generate or manipulate image, audio or video content, texts or scripts that appreciably resembles existing persons, places or events and would falsely appear to a person to be authentic, should disclose in a clear manner that the content has been artificially created or manipulated by labelling the artificial intelligence output accordingly and disclosing its artificial origin.

Amendment 19

Proposal for a regulation

Recital 73

Text proposed by the Commission

Amendment

(73) In order to promote and protect innovation, it is important that the interests of small-scale providers and users of AI systems are taken into particular account. To this objective, Member States should develop initiatives, which are targeted at those operators, including on awareness raising and information communication. Moreover, the specific interests and needs of small-scale providers shall be taken into account when Notified Bodies set conformity assessment fees. Translation costs related to mandatory documentation and communication with authorities may constitute a significant cost for providers and other operators, notably those of a smaller scale. Member States should possibly ensure that one of the languages determined and accepted by them for relevant providers’ documentation and for communication with operators is one which is broadly understood by the largest possible number of cross-border users.

(73) In order to promote and protect innovation, it is important that the interests of small-scale providers and users of AI systems are taken into particular account. To this objective, Member States should develop initiatives, which are targeted at those operators, including on AI literacy, awareness raising and information communication. Moreover, the specific interests and needs of small-scale providers shall be taken into account when Notified Bodies set conformity assessment fees. Translation costs related to mandatory documentation and communication with authorities may constitute a significant cost for providers and other operators, notably those of a smaller scale. Member States should possibly ensure that one of the languages determined and accepted by them for relevant providers’ documentation and for communication with operators is one which is broadly understood by the largest possible number of cross-border users.

Amendment 20

Proposal for a regulation

Recital 74

Text proposed by the Commission

Amendment

(74) In order to minimise the risks to implementation resulting from lack of knowledge and expertise in the market as well as to facilitate compliance of providers and notified bodies with their obligations under this Regulation, the AI-on demand platform, the European Digital Innovation Hubs and the Testing and Experimentation Facilities established by the Commission and the Member States at national or EU level should possibly contribute to the implementation of this Regulation. Within their respective mission and fields of competence, they may provide in particular technical and scientific support to providers and notified bodies. It is necessary for the Commission to also create a pan-European network of universities and researchers focused on AI for enhanced studying and research on the impact of AI and to update the Digital Education Action Plan established in the communication of the Commission of 30 September 2020 entitled ‘Digital Education Action Plan 2021-2027 – Resetting education and training for the digital age’, in order to integrate AI and robotics innovation in education.

Amendment 21

Proposal for a regulation

Recital 76

Text proposed by the Commission

Amendment

(76) In order to facilitate a smooth, effective and harmonised implementation of this Regulation a European Artificial Intelligence Board should be established. The Board should be responsible for a number of advisory tasks, including issuing opinions, recommendations, advice or guidance on matters related to the implementation of this Regulation, including on technical specifications or existing standards regarding the requirements established in this Regulation and providing advice to and assisting the Commission on specific questions related to artificial intelligence.

(76) In order to facilitate a smooth, effective and harmonised implementation of this Regulation a European Artificial Intelligence Board should be established. The Board should be responsible for a number of advisory tasks, including issuing opinions, recommendations, advice or guidance on matters related to the implementation of this Regulation, including on technical specifications or existing standards regarding the requirements established in this Regulation and providing expert advice to and assisting the Commission on specific questions related to artificial intelligence and to addressing the challenges rising from the fast evolving development of AI technologies.

Amendment 22

Proposal for a regulation

Recital 83

Text proposed by the Commission

Amendment

(83) In order to ensure trustful and constructive cooperation of competent authorities on Union and national level, all parties involved in the application of this Regulation should respect the confidentiality of information and data obtained in carrying out their tasks. It is appropriate for a new set of common European guidelines and standards to be set up in order to protect privacy while making an effective use of the data available.

Amendment 23

Proposal for a regulation

Recital 85

Text proposed by the Commission

Amendment

(85) In order to ensure that the regulatory framework can be adapted where necessary, the power to adopt acts in accordance with Article 290 TFEU should be delegated to the Commission to amend the techniques and approaches referred to in Annex I to define AI systems, the Union harmonisation legislation listed in Annex II, the high-risk AI systems listed in Annex III, the provisions regarding technical documentation listed in Annex IV, the content of the EU declaration of conformity in Annex V, the provisions regarding the conformity assessment procedures in Annex VI and VII and the provisions establishing the high-risk AI systems to which the conformity assessment procedure based on assessment of the quality management system and assessment of the technical documentation should apply. It is of particular importance that the Commission carry out appropriate consultations during its preparatory work, including at expert level, and that those consultations be conducted in accordance with the principles laid down in the Interinstitutional Agreement of 13 April 2016 on Better Law-Making58 . In particular, to ensure equal participation in the preparation of delegated acts, the European Parliament and the Council receive all documents at the same time as Member States’ experts, and their experts systematically have access to meetings of Commission expert groups dealing with the preparation of delegated acts.

(85) In order to ensure that the regulatory framework can be adapted where necessary, the power to adopt acts in accordance with Article 290 TFEU should be delegated to the Commission to amend the techniques and approaches referred to in Annex I to define AI systems, the Union harmonisation legislation listed in Annex II, the high-risk AI systems listed in Annex III, the provisions regarding technical documentation listed in Annex IV, the content of the EU declaration of conformity in Annex V, the provisions regarding the conformity assessment procedures in Annex VI and VII and the provisions establishing the high-risk AI systems to which the conformity assessment procedure based on assessment of the quality management system and assessment of the technical documentation should apply. It is of particular importance that the Commission carry out appropriate consultations during its preparatory work, including at expert level, and that those consultations be conducted in accordance with the principles laid down in the Interinstitutional Agreement of 13 April 2016 on Better Law-Making58. Such consultations should involve qualified specialists from different areas of society including from the private sector, researchers and scientists, education, media and culture representatives, trade unions, consumer, parental and data protection organizations with skills and knowledge relevant to the task. In particular, to ensure equal participation in the preparation of delegated acts, the European Parliament and the Council receive all documents at the same time as Member States’ experts, and their experts systematically have access to meetings of Commission expert groups dealing with the preparation of delegated acts.

__________________

58 OJ L 123, 12.5.2016, p. 1.

Amendment 24

Proposal for a regulation

Recital 86 a (new)

Text proposed by the Commission

Amendment

(86a) Given the rapid technological developments and the required technical expertise in conducting the assessment of high-risk AI systems, the delegation of powers and the implementing powers of the Commission should be exercised with as much flexibility as possible. The Commission should regularly review Annex III without undue delay, at least every six months, while consulting with the relevant stakeholders, including ethics experts, anthropologists, sociologists, mental health specialists and any other relevant scientists and researchers, as well as with parent associations.

Amendment 25

Proposal for a regulation

Article 1 – paragraph 1 – point c

Text proposed by the Commission

Amendment

(c) harmonised transparency rules for AI systems intended to interact with natural persons, emotion recognition systems and biometric categorisation systems, and AI systems used to generate or manipulate image, audio or video content;

Amendment 26

Proposal for a regulation

Article 2 – paragraph 4 a (new)

Text proposed by the Commission

Amendment

4a. This Regulation shall not affect or undermine academic research or development of AI systems and their outputs for the purpose of academic research.

Amendment 27

Proposal for a regulation

Article 3 – paragraph 1 – point 4

Text proposed by the Commission

Amendment

(4) ‘user’ means any natural or legal person, public authority, agency or other body using an AI system under its authority, except where the AI system is used in the course of a personal non-professional activity;

(4) ‘user’ means any natural or legal person, public authority, educational and training institution, agency or other body using an AI system under its authority, except where the AI system is used in the course of a personal non-professional activity;

Amendment 28

Proposal for a regulation

Article 3 – paragraph 1 – point 35

Text proposed by the Commission

Amendment

(35) ‘biometric categorisation system’ means an AI system for the purpose of assigning natural persons to specific categories, such as sex, age, hair colour, eye colour, tattoos, ethnic origin or sexual or political orientation, on the basis of their biometric data;

(35) ‘biometric categorisation system’ means an AI system that uses biometric data, or other physical, physiological or behavioural data, for the purpose of assigning natural persons to specific categories, such as sex, age, hair colour, eye colour, tattoos, ethnic origin or sexual or political orientation, on the basis of their biometric data;

Amendment 29

Proposal for a regulation

Article 3 – paragraph 1 – point 39

Text proposed by the Commission

Amendment

(39) ‘publicly accessible space’ means any physical place accessible to the public, regardless of whether certain conditions for access may apply;

(39) ‘publicly accessible space’ means any place accessible to the public, regardless of whether certain conditions for access may apply;

Amendment 30

Proposal for a regulation

Article 3 – paragraph 1 – point 44 – point a

Text proposed by the Commission

Amendment

(a) the death of a person or serious damage to a person’s health, to property or the environment,

(a) the death of a person or serious damage to a person’s fundamental rights, health, to property or the environment, to democracy or the democratic rule of law,

Amendment 31

Proposal for a regulation

Article 3 – paragraph 1 – point 44 a (new)

Text proposed by the Commission

Amendment

(44a) ‘education and training institutions’ means providers of education and training, irrespective of the age of the persons receiving the education and training, including preschools, childcare, primary schools, secondary schools, tertiary education providers, vocational education and training and any type of lifelong learning providers;

Amendment 32

Proposal for a regulation

Article 3 – paragraph 1 – point 44 b (new)

Text proposed by the Commission

Amendment

(44b) ‘cultural institutions’ means institutions such as libraries, museums, theatres, concert halls, exhibition centres, architectural ensembles and multi-purpose arts venues, as well as their virtual sections, which organise cultural education, democratic exchanges and research and provide ways and means of engaging with cultural heritage;

Amendment 33

Proposal for a regulation

Article 3 – paragraph 1 – point 44 c (new)

Text proposed by the Commission

Amendment

(44c) 'deep fake' means manipulated or synthetic audio, visual or audiovisual content, text or scripts which feature persons purported to be authentic and truthful;

Amendment 34

Proposal for a regulation

Article 3 – paragraph 1 – point 44 d (new)

Text proposed by the Commission

Amendment

(44d) 'AI literacy' means the skills, knowledge and understanding regarding AI systems.

Amendment 35

Proposal for a regulation

Article 4 a (new)

Text proposed by the Commission

Amendment

Article 4a

Trustworthy AI

1. All AI systems in the Union shall be developed, deployed and used in full respect of the Charter of Fundamental Rights of the European Union (the Charter).

2. In view of promoting trustworthy AI in the Union, and without prejudice to the requirements set out in Title III for high-risk AI systems, all AI systems shall be developed, deployed and used:

(a) in a lawful, fair and transparent manner (‘the principle of lawfulness, fairness and transparency’);

(b) in a manner that ensures that natural persons are always able to make informed decisions regarding such systems and that such systems do not undermine or override human autonomy (‘the principle of human agency and oversight’);

(c) in a manner that ensures their safe, accurate and reliable performance, with embedded safeguards to prevent any kind of individual or collective harm (‘the principle of safety, accuracy, reliability and robustness’);

(d) in a manner that guarantees privacy and data protection (‘the principle of privacy’);

(e) in a manner that privileges the integrity and quality of data, including with regard to access (‘the principle of data governance’);

(f) in a traceable, auditable and explainable manner that ensures responsibility and accountability for their outcomes and supports redress (‘the principle of traceability, auditability, explainability and accountability’);

(g) in a manner that does not discriminate against persons or groups of persons on the basis of unfair bias and that includes, to that end, the participation and input of relevant stakeholders(‘the principle of non-discrimination and diversity’);

(h) in an environmentally sustainable manner that minimises their environmental footprint, including with regard to the extraction and consumption of natural resources (‘the principle of environmental sustainability’);

(i) in a socially responsible manner that minimises their negative societal impact, especially with regard to social and gender inequalities and democratic processes (‘the principle of social responsibility’).

Amendment 36

Proposal for a regulation

Article 4 b (new)

Text proposed by the Commission

Amendment

Article 4b

AI literacy

1. When implementing this Regulation, the Union and the Member States shall promote measures and tools for the development of a sufficient level of AI literacy, across sectors and groups of operators concerned, including through education and training, skilling and reskilling programmes and while ensuring a proper gender and age balance, in view of allowing a democratic control of AI systems.

2. Providers and users of AI systems shall promote tools and shall take measures to ensure a sufficient level of AI literacy of their staff and any other persons dealing with the operation and use of AI systems on their behalf, taking into account their technical knowledge, experience, education and training and the environment in which the AI systems are to be used, and considering the persons or groups of persons on which the AI systems are to be used.

3. Such literacy tools and measures shall consist, in particular, of the teaching and learning of basic notions and skills about AI systems and their functioning, including the different types of products and uses, their risks and benefits and the severity of the harm they can cause and its probability of occurrence.

4. The level of AI literacy shall be considered to be sufficient where it contributes to the ability of operators to fully comply with and benefit from trustworthy AI, and in particular with the requirements laid down in this Regulation.

Amendment 37

Proposal for a regulation

Article 6 – paragraph 2

Text proposed by the Commission

Amendment

2. In addition to the high-risk AI systems referred to in paragraph 1, AI systems referred to in Annex III shall also be considered high-risk.

2. In addition to the high-risk AI systems referred to in paragraph 1, AI systems referred to in Annex III shall also be considered high-risk due to their risk to cause harm to health, safety, the environment, fundamental rights or to democracy and the rule of law.

Amendment 38

Proposal for a regulation

Article 7 – paragraph 1 – point a

Text proposed by the Commission

Amendment

(a) the AI systems are intended to be used in any of the areas listed in points 1 to 8 of Annex III;

(a) the AI systems are intended to be used in any of the areas listed in Annex III;

Amendment 39

Proposal for a regulation

Article 7 – paragraph 1 – point b

Text proposed by the Commission

Amendment

(b) the AI systems pose a risk of harm to the health and safety, or a risk of adverse impact on fundamental rights, that is, in respect of its severity and probability of occurrence, equivalent to or greater than the risk of harm or of adverse impact posed by the high-risk AI systems already referred to in Annex III.

(b) the AI systems pose a risk of harm to the health and safety, or a risk of adverse impact on fundamental rights, democracy and the rule of law, or the environment that is, in respect of its severity and probability of occurrence, equivalent to or greater than the risk of harm or of adverse impact posed by the high-risk AI systems already referred to in Annex III.

Amendment 40

Proposal for a regulation

Article 7 – paragraph 2 – introductory part

Text proposed by the Commission

Amendment

2. When assessing for the purposes of paragraph 1 whether an AI system poses a risk of harm to the health and safety or a risk of adverse impact on fundamental rights that is equivalent to or greater than the risk of harm posed by the high-risk AI systems already referred to in Annex III, the Commission shall take into account the following criteria:

2. When assessing for the purposes of paragraph 1 whether an AI system poses a risk of harm to the health and safety or a risk of adverse impact on fundamental rights, democracy and the rule of law, or the environment that is equivalent to or greater than the risk of harm posed by the high-risk AI systems already referred to in Annex III, the Commission shall take into account the following criteria:

Amendment 41

Proposal for a regulation

Article 7 – paragraph 2 a (new)

Text proposed by the Commission

Amendment

2a. The Commission shall conduct the assessment referred to in paragraph 2 annually under the consultation conditions laid down in Article 73.

Amendment 42

Proposal for a regulation

Article 9 – paragraph 4 – subparagraph 2 – point d a (new)

Text proposed by the Commission

Amendment

(da) provision of a sufficient level of AI literacy;

Amendment 43

Proposal for a regulation

Article 9 – paragraph 8

Text proposed by the Commission

Amendment

8. When implementing the risk management system described in paragraphs 1 to 7, specific consideration shall be given to whether the high-risk AI system is likely to be accessed by or have an impact on children.

Amendment 44

Proposal for a regulation

Article 10 – paragraph 2 – point f

Text proposed by the Commission

Amendment

(f) examination in view of possible biases;

(f) examination in view of possible biases, in particular deviations that could affect the health and safety of people or could lead to discrimination;

Amendment 45

Proposal for a regulation

Article 10 – paragraph 2 – point g a (new)

Text proposed by the Commission

Amendment

(ga) the purpose and the environment in which the system is to be used;

Amendment 46

Proposal for a regulation

Article 13 – paragraph 1

Text proposed by the Commission

Amendment

1. High-risk AI systems shall be designed and developed in such a way to ensure that their operation is sufficiently transparent to enable users to interpret the system’s output and use it appropriately. An appropriate type and degree of transparency shall be ensured, with a view to achieving compliance with the relevant obligations of the user and of the provider set out in Chapter 3 of this Title.

1. High-risk AI systems shall be designed and developed in such a way to ensure that their operation is sufficiently transparent to enable providers and users and other relevant stakeholders to easily interpret the system’s functioning and output and use it appropriately on the basis of informed decisions, with a view to achieving compliance with the relevant obligations set out in Chapter 3 of this Title.

Amendment 47

Proposal for a regulation

Article 13 – paragraph 3 a (new)

Text proposed by the Commission

Amendment

3a. In order to comply with the obligations laid down in this Article, providers and users shall ensure a sufficient level of AI literacy in accordance with Article 4b.

Amendment 48

Proposal for a regulation

Article 14 – paragraph 5 a (new)

Text proposed by the Commission

Amendment

5a. In order to comply with the obligations laid down in this Article, providers and users shall ensure a sufficient level of AI literacy in accordance with Article 4b.

Amendment 49

Proposal for a regulation

Article 29 – paragraph 1 a (new)

Text proposed by the Commission

Amendment

1a. In order to comply with the obligations laid down in this Article, as well as to be able to justify their possible non-compliance, users of high-risk AI systems shall ensure a sufficient level of AI literacy in accordance with Article 4b.

Amendment 50

Proposal for a regulation

Article 41 – paragraph 2

Text proposed by the Commission

Amendment

2. The Commission, when preparing the common specifications referred to in paragraph 1, shall gather the views of relevant bodies or expert groups established under relevant sectorial Union law.

2. The Commission, when preparing the common specifications referred to in paragraph 1, shall gather the views of relevant stakeholders, including industry representatives, SMEs and other relevant bodies or expert groups established under relevant sectorial Union law.

Amendment 51

Proposal for a regulation

Article 52 – paragraph 1

Text proposed by the Commission

Amendment

1. Providers shall ensure that AI systems intended to interact with natural persons are designed and developed in such a way that natural persons are informed that they are interacting with an AI system, unless this is obvious from the circumstances and the context of use. This obligation shall not apply to AI systems authorised by law to detect, prevent, investigate and prosecute criminal offences, unless those systems are available for the public to report a criminal offence.

1. Providers shall ensure that AI systems used to interact with natural persons are designed and developed in such a way that natural persons are informed that they are interacting with an AI system, unless this is obvious from the circumstances and the context of use. This obligation shall not apply to AI systems authorised by law to detect, prevent, investigate and prosecute criminal offences, unless those systems are available for the public to report a criminal offence.

Amendment 52

Proposal for a regulation

Article 52 – paragraph 2

Text proposed by the Commission

Amendment

2. Users of an emotion recognition system or a biometric categorisation system shall inform of the operation of the system the natural persons exposed thereto. This obligation shall not apply to AI systems used for biometric categorisation, which are permitted by law to detect, prevent and investigate criminal offences.

2. Users of an emotion recognition system or a biometric categorisation system shall inform, in a timely, clear and intelligible manner, of the operation of the system the natural persons exposed thereto. That information shall also include, as appropriate, the rights and processes to allow natural persons to appeal against the application of such AI systems to them. This obligation shall not apply to AI systems used for biometric categorisation, which are permitted by law to detect, prevent and investigate criminal offences.

Amendment 53

Proposal for a regulation

Article 52 – paragraph 3 – subparagraph 1

Text proposed by the Commission

Amendment

Users of an AI system that generates or manipulates image, audio or video content that appreciably resembles existing persons, objects, places or other entities or events and would falsely appear to a person to be authentic or truthful (‘deep fake’), shall disclose that the content has been artificially generated or manipulated.

Users of an AI system that generates or manipulates image, audio, text, scripts or video content that appreciably resembles existing persons, objects, places, text, scripts or other entities or events and would falsely appear to a person to be authentic or truthful (‘deep fake’), shall disclose, in an appropriate clear, repetitive, timely and visible manner, that the content has been artificially generated or manipulated.

Amendment 54

Proposal for a regulation

Article 52 – paragraph 3 – subparagraph 2

Text proposed by the Commission

Amendment

However, the first subparagraph shall not apply where the use is authorised by law to detect, prevent, investigate and prosecute criminal offences or it is necessary for the exercise of the right to freedom of expression and the right to freedom of the arts and sciences guaranteed in the Charter of Fundamental Rights of the EU, and subject to appropriate safeguards for the rights and freedoms of third parties.

However, the first subparagraph shall not apply where the use forms part of an evidently artistic, creative or fictional cinematographic or analogous work or where it is necessary for the exercise of the right to freedom of expression and the right to freedom of the arts and sciences guaranteed in the Charter of Fundamental Rights of the EU, and subject to appropriate safeguards for the rights and freedoms of third parties.

Amendment 55

Proposal for a regulation

Article 52 – paragraph 3 a (new)

Text proposed by the Commission

Amendment

3a. Providers and users of AI systems that recommend, disseminate and order news or creative and cultural content shall disclose, in an appropriate, easily accessible, clear and visible manner, the main parameters used for the moderation of content and personalized suggestions. That information shall include a disclaimer.

Amendment 56

Proposal for a regulation

Article 52 – paragraph 3 b (new)

Text proposed by the Commission

Amendment

3b. The information referred to in this Article shall be provided to the natural persons in a timely, clear and visible manner, at the latest at the time of the first interaction or exposure. Such information shall be made accessible when the exposed natural person is a person with disabilities, a child or where he or she belongs to a vulnerable group. It shall be complete, where possible, with intervention or flagging procedures for the exposed natural person, taking into account the generally acknowledged state of the art and relevant harmonised standards and common specifications.

Amendment 57

Proposal for a regulation

Article 56 – paragraph 2 – point a a (new)

Text proposed by the Commission

Amendment

(aa) work towards promoting uptake of AI within the Union, especially amongst SMEs;

Amendment 58

Proposal for a regulation

Article 57 – paragraph 1

Text proposed by the Commission

Amendment

1. The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority, and the European Data Protection Supervisor. Other national authorities may be invited to the meetings, where the issues discussed are of relevance for them.

1. The Board shall be composed of the national supervisory authorities, who shall be represented by the head or equivalent high-level official of that authority, and the European Data Protection Supervisor. Other national or international authorities and relevant stakeholders, including from the private sector, shall be invited to the meetings, where the issues discussed are of relevance for them.

Amendment 59

Proposal for a regulation

Article 57 – paragraph 4

Text proposed by the Commission

Amendment

4. The Board may invite external experts and observers to attend its meetings and may hold exchanges with interested third parties to inform its activities to an appropriate extent. To that end the Commission may facilitate exchanges between the Board and other Union bodies, offices, agencies and advisory groups.

4. The Board shall, where relevant, invite external experts and observers to attend its meetings and may hold exchanges with interested third parties to inform its activities to an appropriate extent. To that end the Commission may facilitate exchanges between the Board and other Union bodies, offices, agencies and advisory groups, including the High-Level Expert Group on AI.

Amendment 60

Proposal for a regulation

Article 69 – paragraph 3

Text proposed by the Commission

Amendment

3. Codes of conduct may be drawn up by individual providers of AI systems or by organisations representing them or by both, including with the involvement of users and any interested stakeholders and their representative organisations. Codes of conduct may cover one or more AI systems taking into account the similarity of the intended purpose of the relevant systems.

3. Codes of conduct may be drawn up by individual providers of AI systems or by organisations representing them or by both, including with the involvement of users and any interested stakeholders and their representative organisations, including in particular trade unions and consumers organisations. Codes of conduct may cover one or more AI systems taking into account the similarity of the intended purpose of the relevant systems.

Amendment 61

Proposal for a regulation

Article 69 – paragraph 3 b (new)

Text proposed by the Commission

Amendment

3b. In order to comply with the obligations laid down in this Article, providers and users shall ensure a sufficient level of AI literacy in accordance with Article 4b.

Amendment 62

Proposal for a regulation

Article 71 – paragraph 1

Text proposed by the Commission

Amendment

1. In compliance with the terms and conditions laid down in this Regulation, Member States shall lay down the rules on penalties, including administrative fines, applicable to infringements of this Regulation and shall take all measures necessary to ensure that they are properly and effectively implemented. The penalties provided for shall be effective, proportionate, and dissuasive. They shall take into particular account the interests of small-scale providers and start-up and their economic viability.

1. In compliance with the terms and conditions laid down in this Regulation, Member States shall lay down the rules on penalties, including administrative fines, applicable to infringements of this Regulation and shall take all measures necessary to ensure that they are properly and effectively implemented. The penalties provided for shall be effective, proportionate, and dissuasive. They shall take into particular account the interests and market position of small-scale providers and start-up and their economic viability.

Amendment 63

Proposal for a regulation

Article 73 – paragraph 3 a (new)

Text proposed by the Commission

Amendment

3a. Before adopting a delegated act, the Commission shall consult with the relevant institutions and stakeholders in accordance with the principles laid down in the Interinstitutional Agreement of 13 April 2016 on Better Law-Making.

Amendment 64

Proposal for a regulation

Annex III – paragraph 1 – point 2 – point a

Text proposed by the Commission

Amendment

(a) AI systems intended to be used as safety components in the management and operation of road traffic and the supply of water, gas, heating and electricity.

(a) AI systems intended to be used as safety components in the management and operation of road traffic and the supply of water, gas, heating, telecommunications, and electricity.

Amendment 65

Proposal for a regulation

Annex III – paragraph 1 – point 3 – introductory part

Text proposed by the Commission

Amendment

3. Education and vocational training:

3. Education and training:

Amendment 66

Proposal for a regulation

Annex III – paragraph 1 – point 3 – point a

Text proposed by the Commission

Amendment

(a) AI systems intended to be used for the purpose of determining access or assigning natural persons to educational and vocational training institutions;

(a) AI systems intended to be used for the purpose of determining access or assigning natural persons to educational and training institutions;

Amendment 67

Proposal for a regulation

Annex III – paragraph 1 – point 3 – point b

Text proposed by the Commission

Amendment

(b) AI systems intended to be used for the purpose of assessing students in educational and vocational training institutions and for assessing participants in tests commonly required for admission to educational institutions.

(b) AI systems intended to be used for the purpose of assessing students in educational and training institutions and for assessing participants in tests commonly required for admission to educational institutions.

Amendment 68

Proposal for a regulation

Annex III – paragraph 1 – point 3 – point b a (new)

Text proposed by the Commission

Amendment

(ba) AI systems intended to be used for the purpose of determining the study programme or areas of study to be followed by students in education and training institutions;

Amendment 69

Proposal for a regulation

Annex III – paragraph 1 – point 3 – point b b (new)

Text proposed by the Commission

Amendment

(bb) AI systems intended to be used for monitoring and detecting prohibited behaviour of students during tests at education and training institutions;

Amendment 70

Proposal for a regulation

Annex III – paragraph 1 – point 4 – introductory part

Text proposed by the Commission

Amendment

4. Employment, workers management and access to self-employment:

4. Employment and employment support, workers management and access to self-employment:

Amendment 71

Proposal for a regulation

Annex III – paragraph 1 – point 4 – point b

Text proposed by the Commission

Amendment

(b) AI intended to be used for making decisions on promotion and termination of work-related contractual relationships, for task allocation and for monitoring and evaluating performance and behavior of persons in such relationships.

(b) AI intended to be used for making decisions on establishment, promotion and termination of work-related relationships, for task allocation, for monitoring compliance with workplace rules and for monitoring and evaluating performance and behavior of persons in such relationships.

ANNEX: LIST OF ENTITIES OR PERSONS FROM WHOM THE RAPPORTEUR HAS RECEIVED INPUT

The following list is drawn up on a purely voluntary basis under the exclusive responsibility of the rapporteur. The rapporteur has received input from the following entities or persons in the preparation of the opinion, until the adoption thereof in committee:

Entity and/or person

AccessNow

Baptiste Caramiaux, CNRS researcher, Sorbonne Université in Paris

BEUC

Defenddigitalme

EDRi

European Disability Forum

Huawei

International Federation of Library Associations and Institutions

Joanna Bryson, Professor of Ethics and Technology, Hertie School

Liber Europe

Maldita.es

Martin Senftlebe, Professor of Intellectual Property Law, University of Amsterdam

Meaning Processing Ltd.

Saidot Ltd.

Scio.cz

PROCEDURE – COMMITTEE ASKED FOR OPINION

Title

Harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union Legislative Acts

References

COM(2021)0206 – C9-0146/2021 – 2021/0106(COD)

Committees responsible

Date announced in plenary

IMCO

7.6.2021

LIBE

7.6.2021

Opinion by

Date announced in plenary

CULT

7.6.2021

Associated committees - date announced in plenary

16.12.2021

Rapporteur for the opinion

Date appointed

Marcel Kolaja

13.7.2021

Rule 58 – Joint committee procedure

Date announced in plenary

16.12.2021

Discussed in committee

15.3.2022

Date adopted

15.6.2022

Result of final vote

–:

Members present for the final vote

Asim Ademov, Ilana Cicurel, Tomasz Frankowski, Romeo Franz, Catherine Griset, Sylvie Guillaume, Hannes Heide, Petra Kammerevert, Niyazi Kizilyürek, Predrag Fred Matić, Dace Melbārde, Peter Pollák, Diana Riba i Giner, Andrey Slabakov, Massimiliano Smeriglio, Michaela Šojdrová, Sabine Verheyen, Theodoros Zagorakis, Milan Zver

Substitutes present for the final vote

Loucas Fourlas, Ibán García Del Blanco, Marcel Kolaja, Domènec Ruiz Devesa

Substitutes under Rule 209(7) present for the final vote

Karen Melchior

FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION

Key to symbols: