2024/436

2.2.2024

COMMISSION DELEGATED REGULATION (EU) 2024/436

of 20 October 2023

supplementing Regulation (EU) 2022/2065 of the European Parliament and of the Council, by laying down rules on the performance of audits for very large online platforms and very large online search engines

THE EUROPEAN COMMISSION,

Having regard to the Treaty on the Functioning of the European Union,

Having regard to Regulation (EU) 2022/2065 of the European Parliament and of the Council of 19 October 2022 on a Single Market For Digital Services and amending Directive 2000/31/EC (Digital Services Act) (1), and in particular Article 37(7) thereof,

Whereas:

(1) Independent audits are an important tool in the supervision of the compliance of providers of very large online platforms and of very large online search engines with their obligations under Regulation (EU) 2022/2065. While other accountability tools are provided for in that Regulation, not least through enhanced public scrutiny of transparency reports and other data disclosure requirements, independent auditing organisations have a special role in assessing such providers’ compliance with that Regulation early on. The conclusions and findings of such independent audits and their recommendations can meaningfully inform regulatory supervision. At the same time, independent audits represent one among several sources of information and analysis that regulators can use in their supervisory and enforcement role.

(2) In order to ensure that independent audits are carried out in an effective, efficient, timely and comparable manner from the date of application of Regulation (EU) 2022/2065, as defined in Articles 92 and 93 thereof, the Commission should lay down rules on the performance of audits, in particular as regards the legal obligations for the audited providers and the procedural steps for ensuring that organisations performing audits fulfil the conditions of independence, no conflict of interest, expertise and professional ethics laid down in Article 37(3) of Regulation (EU) 2022/2065.

(3) In order to facilitate the appropriate performance of audits with a high level of expertise, and to pre-empt unintended consequences in the market for auditing services, it should be clarified that audits conducted in accordance with Article 37 of Regulation (EU) 2022/2065 may be carried out by several auditors. Where necessary, for example because of the need for specific expertise in auditing certain obligations or commitments, such as those relating to the design and functioning of algorithmic systems, an understanding of risks to fundamental rights, or the spread of illegal content, the audited provider may contract different auditing organisations, or a consortium of organisations, to conduct the audit. Auditing organisations may also subcontract the necessary expertise, provided that both the auditing organisation and the subcontractors comply with the necessary conditions on independence, non-conflict of interest, proven objectivity and professional ethics, and that they jointly comply with conditions on technical expertise. In such cases, the audited provider should still ensure that its compliance with all obligations and commitments referred to in Article 37(1) of Regulation (EU) 2022/2065 is audited at least once per year.

(4) Audit opinions referred to in Article 37(4), point (g), of that Regulation (EU) 2022/2065 should be asserted by auditing organisations with a reasonable level of assurance. To reach a reasonable level of assurance, the auditing organisation should have a high, but not absolute, level of confidence that there have been no misstatements, such as omissions, misrepresentations, or errors, which were not detected in the audit. To ensure that level of assurance, the auditing organisation should, amongst others, obtain sufficient evidence and use appropriate auditing methodologies in its assessment.

(5) Pursuant to Article 37(1) of Regulation (EU) 2022/2065, independent audits should be conducted at least annually, aligned with the yearly cycle of risk assessments referred to in Article 34 of that Regulation. However, more frequent audits may be necessary in certain cases. The sequencing of audits should ensure a continuum of supervision for the audited providers’ compliance with Regulation (EU) 2022/2065 and relevant codes of conduct and crisis protocols. The audited provider should ensure that the period for which a given audit assesses compliance with the audited obligations and commitments complements the period covered by the previous audit of the provider’s compliance with those obligations and commitments and starts at the latest where the period covered in the previous audit has ended. As the conclusion of an audit includes both the assessment done by the auditing organisation and the establishment of an audit report, audited providers should ensure that the duration of the audit allows for audits to be concluded at least once per year and the submission of the audit reports to the Commission and the Digital Service Coordinator follows without undue delay, pursuant to Article 42(4) of Regulation (EU) 2022/2065.

(6) While audited providers should in no circumstance interfere with the performance of the audit and its conclusions, they should fulfil their obligations under Article 37 of Regulation (EU) 2022/2065 including by agreeing contractual terms with the auditing organisation and verifying, prior to selecting an auditing organisation, that that organisation fulfils the conditions laid down in Article 37(3) of Regulation (EU) 2022/2065.

(7) The audited provider should, for example, assess contracts it has previously concluded with the auditing organisation or contracts concluded between the auditing organisation and legal persons connected to the audited provider. The audited provider should also include clauses in contracts with auditing organisations to guarantee the respect of the conditions laid down in Article 37(3) of Regulation (EU) 2022/2065. Where the auditing organisation consists of several entities, the audited provider should ascertain that all those entities fulfil those conditions, including, where applicable, any sub-contractors hired by the auditing organisation for the purpose of supporting the performance of the audit in any way. Whereas each entity performing the audit should individually fulfil the independence requirements and the requirements on no conflict of interests, those entities should fulfil jointly the requirements related to competence, expertise, or technical resources, thereby allowing different entities to perform different parts of the audit and contribute with the capabilities, competence and expertise needed to perform the audit. The audit report should specify the responsibility of each of those entities for the respective parts of the audit.

(8) Pursuant to Article 37(3), point (a)(i) of Regulation (EU) 2022/2065, the audited provider should pay particular attention to avoid that the auditing organisation provides non-audit services to the audited provider when verifying whether an auditing organisation fulfils the independence requirements and the requirements on no conflict of interests. The audited provider should, for example, assess whether services were provided, such as those linked to any system, software or process involved in matters relevant to the audited obligation or commitment, such as consultancy services for assessments of performance, of governance and of software, training services, development or maintenance of systems, or subcontracting content moderation. Such services also include services provided to the audited provider which consist in consulting on, or developing internal controls, or assessing, for internal purposes, the audited provider’s compliance with Regulation (EU) 2022/2065 or codes of conduct and crisis protocols, including when this is limited to punctual tests, such as third-party tests on the performance of content moderation systems. This should not exclude auditing organisations who have performed statutory financial audits.

(9) Given the complexity and particular nature of compliance audits for Regulation (EU) 2022/2065, the subject-matter expertise of the auditing organisation is key for performing audits with a reasonable level of assurance and for exercising the professional judgment and scepticism that enables that organisation to know, for example, which information it needs to perform the audit procedures or to challenge contradictory information. The audited provider should therefore verify whether the auditing organisation provides such expertise, including in the area of risk management, both with regard to audit risks and the subject-matter of Regulation (EU) 2022/2065 and, in particular, the systemic societal risks referred to in Article 34 of that Regulation. In addition, the audited provider should verify the technical competence and capabilities of the auditing organisation in view of the specific audited service, including its subject-matter expertise, for example as regards the functioning and effects of algorithmic systems, such as recommender systems and other socio-technical systems maintained by the provider. The auditing organisation should have the possibility to subcontract or otherwise obtain and deploy the necessary expertise and capabilities, and the audited provider should verify and ensure that the auditing organisation is able to acquire that expertise and those capabilities in time for the performance of the audit.

(10) In verifying that auditing organisations fulfil the conditions laid down in Article 37(3) of Regulation (EU) 2022/2065, the audited provider should assess relevant evidence, including, as appropriate, certifications, declarations and audit reports issued by the auditing organisation. The appropriate expertise could be proven, for example, by practical experiences in assessing and managing risks, as well as by academic activity, scientific publications, and experience with relevant audits. Audit reports should contain all the relevant supporting documentation attesting that the auditing organisation fulfils the necessary conditions.

(11) Pursuant to Article 37(2) of Regulation (EU) 2022/2065, the audited provider is to afford all the necessary cooperation and assistance for the auditing organisation to conduct the audit in an effective, efficient, and timely manner, as well as refrain from interfering in any way with independent decisions of the auditing organisation. For example, the audited provider should not impose, give any guidance, or otherwise influence the auditing organisation through any kind of contractual or other limitations or incentives in their choice and execution of audit procedures, methodologies, collection and processing of information and audit evidence, analysis, tests, audit opinion or elaboration of audit conclusions.

(12) To guarantee the necessary cooperation and assistance during the audit, the audited provider should ensure that the auditing organisation is granted access to all information necessary for the performance of the audit. The audited provider should send, as early as possible and in any case before the auditing organisation starts performing audit procedures, all the necessary documents and explanations. For example, pursuant to Article 41(3), points (d) and (e), of Regulation (EU) 2022/2065, the compliance function of the audited provider is to monitor compliance with all audited obligations and commitments, which should result in the elaboration of internal controls. The auditing organisation should therefore be granted access to all information related to such controls, and any other information outlining the audited provider’s strategy to ensure compliance. In particular, the audited provider should make available to the auditing organisation the benchmarks it relies upon to ensure compliance with Regulation (EU) 2022/2065 so that the auditing organisation can base the audit criteria on this information. In addition, the auditing organisation should be granted access to any analysis the audited provider might have conducted on inherent risks and control risks. That provider should make available to the auditing organisation information that facilitates the understanding of the audited service, its governance, the competence of respective teams and decision-making structures, including its compliance function, as well as presentations of its information technology (IT) systems, data and records structures, and the interplay between different algorithmic systems of relevance to the audit.

(13) The auditing organisation should be able to request, at any time in the performance of the audit, any other necessary information. Access to that information should be granted without undue delay in a manner that does not in any way hamper the performance of the audit. This should include access to data, including personal data, collected from various sources, such as documents, algorithmic systems, databases or interviews, as appropriate. The audited provider should also grant the auditing organisation access to procedures and processes, IT systems, such as algorithmic and information systems, including testing environments. In order to allow the auditing organisation to meaningfully inspect such systems, the audited provider should make all necessary resources available to assist that organisation in accessing and assessing the systems, such as by making available the provider’s competent personnel to answer questions or operate test environments and provide explanations about their functioning, or facilitate any other necessary access to personnel and premises, such as buildings. Access to procedures and processes could imply, for example, access to descriptions or documents concerning the audited provider’s internal decision-making process. Access to relevant information may also require other ancillary actions from the audited provider to fulfil their obligation for cooperation and assistance. For example, interviews with personnel may require secure facilities provided by the audited provider. Where it is necessary for the performance of the audit, audited providers should fulfil the cooperation and assistance obligation towards auditing organisation among other things by facilitating access to relevant data related to their operations held by their third-party contractors. This might be the case, for example, as regards results of content moderation actions, training material or guidance used by third-party contractors who moderate content, or vendors and service providers for IT solutions, including, for example algorithms and applications used in recommender systems or advertising systems used by the audited provider.

(14) To facilitate meaningful transparency of the audit findings and to provide a comprehensive and comparable format of audit reports, referred to in Article 37(4) of Regulation (EU) 2022/2065 and audit implementation reports referred to in Article 37(6) of that Regulation, this Regulation should establish templates for those reports and require a number of annexes for each of the reports. While the templates laid down in this Regulation require comprehensive reporting, they should not affect the requirements on the publication of reports provided for in Article 42(4) and (5) of Regulation (EU) 2022/2065.

(15) In order to ensure that the auditing organisation receives all the necessary assistance from the audited provider, without interference in the performance of the audit, and that the auditing organisation fulfils all conditions for the preparation of the audit and delivers the audit report in due time and with the quality necessary to reach a reasonable level of assurance, certain rules should specify the procedures for the preparation of the audit. The duties and responsibilities of the audited provider and the auditing organisation, including all sub-contractors or partner organisations and the staff responsible for carrying out the audit, should be set out in a written agreement, including through contractual terms. The written agreement should also specify the audited obligations and commitments, the allocation of resources, and the rules of interaction and contact points between the auditing organisation and the audited provider. All supporting documentation and contracts should be annexed to the audit report, including when the documents take the form of an audit engagement letter or other contractual terms.

(16) In order to provide a comprehensive overview and facilitate the accountability of audited providers, the audit report should include a conclusion of the auditing organisation’s assessment of compliance of the audited provider with each audited obligation or commitment. Each audit conclusion should be based on a reasonable level of assurance and should be either ‘positive’, ‘positive with comments’ or ‘negative’, in order to appropriately inform the audit opinion. Conclusions that are ‘positive with comments’ should not concern the assessment of compliance itself. Such comments could refer, for example, to the production of information by the provider at the request of the auditing organisation, or to improvements in the maintenance or controls put in place by the audited provider, or take note of further mitigation plans and improvements that the provider intends to make. In any event, where the auditing organisation deems the audited provider compliant with an audited obligation or commitment according to the benchmarks reported by the audited provider, but considers it necessary to include remarks on those benchmarks, the audit conclusion should be ‘positive with comments’, since such comments could usefully inform the provider about opportunities for potential changes to its benchmarks, based on the auditing organisation’s knowledge and expertise, as well as information from external sources. For example, the comments could be informed by guidance from the Commission, including through guidelines from the Commission referred to in Article 35(3) of Regulation (EU) 2022/2065, and any other relevant guidelines issued by the Commission with respect to the application of that Regulation, reports from the European Board for Digital Services referred to in Article 35(2) of that Regulation, enforcement actions, decisions taken by the Commission pursuant to that Regulation, relevant case law, especially from the Court of Justice of the European Union, public consultations, or relevant authoritative sources.

(17) In order to enable public scrutiny and regulatory supervision, where an audit conclusion is ‘negative’ but applies only for a limited period of time and the auditing organisation deems that the audited provider complied with the obligation or commitment for the rest of the audited period, this should be reflected in the audit report for each concerned obligation or commitment. The report should include the auditing organisation’s observations on any information made available to them by the audited provider as regards mitigation plans in place or planned to remedy non-compliance.

(18) In light of the different nature of the legal obligations laid down in Chapter III of Regulation (EU) 2022/2065 and the voluntary commitments made under codes of conduct and crisis protocols pursuant to Articles 45, 46 and 48 of that Regulation, the auditing organisation should issue audit opinions on the compliance with that Chapter and with each code and protocol.

(19) In order to perform the audit with a reasonable level of assurance and to design the appropriate audit procedures according to methodologies that minimise the audit risk to a low level, a key part of the methodology for performing the audit should be the estimation of the audit risks, namely the risk that the auditing organisation expresses an inappropriate audit opinion or conclusion. Therefore, the auditing organisation should assess the audit risk at the very beginning of the audit, before designing the precise methodology and performing audit procedures. The audit risk analysis is necessary to allow the auditing organisation to select the precise methodologies for the audit and determine how comprehensive the audit procedures must be so as to attain the reasonable level of assurance for the audit opinion. The auditing organisation should perform the audit risk analysis for the assessment of compliance with each audited obligation or commitment, considering inherent risks, control risks and detection risks.

(20) In order to correctly evaluate the audit risks, the audit risk analysis should take into account the nature of the audited service, notably its risk profile, and the scope and complexity of the audit. For example, it is likely that online platforms that allow the conclusions of distance contracts between consumers will have different inherent risks than video-sharing platforms or search engines. Furthermore, the societal and the economic context in which the audited service is provided should be considered, for example as regards typical user groups such as minors, or frequent behaviour such as a high incidence of inauthentic use and coordinated behaviours in disinformation campaigns. The societal and the economic context to be considered should also include the probability and, independently, the severity of exposure to crisis situations and unexpected events, as referred to in Regulation (EU) 2022/2065.

(21) In order to ensure that the audit risk analysis reflects the evolution of the risks to which the service is subject, the audit risk analysis should also be based on information from previous audits to which the audited provider was subject, where applicable, and build upon information from sources such as audit reports of other providers with a similar risk profile. To ensure that the audit risk analysis is fully informed by state-of-the-art evidence of risks in contexts similar to those in which the audited provider operates, and by authoritative sources of direct relevance for the application of Regulation (EU) 2022/2065, the analysis should also rely on information from reports issued by the European Board for Digital Services or guidelines from the Commission, where applicable. Other information could also include information from audit reports published pursuant to Article 42(4) of Regulation (EU) 2022/2065 by other providers of very large online platforms or of very large online search engines.

(22) The auditing organisation should draw up, without influence from the audited provider, the audit methodologies used to assess compliance with the audited obligations and commitments. The audit criteria should be based on the information submitted by the audited provider as regards benchmarks used by the audited provider for monitoring compliance. The methodology may also take into consideration other information made available by the audited provider, such as the analysis of inherent risks, when the audited provider has done such analysis, for example through measures developed by the compliance officer or the management body in accordance with Article 41 of Regulation (EU) 2022/2065 or other measures embedded in the functioning of the service for the systemic risk assessments referred to in Article 34 of that Regulation.

(23) In order to ensure that the audit methodologies are appropriate for reaching a reasonable level of assurance for the audit opinions, the choice of methodology for the audit procedures should address the specificities of the audited obligation or commitment and should be adapted, for example, to the nature of the audited obligation as an obligation of means, or an obligation of results that the provider must achieve in order to be compliant. For example, auditing procedures for assessing compliance with transparency reporting pursuant to Article 15 of Regulation (EU) 2022/2065 could allow the auditing organisation to conclude whether the reports were published within the delays and formats requested in that Regulation, as well as whether they were complete and the data reported was accurate, representative, and appropriately broken down, for example, per category of illegal content actioned.

(24) The choice of methodology should also depend on whether the compliance assessment requires contextual interpretations by the auditing organisation. The selection of methodologies should also be adapted to the inherent risks linked to the activities carried out in providing the service and the context in which the service is provided, for example, whether the service involves the sale of goods that could be illegal or whether the service is primarily used by minors. For example, methodologies for assessing compliance with obligations to put in place appropriate and proportionate measures to ensure a high level of privacy, safety, and security of minors pursuant to Article 28(1) of Regulation (EU) 2022/2065 should allow the auditing organisation to have a sufficient understanding of how the audited service is used by minors and the risks to their privacy, safety and security that may be incurred, as well as what constitutes an appropriate and proportionate measure in the specific context of the audited service and its use by minors. To this end, auditing organisations should break down the assessment into appropriate steps. They should assess the audit risks according to the risk profile of the audited provider, notably whether it is available to or predominantly used by minors. They should assess, for example, whether the provider has put in place age assurance tools, whether these are effective and how the audited provider assesses and monitors their effectiveness. They should assess whether the audited provider has put in place appropriate measures for detecting adversarial use of their service and behavioural patterns that seek to harm or are likely to harm minors.

(25) The selection of methodologies should be adapted to the control risks linked to the compliance measures put in place by the audited provider, as well as to the detection risks, namely the risk not to detect misstatements in the information the provider makes available to the auditing organisation. For example, where an audited obligation could involve the audit of an algorithmic system based on personalisation for individual recipients of the audited service and on recurrent updates of the algorithmic system, such as the disclosure obligations for recommender systems pursuant to Article 27 of Regulation (EU) 2022/2065, the choice of methodology should allow the auditing organisation to design the appropriate tests to minimise detection risks. Similarly, where the auditing organisation seeks to assess whether all relevant risks were mitigated in the design, functioning and use of applications based on large-scale language models, such as chat functionalities or recommender systems deployed by the audited provider, the auditing organisation should first assess the appropriateness of the controls put in place by the provider. The choice of tests should be informed by the robustness of those internal controls. In particular, but not limited to the cases where the internal controls are weak, incomplete or inconclusive to assess whether the rules are complied with when considering the population of recipients of the audited service, the audit procedures should rely on a combination of methodologies. For example, methodologies could include substantive analytical procedures, such as the analysis of the interactions between all algorithmic systems involved in the recommender systems and related decision-making rules and processes for establishing the main parameters of those recommender systems, observations of digital records and logs. Methodologies should also include tests of the system, such as tests in simulated environments.

(26) In order to ensure that the methodology is relevant and adapted to new findings during the performance of the audit, the selection of methodologies should be guided by the professional judgment of the auditing organisation and should be adjusted to address those new findings, in particular when the auditing organisation has reasonable doubts in relation to the information submitted by the audited provider. The professional scepticism of the auditing organisation should be based on its expertise, as well as on other sources of information of particular relevance for the application of Regulation (EU) 2022/2065 such as reports from the European Board for Digital Services, guidance from the Commission, audit reports issued from codes of conduct or crisis protocols referred to in Articles 45, 46 and 48 of that Regulation or information emerging during the performance of the audit, including when related to events, in particular crisis situations, that require additional actions from the audited provider to ensure compliance with certain audited obligations or commitments.

(27) In order to ensure that sufficient audit evidence is gathered during the audit, auditing organisations should assess both the internal controls of the audited provider and perform substantive audit procedures for assessing the audited provider’s compliance. In certain cases, the auditing organisation should also perform tests.

(28) Given the complexity of algorithmic systems used by providers of online platforms and their important role in complying with several obligations laid out in Regulation (EU) 2022/2065, particular attention should be paid to the necessary and appropriate methodological choices for auditing algorithmic systems. This is the case both when algorithmic systems are part of the controls put in place by the audited provider, and when they are themselves the subject-matter of the audited obligations or commitments, such as with respect to recommender systems, for example pursuant to Articles 27, 34, 35, and 38 of Regulation (EU) 2022/2065, or advertising systems, for example pursuant to Articles 26, 28, 34, 35, and 39 of that Regulation, content moderation systems, for example pursuant to Articles 14, 15, 34 and 35 of that Regulation, or any other algorithmic system that contributes to the risks referred to in Article 34 of that Regulation.

(29) A combination of substantive analytical procedures should also be used, including, as appropriate, based on observations of processes and activities of the audited provider in designing, developing, operating, testing, and monitoring algorithmic systems, or observations of digital records and logs produced by the systems. The methodologies should be adapted to the particularities of algorithmic systems, including their governance, the interaction between different algorithmic systems as well as the related data management systems, and to the technologies underlying those algorithmic systems, such as generative models or other classifiers, selection or search algorithms.

(30) Furthermore, audit methodologies for algorithmic systems should include tests, for example, to gather information that the audited provider has not previously documented, or to independently reproduce and assess results of accuracy indicators, tests in sandboxes or simulated environments, or tests in production systems, including through data scraping or adversarial testing.

(31) Given that the high quality of the audit evidence is a necessary condition for an auditing organisation to form an audit opinion with a reasonable level of assurance, the information that the auditing organisation decides to use as audit evidence should be appropriate and sufficient to reduce audit risks. In addition, the audit evidence should be reliable according to the auditing organisation’s professional judgment and scepticism and, where appropriate, in the light of alternative sources of information. Professional judgment and scepticism should include a critical assessment of audit evidence and possible misstatements. Those quality standards should apply to all audit evidence regardless of whether it has been provided by the audited provider or collected from other sources.

(32) A range of sources of information should be considered by the auditing organisation and could include, for example, interviews with the personnel or contractors of the audited provider, including compliance officers, engineers, data scientists, software architects, or members of internal audit teams. They could also include technical documentation on the design, implementation, testing and monitoring of a relevant system, including on data quality and governance and on updates and versions of the system, and other documents on the audited provider’s governance and decision-making processes, including in view of priorities, resources, allocations of tasks and responsibilities, or the expertise of relevant personnel.

(33) In order to ensure efficiency and proportionality in the performance of the audit, the auditing organisation should be allowed to sample data and information, with due regard to reaching a representative sample, to enable the auditing organisation to reach an audit opinion with a reasonable level of assurance. To ensure transparency and reproducibility of the audit procedures, the auditing organisation should provide justifications of the choices of sample size and method of the sampling in the audit report. For example, the size of the sample and methodology should be selected considering what is effective in meeting the purpose of auditing the specific audited obligation or commitment, and to minimise the risk that the conclusion of auditing the specific sample is different from what the conclusion would be if the entire population of evidence were subjected to the auditing procedure. The size and methodology for the sample should be selected considering the full scope of the audit, as well as internal or external changes to the audited service during this time. They should also be adapted to the particularities of algorithmic systems including as regards personalisation by profiling. As a part of this consideration the auditing organisation should, for example, appropriately sample from the different cohorts or partitions that may result from personalisation techniques, or identify the margin of error and justify why it is at an acceptable level.

(34) Given the novelty of certain provisions of Regulation (EU) 2022/2065, it is necessary to set out methodological principles, including audit questions, and further orientations for the selection of the audit methodologies and audit evidence for the assessment of compliance with those provisions, namely for assessing compliance with Articles 34, 35 and 36 of Regulation (EU) 2022/2065 on the performance of risk assessments and the adoption of risk mitigation measures by audited providers and on the application of obligations with respect to crisis response.

(35) Given that auditing organisations should also assess compliance of the audited provider with Article 37 of Regulation (EU) 2022/2065, further specifications should also be provided on the precise audit with respect to which compliance should be assessed, in particular to avoid any conflicts of interests for the auditing organisation.

(36) In view of the voluntary nature of codes of conduct and crisis protocols, it is necessary to provide specific rules for auditing compliance with Articles 45, 46 and 48 of Regulation (EU) 2022/2065, in particular to ensure that auditing organisations dispose of all the necessary information to perform audits specific to the commitments under each code of conduct and crisis protocol,

HAS ADOPTED THIS REGULATION:

SECTION I

General provisions

Article 1

Subject matter

This Regulation lays down rules on the performance of audits pursuant to Article 37 of Regulation (EU) 2022/2065, as regards:

(a) the procedural steps for ensuring that the auditing organisation to be selected fulfils the conditions laid down in Article 37(3) of Regulation (EU) 2022/2065;

(b) the procedural steps for cooperation and assistance by the audited provider in the performance of audits, including accessing relevant information with a view to obtaining audit evidence;

(c) the definition and selection of auditing methodologies;

(d) the templates for the audit report and the audit implementation report.

Article 2

Definitions

For the purpose of this Regulation, the following definitions shall apply:

(1) ‘auditing organisation’ means an individual organisation, a consortium or other combination of organisations, including any sub-contractors, that the audited provider has contracted to perform an independent audit in accordance with Article 37 of Regulation (EU) 2022/2065;

(2) ‘audited service’ means a very large online platform or a very large online search engine designated in accordance with Article 33 of Regulation (EU) 2022/2065;

(3) ‘audited provider’ means the provider of an audited service which is subject to independent audits pursuant to Article 37(1) of that Regulation;

(4) ‘audited obligation or commitment’ means an obligation or commitment referred to in Article 37(1) of Regulation (EU) 2022/2065 which forms the subject matter of the audit;

(5) ‘audit criteria’ means the criteria against which the auditing organisation assesses compliance with each audited obligation or commitment;

(6) ‘audit evidence’ means any information used by an auditing organisation to support the audit findings and conclusions and to issue an audit opinion, including data collected from documents, databases or IT systems, interviews or testing performed;

(7) ‘misstatement’ means an intentional or unintentional omission, misrepresentation or error in the declarations or data reported or provided by the audited provider to the auditing organisation, or in the testing environment made available by the audited provider to the auditing organisation;

(8) ‘audit risk’ means the risk that the auditing organisation issues an incorrect audit opinion or reaches an incorrect conclusion concerning the audited provider’s compliance with an audited obligation or commitment, considering detection risks, inherent risks and control risks with respect to that audited obligation or commitment;

(9) ‘detection risk’ means the risk that the auditing organisation does not detect a misstatement that is relevant for the assessment of the audited provider’s compliance with an audited obligation or commitment;

(10) ‘inherent risk’ means the risk of non-compliance intrinsically related to the nature, the design, the activity and the use of the audited service, as well as the context in which it is operated, and the risk of non-compliance related to the nature of the audited obligation or commitment;

(11) ‘control risk’ means the risk that a misstatement is not prevented, detected and corrected in a timely manner by means of the audited provider’s internal controls;

(12) ‘materiality threshold’ means the threshold beyond which deviations or misstatements by the audited provider, individually or aggregated, would reasonably affect the audit findings, conclusions and opinions;

(13) ‘reasonable level of assurance’ means a high but not absolute level of assurance, which allows the auditing organisation to assert in its audit opinion and audit conclusions whether the audited provider complies with the audited obligations or commitments, based on sufficient and appropriate evidence;

(14) ‘internal control’ means any measures, including processes and tests, that are designed, implemented and maintained by the audited provider, including its compliance officers and management body, to monitor and ensure the audited provider’s compliance with the audited obligation or commitment;

(15) ‘vetted researcher’ means a researcher vetted in accordance with Article 40(8) of Regulation (EU) 2022/2065;

(16) ‘audit procedure’ means any technique applied by the auditing organisation in the performance of the audit, including data collection, the choice and application of methodologies, such as tests and substantive analytical procedures, and any other action taken to collect and analyse information to collect audit evidence and formulate audit conclusions, not including the issuing of an audit opinion or of the audit report;

(17) ‘test’ means an audit methodology consisting in measurements, experiments or other checks, including checks of algorithmic systems, through which the auditing organisation assesses the audited provider’s compliance with the audited obligation or commitment;

(18) ‘substantive analytical procedure’ means an audit methodology used by the auditing organisation to assess information to infer audit risks or compliance with the audited obligation or commitment.

Article 3

Scope of the audit and reasonable level of assurance

1.   The audit shall be performed in a manner and for a duration that allows the auditing organisation to assess the audited provider’s compliance with all audited obligations and commitments with a reasonable level of assurance.

2.   The audit shall cover the period starting immediately after the period covered by the previous audit and ending on a date that allows the auditing organisation to perform the audit within the time frame required by Article 37(1) of Regulation (EU) 2022/2065, including by asserting its assessment pursuant to paragraph 1 based on the evidence collected and audit procedures conducted during that period, and by completing and submitting the audit report pursuant to Article 37(4) of that Regulation to the audited provider.

3.   Where no previous audit was performed, the audit shall cover the period starting four months after the notification referred to in Article 33(6), first subparagraph, of Regulation (EU) 2022/2065, and the duration of the audit shall allow for the audit report pursuant to Article 6(1) to be completed at the latest within a year as from the start of the audited period.

SECTION II

Conditions for the performance of the audit

Article 4

Selection of the auditing organisation

1.   Prior to selecting an auditing organisation with a view to performing the audit, the audited provider shall check whether the organisation to be selected fulfils the requirements laid down in Article 37(3) of Regulation (EU) 2022/2065.

2.   Where the auditing organisation to be selected consists of more than one legal person or intends to have recourse to one or several sub-contractors, the audited provider shall check whether all those legal persons or subcontractors:

(a) individually fulfil the requirements laid down in Article 37(3), points (a) and (c), of Regulation (EU) 2022/2065;

(b) jointly fulfil the requirement laid down in Article 37(3), point (b), of Regulation (EU) 2022/2065.

Article 5

Cooperation and assistance between the audited provider and the auditing organisation

1.   At a time agreed with the auditing organisation, and in any event prior to the performance of any audit procedure, the audited provider shall transmit to the selected auditing organisation at least the following information:

(a) a description of the internal controls put in place with respect to each audited obligation and commitment, including related indicators and all present and historical measurements, and benchmarks used by the audited provider to assert or monitor compliance with the audited obligations and commitments, as well as any supporting documentation;

(b) its preliminary analysis of inherent and control risks, where the audited provider has performed such an analysis, and any supporting documentation;

(c) information about any relevant decision-making structures, competences of departments of the provider, including the compliance function pursuant to Article 41 of Regulation (EU) 2022/2065, relevant IT systems, data sources, processing and storage, as well as explanations of relevant algorithmic systems and their interactions.

2.   The audited provider shall grant the auditing organisation, without undue delay, access to all data necessary for the performance of the audit, including personal data, documentation, information on procedures and processes, and to the information technology systems, testing environments, personnel and premises of that provider, and any relevant sub-contractors.

3.   The audited provider shall make all necessary resources available and provide the auditing organisation with the assistance and explanations necessary for the auditing organisation to analyse the relevant information and to carry out tests, including where the information requested by the auditing organisation in accordance with Article 37(3) of Regulation (EU) 2022/2065 is held by a third-party contracted by the audited provider.

SECTION III

Performance of audits

Article 6

Audit report and audit implementation report

1.   The audit report referred to in Article 37(4) of Regulation (EU) 2022/2065 shall be established by the auditing organisation, without interference from the audited provider. That audit report shall be drawn up in accordance with the template in Annex I, and shall contain detailed and substantiated conclusions in relation to all elements of the template.

2.   Where applicable, the audit implementation report referred to in Article 37(6) of Regulation (EU) 2022/2065 shall be drawn up in accordance with the template in Annex II.

Article 7

Procedures for the preparations for the audit

1.   The audited provider and the auditing organisation shall conclude a written agreement setting out:

(a) the exhaustive list of audited obligations and commitments;

(b) the responsibilities of the audit organisation, including, where applicable, detailed for each legal person constituting the auditing organisation, and the parties empowered to sign the audit report;

(c) the procedures and contact points made available by the audited provider for the auditing organisation to request access to data referred to in Article 5(2);

(d) the timeframe for the audit, including the start and end date of the audit procedures and the completion of the audit report;

(e) a procedure on how disputes between the audited provider and the auditing organisation arising from the performance of the audit shall be resolved.

2.   The agreement referred to in paragraph 1, as well as any other agreements or engagements letters between the auditing organisation and the audited provider related to the performance of the audit, shall be annexed to the audit report.

3.   Where changes are made to the agreement referred to in paragraph 1 during the performance of the audit, they shall be made explicit in the audit report.

Article 8

Audit opinion, audit conclusions and recommendations

1.   The audit report shall include the audit conclusions that the auditing organisation has reached on the audited provider’s compliance with each of the audited obligations and commitments. The audit conclusions shall be either:

(a) ‘positive’, where the auditing organisation concludes with a reasonable level of assurance that the audited provider has complied with an audited obligation or commitment;

(b) ‘positive with comments’, where the auditing organisation concludes with a reasonable level of assurance that the audited provider has complied with an audited obligation or commitment, but:

(i) the auditing organisation includes remarks on the benchmarks provided by the audited provider pursuant to Article 5(1), point (a); or

(ii) the auditing organisation recommends improvements that do not have a substantive effect on its conclusion;

(c) ‘negative’, where the auditing organisation concludes with a reasonable level of assurance that the audited provider has not complied with an audited obligation or commitment.

2.   Where an audit report includes operational recommendations pursuant to Article 37(4), point (h) of Regulation (EU) 2022/2065, those recommendations and their recommended timeframe shall be specific to each audited obligation or commitment for which the audit conclusion pursuant to paragraph 1 is ‘positive with comments’ or ‘negative’.

3.   Where the operational recommendations referred to in paragraph 2 include specific measures to achieve compliance, they shall be formulated in a way that explains the auditing organisation’s assessment of how such measures would affect the materiality threshold by comparison with the audit conclusion for the respective audited obligation or commitment.

4.   On the basis of the audit conclusions, the audit report shall include an audit opinion on the audited provider’s compliance with all audited obligations referred to in Article 37(1), point (a), of Regulation (EU) 2022/2065.

5.   On the basis of the conclusions of all audited commitments, the audit report shall include an audit opinion or opinions, as applicable, on the audited provider’s compliance with all audited commitments made by the audited provider under each code of conduct and crisis protocol referred to in Article 37(1), point (b), of Regulation (EU) 2022/2065.

6.   Audit opinions pursuant to paragraphs 4 and 5 shall be either:

(a) ‘positive’ if the auditing organisation has reached a ‘positive’ audit conclusion for all of the audited obligations or commitments;

(b) ‘positive with comments’ if the auditing organisation has reached at least one audit conclusion that is ‘positive with comments’ for an audited obligation or commitment and has not reached a ‘negative’ audit conclusion for any of the audited obligations or commitments;

(c) ‘negative’ if the auditing organisation reached a ‘negative’ audit conclusion for at least one audited obligation or commitment.

7.   Where the auditing organisation assesses that, for a limited period during the period referred to in Article 3(2), the provider has not complied with an audited obligation or commitment, the audit report shall duly document that assessment.

8.   Where the auditing organisation cannot issue with a reasonable level of assurance an audit conclusion pursuant to paragraph 1 or an audit opinion pursuant to paragraphs 4 and 5, the audit report shall include an explanation of the circumstances and the reasons why such a level of assurance could not be achieved.

SECTION IV

Audit methodologies

Article 9

Audit risks analysis

1.   The audit report shall include a substantiated audit risk analysis performed by the auditing organisation for the assessment of the audited provider’s compliance with each audited obligation or commitment.

2.   The audit risk analysis shall be carried out prior to the performance of audit procedures and shall be updated during the performance of the audit, in the light of any new audit evidence which, according to the professional judgement of the auditing organisation, materially modifies the assessment of the audit risk.

3.   The audit risk analysis shall consider:

(a) inherent risks;

(b) control risks;

(c) detection risks.

4.   The audit risk analysis shall be conducted taking into account:

(a) the nature of the audited service and the societal and economic context in which the audited service is operated, including probability and severity of exposure to crisis situations and unexpected events;

(b) the nature of the obligations and commitments;

(c) other appropriate information, including:

(i) where applicable, information from previous audits to which the audited service was subjected;

(ii) where applicable, information from reports issued by the European Board for Digital Services or guidance from the Commission, including guidelines issued pursuant to Article 35(2) and (3) of Regulation (EU) 2022/2065, and any other relevant guidance issued by the Commission with respect to the application of Regulation (EU) 2022/2065;

(iii) where applicable, information from audit reports published pursuant to Article 42(4) of Regulation (EU) 2022/2065 by other providers of very large online platforms or of very large online search engines operating in similar conditions or providing similar services to the audited service.

Article 10

Appropriate audit methodologies

1.   Without prejudice to the specific audit methodologies set out in Articles 13, 14, and 15, audits shall be performed by using appropriate auditing methodologies to reduce the assessed audit risks to a level that enables the auditing organisation to reach audit conclusions at a reasonable level of assurance.

2.   The audit report shall include a description of the audit methodologies designed by the auditing organisation prior to performing any audit procedures, including at least:

(a) the audit criteria, for assessing compliance with each audited obligation or commitment, defined on the basis of information pursuant to Article 5(1), point (a), and the materiality threshold tolerated and expressed in qualitative or quantitative terms, as appropriate;

(b) all tests and substantive analytical procedures and audit evidence that the auditing organisation intends to use to assess compliance for each audited obligation or commitment.

The audit report shall include a description of any changes to the methodologies used during the performance of the audit compared to the methodologies designed prior to performing audit procedures.

3.   Where an auditing organisation has reasonable doubts concerning the information assessed in the performance of the audit, in particular as regards information that has been presented by the audited provider, the choice and application of the methodology shall be adapted to afford that organisation the necessary audit evidence in accordance with Article 11.

4.   Reasonable doubts referred to in paragraph 3 shall be deemed to arise, in particular, in the presence of any of the following elements:

(a) professional judgment and scepticism in assessing information, including concerning internal controls of the audited provider, that leads the auditing organisation to formulate reasonable doubts;

(b) external indications pointing to audit risks, in particular reports from the European Board for Digital Services referred to in Article 35(2) of Regulation (EU) 2022/2065, guidance from the Commission including through guidelines referred to in Article 35(3) of that Regulation, and any other relevant guidance issued by the Commission with respect to the application of Regulation (EU) 2022/2065, and audit reports issued pursuant to codes of conduct or crisis protocols referred to in Articles 45, 46 and 48 of that Regulation;

(c) information related to events occurring during the performance of the audit, including crisis situations, that require additional actions from the audited provider to ensure compliance with certain audited obligations or commitments.

5.   Audit procedures shall include at least:

(a) the performance of tests and substantive analytical procedures for the internal controls the audited provider has put in place for each of the audited obligations or commitments;

(b) the performance of substantive analytical procedures to assess compliance with each audited obligation and commitment, including as regards algorithmic systems;

(c) the performance of tests, including with respect to algorithmic systems, concerning the audited obligations and commitments in relation to which the auditing organisation has reasonable doubts, as referred to in paragraph 4, and concerning audited obligations and commitments where the auditing organisation deems necessary to perform tests in its choice of methodology pursuant to paragraph 1.

6.   Where obligations or commitments referred to in Article 37(1) of Regulation (EU) 2022/2065 require the audited provider to report certain information publicly, the auditing methodologies shall include an assessment of whether the reported information is free from material error or omission which might otherwise render them misleading.

Article 11

Quality of audit evidence

The audit conclusions and audit opinions shall be based on audit evidence which fulfils both of the following requirements:

(a) it is relevant and sufficient to reduce audit risks identified in accordance with Article 9, and to enable the auditing organisation to provide audit conclusions and opinions in accordance with Article 8;

(b) it is reliable, according to the auditing organisation’s professional judgment and scepticism.

Article 12

Sampling methods

1.   Where audit evidence is based, partially or entirely, on a sample of data or information, the sample size and methodology for sampling shall be selected with a view to minimising the detection risk and without interference by the audited provider.

2.   The sample size and methodology for sampling shall be selected in a way that ensures representativeness of the data or information and, as appropriate, in consideration of all of the following:

(a) the representativeness of the sample for the period referred to in Article 3(2) and (3);

(b) relevant changes to the audited service during that period;

(c) relevant changes to the context in which the audited service is provided during that period;

(d) relevant features of algorithmic systems, where applicable, including personalisation based on profiling or other criteria;

(e) other relevant characteristics or partitions of the data, information and evidence under consideration;

(f) the representation and appropriate analysis of concerns related to particular groups as appropriate, such as minors or vulnerable groups and minorities, in relation to the audited obligation or commitment.

3.   The audit report shall include a justification of the choice of the sample size and of the methodology for sampling.

Article 13

Specific methodologies for auditing compliance with Article 34 of Regulation (EU) 2022/2065 on risk assessment

1.   The assessment of the audited provider’s compliance with Article 34 of Regulation (EU) 2022/2065 shall include, but not be limited to, an analysis of all of the following:

(a) whether the audited provider has diligently identified, analysed, and assessed the systemic risks in the Union referred to in Article 34(1), first subparagraph, of Regulation (EU) 2022/2065, including by assessing:

(i) how the audited provider identified the risks that are linked to its service, taking into account regional and linguistic aspects of the use made of its service, including when specific to a Member State, and whether the risks are appropriately identified;

(ii) how the audited provider analysed and assessed each risk, including how it considered the probability and severity of the risks, and whether the assessment was appropriate;

(iii) how the audited provider identified, analysed and assessed the factors referred to in Article 34(2), first subparagraph, of Regulation (EU) 2022/2065, whether they were appropriately identified, and to what extent such factors influence the risks identified in paragraph 1 of that Article;

(iv) what sources of information the audited provider used, how it collected the information, including whether and how it relied on scientific and technical insights;

(v) whether and how the audited provider tested assumptions on risks with groups most impacted by the specific risks;

(b) whether the risk assessment was performed within the timeframes set out in Article 34(1), second subparagraph, of Regulation (EU) 2022/2065 and, where applicable, within the timeframes set for activities established as risk mitigation measures for the detection of systemic risks pursuant to Article 35(1), point (f) of that Regulation;

(c) how the audited provider identified functionalities that are likely to have a critical impact on the risks for which risk assessments shall be conducted prior to their deployment, pursuant to Article 34(1), second subparagraph, of Regulation (EU) 2022/2065, whether those functionalities were correctly identified, and whether the risk assessment was appropriately conducted;

(d) whether the audited provider correctly identified the supporting documentation that should be preserved with respect to the risk assessment and whether it has put in place the necessary means to ensure the preservation of that documentation for at least three years, pursuant to Article 34(3) of Regulation (EU) 2022/2065, and whether the documentation was preserved accordingly.

2.   Without prejudice to any other analysis necessary for reaching a reasonable level of assurance, methodologies for auditing compliance with Article 34 of Regulation (EU) 2022/2065 shall include at least an assessment by the auditing organisation of the following elements:

(a) the internal controls that the audited provider has put in place to monitor the performance of risk assessments regarding each factor referred to in Article 34(2), first subparagraph, of Regulation (EU) 2022/2065; such assessment shall:

(i) be based on substantive analytical procedures, for those internal controls;

(ii) be based on tests of whether those internal controls are reliable and diligently conceived, executed and monitored;

(iii) evaluate how the compliance officer or officers performed their tasks with respect to Article 41(3), points (b), (d), (e) and, where applicable, (f), of Regulation (EU) 2022/2065 and how the management body of the audited provider was involved in the decisions related to risk management pursuant to Article 41(6) and (7) of that Regulation;

(b) the actions, means and processes put in place by the audited provider to ensure compliance with Article 34 of Regulation (EU) 2022/2065 and the results thereof; such assessment shall be based on:

(i) substantive analytical procedures;

(ii) tests, including of algorithmic systems, where the auditing organisation has reasonable doubts, following the results of the substantive analytical procedures and the assessment of internal controls, or where the auditing organisation deems necessary to perform tests in its choice of methodology pursuant to article 10(1).

3.   Information analysed by the auditing organisation in support of the assessment carried out pursuant to this Article shall consist of, but not be limited to:

(a) the risk assessment report for the relevant audited period, which has been drawn up by the audited including, where necessary, confidential information that is not part of the information published pursuant to Article 42(2) of that Regulation, and all supporting documents;

(b) where relevant, other risk assessments reports of the audited provider and their supporting documents;

(c) information submitted by the audited provider pursuant to Article 5;

(d) all relevant transparency reports of the audited provider referred to in Article 15(1) of Regulation (EU) 2022/2065;

(e) any other test results, documentation, evidence, statements made in response to written or oral questions addressed by the auditing organisation to the personnel of the audited provider, and observations made on premises, where applicable;

(f) other relevant evidence, including based on information made available by the audited provider;

(g) where available, reports referred to in Article 35(2) of Regulation (EU) 2022/2065 and guidance from the Commission, including guidelines issued pursuant to Article 35(3) of that Regulation and any other relevant guidance issued by the Commission with respect to the application of Regulation (EU) 2022/2065.

4.   Information analysed by the auditing organisation may comprise, as appropriate, information referred to in Article 42(4) of Regulation (EU) 2022/2065, including from audit, risk assessment and risk mitigation reports, concerning other very large online platforms or very large online search engines, or data and research made publicly available by vetted researchers pursuant to Article 40(8), point (g), of the Regulation.

Article 14

Specific methodologies for auditing compliance with Article 35 of Regulation (EU) 2022/2065 on mitigation of risks

1.   The assessment of the audited provider’s compliance with Article 35 of Regulation (EU) 2022/2065 shall include, but not be limited to, an analysis of all of the following:

(a) how the audited provider identified risk mitigation measures for each of the systemic risks referred to in Article 34(1) of Regulation (EU) 2022/2065, and whether the identification of such risk mitigation measures was carried out in a diligent manner;

(b) how the audited provider assessed whether the risk mitigation measures in Article 35(1), points (a) to (k), of Regulation (EU) 2022/2065 were applicable to the audited service and whether the conclusion of that assessment was appropriate, including as regards those measures which were not applied by the audited provider;

(c) whether the mitigation measures put in place by the audited provider are reasonable, proportionate and effective for mitigating the respective risks, including by

(i) assessing whether they respond collectively to all the risks, with particular consideration of the risks concerning the exercise of fundamental rights;

(ii) assessing comparatively how the risks were addressed before and after the specific risk mitigation measures were put in place;

(iii) whether the risk mitigation measures were appropriately designed and executed.

2.   Without prejudice to any other analysis necessary for reaching a reasonable level of assurance, methodologies for auditing compliance with Article 35 of Regulation (EU) 2022/2065 shall include at least an assessment by the auditing organisation of the following elements:

(a) the internal controls the audited provider has put in place to monitor the application of risk mitigation measures referred to in Article 35(1) of Regulation (EU) 2022/2065 and whether they are reasonable, proportionate and effective; such assessment shall:

(i) be based on substantive analytical procedures for those internal controls;

(ii) be based on tests, of whether those internal controls are reliable and diligently conceived, executed and monitored;

(iii) evaluate how the compliance officer or officers performed their tasks with respect to Article 41(3), points (b), (d), (e) and, where applicable, (f), of Regulation (EU) 2022/2065, and how the management body of the provider was involved pursuant to Article 41(6) and (7) of that Regulation;

(b) mitigation measures put in place by audited providers; such assessment shall be based on:

(i) substantive analytical procedures;

(ii) tests, including of algorithmic systems, where the auditing organisation has reasonable doubts, following the results of the substantive analytical procedures and the assessment of internal controls, or where the auditing organisation deems necessary to perform tests in its choice of methodology pursuant to Article 10(1).

3.   Information analysed by the auditing organisation in support of the assessment carried out pursuant to this Article shall consist of, but not be limited to:

(a) the reports on risk assessment and risk mitigation for the relevant audited period, which have been drawn up by the audited provider including, where necessary, confidential information that is not part of the information published pursuant to Article 42(2) of Regulation (EU) 2022/2065, and all supporting documents;

(b) where relevant, other reports on risk assessment and risk mitigation of the audited provider and their supporting documents;

(c) information submitted by the audited provider pursuant to Article 5;

(d) all relevant transparency reports of the audited provider referred to in Article 15(1) of Regulation (EU) 2022/2065;

(e) where relevant, past reports on risk mitigation and their supporting documents, which concern periods not covered by the audited period, including, where necessary, confidential information that is not part of the information published pursuant to Article 42(2) of Regulation (EU) 2022/2065;

(f) any other test results, documentation, evidence, statements made in response to written and or oral questions addressed by the auditing organisation to the personnel of the audited provider, and observations made on premises, where applicable;

(g) other relevant evidence, including based on information made available by the audited provider;

(h) where available, reports referred to in Article 35(2) of Regulation (EU) 2022/2065 and guidance from the Commission, including guidelines issued pursuant to Article 35(3) of that Regulation and any other relevant guidance issued by the Commission with respect to the application of Regulation (EU) 2022/2065.

4.   Information analysed by the auditing organisation may comprise, as appropriate, information referred to in Article 42(4) of Regulation (EU) 2022/2065, including from audit, risk assessment and risk mitigation reports, concerning other very large online platforms or very large online search engines, or data and research made publicly available by vetted researchers pursuant to Article 40(8), point (g), of Regulation (EU) 2022/2065.

Article 15

Specific methodologies for auditing compliance with Article 36 of Regulation (EU) 2022/2065 on crisis response mechanism

1.   The assessment of the audited provider’s compliance with Article 36(1), first subparagraph, point (a) of Regulation (EU) 2022/2065 shall include, but not be limited to, an analysis of whether and how the audited provider performed the required actions, in particular:

(a) whether and how the audited provider identified the relevant systems involved in the functioning and use of their service that significantly contribute to the serious threat and whether those systems were appropriately identified;

(b) whether and how the audited provider defined and monitored the significant contribution to the serious threat and whether its assessment was appropriate;

(c) any other requirement specified in the Commission’s decision referred to in Article 36(1) or (7), second subparagraph, of Regulation (EU) 2022/2065, as appropriate.

2.   The assessment of the audited provider’s compliance with Article 36(1), first subparagraph, point (b), of Regulation (EU) 2022/2065 shall include, but not be limited to, an analysis of whether and how the audited provider performed the required actions, in particular:

(a) whether and how the audited provider identified measures to prevent, eliminate or limit any contribution to the serious threat;

(b) whether and how the measures taken by the audited provider addressed the gravity of the serious threat, the urgency, and whether the measures were appropriate in this respect;

(c) whether and how the audited provider identified the parties concerned by the measures and their legitimate interests, and how the audited provider assessed the actual or potential impact of the measures on those parties’ rights, including fundamental rights, and legitimate interests;

(d) whether the measures taken by the audited provided were effective and proportionate;

(e) any other requirement specified in the Commission’s decision referred to in Article 36(1) or (7), second subparagraph, of Regulation (EU) 2022/2065, as appropriate.

3.   The assessment of the audited provider’s compliance with Article 36(1), first subparagraph, point (c) of Regulation (EU) 2022/2065, shall include, but not be limited to, an analysis of how the audited provider performed the required action, in particular whether the audited provider provided to the Commission the information required in the Commission’s decision referred to in Article 36(1) or (7), second subparagraph, of Regulation (EU) 2022/2065, and whether those reports were accurate.

Article 16

Auditing compliance with Article 37 of Regulation (EU) 2022/2065 on independent audit

1.   Compliance with the obligations laid down in Article 37 of Regulation (EU) 2022/2065 and in this Regulation shall be assessed in relation to the audit or audits performed for the yearly period preceding that of the current audit.

2.   In addition to paragraph 1, the audit shall include an assessment of the audited provider’s compliance with Article 37(2) of Regulation (EU) 2022/2065 with respect to the current audit.

3.   Where the previous audit or audits referred to in paragraph 1 were performed by the same auditing organisation as the current audit, or where the auditing organisation carrying out the current audit comprises at least one legal entity which participated in the previous audit, the audit report shall include an explanation of the steps put in place by the auditing organisation to ensure the objectivity of the assessment.

Article 17

Auditing compliance with codes of conduct and crisis protocols

1.   The audited provider shall make available to the auditing organisation:

(a) a list and the text of all codes of conduct referred to in Articles 45 and 46 of Regulation (EU) 2022/2065 and crisis protocols referred to in Article 48 of that Regulation, to which the audited provider is a signatory;

(b) a detailed list of commitments within those codes of conduct and crisis protocols that the audited provider has taken;

(c) where applicable, the key performance indicators agreed under each code of conduct and crisis protocol;

(d) where applicable, any available measurements, data and documentation, and any reports prepared by the audited provider with respect to the compliance of the audited provider with the commitments taken, including access to all relevant information and data related to the functioning of the services offered by the audited provider relevant to the implementation of the code of conduct or the crisis protocol;

(e) where applicable, other measurements, data and documentation prepared by signatories of the code of conduct or the crisis protocol, and the assessments by the Commission or the Board referred to in Article 45(4) of Regulation (EU) 2022/2065.

2.   The assessment of the audited provider’s compliance with codes of conduct referred to in Article 45 of Regulation (EU) 2022/2065 shall include, but not be limited to, the measurement of key performance indicators agreed in the code of conduct pursuant to Article 45(3) of that Regulation, specifying the materiality threshold of the audit conclusions, and whether the reported data is accurate.

SECTION V

Final provisions

Article 18

Entry into force

This Regulation shall enter into force on the twentieth day following that of its publication in the

Official Journal of the European Union

.

This Regulation shall be binding in its entirety and directly applicable in all Member States.

Done at Brussels, 20 October 2023.

For the Commission

The President

Ursula VON DER LEYEN

(1)  

OJ L 277, 27.10.2022, p. 1

.

ANNEX I

TEMPLATE FOR THE AUDIT REPORT REFERRED TO IN ARTICLE 6

Table of contents

[Bild bitte in Originalquelle ansehen]

[Bild bitte in Originalquelle ansehen]

[Bild bitte in Originalquelle ansehen]

[Bild bitte in Originalquelle ansehen]

[Bild bitte in Originalquelle ansehen]

[Bild bitte in Originalquelle ansehen]

[Bild bitte in Originalquelle ansehen]

Annexes to the Audit Report (as applicable):

Documents requested pursuant to Article 7(2) of this Regulation.

Documents relating to the audit risk analysis pursuant to Article 9 of this Regulation.

Documents attesting that the auditing organisation complies with the obligations laid down in Article 37(3), point (a) of Regulation (EU) 2022/2065.

Documents attesting that the auditing organisation complies with the obligations laid down in Article 37(3), point (b) of Regulation (EU) 2022/2065.

Documents attesting that the auditing organisation complies with the obligations laid down in Article 37(3), point (c) of Regulation (EU) 2022/2065.

Documentation and results of any tests performed by the auditing organisation, including as regards algorithmic systems of the audited provider.

Codes of conduct referred to in Article 45 and 46 of Regulation (EU) 2022/2065 under which the audited provider made commitments, including a clear indication of any commitment undertaken and of any agreed key performance indicator for that commitment.

Crisis protocols referred to in Article 48 of Regulation (EU) 2022/2065 implemented by the audited provider.

Any other annex the auditing organisation wishes to include.

ANNEX II

TEMPLATE FOR THE AUDIT IMPLEMENTATION REPORT REFERRED TO IN ARTICLE 6

Table of contents

[Bild bitte in Originalquelle ansehen]

[Bild bitte in Originalquelle ansehen]

[Bild bitte in Originalquelle ansehen]

ELI: http://data.europa.eu/eli/reg_del/2024/436/oj

ISSN 1977-0677 (electronic edition)