text: www.enisa.europa.eu European Union Agency For Network And Information Security

text: Impact evaluation on the implementation of

text: Article 13

text: a incident reporting scheme within EU

text: ISBN: 978-92-9204-149-6; DOI: 10.2824/491369

text: Article 13

text: a incident reporting scheme within EU

text: Article 13

text: a incident reporting scheme within EU

text: Article 13

text: a incident reporting scheme within EU

text: Directive

text: of the European Parliament and of the Council of 25 November 2009 amending Directives

text: on a common regulatory framework for electronic communications networks and services,

text: on access to, and interconnection of, electronic communications networks and associated facilities, and

text: on the authorisation of electronic communications networks and services

text: Article 13

text: a incident reporting scheme within EU

text: Article 13

text: a incident reporting scheme within EU

text: Article 13

text: a incident reporting scheme within EU

text: https://ec.europa.eu/digital-agenda/en/telecoms-rules (November 2015) https://resilience.enisa.europa.eu/article-13/guideline-for-incident-reporting

text: Article 13

text: a incident reporting scheme within EU

text: https://resilience.enisa.europa.eu/article-13/guideline-for-minimum-security-measures https://resilience.enisa.europa.eu/article-13/guideline_on_threats_and_assets

text: Article 13

text: a incident reporting scheme within EU

text: Ward, K, 1995, ‘The Impact of Network Interconnection on Network Integrity’. British Telecommunications Engineering, 13:296–303. In information security literature the term ‘integrity’ usually refers to the property that data or communications cannot be altered or tampered with.

text: Article 13

text: a incident reporting scheme within EU

text: Figure 1: Scope of Art. 13a national implementation (security measures)

text: 0% Continuity/ availability of Measures to protect the Protecting customers against Protecting customers against electronic communications confidentiality of information/ denial of service attacks malware attacks networks and services data carried over electronic communications networks and services

text: Figure 2: Scope of Art. 13a national implementation (mandatory incident reporting)

text: 0% Disruption/ outages in Breaches impacting the Denial of service attacks Malware attacks not electronic communications confidentiality of not impacting the impacting the availability networks and services information/ data carried availability of the of the electronic over electronic electronic communications communications service communications networks service but impacting but impacting customers and services customers

text: Article 13

text: a incident reporting scheme within EU

text: Figure 3: Services in scope of national implementation of Art. 13a

text: 80% 70% 60% 50% 40% 30% 20% 10% 0% SMS TV Radio Other IXPs - Satellite Commercial Webmail ccTLDS - Webbased Broadcastingbroadcasting Internal telephony with and Country voice not Exchange hotspots messaging Code Top using the Points services Level E.164 tel Domains number plan

text: Article 13

text: a incident reporting scheme within EU

text: Figure 4: Coverage of telecom security by Art. 13a

text: Yes, together with

text: Article 4

text: in the ePrivacy Directive No

text: Article 13

text: a incident reporting scheme within EU

text: Article 13

text: a incident reporting scheme within EU

text: Figure 5: NRA satisfaction regarding stronger security measures in the sector

text: 32% Yes 45% No Don't Know

text: Article 13

text: a incident reporting scheme within EU

text: Article 13

text: a incident reporting scheme within EU

text: Figure 6: Main NRA challenges during the implementation process

text: 0% Administrative Providers Limited value Insufficient Insufficient Inadequate Lack of Rigid process Poor initial Poor initial legal Other burden reluctance for money budget process for planning communication security framework auditing/ follow between measures at up of security relevant parties provider's level measures (NRA - Providers / NRA - ENISA) No challenge Small challenge Medium challenge Big challenge

text: FTE = Full time employee

text: Article 13

text: a incident reporting scheme within EU

text: Figure 7: Areas in which providers improved the most

text: 0% D1: D2: Human D3: Security of D4: Operations D5: Incident D6: Business D7: Monitoring, Other Governance Resources systems and Management management continuity auditing and and risk security facilities management testing management*

text: No information No improvement Some improvement Fair amount of improvement a lot of improvement

text: *based on Art. 13a Technical Guideline on Security Measures

text: Article 13

text: a incident reporting scheme within EU

text: Figure 8: Asset classes most improved by providers

text: 35% 30% 25% 20% 15% 10% 5% 0% Core network Metropolitan network Access network (the Peering points (IXP, Other (backbone network, (backhaul, municipality last mile to the interconnection points core switches and networks, etc.) customers, ex base between providers, routers, etc.) stations, fibre to the etc.) customers, etc.) No information No improvement Some improvement Fair amount of improvement A lot of improvement

text: Article 13

text: a incident reporting scheme within EU

text: Article 13

text: a incident reporting scheme within EU

text: Figure 9: Provider’s attitude towards mandatory incident reporting

text: 80% 70% 60% 50% 40% 30% 20% 10% 0% Reluctant Neutral Collaborative Independent Needed Other (providers did support not request assistance from NRAs' side)

text: Figure 10: Measures for promoting transparency by NRAs

text: 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Issue Provide Require Inform otherInform ENISA Inform the Controlling/ Require Provide/ Inform other Publish Other Publish guidelines/ online affected relevant about major public about auditing affected promote relevant results incidents secondary incident providers to NRAs about incidents ongoing providers providers to tools for providers naming the reports legislation on reporting inform their major with a incidents regarding publicly showing about major providers naming the terms for tool customers incidents possible reporting announce ongoing ongoing after a providers on incident about major with a impact in routines major incidents incidents follow-up websites reporting ongoing possible other ongoing supervision incidents impact in Member incidents other States Member States

text: Article 13

text: a incident reporting scheme within EU

text: Figure 11: Measures that contributed most to transparency

text: 80% 70% 60% 50% 40% 30% 20% 10% 0% Issue Controlling/ Provide Inform the Publish Require Require Inform Provide/ Inform Inform Publish Other Publish guidelines/ auditing online public about annual/ affected affected other promote other ENISA about results incidents secondary providers incident major periodic providers toproviders to relevant tools for relevant major naming the reports legislation regarding reporting ongoing reports publicly inform their providers showing NRAs about incidents providers naming the on terms for reporting tool incidents regarding announce customers about major ongoing major with a after a providers on incident routines past major about major ongoing incidents incidents possible follow-up websites reporting incidents ongoing ongoing incidents with a impact in supervision incidents incidents possible other impact in Member other States Member States

text: Figure 12: Opinion regarding the information quality

text: 23% Yes, enough information and of good quality

text: No, insufficient or irrelevant information to build sufficient knowledge upon

text: We get more information than needed in the reports 77%

text: Article 13

text: a incident reporting scheme within EU

text: Figure 13: Appropriate numbers of incidents collected

text: Too few reports - under 5% reporting 27% Reporting as expected

text: Too many reports to handle/ 68% too many non significant reports

text: Article 13

text: a incident reporting scheme within EU

text: Figure 14: Informing the stakeholders

text: 100% 80% 60% 40% 20% 0% The public Affected 112 - Emergency Important societal Policy makers Other providers customers services functions dependent on electronic communications Under informed Sufficiently informed Very well informed

text: Article 13

text: a incident reporting scheme within EU

text: Figure 15: Additional sources of information

text: 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% We follow media We ask for We have 24/7 Other We have a hotline/ We have our own voluntary personnel to webpage for the probes/ sensors notification during discover incidents/ public for reporting following on the incident receive incidents incidents although this is not notifications mandatory

text: Figure 16: Incident notifications for ongoing incidents

text: Yes 23%

text: No, we only receive notifications/ reports after the incidents 77% have been resolved

text: Article 13

text: a incident reporting scheme within EU

text: Figure 17: Main costs associated with improving transparency

text: 80% 70% 60% 50% 40% No cost 30% Low cost 20% Medium cost 10% High cost 0% Educate Provide Develop tools Gather data Meet Meet other Increased Other providers guidelines/ to support from providers on NRAs on a FTEs about the secondary providers in providers to a regular regular basis dedicated need to legislation to the incidents report to basis to discuss team conduct providers to reporting ENISA incidents incidents report process report reporting incidents

text: Article 13

text: a incident reporting scheme within EU

text: Figure 18: Costs vs. benefits on implementing Art. 13a incident reporting

text: Neutral, the costs outweigh the benefits 23% No, costs are higher than benefits 63% Don't know

text: Article 13

text: a incident reporting scheme within EU

text: Figure 19: Areas where incident reporting is used for learning

text: 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% As a basis for As a basis for As a basis for As a basis for As a basis for As a basis for As a basis for As an input for As a basis for Other maintaining post incident publishing developing studies on root workshops sharing sector wirde participating in statistics supervision annual/ guidelines/ causes with providers experiences risk and the periodic secondary with other vunerability management/ reports on legislation NRAs mitigation of reported the incident incidents

text: Article 13

text: a incident reporting scheme within EU

text: Figure 20: Benefits of incident reporting

text: 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Facilitate Give direction Can support Inform the Input for The notion of Can provide Input for Input for Input for Input for Input for Input for Other appropriate to planned fast and public, providing and a reporting information discussing illustrating developing thematic sharing sector wide follwo up supervision/ appropriate customers maintaining obligation back to the challenges root causes appropriate studies on experiences risk and supervision auditing incident and statistics pushes providers on and issues and trends guidelines/ root causes with other vulnerability response/ important providers to root causes with for secondary NRAs assessments management functions improve and issues providers and appropriate legislation about the other sectors policy current decisions situation

text: Not important at all Not so important Quite important Very important

text: Article 13

text: a incident reporting scheme within EU

text: Figure 21: Collaboration between NRAs and ENISA

text: 60% 50% 40% 30% 20% 10% 0% Collaboration with ENISA Collaboration among Useful and accurate On average, the On average, the took place along the NRAs took place along guidelines were provided relationship with ENISA collaboration within the whole implementation the whole by ENISA through the was beneficial for the

text: Article 13

text: a Expert Group process implementation process

text: Article 13

text: a Expert Group implementation process was beneficial for the implementation N/A Strongly disagree Disagree Neutral Agree Strongly agree

text: Article 13

text: a incident reporting scheme within EU

text: Figure 22: FTE spend by NRAs during implementation phase

text: Other Supervising/ auditing the providers Participate in surveys and in the drafting/ review of ENISA… Inform/ collaborate with NRAs in other Members States impacted… Meet ENISA and other NRAs on a regular basis to discuss the… Gather data from providers to report to ENISA Develop tools for providers in the incident reporting process Meet providers on a regular basis Educate providers about the need to conduct incident reporting Internal educational activities (workshop, training, etc.) Draft and provide guidelines/ secondary legislation to providers 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% above 76% 51% - 75% 26% - 50% 11% - 25% Below 10%

text: Article 13

text: a incident reporting scheme within EU

text: Figure 23: Collaboration between providers and NRAs during the implementation of Art. 13

text: 70% 60% 50% 40% 30% 20% 10% 0% Providers were reluctant to Providers requested guidelines Collaboration with the implement the requirements and advices from the NRA providers took place along the of

text: Article 13

text: a whole implementation process

text: N/A Strongly disagree Disagree Neutral Agree Strongly agree

text: Article 13

text: a incident reporting scheme within EU

text: Figure 24: Current collaboration NRAs – ENISA after the implementation of Art. 13a

text: 80% 70% 60% 50% 40% 30% 20% 10% 0% Providers are still Providers are open Providers request Useful and On average, the On average, the Other reluctant to to share relevant guidelines and accurate relationship with collaboration implement the information about advices from NRAs guidelines were ENISA is beneficial within the Article requirements of incidents provided by ENISAfor the work within 13a Expert Group

text: Article 13

text: a through the Article

text: Article 13

text: a is beneficial for 13a Expert Group your

text: Article 13

text: a related work N/A Strongly disagree Disagree Neutral Agree Strongly agree

text: Article 13

text: a incident reporting scheme within EU

text: Figure 25: FTE spend by NRAs after implementation phase

text: Other Supervising/ auditing the providers Participate in surveys and in the drafting/ review of ENISA Technical… Inform/ collaborate with NRAs in other Members States impacted by… Meet ENISA and other NRAs on a regular basis to discuss the… Gather data from providers to report to ENISA Develop tools for providers in the incident reporting process Meet providers on a regular basis Educate providers about the need to conduct incidents reporting Internal educational activities (workshop, training, etc.) Draft and provide guidelines/ secondary legislation to providers 0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

text: above 76% 51% - 75% 26% - 50% 11% - 25% Below 10%

text: Article 13

text: a incident reporting scheme within EU

text: Directive

text: Article 6

text: Article 13

text: a incident reporting scheme within EU

text: Figure 26: The use of ENISA’s technical guidelines by NRAs

text: Article 13

text: a

text: Article 13

text: a

text: Article 13

text: a

text: Article 13

text: a and ENISA Annual ENISA studies Technical Technical Technical

text: Article 4

text: reports on Guideline on Guideline on Guideline on Technical reported Incident Security Threats and Guideline on incidents Reporting Measures Assets Security Measures

text: Hardly/ not at all Not so much but we refer to it in our work We get enough inspiration and use a fair amount of it in our work We use mot/ all of it in our work

text: Article 13

text: a incident reporting scheme within EU

text: Figure 27: Appropriate level of harmonization

text: 14% Yes, a harmonized approach allowing for national differences is appropriate No, more harmonization across the Member States would be 86% preferable

text: Article 13

text: a incident reporting scheme within EU

text: Figure 28: Ways of carrying harmonization

text: 0% Yes, the way we work through the ENISA

text: Article 13

text: a Expert Group drafting and agreeing on common 18% technical guidelines and sharing experiences, is appropriate Yes, but ENISA could push more for aligning the NRAs with each other regarding secondary legislation and supervision methodologies

text: No, we would prefer the Commission to adopt 82% technical implementing measures for us to follow on an equal basis

text: Article 13

text: a incident reporting scheme within EU

text: Figure 29: Total incidents reported 2011 - 2014

text: Directive

text: Article 13

text: a incident reporting scheme within EU

text: Figure 30: Root causes for reported 2011 - 2014

text: Article 13

text: a incident reporting scheme within EU

text: Article 13

text: a incident reporting scheme within EU

text: Article 13

text: a incident reporting scheme within EU

text: Article 13

text: a incident reporting scheme within EU

text: ks s s s or ce m g ce te w rvi s e in g rvi ur ) y m in et se ce l e sys s n e se vi ct al y e oa st g ia bl ri et on ce on ic ng er ru on bl l r ca in st tr ca st et rn ph vi st si vo gi l s es al e cs re ph rn ca le er na l ad vi d ai ra te ca FI sa rr ri in ti er le te s io ai ro le se m nf ity in te b ad es te ae ar op (t te Te in ic te at m te s WI m e l I e e r- o lli ile ile S lli e IO ro e D ic ba ic d d tr S rn b b- ca bl bl bm di te ob ob M te te ic D b bl Ps TL TV D bl IP bl iti be xe xe ec Ca Ca Su Fi Ra Sa Fi Fi El M M SM M Sa In Vo RA TV Ca IX cc IP Vo Pu We We Vo Pu Cr

text: NETWORKS SERVICES INTERNET RELATED CI Nr. Flag Country

text: Finland ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 2 Slovak republic ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 3 Estonia ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 4 Hungary ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 5 Lithuania ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 6 Portugal ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 7 Czech Republic ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 8 Sweden ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 9 UK ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 10 Romania ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 11 Belgium ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 12 Croatia ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 13 Cyprus ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 14 Germany ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 15 Slovenia ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 16 Norway ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 17 Switzerland ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 18 Ireland ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 19 Poland ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 20 Greece ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 21 Malta ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 22 Austria ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 23 Luxembourg ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦ 24 Bulgaria ♦ ♦ ♦ ♦ ♦ ♦ ♦ ♦

identifier: Impact evaluation on the implementation of Article 13a incident reporting scheme within EU

publisher: Europeiska unionens cybersäkerhetsbyrå

sprak: en

title: Impact evaluation on the implementation of Article 13a incident reporting scheme within EU

serie: rapporter

text: About ENISA

type: rubrik

text: The European Union Agency for Network and Information Security (ENISA) is a centre of network and information security expertise for the EU, its member states, the private sector and Europe’s citizens. ENISA works with these groups to develop advice and recommendations on good practice in information security. It assists EU member states in implementing relevant EU legislation and works to improve the resilience of Europe’s critical information infrastructure and networks. ENISA seeks to enhance existing expertise in EU member states by supporting the development of cross-border communities committed to improving network and information security throughout the EU. More information about ENISA and its work can be found at www.enisa.europa.eu.

type: stycke

text: Authors

text: Dan Tofan (ENISA) Konstantinos Moulinos (ENISA) Christoffer Karsberg (ENISA) – survey phase

text: Contact

text: For contacting the authors please use resilience@enisa.europa.eu For media enquiries about this paper, please use press@enisa.europa.eu.

text: Acknowledgements

text: The analysis in this document was produced in collaboration with EY Luxembourg (Brice Lecoustey, Alexandre Minarelli, George Tountas and Cédrine Herbin) and based on the input of the following experts: Vassilis Stathoupoulos (ADAE), Manuel Barros (ANACOM), Costin Masiliev (ANCOM), Heidi Kivekas (FICORA/NCSC-FI), Tamas ROKA (NMHH), Vasiliki Mylona (OCECPR), Marios Pantazi (OCECPR), Antonis Antoniades (OCECPR), Matthew Bryant (OFCOM), Karin Lodin and Björn Scharin (PTS), Ulrich Latzenhofer (RTR), Francois Zamora (Orange France), Zerboni Marcello Alessandro Michele (Telecom Italia), Marnix Dekker (former ENISA expert in NIS).

text: Legal notice

text: Notice must be taken that this publication represents the views and interpretations of the authors and editors, unless stated otherwise. This publication should not be construed to be a legal action of ENISA or the ENISA bodies unless adopted pursuant to the

text: Regulation (EU) No 526/2013

text: . This publication does not necessarily represent state-of the-art and ENISA may update it from time to time.

text: Third-party sources are quoted as appropriate. ENISA is not responsible for the content of the external sources including external websites referenced in this publication.

text: This publication is intended for information purposes only. It must be accessible free of charge. Neither ENISA nor any person acting on its behalf is responsible for the use that might be made of the information contained in this publication.

text: Copyright Notice

text: © European Union Agency for Network and Information Security (ENISA), 2015 Reproduction is authorised provided the source is acknowledged.

text: Contents

text: Executive Summary 6

text: 1. Introduction 9

text: 1.1. Background 9

text: 1.2. Scope 10

text: 1.3. Target audience 10

text: 1.4. Impact evaluation methodology 10

text: 2. Status of Art. 13a implementation 11

text: 2.1. Introduction to Art. 13a 11

text: 2.1.1. Requirements 11 2.1.2. Networks and services in scope of Art. 13a 13

text: 2.2. Situation of the countries before the implementation of Art. 13a 14

text: 2.2.1. Approach of the NRAs and providers 14

text: 3. Appropriate security measures for providers 16

text: 3.1. Specific objective context 16

text: 3.2. Level of satisfaction with the security measures implementation 16

text: 3.3. Challenges encountered during the implementation 17

text: 3.4. Outcome/benefit analysis of stronger security measures 18

text: 3.5. Areas of further improvement 20

text: 4. Transparency in incident reporting 21

text: 4.1. Specific objective context 21

text: 4.2. Level of satisfaction regarding the transparency in the incident reporting 21

text: 4.2.1. Providers’ level of collaboration 21 4.2.2. Regarding the information quality 23 4.2.3. Regarding the number of incident reports collected 24

text: 4.3. Level of communication 25

text: 4.4. Challenges 26

text: 4.5. Outcome/Benefits analysis regarding transparency 27

text: 4.5.1. Costs for the incident reporting from transparency requirements 27 4.5.2. Benefits of the incident reporting from transparency requirements 27

text: 4.6. End – user feedback regarding Art. 13a 28

text: 4.7. Area of further improvement 28

text: 5. Learning and improving based on reported incidents 29

text: 5.1. Incident reporting as a tool for learning 29

text: 5.2. Outcomes/benefits of the learning process 29

text: 5.3. Areas of further improvement 30

text: 6. Collaboration between the actors 31

text: 6.1. Specific objective context 31

text: 6.2. Collaboration during the implementation of Art. 13a requirements 31

text: 6.2.1. Collaboration between ENISA and NRAs 31 6.2.2. Collaboration between NRAs and providers 32

text: 6.3. Collaboration after the implementation of Art. 13a requirements 33

text: 6.4. Areas of further improvement 35

text: 7. Harmonization of practices within the EU 36

text: 7.1. Paradox: harmonization at EU level vs fragmentation at country level 36

text: 7.1.1. Harmonization vs differentiation 36 7.1.2. Different approaches in implementing Art. 13a requirements 37

text: 7.2. NRAs level of satisfaction regarding harmonization 38

text: 7.3. Areas of further improvement 39

text: 8. Impact evaluation based on reported incidents 40

text: 8.1. ENISA’s annual incident reports 40

text: 8.2. Areas of further improvement 41

text: 9. Key findings and conclusions 42

text: Annex A: Network and services covered by national implementations of Art. 13a 45

text: Executive Summary

text: In today’s interconnected world, telecommunications are transforming the way people engage in their everyday lives. Economic development is strongly related to the existence and well-functioning of the telecommunication networks. Electronic communications services guarantee the smooth transmission of data in this strongly interconnected world by providing the infrastructure for businesses and critical services to run. Electronic communication services also play a significant role in national security, emergency response and in the economic development of a country. As a result, a security incident affecting one of these areas can result in severe consequences. Art. 13a gave a new momentum in the telecom industry at European level. Being part of the Framework Directive 2009/140 EC within the Telecom Package, the set of obligations in the article aims at ensuring the security and integrity of electronic communication networks and services, dealing mostly with prevention of outages or service disruption (availability of the service). Although availability seems to be the main concern of Art. 13a, some countries (as you will notice later) have decided to cover also other types on security incidents within their national legislation (e.g. privacy). By amending the 2002 legislation, the Art. 13a within the 2009 Telecom Package, addresses specific security and resilience obligations for the telecom sector, for the first time in EU, as the 2002 directive had only vague provisions on this area.

text: As several years have passed since the publication and implementation of the Framework Directive including Art. 13a, an impact evaluation of the new article was the proper thing to do. The evaluation has the purpose of assessing the changes in outcome that can directly be attributed to the provision of Art. 13a, the effects caused by this particular set of obligations within the Telecom Package. The evaluation focused on 5 key areas, where we tried to identify possible outcomes:

text:  The new security measures implemented in the member states ;  The transparency resulting from the incident reporting process;  The learning process resulting from incidents ;  The level of collaboration between the stakeholders ;  The harmonization of the procedures within the European Union.

text: The compendious evaluation we have done within this project has brought to light some important outcomes that have definitely contributed to increasing the resilience and security of the telecommunications infrastructures in Europe. In a European Union which was highly diversified in terms of security measures, Art. 13a brought a certain amount of uniformity in the approach taken regarding security of telecommunication services, but more importantly contributed to strengthening the European telecom infrastructure’s resilience and services availability across the EU. The role of ENISA, especially in the coordination of Art. 13a expert group, was most beneficial as it helped considerably in bringing more harmonization within the implementation process and collaboration among stakeholders (NRAs and providers).

text: Overall, Art. 13a has undeniably contributed to improving the level of security in the telecommunication sector but in a balanced way as some countries were already in line or even ahead of the requirements and

text: were already experiencing the expected benefits, whereas other less advanced countries or providers experienced strong benefits in spite of the costs and effort provided.

text: It was also noted throughout this project that further analysis is needed in order to draw some strong conclusions on next steps that are needed in this area. This report only covers a set of findings that must be further analyzed, by responsible parties, in order to propose concrete recommendations and next steps to be considered by different types of stakeholders.

text: The main findings of the study are the following, grouped into categories:

text: The scope of Art. 13a

text: 1. Current lack of precise information within Art. 13a and Telecom Directive, as regards the types of networks and services that should be covered, has led to some differences among national implementations within member states. Although, the level of harmonization seem to be satisfactory, the differences within services covered by member states could represent obstacles in achieving the overall or specific objectives stated within the Telecom Package. Further assessments in this area are needed (more details in section 2.1.2. and Annex I) in order to establish possible next steps. 2. More than half of the respondents (54%) considered that Art. 13a cannot sufficiently and clearly cover by itself security of electronic communications, but together with Art. 4 in the e-Privacy Directive.

text: Appropriate security measures for providers:

text: 1. The majority (45%) of the respondents (NRAs) considered that Art. 13a has led to stronger security measures within the sector, but further analyses are needed as more than half of them (55%) do not share this opinion, 23% stating “no” and 32% “do not know”. 2. Almost 60% of the respondent NRAs are not aware of the areas where the providers have improved the most, in terms of security measures.

text: Transparency in incident reporting

text: 1. The approach of providers towards NRAs as regards the implementation of Art. 13a mandatory incident reporting regulations was mostly collaborative. Withal the majority of the respondent NRAs are satisfied with the quality of the information provided by the operators in their incident reports and declared that they are receiving reports as expected. 2. Bringing more clarity to the incident reporting process, by issuing guidelines and additional legislation – 73%, was by far the most effective method of improving transparency.

text: Learning and improving based on reported incidents

text: 1. NRAs mostly use the incident reporting process as a tool in learning/improvement, but mostly for internal purposes such as compliance, internal statistics and improve regulations. The use of annual incident statistics as an input for evaluating risk at national or sectorial level is not a common approach among NRAs (more details in section 5.1. and 5.3).

text: Collaboration between the actors

text: 1. The establishment and development of Art. 13a expert group, under ENISA coordination, turned out to be a successful and helpful experience (more details in section 6.3.), as appreciated by 80% of the respondents. The operation and development of the group should definitely be continued, under ENISA’s coordination.

text: 2. Bi-directional communication with the population in the incident reporting process is poorly addressed by both NRAs and providers. Further analyses should be carried in order to determine the necessity of developing such processes (more details in section 4.3.). 3. The amount of resource employed by NRAs in the cross-border collaboration area appears to be low (more details in section 6.3.).

text: Harmonization of practices within the EU

text: 1. Over 80% of the surveyed NRAs declare that they are satisfied with the current level of harmonization within the EU, which sustains also national specificities, and do not think that an improvement is needed in present. 2. ENISA’s work, together with Art. 13a expert group, in the area of guidelines and good practices, was considered useful by up to 70% of the respondent NRAs, as it supported the achievement of a mature level of harmonization.

text: Impact evaluation based on reported incidents

text: 1. More and more reported incidents (at ENISA level) are caused by third party failures (more details in section 8.2.), meaning they are caused by parties out of provider’s direct control, but within the provider’s supply chain. 2. The main root cause for incidents at EU level in 2014 and years before, is “system failures”. Further assessment needs to be done in this area, in order to identify more detailed causes and security measures that can be adopted.

text: 1. Introduction

text: 1.1. Background

text: In today’s interconnected world, telecommunications are transforming the way people engage in their everyday lives. Economic development is strongly related to the existence and well-functioning of the telecommunication networks. Electronic communications services guarantee the smooth transmission of data in this strongly interconnected world by providing the infrastructure for business services to run. Electronic communication services also play a significant role in national security, emergency response and in the economic development of a country. As a result, an outage in any one of these areas can result in severe consequences. The Telecom Package represents the EU's regulatory framework for electronic communications, and is, according EU Commission’s website, “[…] a series of rules which apply throughout the EU member states. It encourages competition, improves the functioning of the market and guarantees basic user rights. The overall goal is for European consumers to be able to benefit from increased choice thanks to low prices, high quality and innovative services” . The Telecom Package was adopted in November 2009, as a review of the European Union Telecommunications Framework 2007 – 2009. Art. 13a, of the Directive 2009/140 EC, is part of the Telecom Package and aims at ensuring the security and integrity of electronic communication networks and services, dealing mostly with prevention of outages or service disruption (availability of the service). This is partially achieved through requiring telecommunication service providers to take the appropriate technical and organizational measures to manage the risks posed to security of networks and services, guarantee the integrity of their networks (ensure the continuity of supply of services provided over those networks) and notify the competent national regulatory authority (NRA) of a breach of security or loss of integrity that has had a significant impact on the operation of networks or services. Published in 2009, Art. 13a required that the deadline for the transposition should be up to 2011. However, the transposition timeframe and process significantly varied from one country to the other. As a matter of fact, countries maturity level, national legislation complexity and process impacted some countries in their ability to comply with the deadlines, resulting in certain gaps between countries overtime. Today, the majority of countries implemented the provisions of the Art. 13a, in one way or another, besides one country. Art. 13a also designates ENISA, along with the European Commission (EC), as responsible bodies for collecting notifications received and actions taken within member states, under the provisions of national implementations of Art. 13a. Besides this specific mandate, according to the directive, ENISA should also contribute to the “harmonization of appropriate technical and organizational security measures by providing expert advice” and by “promoting the exchange of best practices”.

text: As a response to the directive’s requirements, in 2010, ENISA, Ministries and NRAs from member states, initiated a series of meetings (workshops, conference calls) in order to achieve a harmonized implementation of Art. 13a of the Framework directive. As a result of these meetings, a group of experts from NRAs, now entitled the Art. 13a Expert Group, reached agreement on three non-binding technical documents providing guidance to the NRAs in the EU member states:

text:  Technical Guideline on Incident Reporting

text:  Technical Guideline on Security Measures  Technical Guideline on Threats and Assets .

text: The Art. 13a Expert Group continues to meet several times a year, to develop and improve technical guidelines, to discuss upon the implementation of Art. 13a and to share knowledge and exchange views about past incidents, and how to address them.

text: 1.2. Scope

text: Considering the above, as several years have passed since the implementation of Art. 13a, an impact evaluation of the new regulation is the proper thing to do. The evaluation has the purpose of assessing the changes in outcome that can directly be attributed to the provision of Art. 13a, the effects caused by this particular regulation within the Telecom Package. The ultimate goal of this study is to assess the “as-is” situation and the outcomes produced through implementations of Art. 13a. The results of this study are intended for understanding the impact of the regulation on the European telecommunications industry, and for further improving the security and resilience of the networks and services within the sector.

text: 1.3. Target audience

text: This report is addressed to:  policy makers at EU level, including the European Commission and ENISA, in order to help them understand the impact of the regulation on the telecommunications market and EU digital single market and further improve the security and resilience of the sector;  policy makers within member states, in order to help them better understand the impact of the regulation on their internal market;  telecommunications providers, in order to help them better understand the impact of the regulation on their business;  any other stakeholder interested in this policy field;

text: 1.4. Impact evaluation methodology

text: This report is based on a qualitative survey type approach, consisting of a desktop research, an online surveys and telephone interviews. The approach aimed to collect all the available information regarding the national implementations of Art. 13a and identify the outcomes produced at national and European level. This helped in building a baseline on which the interviews and survey relied upon. With the background set from the desktop research, the interview guides and the survey focused on understanding the point of view of both NRAs and providers in their respective countries. Focus was given in understanding the differentiating factors in each country as well as the successes and failures. The interviews were structure around 5 key areas in order to provide the fullest picture:  The new security measures implemented in the member states ;  The transparency resulting from the incident report process;  The learning process resulting from the incidents ;  The level of collaboration between the stakeholders ;  The harmonization of the procedures within the European Union.

text: Respondents: 22 respondents (exclusively NRAs) for the online survey, 14 participants for telephonic interviews (both NRAs and providers).

text: 2. Status of Art. 13a implementation

text: 2.1. Introduction to Art. 13a 2.1.1. Requirements

text: Art. 13a states that the EU member states shall ensure that telecom service providers “take appropriate technical and organisational measures to appropriately manage the risks posed to security of networks and services” and “take all appropriate steps to guarantee the integrity of their networks, and thus ensure the continuity of supply of services provided over those networks”.

text: A closer look on the article, will reveal 3 terms: “security incidents”, “security breaches” and “integrity losses”:

text:  Paragraph 1 requires “that measures shall be taken to prevent and minimize the impact of security incidents on users and interconnected networks”  Paragraph 2 requires providers to “take all appropriate steps to guarantee integrity of their networks, and thus ensure the continuity of supply of services”.  Paragraph 3 requires “to notify the competent national regulatory authority of a breach of security or loss of integrity that has had a significant impact on the operation of networks or services”

text: The use of the term integrity in the article text may be confusing to some readers. We refer to the definition in technical literature on networks and network interconnections , which defines integrity “as the ability of the system to retain its specified attributes in terms of performance and functionality”. Integrity of networks would be called availability or continuity in most information security literature . In this document we call these types of incidents simply “security incidents” or “incidents” and we use the following definition in this document: a breach of security or a loss of integrity that could have an impact on the operation of

text: electronic telecommunications networks and services.

text: In this respect, although we may assume that Art. 13a relates mostly with the availability of the service, as its main purpose is to “ensure the continuity of supply of services”, the text of the article allows certain interpretations as regards to the scope. According to the graph below (Fig. 1), the requirements related to continuity/availability of the services has been implemented by 100% of the surveyed NRAs. Nevertheless, some countries have made a step further and covered in their national implementation other security concepts than availability, such as confidentiality (Fig. 1).

text: Furthermore, telecom service providers have the obligation to notify the competent NRA regarding any breach of security or loss of network integrity that has a significant impact on the operation of networks or services. In this case also, 100% of the NRAs implemented an incident reporting process for any disruption/ outages in the electronic communications networks and services, while a minority (20%) also included breaches impacting information confidentiality or denial of services attacks not necessarily impacting the availability of the electronic communications (as shown in Fig. 2). This requirement is also followed by all EU member states along with some EFTA members also.

text: Among these examples, what does your national implementation of

text: Article 13

text: a mainly address in terms of scope regarding the security measures?

text: Article 13

text: a mainly address in terms of scope regarding the incident reporting?

text: After receiving reports/notifications at a national level, NRAs shall report to ENISA any incident that had a significant impact on the operation of networks or services. As the term “significant incident” has not been defined by the directive, neither has been defined within the national implementations of Art. 13a in most of the countries, ENISA, in collaboration with the Art. 13a Expert Group has defined “significance” by adopting certain thresholds through a set of informal guidelines, adopted/referred to by all member states, as shown in Chapter 1 - Introduction.

text: 2.1.2. Networks and services in scope of Art. 13a

text: Art. 13a aims at safeguarding network availability through minimizing the impact of security incidents and ensuring the continuity of service supplying provided over those networks. The article applies for several types of networks and services, depending on the national implementation of the directive by each member state. Services such as mobile and fixed telephony but also to mobile and fixed internet are covered by all member states. In addition to these networks and services, NRAs included in Art. 13a several other services such as SMS (almost 70%), TV and radio broadcasting (45% and 31% respectively), according to their national implementations of the framework directive. For a full set of networks and services covered by national implementations of Art. 13a please see Annex I.

text: What publicly available networks and services are in scope on your national implementation of

text: Article 13

text: a, in addition to fixed/ mobile telephony and fixed/ mobile internet

text: As telecommunication networks and services can also be affected by other types of incidents that may or may not have an impact on the availability of the service, we have also tried to understand if stakeholders (meaning only NRAs) are satisfied with the current types of incidents covered by Art. 13a as regards to the overall security of electronic communications within a member state. 54% of the surveyed NRAs consider

text: that appropriate level of security within the telecom sector is achievable together with Article 4 of e-

text: Privacy Directive, so that more types of incidents could be covered.

text: Do you think that

text: Article 13

text: a is sufficiently clear and covers security of electronic communications?

text: 2.2. Situation of the countries before the implementation of Art. 13a

text: Before the implementation of Art. 13a, the telecom security landscape was fragmented and very specific to each country, with no pan European regulation. In other words, the legal maturity, practices and governance of the network resilience of each country were very heterogeneous which impacted the transposition and implementation process according to the level of maturity.

text: Although the 2002 Telecom Package tended to obliterate the legal differences among the countries at EU level by providing harmonized guidelines, countries kept their national prerogative and transposed the requirements into national law according to their countries specificities, market and maturity. However, the differences observed during the implementation process of the 2002 Telecom Package resurfaced in the approach taken by each country when implementing Art. 13a requirements from 2009. As you may see from Fig. 3 (or the Annex 1) although all countries have implemented mandatory incident reporting for service

text: disruptions on telecom providers, they do cover differently the types of networks and services in scope of

text: Art. 13a, meaning that incidents affecting different types of services are reported by each state.

text: 2.2.1. Approach of the NRAs and providers

text: Art. 13a has been received in many different ways by the consulted stakeholders, ranging from reluctance to enthusiasm but most commonly by skepticism. NRAs and providers were challenged to have a clear understanding of the scope and how to assess the criteria defining the impact of an incident or the appropriate level of security measures as per Art. 13a requirements, although the provisions of the European regulations were quite unclear.

text: NRAs

text: When Art. 13a was initially published, many NRAs were uncertain about the level of details and precision of the Directive, because several aspects of it were left – on purpose – open for the NRAs’ interpretation. For example, Art. 13a states that member states shall ensure that providers take all appropriate steps to guarantee the integrity of their networks, but does not specify what is meant by “appropriate steps” or how “integrity” is defined. In this situation, NRAs referred in their replies to the Technical Guideline on Incident

text: Reporting for further clarity in aligning as much as possible the domestic practices with the expectations at European level. Following the consultation of these guidelines, some NRAs issued second level regulations and/or some specific guidelines to support their national providers in their national implementation process.

text: In addition, NRAs were invited on a regular basis to participate in the Art. 13a Expert Group, an information exchange group especially created in this context by ENISA. This group had a critical role in federating NRAs during and after the implementation process. During these meetings, NRAs shared their point of view, experiences and thoughts about Art. 13a requirements.

text: Providers

text: As far as providers are concerned, the reaction of those consulted varied significantly and it can be directly correlated with their level of maturity in terms of country legislation and practices as well as to the level and maturity of collaboration with the NRA. Therefore, in the cases where providers were very mature, they were for most indifferent to Art. 13a, as such requirements were already included in their day-to-day practices and obligations.

text: As far as was shown from the qualitative approach of interviewing providers, those with a good level of maturity were either eager to improve their process and security measures, or reluctant due to the additional cost and due to stricter rules. Small providers or immature ones were either welcoming Art. 13a, identified as a strong opportunity to improve or reluctant to these new requirements, due to the additional effort to be provided while they face limited resources.

text: As a conclusion regarding the providers, whatever their maturity and development was, had whether a reluctant or embracing attitude towards adopting the new requirements, depending more on other causes (like business processes) than their maturity. Unfortunately, the survey could not capture more insights on this area, future work needing to be done.

text: 3. Appropriate security measures for providers

text: 3.1. Specific objective context

text: Art. 13a requires NRAs to ensure that providers take the appropriate security measures to protect and guarantee the integrity and service continuity of their networks. In that regard, ENISA collected feedbacks regarding the level of satisfaction with the security measures implemented and assessed the resulting outcomes for providers and NRAs.

text: 3.2. Level of satisfaction with the security measures implementation

text: It is noted that 45% of the interviewed NRAs are satisfied with the level of security achieved by

text: implementing new security measures in their respective countries. Telecom service providers share the same opinion, although some of them adopted an approach that goes beyond the current legal requirements, with continuous improvement process and more demanding security measures.

text: In your opinion, have

text: Article 13

text: a requirements led to stronger security measures in the sector in your country?

text: For 45% of the respondents, Art. 13a requirements led to stronger security measures in the sector and the country. But, the majority of interviewed NRAs think that it is not possible to attribute this improvement only to the new security regulations. This is reflected by the heterogeneity of the answers, which can be attributed mostly to the fact that:  The level of implementation differs from one country to another. Art. 13a requirements had an important impact on harmonizing the level of security measures among countries, as the general approach was highly different in some cases before the enforcement of the regulation. Although harmonizing has been achieved at the higher level, the situation still remains fragmented at lower levels of detail from one country to another.  NRAs do not have sufficient information from the providers to assess the impact of Art. 13a on the security measures already in place.  Providers’ maturity and the level of security measures vary from one country to another. Thus Art. 13a requirements will impact the level of security measures in a heterogeneous manner, with more improvement and stronger impact in countries where the providers are less mature and where the legal framework is weak.

text:  The variety in size between providers lead to a difference in impact. For instance, Art. 13a requirements had a much stronger impact on small providers than on large providers. Small providers experienced an increase in security severity and overhead, while large providers did not face any particular issue or strong benefits.

text: Based on opinions expressed, providers, on the other hand, expressed two different. Either (a) they were already experiencing such level of security measures and did not see any particular changes, resulting in indifference or neutral opinion, or (b) they were quite satisfied with the level of security measures included in Art. 13a. The responses of the providers only underlined linear development in the industry, as more

text: mature operators already had similar measures implemented, whereas the more immature ones had to

text: invest some resources in adopting the new measures.

text: 3.3. Challenges encountered during the implementation

text: On average, most of the NRAs faced relatively small challenges with regards to the implementation of the security measures. The most common were:  37% of the NRA respondents indicated challenges within the process of auditing and following-up on the security measures, meaning that after implementation of the regulations and the

text: subsequent security measures, authorities found it difficult to check whether those

text: measures/requirements were properly implemented.  27% of the NRAs indicated as a medium to big challenge the reluctance of some of the providers in implementing the new regulations;  over 25% of the respondents mentioned as a medium to big challenge the available budget and the administrative burdens;  21% indicated as a medium to big challenge the limited value for money as an outcome for the overall process, meaning that the expected benefits were not as consistent as expected.

text: It has been mentioned in the interviews that in most cases smaller providers were lacking the appropriate internal processes and methodology to implement these requirements. On top of that, most of them faced some operational challenges as they did not have sufficient resources and knowledge about the security measures to implement them without struggling at some point.

text: As revealed from the interview, another challenge in the implementation process of the security measures has to do with the lack of precision in the definition of “critical assets” and other key concepts such as “appropriate level of security”. Indeed, Art. 13a does not specify or define what is to be considered as a critical asset leaving the interpretation to NRAs. In order to overcome this challenge, providers and NRAs consulted the ENISA’s technical guidelines (see Introduction). Based on those some NRAs issued a second level of legislation as well as other guidelines.

text: As indicated within the interviews, the activities developed by ENISA within the area of Art. 13a, including

text: hosting and coordination of the Art. 13a expert group, was perceived as an essential support in easing the

text: implementation process and supporting NRAs and providers during their transition phase.

text: What were the main challenges that you faced when supporting providers in the implementation of security measures?

text: 3.4. Outcome/benefit analysis of stronger security measures

text: The implementation of the Art. 13a requirements affected both NRAs and providers, in terms of resources needed. Even though policy making and implementation is the day-to-day business of most NRAs, they faced additional costs such as: educational costs (for training the providers on the new regulations), costs for developing secondary legislation or other guidelines, follow-up and audit on the progress of the implementation. Providers faced implementation, maintenance and management costs. In their case, the size of the costs is largely depending on:  the average level of security measures already implemented in the providers in the country  the flexibility / ability of the providers to comply with the requirements within a short time frame  the complexity of the process for security measures implementation  the level of involvement of the NRA  the availability and allocation of resources (FTEs ) necessary for the implementation process. Some providers have been advertising security measures and resilience, as a competitive advantage. Especially in the countries where the level of the security measures was weak, providers emphasized the quality of their infrastructures, network and connections.

text: It is difficult to link the improvement of security measures with the evolution of the number of incidents experienced. Implementing better monitoring mechanisms may mean more incidents reported, which does

text: not necessarily mean that the security measures have worsened. Providers deal with some external risks that cannot be controlled nor prevented by an increase of the security measures. Such risks mainly include environmental risks and malicious actions. As such it becomes difficult to measure direct benefits attributed to Art. 13a.

text: Nevertheless, according to the consulted parties, Article13a has definitely helped in reducing the risks related to infrastructure resilience through reporting and learning. That is especially true for the smaller or less advanced operators, where technical assistance and know-how were needed. For larger operators, especially in more developed countries, the security measures where already implemented as part of their corporate strategy.

text: Based on your information, in which area did the providers improve the most?

text: The numbers from the Figure 7 discount the fact that in many areas, NRAs do not have sufficient information to assess whether Art. 13a brought improvements. For instance, 28% of the surveyed NRAs do not have information on the improvement level regarding human resources security and 18% don’t know if Art. 13a positively impacted business continuity management.

text: Moreover, NRAs estimate that in general, stronger security measures have indeed improved the overall security in their countries. As shown in Fig. 7, NRAs consider that most of the improvements were done in the following areas:

text:  incident management (27% fair amount or a lot of improvement)  operations management (18% fair amount or a lot of improvement)  security of systems and facilities (18% fair amount or a lot of improvement)  business continuity management (14% fair amount or a lot of improvement)  governance and risk management (14% fair amount or a lot of improvement)

text: It is reported that for front-runner providers, when Art. 13a came out, it did not bring strong benefits compared to the practices already in place. In other words, as the security measures and practices were

text: already strong and well implemented, front-runners did not gain strong benefits from Art. 13a security measures. Security measures positively impacted the quality of the infrastructures and thus of the services provided, which is the core business of telecom providers.

text: Regarding the improvements on the assets operated by providers, Fig. 8 expresses the opinion of NRAs on this matter.

text: Based on your information, in which asset class did the providers improve the most?

text: 3.5. Areas of further improvement

text: Suppliers of telecom components have a key role in the resilience of the infrastructures, but are not

text: included in Art. 13a requirements. In case of a security incident, only the telecom service provider is responsible while the telecom components suppliers are not part of the legal action/ incident action. Further assessments in this area are needed, with the involvement of responsible stakeholders, in order to develop an optimal approach to addressing this issue.

text: As more than half of the respondents (55%) haven’t indicated that Art. 13a has led to stronger security measures in the sector, and 60% of the respondents are not aware of areas where providers have improved the most, further analyses must be carried out in this area in order to determine next steps to be taken.

text: Adoption of common baseline security requirements for all EU telecommunication providers has been

text: indicated as a possible further development. ENISA has already done some work in this area, please check the Technical Guideline on Security Measures.

text: 4. Transparency in incident reporting

text: 4.1. Specific objective context

text: Transparency is achieved through the reporting obligation that falls on providers experiencing incidents and is driven by the quality of information provided. Transparency is fundamental for the NRAs as they need correct and relevant information in order to take adequate and proper supervision measures. NRAs also have a reporting obligation towards ENISA and the European Commission, as information collected at national level is sent at European level for aggregated analysis. Therefore the more the information is detailed and transparent, the more the reporting flow is straight forward and the less NRAs request complementary information, which is time and resource consuming. All in all, transparency is an asset for providers, NRAs and ENISA as it enables the stakeholders to understand the issues faced in order to take appropriate actions (by the providers themselves, but also by the NRAs who can assist the providers or amend the national regulations in place).

text: In order to promote transparency, NRAs usually issue templates for incident reporting with a list of the mandatory information and the level of details required. Besides, providers have a specific point of contact at the responsible NRA to enhance communication and anticipate any need for further information.

text: 4.2. Level of satisfaction regarding the transparency in the incident reporting

text: On average, NRAs are satisfied with the level of transparency reflected in the incident reports filled by the providers, meaning that the information quality of the incident reports is satisfying. This part assesses in more detail the level of satisfaction regarding the level of transparency, based on 3 perspectives:

text:  the provider’s compliance with the expected level of information transparency,  the information quality,  the number of incident reports collected.

text: 4.2.1. Providers’ level of collaboration

text: Providers’ attitude towards Art. 13a mandatory incident reporting regulations was mostly collaborative, as 68% of the respondent NRAs attest that providers were overall collaborative with the incident reporting compliance requirements (Fig. 9). Nevertheless NRAs also faced two other distinct behaviors. On the one hand, most of the providers were neutral or even indifferent (up to 40% of the responses) regarding the new incident reporting requirements and provided the reports as required as per the defined thresholds. This is due to the fact that some providers were already using such process at internal level and that Art. 13a did not bring major changes, constraints or benefits on this point. On the other hand, a minority of providers (21% of the responses) were reluctant towards the changes imposed by the new incident reporting process. These providers are mainly from countries were the practices are not as mature as in the most advanced countries and are, in most cases, small providers. This attitude can be explained by the investments that providers were asked to do in order to complete the requirements. Such investments can prove to be a real challenge for smaller providers with limited resources (financial or otherwise).

text: As an incentive to promoting a more collaborative and transparent approach for providers, 86% of NRA respondents stated they have issued several guidelines or country level legislation to fill in the lack of precision in the directive and mitigate the level of reluctance due to the lack of information. As shown by Fig. 10 below other measures adopted by NRAs to promote transparency are: provide online tools for incident reporting (59%), issue second level regulation on top of the standard requirements (inform customers (59%), inform general public on major disruptions (41%)), inform other member states, ENISA or the public for major disruptions.

text: What was the general response of the providers when they had to comply with incident reporting?

text: How do you promote transparency with regards to incidents?

text: Fig. 11 reveals that, in NRAs opinion, bringing more clarity to the process (by issuing guidelines and additional legislation – 73%), was by far the most effective method of improving transparency.

text: In your opinion, what contributed the most to improve transparency?

text: 4.2.2. Regarding the information quality

text: 77% of the respondent NRAs are satisfied with the quality of the information provided by the operators in

text: their incident reports. Information collected by NRAs mainly follows the ENISA guidelines on incident reporting. Most relevant pieces of information reported are: numbers of users/connections affected, duration of the incidents, services affected, assets affected, measures taken etc.

text: Is the information in the reports generally of enough quality in terms of details and descriptions?

text: In cases where the information provided is insufficient, usually NRAs contacted the providers to obtain the complementary information. Some common issues which were reported, include situations such as the incident description being too short, or the vocabulary used being too technical or too specific at the internal

text: lingo of the provider’s organization. Due to this feedback process, the level of the quality of the information self-maintains itself.

text: In the special case of emergency services incident or outage, providers are requested to provide more details and information in their incident reports, increasing transparency requirements for the providers.

text: 4.2.3. Regarding the number of incident reports collected

text: The number of reports collected by the NRAs is linked to several factors. First and most important is the level of thresholds that determines whether an incident report is triggered or not. As the directive does not specify details regarding thresholds or how “significance” should be detailed, the Art. 13a expert group, has issued a non-binding document to help member states define significant incidents. The ENISA’s Technical Guideline on Incident Reporting establishes absolute and relative thresholds, so that countries in different sizes in terms of population or providers in different sizes in terms of customers can be covered.

text: Do you get the appropriate number of incident reports by the providers?

text: 68% of the surveyed NRAs reported that they are receiving reports as per the expected levels, however

text: one third of the surveyed NRAs do not receive as many reports as expected. One of the reasons could be that the thresholds are not fine-tuned to the number of incidents, market or provider size; therefore the reports do not capture as many incidents as expected in the first place. The second reason has to do with the size of the provider. Outages from small providers do not impact as many users as the larger ones, and as such thresholds do not trigger incident reporting in their cases. As no additional information was obtained in this area, this remains still to be further analyzed.

text: Another challenge addressed by the majority of the interviewed NRAs, is that 90% of the telecom services providers are SMEs, which represent only 10% of the incident reports received by the NRAs. Given that smaller providers (SMEs) – usually – do not have enough customers to breach the reporting thresholds, the vast majority of incident reports come from the large operators (the 10%). Consequently, the number of incidents reported could represent a distorted view of the status of each country.

text: 4.3. Level of communication

text: Although according to the surveyed NRAs, all stakeholders are sufficiently informed about the incidents (at least 50% of all stakeholders are sufficiently informed), some nuances are to be mentioned. After a deeper look, it appears that if sufficient communication takes place, it is often upon affected users request that information is disclosed and exclusively to this specific user. Instead of receiving information from providers, affected users often have to inquire themselves on the experienced incident. In some cases, it also happens that incident notifications are not published in the media or on the provider’s website, whereas the thresholds have been reached.

text: Do you believe that stakeholders are sufficiently informed about incidents?

text: Communication with affected users is not an obligation triggered by Art. 13a requirements, but can be requested by the NRA if the disclosure of the breach is in public’s interest. Many NRAs took the opportunity to introduce a transparent communication with the population.

text: Besides the mandatory incident reporting, do you get information about incidents in other ways?

text: 4.4. Challenges

text: From the interviews conducted, it appeared that the main challenge for providers, in terms of incident reporting and transparency is to compile a report while the incident handling process is in progress (77% of the NRAs answered that they require notifications from providers during ongoing incidents). In this case resources are allocated to the reporting process whereas they could be affected to the resolution of the incident. The full use of the resources for reaching other providers practices, level and maturity can be detrimental for the providers which are still in the maturation process. The incident may last longer and impact the providers’ reputation and finances.

text: Do you require notifications from providers during ongoing incidents?

text: 4.5. Outcome/Benefits analysis regarding transparency 4.5.1. Costs for the incident reporting from transparency requirements

text: Based on the graph below, for the majority of the NRAs, transparency requirements as per Art. 13a had a relatively low cost. The main costs regarding the improvement of transparency are the development of a new tool to support the providers in their incident reporting process and the creation (in some cases) of a dedicated team. Indeed 60% of the NRAs had to increase their workforce and faced this issue with a medium or high cost.

text: What were the main costs for the NRA associated with improving transparency?

text: The burden supported by the providers with regards to the transparency highly depends on whether the indicators used by the providers are the same as the indicators used by the NRAs to assess the impact of the incident. When the indicators used are not the same between providers and NRAs, providers have to generate two reports with different indicators to assess, which is time and resources consuming. As a consequence, the quality of the report differs from one provider to another.

text: 4.5.2. Benefits of the incident reporting from transparency requirements

text: Although for more than 60% of the surveyed NRAs, the implementation of an incident reporting process had greater benefits than costs, the rest (40%) remains quite neutral on the question. It is certain that such process has been beneficial for the providers and NRAs, but 40% of neutral opinions show that there is still some space for further improvement.

text: Do you think that the costs of implementing incident reporting are less than the resulting benefits?

text: Incident reporting and transparency requirements, as requested by NRAs in their incident report templates, do contribute to the improvement of the information quality and quantity. Such transparency requirements benefit the NRAs because they have uniform and detailed information about the incidents in order to generate reports and country level analyses.

text: 4.6. End – user feedback regarding Art. 13a

text: As part of this survey NRAs were inquired on whether the end-users have expressed any opinion on Art. 13a and whether there’s any type of communication regarding improvements. The general consensus is that NRAs do not have feedback from end-users regarding these effects. Most of the public remains unaware of the initiative, and even if some improvement in the communications has been noticed, it would be difficult for the NRAs to gather the sentiment, and even more so, to attribute it to Art. 13a – related procedures (rather than advances of technology for example). Consequently, no feedback has been collected from the end – user (citizen) perspective.

text: 4.7. Area of further improvement

text: Providers only officially report incidents to the NRAs but end-users do not have access to such information. Being business-impact orientated, in opposition to NRAs and ENISA which are more end-users orientated, some providers do not proactively communicate directly to their customers on the on-going faced incident. End-users have to either request the information or wait for the yearly report from the NRAs and ENISA to obtain information on the past incidents. Further developments in this area are needed to add more transparency to customers and the general public, by periodically informing them about outages, measures taken etc.

text: Current thresholds are more likely to trigger an incident report from large providers than from the small providers, resulting in NRAs receiving more incident reports from large providers than from the small providers. Consequently, NRAs have an overview only of the incidents that impact large providers. Thus NRAs should adapt the thresholds to capture incidents experienced also by small providers. However this option is difficult to implement because it assumes a very delicate fine-tuning of the thresholds in order to cover incidents affecting smaller providers.

text: 5. Learning and improving based on reported incidents

text: 5.1. Incident reporting as a tool for learning

text: The incident reporting process is a tool itself to learn from incidents and to sustain a continuous and dynamic internal learning process for both providers and NRAs. According to the surveyed NRAs, incident reporting is used as a tool for learning in many areas, as shown in Fig. 19. Statistical purposes, post incident supervision, periodic reports, developing guidelines and root causes studies are areas where incident reporting is mostly used for learning, confirmed by 70% or more of the NRA respondents.

text: How do you use the incident reports you receive in the learning/ improvement process?

text: 5.2. Outcomes/benefits of the learning process

text: Incident reporting and the resulting learning process contribute to continuously strengthening and maintaining providers’ infrastructures. As reflected in the graph below, the notion of the reporting obligation pushes providers to improve (identified as an important benefit by 77% of the surveyed NRAs), and is an input for sector wide risk and vulnerability assessment (qualified as “quite important benefit” by 69% of the surveyed NRAs). 87% of the respondents also stated that the incident reporting process is used as an input for appropriate policy decision, meaning improving regulations within the sector.

text: In your opinion, what are the main benefits from incident reporting?

text: However incident reporting and a learning process cannot guarantee a drop in the number of incidents, especially in case of incidents that are due to environmental causes, although they surely contributed to improving the overall process of addressing different types of incidents.

text: 5.3. Areas of further improvement

text: In this part, the suggestions made in order to further improve the learning process, as communicated by the surveyed NRAs and providers, are conflicting. Some NRAs and providers think that sharing more information regarding the learning and preventive actions taken could help other providers in their own incident solution process and would bring stronger knowledge to small providers. In addition, cross country collaboration on incidents and supervision matters for NRAs could be reinforced by more information sharing.

text: However, this suggestion raises the issue related to confidential information disclosure and a concern from providers regarding the risk of being penalized for the incident that they have experienced. In addition, such information sharing may not bring as many benefits as expected to the other providers as the impact and the corresponding solution may be specific to the infrastructures of the providers. Therefore driven solutions and lesson learnt may not be compatible from one situation to another or from one provider to the other.

text: As you may notice NRAs mostly use the incident reporting process as a tool in learning/improvement but mostly for compliance, internal statistics and improve regulations (Fig. 19). The percentage of those who use the results of the process also as an input for sector wide or nationwide risk assessment is rather low.

text: Considering widening the usage of the results to other areas, related to national security or industry

text: related risk analysis, could be a further improvement in this area.

text: 6. Collaboration between the actors

text: 6.1. Specific objective context

text: Collaboration is at the very heart of the implementation process, especially when the telecommunications’ market landscape is quite different from one country to another. A successful implementation at national level is possible only if the NRA and the national providers work together, helping each other to overcome the challenges. NRAs involvement is all the more important given that small providers do not have as much experience as the front-runners and may face more difficulties in the implementation process. However, such collaboration faces some limits when brought at EU level. Indeed, providers and NRAs are hesitant in sharing confidential and sensitive data to other stakeholders, even in the context of Art. 13a and for collaboration and improvement purposes.

text: 6.2. Collaboration during the implementation of Art. 13a requirements 6.2.1. Collaboration between ENISA and NRAs

text: ENISA’s role has been to support NRAs along the implementation process of the requirements mandated in Art. 13a, to provide expertise, advice and to promote the exchange of good practices among the member states. In order to do so, ENISA built a strong relationship and collaborative working environment. As a matter of fact, 68% of the surveyed NRAs agreed/strongly agreed that ENISA‘s role was beneficial for the implementation process.

text: How would you describe the collaboration between the NRAs and ENISA during the implementation of

text: Article 13

text: a?

text: ENISA worked closely with the NRAs to provide more detailed information to them as well as the providers and has issued several guidelines. These guidelines have been helpful for the NRAs to better understand the requirements of Art. 13a, especially the scope and definitions, and to have a concrete perception of the threshold levels to set up. Indeed, ENISA’s guidelines provide definition and advice to NRAs and providers

text: on how to consider the interpretation of the requirements, and were very well received and appreciated

text: by the community (as stated by 68% by the respondents). Exchange of information between NRAs is facilitated by the Art. 13a Expert Group meetings, which are organized on a regular basis, and aim at involving all the EU member states in an open discussion about Art. 13a. This observation includes discussing implementation in detail, sharing knowledge and exchanging views, in order to achieve a harmonized approach of implementing Art. 13a.

text: 6.2.2. Collaboration between NRAs and providers

text: As shown by Fig. 22 most of the NRAs indicated some overloading as regards to the implementation

text: process of Art. 13a, but most of them rarely indicated overloads of more than 25% of the total time of the

text: responsible personnel involved (FTE). In order to support their domestic telecom service providers, dedicated personnel within 20% of the NRAs spent half or more than half of their time to issue second level regulations, guidelines and educate providers. Developing reporting tools for providers was also indicated as time consuming by less than 20% of the respondent NRAs. Regarding the collaboration between NRAs and providers, as described in Fig. 23, the majority (77%) stated that the collaboration lasted for the whole implementation period, and 64% of them declared that the providers requested additional supporting documentation, as guidelines or advice. Given the level of detail provided within the directive, we can certainly say that collaboration was more than necessary at this level, both at European level and at national level.

text: How much time did your NRA's FTEs spend during the implementation phase on the following activities related to

text: Article 13

text: a requirements implementation?

text: How would you describe the collaboration between providers and the NRAs during the implementation of

text: Article 13

text: a? 6.3. Collaboration after the implementation of Art. 13a requirements

text: As regards to the situation after the implementation period of Art. 13a, we can notice from Fig. 24 that collaboration between NRAs and ENISA was still needed, as by the majority of the respondents. More specifically guidelines on particular areas were still needed, and the overall activities carried out by ENISA alone or within the Art. 13 Expert Group are considered beneficial by more than 80% of the respondents. ENISA continues to hold the Art. 13a Expert Group Meetings with the intent to keep a high level of collaboration and information exchange between the member states. The Expert Group is also a way for ENISA to measure the level of harmonization in regards to the member states practices.

text: The estimated time spent by the NRAs’ FTE dedicated personnel during the implementation phase of Art. 13a (Fig. 22), with the time spent after the implementation (Fig. 25), is more or less the same, highlighting that Art. 13a has developed to a continuous process within NRAs. Although the implementation process is complete, providers still need the support of the NRAs, as further improvements to the incident reporting process are developed periodically and, as statistics show, the number of incidents is increasing. The only activity that sees its allocated time decreasing is the supplying of guidelines/secondary legislation to providers, as the market has reached a certain level of maturity within this area.

text: How would you describe the current collaboration between providers, NRAs and ENISA after the implementation of

text: Article 13

text: a?

text: However the need for developed tools and supervision/audit of the providers remains almost the same as during the implementation process. This is the consequence of a need for a continuous effort in monitoring the measures implemented for compliance with Art. 13a.

text: This graph (Fig. 25) confirms that the cross border collaboration between NRAs remains quite limited as

text: 82% of the surveyed NRAs estimate spending less than 10% in exchanging information with NRAs in other

text: member states impacted by the same type of incident. In addition interviewed NRAs confirmed that they did not often collaborate or exchange information with other NRAs on a regular basis.

text: Regarding the supervision and audit of the providers, 63% of the dedicated personnel within NRAs spent up to 50% of their total time doing this activity. Other time consuming activities are gathering data for reporting to ENISA, develop tools and meet with providers on a regular basis etc.

text: How much time are your NRA's FTEs spending currently on the following activities related to

text: Article 13

text: a? 6.4. Areas of further improvement

text: The opinions on the possibility and benefits of improving the current level of collaboration differ among different types of stakeholders. On the one hand, for some NRAs, it would be useful to collect incident reporting and information from national providers, which will be anonymized, in order to share this information with the other NRAs, so their domestic providers have access to other providers’ incident reports, experiences and practices. On the other hand, some NRAs do not agree with this proposition for several reasons. First, if the incident is a major incident and has been related in the media, it is always easy to find the identity of the provider. Secondly, some NRAs state that each incident is specific to a provider, a market structure, the providers’ infrastructures, etc. and that it is not possible to adapt preventive measures from one country to another where the culture, infrastructures and market structure is not the same. Meanwhile ENISA is trying to develop, along with the Art. 13a expert group, a provider to provider sharing capability in order to facilitate this kind of cooperation and, of course, satisfy the majority of requests by member states. However, providers do share their need for more information exchange on the incidents experienced by other providers. This information sharing is already in place in some countries, under the NRA’s supervision and in project in some others. Providers also underline their continuous need for feedback and support from their NRAs on incident reporting quality and assessment, and a true bi-directional beneficial communication. Last but not least, collaboration could be improved further, as the different indicators used by NRAs and providers to measure the impact of an incident are not aligned. If this situation is persistent, it shows that communication and collaboration are not optimum between NRAs and providers.

text: 7. Harmonization of practices within the EU

text: 7.1. Paradox: harmonization at EU level vs fragmentation at country level 7.1.1. Harmonization vs differentiation

text: In the European Union, directives prevail on national law and bind upon the EU member states as to the implementation of the requirements, regardless the specificities of the national law. EU member states have an obligation of transposition into domestic legislation, although they are free to choose the means and the ways to reach this objective within the given time frame. Lastly, countries are always free to adapt the requirements on the national level. Differences between countries can include: difference of maturity, different practices and cultures, etc. For example, some countries issue one or two incident reports annually, while other countries can issue hundreds.

text: If the directive sets up minimum standards to be implemented within national legal framework, member states are free to apply more demanding national measures, as long as they do not conflict with the requirements of the directive and with the free movement and free market rules. Although this principle wants to respect countries’ sovereignty, this is a source of differentiation among countries. If the legal framework is by definition harmonized at EU level, the content of the domestic law is impacted by the countries specificities and needs. For instance, thresholds that trigger the incident reporting process are different from one country to another, although the NRAs referred to the ENISA’s Technical Guideline on Incident Reporting. This guideline and the others aim at providing more details and indications to the NRAs in order to facilitate the implementation of Art. 13a requirements.

text: As per the Directive

text: , member states were required to conduct a national public consultation to “give interested parties the opportunity to comment on the draft measure within a reasonable period ” but remain free to take the results more or less into consideration.

text: In the context of Art. 13a, the transposition of the directive into domestic law has been successfully achieved by all the member states but one, which is still under progress. Furthermore, the implementation process has not been completed within the same timeline. For example, some countries implemented Art. 13a requirements by 2011, while some countries needed an additional time period to complete the process by 2012, 2013 and 2014.

text: If countries kept their specificities and regulation prerogatives, Art. 13a definitively helped to harmonize the provider and member states general approach regarding incident reporting.

text: To what extent are you using

text: Article 13

text: a Technical Guideline and studies in your national implementation and work?

text: However, if the NRAs do refer to the ENISA technical guidelines, reports and studies, most of them use those documents as an inspiration or guidance when the European regulations or domestic laws are not clear enough (on average 30% of the surveyed NRAs use most or all of ENISA’s technical guidelines on incident

text: reporting and on security measures in their work, and 40% of the NRAs use ENISA’s technical guidelines

text: on incident reporting and on threats and assets as a source of inspiration in their work). Nevertheless, ENISA’s technical guidelines are the most important source of harmonization, given the fact that the Directive does not provide sufficient level of details regarding the scope and thresholds for the incident reporting process, and also lack of definition of what to consider as an asset.

text: On the other hand, the frequent meeting groups held by ENISA enabled NRAs to share good practices, experiences and advices among themselves, resulting in a trend for harmonization of the practices within the member states. This harmonization is driven indirectly but does not reach every country.

text: 7.1.2. Different approaches in implementing Art. 13a requirements

text: Most of the countries held a public consultation, according to Article 6 of the Directive

text: , before adopting the new regulations; the NRAs as regulatory authorities are free to take the results of the consultation into consideration or to follow their own prerogatives. Therefore, the levels of input from the providers are different from one country to another and impact more or less the domestic legislation.

text: Furthermore, if the majority of the countries benefit from a second level legislation, some countries do not have such legal framework and are keener in interpreting the Directive and refer to the guidelines than countries which have such level of regulation. ENISA’s technical guidelines remain in each situation the main point of information, although countries did not implement Art. 13a through the same level of regulation and legal means.

text: In most cases, countries with a second level legislation are the most advanced countries with regards to their legal framework and good practices, resulting in different level of efforts required in comparison with countries that have a very young or immature legal framework. Given that some countries had already a more mature legal framework in place and a NRA issuing supportive guidelines, there were only limited efforts to be provided for the implementation of Art. 13a requirements.

text: Consequently, as the ways and means used to implement Art. 13a requirements differ from one country to another, the thresholds for incidents reporting defined and implemented in the countries differ. First some NRAs set up their thresholds based only on ENISA’s Technical Guideline on Incident Reporting, while some other countries took in consideration the comments collected during the public consultation to adapt the level of thresholds proposed in the guidelines. Last but not the least, some countries implemented the thresholds based on the guideline but updated them later to better fit the market specificities, sector practices and the need to collect more information.

text: Harmonization of the regulation within the European Union will facilitate the continuous development of a single EU market. Furthermore, diverging regulatory requirements across EU member states may deter and hinder effective competition and thus limit the benefits for some customers in specific countries. Harmonized procedures in terms of incident reporting will also guarantee customers getting access to information related to interruption of service or other types of incidents. On the other hand, harmonized security measures and incident reporting processes will enable cross-border providers to realize economies of scale, as they are already familiar with the incident reporting process, the requirements and then benefit from synergies in the training materials, tool usage, incident management, lessons learnt on passed incident in other countries, etc.

text: For ENISA, a harmonized approach in the incident reporting is necessary to compare, benchmark and assess the situation within the European Union. Without a common template, indicators and process, the assessment and benchmark of the incidents would be complicated.

text: 7.2. NRAs level of satisfaction regarding harmonization

text: Taking into account the mandatory level or harmonization imposed by the directive, the necessary level of harmonization induced by ENISA’s guidelines and the common specificities noticed at member states level,

text: 90% of the surveyed NRAs declare that they are satisfied with the level of harmonization within the EU,

text: which sustains also national specificities (as shown in Fig. 27).

text: Do you think that the level of harmonization is appropriate up to date? 7.3. Areas of further improvement

text: The interviews and the online survey revealed that the 82% of the NRAs are currently satisfied with the

text: level of harmonization, and do not think that an improvement is needed for the member states and the

text: telecom market.

text: Do you think the way the harmonization has been carried out at EU level is appropriate?

text: However, 18% of the surveyed NRAs do think that ENISA could improve the current level of alignment among the NRAs regarding the secondary level of legislation and supervision methodology. Also, an enhanced level of alignment would reduce the differences between the countries and enable cross-border service providers to reduce the current faced costs and barriers.

text: Nonetheless, improving the level of harmonization might be a challenge as the current achieved result is already quite satisfying for ENISA also. Besides the more the harmonization level is raised, the more countries tend to lose their particularities, as created by their internal markets. The risk is that countries start to be reluctant to the implementation of new and stricter regulations.

text: On the other hand, it may be more difficult for cross border operators to continue to expand and to grow with these differences among the countries, regarding the security measures, the security incident reporting triggering thresholds and the security incident indicators. Complying with all the different legislations requires considerable much effort and could impact the decision to enter the market and compete with the already established national providers.

text: 8. Impact evaluation based on reported incidents

text: 8.1. ENISA’s annual incident reports

text: For four years now, ENISA publishes the annual report about significant disruption/outage incidents in the EU electronic communications sector, which are reported to ENISA and the European Commission under Art. 13a of the Framework Directive (

text: ) , by the National Regulatory Authorities (NRAs) from all member states. This annual report covers the incidents that occurred in a year and it gives an aggregated analysis about severe outages across the EU, keeping details about individual countries or providers confidential.

text: As you may notice from the chart below (Fig. 29), the number of incidents reported every year is continuously increasing, showing year by year, the stakeholder’s maturity in identifying, collecting and processing more and more incidents. The number of reported incidents has grown along with the number of reporting countries. Only 51 significant incidents in 2011 reported by 11 countries, and 137 incidents reported by 25 countries in 2014.

text: 137 90 79 51 Incidents reported 2011 2012 2013 2014

text: Looking at the root causes, as shown in the figure below, system failures is one of the main causes that triggers disruptions in Europe. On a closer look, we will notice that a part of the system failures were actually third party failures (16% from all incidents in 2014, 11% in 2013), meaning failures caused by other parties along the supply chain and not the providers that suffered the actual incident. These findings raise other issues regarding causes of disruption incidents around Europe, and how to address them.

text: 80 76 70 66 61 60 47 50 30 19 20 20 14 12 12 8 9 10 6 6 5 6 5 0 2011 2012 2013 2014 Natural phenomena Human errors Malicious actions System failures

text: Supply chain security and responsibility is one of the issues to be taken into consideration in future developments within the area.

text: 8.2. Areas of further improvement

text: In agreement with the NRAs in the Art. 13a expert group, at the moment incidents affecting only 4 types of services are expected to be reported to ENISA:

text:  Fixed telephony  Fixed internet  Mobile telephony  Mobile internet

text: However, the types of services offered by modern telecommunications providers have developed over the years (e.g. messaging services, software based voice services). Consequently the range of covered services within ENISA’s annual report needs to be updated, by adopting the modern services offered by providers.

text: As third party failures represent an important part of reported incidents, signaling different issues within this area, further actions need to be considered in the near future. Developing requirements regarding the

text: supply chain security and responsibility could be a further action to be considered by responsible stakeholders.

text: 9. Key findings and conclusions

text: Art. 13a gave a new momentum in the telecom industry at European level. By amending the 2002 regulations, the Art. 13a within the Framework Directive of the 2009 Telecom Package, addresses security and resilience issues for the first time in the EU.

text: As several years have passed since the publication and implementation of the new regulations, an impact evaluation of the new provisions was the proper thing to do. The evaluation has the purpose of assessing the changes in outcome that can directly be attributed to the provision of Art. 13a, and the effects caused by this particular regulation within the Telecom Package.

text: Despite the obvious positive outcomes that have been expressed by the majority of the respondents, there are also some areas of further improvement. Clarifying the scope of art. 13a in order to provide clear information on the types of networks and services that should be covered is one of them, along with reinforcing cross-border collaboration and also other areas that can contribute to a higher degree of resilience of European networks and services.

text: Overall, Art. 13a has contributed to improving the level of security in the telecommunication sector but in a balanced way as some counties were already in line or even ahead of the requirements and were already experiencing the expected benefits. By opposition, the less advanced countries and providers experimented strong benefits and improvement in their security measures and infrastructures resilience in spite of the costs and efforts provided.

text: It has been also noted throughout this project that further analyses are needed in order to draw some strong conclusions on next steps that are needed in this area. The short period of time allocated to this project along with the complexity of the area, prevented the project team to further analyze additional details and areas that could also have influenced the result of this evaluation. In this respect, this report will only cover a set of findings that must be further analyzed in order to propose concrete recommendations and next steps to be considered by different types of stakeholders. Main findings of the study are the following, grouped into categories:

text: The scope of Art. 13a

text: 2. More than half of the respondents (54%) considered that Art. 13a cannot sufficiently and clearly cover by itself security of electronic communications, but together with Art. 4 in the e-Privacy Directive.

text: 1. The majority (45%) of the respondents (NRAs) considered that Art. 13a has led to stronger security measures within the sector, but further analyses are needed as more than half of them (55%) do not share this opinion, 23% stating “no” and 32% “don’t know”. 2. Almost 60% of the respondent NRAs are not aware of the areas where the providers have improved the most, in terms of security measures.

text: Transparency in incident reporting

text: 1. The approach of the providers towards NRAs as regards the implementation of Art. 13a mandatory incident reporting regulations was mostly collaborative. Withal the majority of the respondent NRAs are satisfied with the quality of the information provided by the operators in their incident reports and declared that they are receiving reports as expected. 2. Bringing more clarity to the incident reporting process, by issuing guidelines and additional legislation – 73%, was by far the most effective method of improving transparency.

text: Collaboration between the actors

text: 1. The establishment and development of Art. 13a expert group, under ENISA coordination, turned out to be a successful and helpful experience (more details in section 6.3.), as appreciated by 80% of the respondents. The operation and development of the group should definitely be continued, under ENISA’s coordination. 2. Bi-directional communication with the population in the incident reporting process is poorly addressed by both NRAs and providers. Further analyses should be carried in order to determine the necessity of developing such processes (more details in section 4.3.). 3. The amount of resource employed by NRAs in the cross-border collaboration area appears to be low (more details in section 6.3.).

text: 1. More and more reported incidents (at ENISA level) are caused by third party failures (more details in section 8.2.), meaning they are caused by parties out of provider’s direct control, but within the provider’s supply chain.

text: 2. The main root cause for incidents at EU level in 2014 and years before, is “system failures”. Further assessment needs to be done in this area, in order to identify more detailed causes and security measures that can be adopted.

text: Annex A: Network and services covered by national implementations of Art. 13a

text: * Countries that have responded within the given time period. 44

text: ENISA

text: European Union Agency for Network and Information Security Science and Technology Park of Crete (ITE) Vassilika Vouton, 700 13, Heraklion, Greece

text: Athens Office

text: 1 Vass. Sofias & Meg. Alexandrou Marousi 151 24, Athens, Greece Catalogue Number TP-04-15-873-EN-N

text: PO Box 1309, 710 01 Heraklion, Greece ISBN: 978-92-9204-149-6 DOI: 10.2824/491369 Tel: +30 28 14 40 9710 info@enisa.europa.eu www.enisa.europa.eu