text: www.enisa.europa.eu European Union Agency For Network And Information Security

text: Security incidents indicators - measuring the impact of incidents affecting electronic communications

text: Figure 1: Challenges of measuring security incidents - NRAs

text: 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Difficult to Difficult to collect Difficult to clearly Difficult to Different Administration Other identify appropriate data define indicators accurately measuring burden appropriate measure the methodologies indicators indicators between provider and NRA No Challenge Small Challenge Medium Challenge Big Challenge No answer

text: Figure 2: Goals and benefits of measuring security incidents – NRAs

text: For you as NRA, what are your benefits and goals when using such indicators ?

text: Other Have an overview of the main types of incidents… Input for discussing challenges and issues with… Identify the cause of failure and/or isolation in… Improve security nationwide Monitoring purposes (i.e. have a detailed log of… No answer Issue guidelines on managing incidents Not applicable Issue guidelines to providers on strengthening… Indirect Comply with EU regulations goals/benefits Secondary Compare impact with different operators or NRAs goals/benefits Key Identify the impact of incidents on citizens goals/benefits Perform risk assessments at national level Improve transparency and incidents reporting Better incident response management Used as a basis for maintain statistics Identify the type of incident

text: Figure 3: Challenges of measuring security incidents - Providers

text: 60% 50% 40% 30% 20% 10% 0% Difficult to Difficult to collectDifficult to clearly Difficult to Administration Limited value for Insufficient Time consuming Other identify appropriate data define indicators accurately burden money budget appropriate measure the indicators indicators No Challenge Small Challenge Medium Challenge Big Challenge No answer

text: Figure 4: Goals and benefits for measuring security indicators - Providers

text: For you as a provider, what are your main goals or benefits when using such indicators?

text: Minimize impact for customers Have an overview of the main types of incidents in the network (kind of risk assessment) Input for discussing challenges and issues with other providers and other sectors Identify the cause of failure and/or isolation in order to prevent them from happening again Improve security and resilience of networks and services Assess whether you need to invest in stronger infrastructure No answer Strongly agree Comply with national regulations Agree Perform risk assessments (network level) Neutral Inform other providers about major impact following Disagree incidents Inform the NRA about major impact following incidents Strongly disagree which could have a possible impact in other Member States Achieve greater transparency by improving the incidents reporting process Better incident response management

text: Used as a basis for maintain statistics Identify the type of incident

text: Interexchange Carrier or long-distance telephone company.

text: Figure 5: Similarity of indicators among NRAs and providers – NRAs perspective

text: 0% Yes, the indicators used by the different providers are similar

text: 23% Yes, the indicators used by the different providers are similar but some providers use additional indicators 54% 15% No, the indicators used by different providers vary quite a lot 8%

text: I don't know the indicators used by the provider

text: Of providers felt apparent, therefore making it much more difficult for the NRAs to come to know of that there was a it. good level of collaboration 15% of survey respondents mentioned that the indicators used by different between them providers, varied quite a lot from one provider to another and 8% expressed that and their NRAs the indicators used were similar, but that some providers were using additional indicators. From this, it is reasonable to say that NRAs had differing opinions on the level of harmonization between them and their providers. Some felt that the harmonization levels were good and that there existed a good alignment between the two, whereas others expressed that this level of harmonization could be further improved. Differing opinions in harmonization levels was also perceived among providers as well. 54% of the respondents from the survey felt that there was a good level of collaboration between them and their NRAs. That being said, 13% described that they were in fact using a greater number of indicators than what had been defined by their NRAs. As resulted from the response, it’s quite normal that a large provider uses more indicators that those requested by the NRA. While the NRA is interested in “major” indicators of impacts on citizens, the provider needs a lot more indicators to help manage the day-to-day operations. Nevertheless, it cannot be assumed that the indicators that the NRA requests are within the indicators already used by the provider.

text: Figure 6: Similarity of the indicators among NRAs and providers –Provider’s perspective

text: 0% Yes, we collaborate with the NRA to define adequate indicators to measure 13% the impact of security incidents Yes, but we use a greater number of indicators than what is defined by the NRA 13% 44% Yes, but we use a fewer number of indicators than what is defined by the NRA No, we have two different methodologies of measuring the 17% impact incidents I don't know the indicators used by the 13% NRA

text: Figure 7: Alignment of the objectives between NRAs and providers– NRA’s perspective

text: 0% Yes, both us and the providers have the same objectives while 15% measuring the impact of security 23% incidents

text: 8% Somewhat, both us and the providers have some objectives which are the same while measuring the impact of security incidents 54% No, objectives are different

text: I do not know the objectives of the providers

text: Figure 8: Alignment of the objectives between NRAs and providers – Provider’s perspective

text: Yes, both us the NRA have the 0% same objectives while measuring the impact of security incidents

text: 17% Somewhat, both us and the NRA 30% have some objectives which are the same while measuring the 9% impact of security incidents No, objectives are different

text: I do not know the objectives of 44% the NRA

text: Figure 9: Satisfaction of security measurement methods – NRA’s perspective

text: 0% Yes, providers timely and accurately assess the impact of 15% security incidents 23% Somewhat, providers assess the impact of security incidents

text: No, We are missing details on the impact of incidents affecting the sector No answer 62%

text: Figure 10: Satisfaction of security measurement methods – Provider’s perspective

text: Yes, the guidelines help to define appropriate indicators and assess the impact of security 0% incidents Somewhat, the guidelines help to define 13% appropriate indicators and assess the impact of security incidents but we would like to have better guidelines No, we would like to have better guidelines to 44% define appropriate indicators and assess the

text: impact of security incidents No, we do not receive any support/guidelines from the NRA

text: We do not need any support/guidelines from 13% the NRA

text: Figure 11: Similarity of goals and objectives between the NRA and provider

text: SIMILARITY OF GOALS AND OBJECTIVES NRA VIEWPOINT PROIVDER VIEWPOINT

text: Objectives of the NRA and provider are somewhat aligned (54%) (44%)

text: Objectives of the NRA and providers are exactly the same (23%) (30%)

text: I don’t know of the indicators used by the NRA/Provider (15%) (17%)

text: Objectives of the NRA and providers are different (8%) (9%)

text: Figure 12: Commonly used indicators among NRAs and providers

text: Average percentage of indicators used among NRAs and providers for all technologies

text: Impact on availability Impact on integrity Impact on confidentiality Impact to recoverability Number of base stations… Impact on interconnections Impact on sensitive areas Impact on emergency services Networks and assets impacted Services affected Geographical area impacted Fluctuating timeperiod Time to recover from an… Specific time at which an… Duration of an incident Number of users impacted

text: Providers NRAs

text: Figure 13: Measuring incidents based on number of users affected

text: Which indicators do you currently use to assess the impact of security incidents affecting the electronic communications sector: Number of users impacted

text: 0% Fixed telephony Mobile telephony Fîxed internet access Mobile internet access Other NRAs Providers

text: Figure 14: Measuring incidents based on time – NRAs perspective

text: Time/Duration related indicators

text: 120% 100% 80% 60% 40% 20% 0% Fixed telephony Mobile telephony Fixed internet access Mobile internet access Other Duration of an incident Specific time at which incident occured Time to recovery Fluctuating time period of an incident

text: Figure 15: Measuring incidents based on time- Providers perspective

text: Time/Duration related indicators

text: 80% 70% 60% 50% 40% 30% 20% 10% 0% Fixed telephony Mobile telephony Fixed internet access Mobile internet access Other Duration of an incident Specific time at which incident occured Time to recovery Fluctuating time period of an incident

text: Figure 16: Measuring incidents based on area impacted

text: Which indicators do you currently use to assess the impact of security incidents affecting the electronic communications sector Geographic Impact

text: Figure 17: Measuring incidents based on services affected

text: Which indicators do you currently use to assess the impact of security incidents affecting the electronic communications sector: Services affected

text: 80% 70% 60% 50% 40% 30% 20% 10% 0% Fixed telephony Mobile telephony Fîxed internet access Mobile internet access Other NRAs Providers

text: Figure 18: Measuring incidents based on emergency services

text: which indicators do you currently use to assess the impact of security incidents affecting the electronic communications sector: Impact on emergency services

text: 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Fixed telephony Mobile telephony Fîxed internet access Mobile internet access Other NRAs Providers

text: Annual Incident Report 2014

text: Figure 19: Economic related indicators to measure security incidents

text: Measuring the impact of an incident based on factors impacting the economy

text: 13% Loss of market share Actual loss of revenue 50% Cost to recover 37% No answer

text: Figure 20: CIA related indicators to measure security incidents – provider’s perspective

text: 30% 40% Confidentiality Integrity Availability

text: Figure 20: CIA related indicators to measure security incidents – NRAs perspective

text: Confidentiality Integrity 55% Availability 27%

text: Figure 21: Indicators used per technology- Fixed technology

text: Indicators per technology: Fixed telephony

text: 120% 100% 80% 60% 40% 20% 0% Number of users Duration of an Time to recovery Geographical Impact on Services affected Networks and impacted incident area impacted emergency assets impacted services NRAs Providers

text: Figure 22: Indicators used per technology- Mobile Telephony

text: Indicators used per technology: Mobile telephony

text: Figure 23: Indicators used per technology – Fixed internet

text: Figure 24: Indicators used per technology – Mobile internet

text: Indicators used per technology: Mobile internet

text: 0% Number of users Duration of an Time to recovery Geographical Impact on Services affected Networks and impacted incident area impacted emergency assets impacted services NRAs Providers

text: Figure 25: Example of thresholds used for the Art.13a annual summary reporting

text: Figure 26: Scales of criticality– provider’s perspective

text: Do you use scale/categories for determining the criticality/significance of an incident/situation

text: 17% Yes No Other No answer 83%

text: Figure 27: Using a combination of indicators – provider’s perspective

text: Do you combine the following indicators among themselves?

text: 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Number of Duration of an Geographic Infrastructures Impact on Root causes Other users affected incident spread and services emergency affected services Number of users affected Duration of the incident Geographic spread Infrastructures and services affected Impact on emergency services

text: Figure 28: “Standard” vs “Important” clients – provider’s view

text: Do you use different indicators depending on whether the users are "standard" or "important"?

text: Yes, we use extra indicators in 13% case of incidents impacting important users No, we always use the same indicators independently from the users No answer 87%

identifier: Security incidents indicators - measuring the impact of incidents affecting electronic communications

nummer: security-incidents-indicators

title: Security incidents indicators - measuring the impact of incidents affecting electronic communications

text: About ENISA

text: The European Union Agency for Network and Information Security (ENISA) is a centre of network and information security expertise for the EU, its member states, the private sector and Europe’s citizens. ENISA works with these groups to develop advice and recommendations on good practice in information security. It assists EU member states in implementing relevant EU legislation and works to improve the resilience of Europe’s critical information infrastructure and networks. ENISA seeks to enhance existing expertise in EU member states by supporting the development of cross-border communities committed to improving network and information security throughout the EU. More information about ENISA and its work can be found at www.enisa.europa.eu.

text: Authors

text: Dan Tofan (ENISA) Konstantinos Moulinos (ENISA) Christoffer Karsberg (ENISA) – survey phase

text: Contact

text: For contacting the authors please use resilience@enisa.europa.eu For media enquiries about this paper, please use press@enisa.europa.eu.

text: Acknowledgements

text: The analysis in this document was produced in collaboration with EY Luxembourg (Alexandre Minarelli, Brice Lecoustey, George Tountas, Cedrine Herbin, and Swathi Selvaraj) and based on the input of the following experts: Vassilios Stathopoulos (ADAE), Manuel Barros (ANACOM), Costin Masiliev (ANCOM), Heidi Kivekäs (FICORA), Vasiliki Mylona and Antonis Antoniades (OCECPR), Karin Lodin (PTS), Ulrich Latzenhofer (RTR), Pedro Gomes Silva and Pedro Gaspar Moreira (NOS).

text: Legal notice

text: Notice must be taken that this publication represents the views and interpretations of the authors and editors, unless stated otherwise. This publication should not be construed to be a legal action of ENISA or the ENISA bodies unless adopted pursuant to the

text: Regulation (EU) No 526/2013

text: . This publication does not necessarily represent state-of the-art and ENISA may update it from time to time.

text: Third-party sources are quoted as appropriate. ENISA is not responsible for the content of the external sources including external websites referenced in this publication.

text: This publication is intended for information purposes only. It must be accessible free of charge. Neither ENISA nor any person acting on its behalf is responsible for the use that might be made of the information contained in this publication.

text: Copyright Notice

text: © European Union Agency for Network and Information Security (ENISA), 2015 Reproduction is authorised provided the source is acknowledged.

text: ISBN: 978-92-9204-156-4; DOI: 10.2824/887699

text: Table of Contents

text: Executive Summary 5

text: 1. Introduction 7

text: Background 7

text: Scope of the document 7

text: Target audience 8

text: Methodology 8

text: 2. Varying approaches on measuring the impact of incidents 9

text: Approaches taken by member states (NRAs) 9

text: 2.1.1 Challenges 9 2.1.2 Goals and benefits 10

text: Approaches taken by e-communications providers 11

text: 2.2.1 Advantages 12 2.2.2 Challenges 12 2.2.3 Goals and benefits 13

text: Approaches taken by non EU states 15

text: Harmonizing NRAs & Providers 16

text: 3. Analysis of identified incident indicators 22

text: User/connection related indicators 23

text: 3.1.1 Number of users affected (fixed telephony) 24 3.1.2 Number of users affected (mobile telephony) 24

text: Time/duration related indicators 25

text: 3.2.1 Duration of an incident 27 3.2.2 Specific time at which an incident occurred 27 3.2.3 Time to recover 27 3.2.4 Fluctuating time period of an incident 28

text: Geographical area related indicators 28

text: 3.3.1 Impact based on location 29 3.3.2 Geographic area based on infrastructure coverage 29 3.3.3 Geographic area based on number of customers affected 29 3.3.4 Geographic area based on number of services affected 29

text: Indicators related to affected infrastructure and services 29

text: 3.4.1 Network infrastructure and assets 30

text: Impact on emergency services 30

text: Root causes dependent indicators 31

text: 3.6.1 Human errors 32

text: 3.6.2 System failures 32 3.6.3 Natural phenomena 32 3.6.4 Internal vs. Third party failure 32 3.6.5 Malicious actions / cyberattacks 32 Source/destination of “attacks” 33 Vulnerabilities within networks and services 33

text: Economic impact related indicators 33

text: Impact on confidentiality, availability and integrity (CIA) 34

text: Indicators used per technology 36

text: 4. Particularities regarding measuring the impact of incidents 39

text: Defining significance of an incident 39

text: Using scales of criticality for assessing the impact 40

text: Using combination of indicators 40

text: Assessing impact of incidents based on customer “importance” 41

text: Assessing impact of incidents based on quality and service degradation 42

text: End-user requirements 42

text: 5. Conclusions 44

text: Executive Summary

text: Telecommunication is an industry that can be considered as the backbone of modern economies. Economic development is strongly related to the existence and well-functioning of the telecommunication networks. Telecommunications demands advanced technologies and processes, playing a vital yet growing role in the European Union (EU). Current developments have determined some changes within customer preferences over the years, changes that have made it crucial for telecommunications operators to illustrate transparency, customer innovation and bring new services to the market. At the same time, operators have to maintain high security and availability standards. Security incidents affecting this sector can have detrimental effects that can manifest themselves in a number of ways. Measuring the impact of incidents has become one of the toughest challenges nowadays, given the multitude of factors/indicators that must be taken into consideration. To address this issue, indicators are used, accompanied by thresholds, to assess the impact of incidents. This approach allows evaluation of incidents from various perspectives, such as business perspective, compliance with regulations, root causes, impact on customers etc. Incidents can vary in nature, and this report tries to include as many indicators as possible, so that as many types of incidents as possible are covered.

text: The overall purpose of the document is to provide guidelines to national regulatory authorities (NRAs) and telecommunications providers (providers) within EU member states, to assist them in the process of measuring the impact of security incidents affecting electronic communication services.

text: This report comes as a practical approach, and contains the view of both NRAs and providers within EU, on real indicators used for measuring significance of incidents affecting telecommunication networks and services. Therefore, interested stakeholders have at their disposal a catalogue of indicators to be used to tailor impact assessment and design the corresponding solutions.

text: As the survey performed had also the objective of analysing the approached taken by NRAs and providers in defining indicators and significance, the results indicated that while there are some discrepancies between NRAs and providers in terms of why they measure security incidents, and for what purpose they use certain indicators over others, it is still plausible to state that the approaches taken and indicators used by both parties are more similar than they are different, as more than half of the respondents have stated this. Further developments regarding harmonisations of the approaches taken are still needed but overall the situation is running smoothly, the processes are in place, and the reporting of significant incidents is being done at national and EU level. From the study, it was realized that approaches taken by NRAs and providers varied depending on certain country-level factors. Some were much more mature than others. Due to several advantageous circumstances, some NRAs experienced strong cooperation with their providers when it came to implementing the necessary changes.

text: Having said this, it is advised that making use of a standardized approach among NRAs and providers can help derive more precise results in the incident reporting process. So, along with the list of indicators to use, the key recommendation from the study is that NRAs and providers should further increase the level of harmonisations in the approach taken to measure the impact of security incidents. Studying the approaches taken, including the benefits and challenges of each measurement method led to gaining an insight of the most common approaches used to assess security incidents. The list below summarises the main classes of indicators, as identified during the study:  User/connection related indicators

text:  Time/duration related indicators  Geographic area related indicators  Indicators related to infrastructure and services affected  Root cause dependent indicators  Economic impact related indicators  Indicators related to cyberattacks  Impact on confidentiality, availability and integrity

text: ENISA recommends the extensive use of this list of indicators, in related activities carried out by both NRAs and telecommunications providers within EU and abroad. Measuring the impact of security incidents, is a rather difficult process and the use of the current study can bring additional clarity.

text: 1. Introduction Background

text: Advances in electronic communication services have resulted in many changes, altering, reshaping and even modernizing the way in which people communicate with each other. Some industries are strongly revolutionized, especially those that deal with human senses, including entertainment, healthcare, education and advertising. Electronic communications services warrant a smooth transmission of data in this strongly interconnected world by providing the infrastructure for other business services to run. Electronic communication services also play a significant role in national security, emergency response and even in the economic well-being of a country. As a result, an outage in any one of these areas can result in severe consequences. For example, impact could range from strong customer dissatisfaction to impairing security and public safety and leading to economic repercussions. In that context, measuring the impact of security incidents affecting telecommunications providers has become a necessity in nowadays interconnected digital market. The European Commission (EC) has made significant steps in this direction, with the 2009 Telecom Package, that included Art. 13a, a regulation enforcing mandatory incident reporting for security incidents in the telecommunication networks. The purpose of

text: Article 13

text: a of the Directive 2009/140 EC is to ensure the security and integrity of electronic communication networks and services in the European Union, mostly by preventing disruption of networks and services among others. As only significant incidents had to be reported, meaning the ones with greater impact, member states have struggled for some time to define “significant”. To be able to identify significant incidents, providers and NRAS need to know how to measure their impact. Measuring the impact of security incidents is not always easy as there are several different variables which play a role. Furthermore, the impact of an incident can be evaluated from different perspectives, making the process rather complicated. By studying the indicators used to measure the impact of security incidents, especially the ones affecting the availability of the service (disruptions), this report aims at collecting good practices and expertise in order to share it across, encourage the exchange of these practices between member states and improve harmonisation of the existing approaches. Evaluating the direct and indirect impact of security related incidents can be crucial as a basis for investment in recovery strategies as well as investment in prevention and mitigation strategies.

text: Scope of the document

text: This report aims to provide guidelines to NRAs and providers, to assist them in the process of measuring the impact of security incidents affecting the availability of electronic communication services. Therefore, interested stakeholders, especially telecommunications providers, will have at their disposal a catalogue of indicators to be used to tailor impact assessment and design the corresponding solutions.

text: The goal of this study is to help policy makers at EU and member states level, along with the European Commission and ENISA, develop better policies in order to further increase the security and resilience of the electronic communications sector.

text: Target audience

text: Electronic communications providers (named providers as for the rest of the document), professionals in the telecommunications industry, along with national regulatory agencies (NRAs) within the EU are the main targets of this report. The report is also addressed to experts within the policy making area, such as European Commission.

text: Methodology

text: The study was carried out in 3 phases. Firstly, a desktop research was performed to review approaches taken by different member states and non-EU countries. Next, a range of NRAs and service providers were identified and selected in order to complete an online survey. A questionnaire was designed and communicated with those who agreed to take part. Finally, telephone interviews were scheduled and conducted. Interview guides were sent to the selected participants prior to the interview in order to foster meaningful conversations and gather useful insights. Conclusions in this report are driven from the desktop research, interviews and the results of the online survey. A total of 9 NRAs and 4 providers were reached for the interviews, whereas 13 NRAs and 23 providers responded to the online survey.

text: 2. Varying approaches on measuring the impact of incidents

text: A milestone in measuring the significance of incidents within the telecommunications sector in the EU was set up along with the adoption of the Framework Directive (Directive 2009/140 EC) within the 2009 Telecom Package, which included Art. 13a.

text: Art. 13a aims at ensuring the security and integrity of electronic communication networks and services in the EU. This is partially achieved through requiring telecommunication service providers to take the appropriate technical and organizational measures to manage the risks posed to security of networks and services, guarantee the integrity of their networks (ensure the continuity of supply of services provided over those networks) and notify the competent national regulatory authority (NRA) of a breach of security or loss of integrity that has had a significant impact on the operation of networks or services.

text: When

text: article 13

text: a was initially published, NRAs and providers were required to make some changes to their internal procedures so as to comply with its requirements. This was especially true with regards to how security incidents were measured and reported. To have a well-established incident reporting method, NRAs and providers need to know how to effectively measure the impact of a security incident. However, measuring the impact of a security incident can often be a lengthy and complex process. In addition to this, there are some differences in the approach taken between member states as well as the approach taken among NRAs and service providers.

text: Approaches taken by member states (NRAs)

text: The approaches taken by NRAs can be classified into two main categories.

text: The first category is one wherein NRAs have taken a reasonably collaborative approach with the providers in adopting indicators to measure security incidents. In these cases, there has been a lot of communication between the various groups involved. Furthermore, the indicators were defined and agreed upon through consultation between the NRAs and providers.

text: The second approach however, is somewhat that of directly implementing the provisions of the directive. Providers did not have much to say in the incident reporting process and instead had to comply with the requirements of their NRAs. Differences in the approach taken can primarily be attributed to country particularities. Regardless of whether the approach was more or less collaborative, NRAs and providers were encouraged to use an additional second-level regulations along with ENISA’s technical guidelines. However, this was not the approach taken by all member states. The reason for which additional measures exists in some cases (but not in all), is related to the fact that the Telecom Package, does not explicitly mention the indicators or reporting procedures that member states must follow.

text: 2.1.1 Challenges

text: Since providers can vary in size, popularity but also in terms of their maturity levels, NRAs can find it difficult to agree on a measurement method, which is applicable to all. Although mentioned as a medium challenge, providers and NRAs are using different measurement methodologies (stated by 45% of the respondents from the survey). A difference in the approach taken to measure security incidents can potentially impact the results of the incident reporting process, and may not always represent a true and fair view of the current situation in the country.

text: In your opinion, what are the challenges of measuring the impact of security incidents?

text: That being said, identifying appropriate indicators and accurately measuring them, is in fact most problematic (38% of the NRAs expressed this to be their medium to big challenge). Although only listed as a medium challenge, NRAs also experienced difficulty in clearly defining an indicator (46% responded that this was the case). The fact that NRAs experienced confusion in defining, identifying and accurately measuring security incident indicators, can, for the most part, be attributed to

text: Article 13

text: a’s relative ambiguity.

text: Article 13

text: a asks NRAs and providers to take appropriate steps to guarantee the integrity of networks, but does not specify what is meant by “appropriate steps” nor does it specify on what basis “integrity” is defined. The directive also lacks necessary details needed for defining “significance” of incidents. To provide more clarity, the Art. 13a Expert Group, coordinated by ENISA, has been established in order to help by issuing good practices on these matters. Accurately measuring the indicators has been mentioned as big to medium challenge by half of the NRA respondents.

text: 2.1.2 Goals and benefits

text: Identifying the type of incident including the reason behind it (i.e. if the incident was accidental or deliberate) and identifying the target of the incident, such as whether the incident affected customers or networks etc. is seen as a key goal and benefit among NRAs (92% of respondents replied this to be the case). This is in line with expectations as NRAs ultimately aim to protect citizens. Therefore, measuring the impact an incident has on the population is necessary. Furthermore, accurately measuring security incidents enables NRAs to have a better overview of the “as-is” situation in their country. By identifying the types of incidents that can occur at a national level, NRAs are also more likely to put in place the right kind of risk assessment and security measures (77% agreed). NRAs were also keen on understanding how security incidents impacted their citizens, with 62% replying that this was another one of their main goals/benefits. 62% of survey respondents also stated that they were measuring the impact of security incidents so as to comply with EU regulations (which was seen as a key goal/benefit).

text: An additional benefit to measuring the impact of a security incident, from the perspective of the NRAs is that the information can be used as a basis for maintaining statistics (as expressed by 38% of the respondents).

text: Issuing guidelines to providers on strengthening infrastructures and using the measurement methodologies as an input for discussing challenges can be of benefit. Furthermore, it can also aid in discussing issues with other providers and other sectors. However, both these indicators were seen as more of a secondary objective rather than a primary one (38% for strengthening infrastructures and 31% for discussing challenges).

text: In terms of the indirect goals, analysing indicators, resulted in the NRAs’ ability to perform better risk assessment procedures at a national level, and to increase the level of security nationwide (38% said that it helped perform risk assessments at national level and 46% agreed that it improved security nationwide).

text: Approaches taken by e-communications providers

text: The approaches taken by providers can be directly correlated with their level of maturity in terms of country legislations and practices (which can be seen when benchmarking one member state with another). Approaches also vary among providers in terms of the providers’ business maturity, as well as their level of maturity as regards the relationship and collaboration with their respective NRAs.

text: For providers who were very mature, the reporting process, including the indicators used to measure security incidents were incorporated to a relatively good standard in their day-to-day practices.

text: Providers with an average level of maturity, including smaller providers were either eager to improve their internal processes and security measures, identifying it as an opportunity to improve or were reluctant to do so due to additional effort and resources needed. These providers were either using

text: fewer or different indicators than the ones defined in the guidelines. In order to comply with the regulations, small providers had to implement these security measures from scratch.

text: 2.2.1 Advantages

text: Benefits in the approach taken by providers were quite similar to the benefits experienced by the NRAs. Having a more transparent affiliation with the NRA, wherein there existed good communication and transfer of information and knowledge between the two, meant that the transition process to the new requirements of

text: Article 13

text: a, was a lot more easy than when compared to provider’s whose relationship was not so strong. Furthermore, providers were able to better understand the importance of

text: Article 13

text: a and how the former could positively benefit them. Effectively using the guidelines meant that providers experienced less implementation costs and efforts taken to comply with the regulations. This was especially true for the bigger telecommunications providers, as well as those who had greater investment capacities.

text: As for the smaller players in the industry, although they may not have had the right capital to invest in, or the relevant expertise and skills to do so, the advantage for them is that they have had the opportunity to learn from the methods used by their counterparts, and based on this, adopt some of the leading practices which would be most applicable to them. By doing so, they can also gain a competitive advantage which is core to the nature of their business.

text: 2.2.2 Challenges

text: Unsurprisingly, providers felt that measuring the impact of a security incident was a process that was not only time consuming but also, expensive (30% said that it was time consuming and 30% mentioned that they had insufficient budget). Providers expressed more in detail, their concerns during the interviews. For example in situations where the NRA had implemented a second level of regulation, providers were asked to issue two

text: incident reports, instead of just one. They were asked to issue the first report Of providers felt that measuring within 24 hours following the identification of an incident. The second report was security incidents requested to be issued within two weeks following the incident fix. Therefore, this was an situation led to some administrative burdens and delay in resolving the incident Administrative (13% stated that this was a big challenge and 57% agreed that it was a medium burden one). Similarly, it was sometimes the case that the same incident was measured twice. The incident was measured once by the provider for internal purposes and was later measured again for the purpose of reporting to the NRA. Given that there are usually limited resources which are usually supposed to be dedicated to resolving the issue; providers felt in such situations that they are “wasting” time in writing reports, and measuring the impact. The former could denote that more transparency and cooperation within the indicators area is required between providers and NRAs.

text: Furthermore, measuring security incident indicators also resulted in limited value for money (48% of the providers responded that this was a big challenge whereas 43% stated that the challenge was a small one). Regardless of maturity levels however, and similar to the NRAs, all providers experienced some level of

text: difficulty when it came to identifying, collecting the relevant data, and accurately measuring security

text: incident indicators.

text: 2.2.3 Goals and benefits

text: Complying with national regulations is one of the main objectives of measuring security indicators (43% of the respondents agreed). Additionally, goals and benefits in the assessment approach means that providers can in consequence also be able to better identify incidents and to establish a more rigorous incident response mechanism (35% strongly felt that this improved identification of security incidents, and 43% replied that this meant ensuring a more rigorous incident response system)

text: Another benefit from the viewpoint of the provider is that measuring security incident indicators can enable them to use the information collected, as a basis for maintaining statistics (70% agreed). Using the indicators also means that providers can identify repeated incidents and thus, put in place certain measures so as to reduce the impact on their customers (52% believed this to be the case).

text: What is interesting to note is that informing other providers of the major impacts following an incident was neither a benefit nor a goal (39% felt this way). At the same time, input for discussing challenges and issues with other providers and other sectors also appeared to be neutral as well. From this information, it can be assumed that providers were not so willing to share or discuss of the security incidents among each other or within different sectors.

text: In addition to this, although complying with national regulations is one of the main aims, many providers expressed that their goals were not always in line with that of their NRAs. Unlike the NRAs, whose ultimate purpose is to protect citizens, providers are much more business oriented. Providers are constantly seeking new ways to grow their business and increase revenue and are also trying to position themselves strategically in the market. Their objective is to deliver a positive experience for their customers. Having said this, being customer centric meant that providers were using security indicators (i.e. root cause of an incident, number of users affected, time duration etc.) to design effective security mechanisms to not lose reputation among the public. The measures were also being used to assess damage of the incidents and to also identify revenue for their business.

text: Approaches taken by non EU states

text: Researching the methodologies used to assess security incident indicators outside of the EU, can help assist NRAs and providers in refining their existing procedures to a further extent. The former can provide useful insights which can eventually be applied, and also leveraged upon among the different members states. A lot of information is available on threats and security issues associated with “information security” in particular. This is especially true for countries such as the United States and India. As a principle, it was noted that three aspects of information security are considered: “availability”, “integrity” and “confidentiality” of information. While availability is the main area of focus for

text: Article 13

text: a, other sectors have directed their attention towards “confidentiality” instead.

text: For example, the Federal Communications Commission (FCC) in the United States measures the impact of security incidents using several key indicators. Furthermore, they have also established an incident reporting process which provides guidelines to telecom operators in the United States on what to report and when to notify of incidents. To facilitate the incident reporting process, the FCC has also set threshold values. Telecommunications operators are requested to report on the following; the date and time of onset of the outage; a brief description of the problem; service effects and the geographic area affected by the outage. A distinction is also made on “initial” and “final” reports meaning that similar to

text: Article 13

text: a, providers report the same incident more than once (once initially and after it is resolved). Outage reporting requirements are categorized based on the service that is affected. Services include cable services, IXC tandem facilities, outages to satellite communications, signaling system, wireless services and interconnected VOIP service providers. For more information, readers can refer to the ecfr.gov website.

text: To achieve efficiency and encourage the public to communicate the incidents, the FCC has issued key information via their online website. They have also established a page which is dedicated to providing information to users including the FCC phone directory, organizational charts, the FCC’s offices and emergency contact information.

text: In India, the Department of Information Technology (DIT) has defined Service Level Indicators for their SWAN operation (State Wide Area Network). Indicators for SWAN are requested to be considered by the respective states within India for incorporation in their Service Level Agreements with operators. Thresholds values are defined which help in measuring the impact of an incident. In addition to this, emphasis is also placed on the time duration of an incident (initial response time and issue resolution time). To assess severity of an impact, the DIT also recommends operators to measure the availability and extent of outage experienced for the following: internet availability, firewall outage, IDS outage and Denial of Service. This is different to the existing indicators used at EU level and assessing the impact of incidents through this viewpoint can allow member states to obtain technical insight on the cause of an incident or vulnerabilities surrounding existing infrastructures and services. For more information on the thresholds including how they are used among different states in India and on the specificities of the technical information, such as bandwidth and availability etc., readers can access this website.

text: The indicators used to measure security incidents in both these situations, can be related to the ones transposed in the European Union, linked to

text: Article 13

text: a.

text: Harmonizing NRAs & Providers

text: To remain a key player in the market, providers should know how to attract and retain customers. For this, they need to be able to provide the best services to their customers. One way of achieving this, is to continuously monitor the functionality of infrastructures and services. In contrast, the approach taken by the NRAs is not necessarily going to be the same (NRAs do not have the same commercial objective as that of a provider). NRAs are concerned with protecting their citizens and on improving security within the country.

text: Regardless of the role and purpose of the NRA and provider, the bottom line is the same. Both need to take appropriate measures to guarantee security, but the main purpose of why they offer security differs from one to the other.

text: Sometimes it appears to be difficult to know of the internal regulations used by operators. Furthermore, it is difficult to say if these regulations are in place in an attempt to comply with country level regulations or if they already existed beforehand. Other than the incidents that are reported to the NRA (as part of the reporting process), several NRAs did not know of the additional indicators that providers were using (23% stated this to be the case).

text: Are the indicators used by the different providers similar for the same services

text: The same was also conveyed more in detail during the interviews, wherein NRAs described that even if they had a good overview of the security measures taken by the big operators within the country, access to relevant information on the smaller players was very much limited. In situations where the NRA did have an indication of the measurement methods however, it was found that providers were using similar indicators for the same kind of services (54% of the NRAs agreed). Nevertheless, there were some notes by providers, stating that although the indicators used may be the same, in some cases the measurement method is not the same. For instance, when measuring affected users on a fixed voice service, the value of the indicator will differ if the NRA uses provisioned customers and the provider uses statistic usage curves.

text: In one example though, the NRA argued the opposite. It explained how 5 of the largest providers in the country represented 90% of the market in terms of customers, while 500 small providers represented only 10%. Indicators were calculated based on affected customers, geographic area and loss of capacity etc. The smaller providers found it difficult to meet the standards of the reporting process, and felt more comfortable with reporting on loss of capacity only. In addition to this, if for example an incident affects a large provider, it would be more likely that the incident is published in the news and/or media. When the 54% incident affects a smaller provider instead, the visibility of that incident is not so

text: Do you use similar indicators than the ones defined by the NRA?

text: Using fewer indicators as opposed to using more is what essentially can lead to some issues ( 17% of the providers mentioned that they were using a fewer number of indicators than their NRAs). This often resulted in additional costs and extra effort. For example, providers already had in place certain indicators to fulfil their own objectives and goals (i.e. build good relationships with customers, establish a good reputation in the marketplace, become the leading provider in the industry etc.). However, complying with the requirements of the NRA meant that they had to make some changes to the way in which they measured security incidents. This also included measuring the same type of incident based on a new set of thresholds. A misalignment in the way in each of the indicators were measured sometimes resulted in a difference in opinion between the NRA and the provider and in some cases also had an impact on the relationship between the NRA and the provider.

text: Furthermore, the “misalignment” most often manifested itself in the incident reporting process. For example, according to the guidelines of 8 of the NRAs interviewed, an incident report should be sent by the provider to the NRA with one hour after identifying an incident. However, the incident is not necessarily resolved within that hour. The time taken to report an incident actually results in a delay in time to resolve the incident. That being said, in the majority of the cases the team who is responsible for reporting the incident to the NRA is also the same team involved in resolving the incident. A lot of effort is required in terms of reporting the incident and additional work is also performed when reporting the same incident for the second time (incident is reported within 1 hour of being identified and then reported again once resolved). While this demands quite a lot of effort from the side of the providers, measuring incidents in this way, actually adds value to the incident reporting process from the viewpoint of the NRAs (NRAs are able to gather more insight on the nature of the incident).

text: To summarize, irrespective of the differing opinions among NRAs and providers and irrespective of if internal procedures differed based on the type of incident in scope, it was found that NRAs and providers mainly use similar indicators to measure security incidents. There are although slight differences regarding the measurement methods, as reported by some providers.

text: To reinforce this point, when asked if objectives of the NRA were aligned with the providers objectives, 54% of NRAs from the survey, felt that objectives were “somewhat aligned”. In other words, even if protecting end users vs building a good reputation, differed to some extent, the means of achieving the objectives are in actual fact similar (Fig. 7). Only 23% of the survey respondents replied that both NRAs and providers had exactly the same objectives when defining indicators to measure the security incidents, whereas only 8% replied that the objectives were different.

text: Do you think your objectives as an NRA are aligned with the providers' objectives?

text: In your opinion are your objectives aligned with your NRAs objectives?

text: The question was also asked to providers, where 44% replied that according to them, their objectives, where somewhat aligned with that of the NRA. To add to this, 30% replied that both they and their NRA had the same objectives. 17% claimed that they did not know of 17% the objectives of their NRA and another 9% stated that the objectives were

text: Of providers different. The key message here is that providers share the same objectives as NRAs. claimed that they did not In the same way, to gain an insight on the satisfaction levels, NRAs and providers know the were asked to also give their opinion as regards to the indicators used by providers objectives of the as well as the guidelines provided by the NRA. From what can be noted, the majority NRA of NRAs were satisfied with the indicators used by providers. 62% of NRAs replied that they were somewhat satisfied whereas only 15% stated that they were missing details on the impact of incidents. Another 23% replied that security incidents were measured in an accurate and timely manner by the providers.

text: According to 44% of the providers, NRAs facilitated them in their process of defining appropriate indicators. However, 13% answered that these guidelines could be somewhat improved. Another 13% answered that they did not need help any from the NRA when measuring security incidents at all. This could because they were already using a good set of indicators, which happened to also be the ones purported by the NRA. There could have also been a certain degree of reluctance on the part of the provider to complying with the recommendations as mentioned in the guidelines. Finally not all providers may have seen the value, or return on investment in complying with regulations.

text: Are you currently satisfied with the way providers assess the impact of securtiy incidents

text: Are you currently satisfied with the guidelines provided by your NRA to

text: assess the impact of security incidents?

text: As mentioned above and to conclude on this part, while there exists some discrepancy between NRAs and providers in terms of why they measure security incidents, and for what purpose they use certain indicators over others, it is still plausible to state that the indicators used by NRAs and providers are more similar than they are different

text: At the same time, while both NRAs and providers may each have their own set of goals which would

text: be unique to their specific situation, the overall and ultimate goal by both parties involved is on minimizing and effectively recovering from any security related incident that affects the electronic

text: communications sector.

text: 3. Analysis of identified incident indicators

text: The indicators presented within this chapter have been obtained and were compiled as the most commonly used indicators from the views of both NRAs and providers. The results from the graph below represent the average percentage of users that replied as using the indicators for measuring security incidents for all technologies. In cases where the user did not answer or the indicator was not used, these percentages were omitted from the calculation.

text: From the graph, it can be noted that “number of users impacted” and “duration of an incident” were the most common indicators which were used by 100% of NRAs, whereas providers used these indicators on average 61% for “number of users impacted” and 63% for “time duration” of an incident. Furthermore, “impact on emergency services”, “networks and assets impacted” as well as “geographical area impacted”, was also used quite frequently. This is especially true among NRAs in particular. To obtain more information on the indicators used by both NRAs and providers for each individual technology however, readers can refer to the following sections of the report.

text: Keeping this in mind, it is important to note that the above does not represent the exhaustive list of indicators. Referring to the summary below, it is possible to also get more information on the measurement methodologies of each indicator including the indicators benefits and challenges.

text:  User/connection related indicators  Number of users affected (fixed telephony)  Number of users affected (mobile telephony)  Time/duration related indicators  Time duration of an incident  Specific time at which an incident occurred  Time to recover  Fluctuating time period of an incident  Geographic area related indicators  Impact based on location  Geographic area based on infrastructure coverage  Geographic area based on number of customers affected  Geographic area based on number of services affected  Indicators related to infrastructure and services affected  Network infrastructures and assets  Impact on emergency services  Root cause dependent indicators  Human error  System failures  Natural phenomena  Internal vs. Third party failure  Economic impact related indicators  Indicators related to cyberattacks  Sources/destination of “attacks” related indicators  Vulnerabilities related to networks and services  Impact on confidentiality, availability and integrity

text: User/connection related indicators

text: Calculating the number of users or connections affected was identified as a key indicator among NRAs and providers. This indicator was being used by all respondents of the surveys, to measure the impact of any given security incident.

text: During the interviews, it came to light however that the measurement method for assessing this particular indicator was in fact, the most challenging one. To add to the difficulty, there are different technologies that are offered to customers, such as fixed telephony and mobile telephony etc. which complicate the measurement even further. A common measurement method for each service cannot be applied. For this reason the measurements are grouped per technology.

text: 3.1.1 Number of users affected (fixed telephony)

text: Identifying the number of users impacted for fixed line telephony is relatively easy to measure when compared with mobile telephony. Fixed telephony service providers can use the number of customers connected to a Switch to estimate the number of affected customers (usually 1 line = 1 subscriber). Furthermore, as providers know the number of users connected to specific equipment, they are in a position to better determine the impact of an incident. However, not all providers are able to apply this method as sometimes 1 line can have many users and this is not always easy to know. The former is often the case when the user is a company (not an individual customer) and unless the providers are able to have access to information about the number of connections/subscribers in the company, the measurement method is not so accurate. When considering the number of users impacted by a security incident, providers can also approach this by measuring both the provisioned number of customers impacted, as well as the actual usage. This allows for a differentiation between the two because the number of users affected by an incident will not be the same at 4PM than at 4AM. This information can be useful for a number of reasons including for example, maintenance purposes. In addition to the former, the provider can also have in place a detailed process for measuring “service usage”. By using a service usage curve, it is possible to examine the number of users using a given service at a specific point in time as opposed to the number of provisioned users for that particular service.

text: 3.1.2 Number of users affected (mobile telephony)

text: For mobile services the guidelines are much more flexible. Therefore, providers can chose in which way they want to perform the analysis to determine the impact on their customers. For example, providers can chose to look at the number of people who are living in the affected area or to look at the market share in the specific area and on how the network is dimensioned in this particular area, to find out how many possible affected subscribers are included.

text: As opposed to fixed telephony, mobile telephony is the provision of telephone services to phones which may move around freely rather than stay fixed in one location, thus making it difficult to measure. An approach taken by some NRAs to overcome the challenge in measuring the number of users affected for mobile services, is to request providers to assess the traffic use of the previous week of the incident or to compare the traffic of the previous week day with the traffic of the current day on the affected equipment and multiply this number with the total number of SIMs they had before the incident took place. (Traffic last Thursday - Traffic this Thursday) x number of SIM cards = number of SIM cards affected While calculating the number of SIM cards can be one way of determining the number of users affected in a geographic area, not all the providers have the ability to obtain the exact number. Therefore most of the operators have a rule that considers the average number of users per base station. In the case of moving disturbances such as a storm, the NRA can request to receive a map from the providers showing the area where the outage occurs at a particular point and the number of affected based stations. To summarize, even if NRAs have equipped their providers with guidelines on measuring the number of users or lines that have been affected following an

text: incident, the choice of methodology has been left to the providers. Of providers measure According to the study, 100% of the NRAs are asking for the number of users incidents for affected for fixed and mobile telephony and internet services. When making the fixed technologies same comparison at the provider level, the majority of providers were measuring the number of users affected for fixed telephony and internet technologies, than they were for mobile telephony and mobile internet technologies (74% for fixed

text: telephony and internet and 48% for mobile telephony and internet technologies). Of providers measure It can be assumed that the reason for which there exists a difference in the figures incidents for at the provider level is because some providers are not so sure on how to effectively mobile technologies measure mobile telephony and internet access services than when compared to fixed lines. As mentioned above, this is because it is easier to detect how many users are connected to fixed line services and also to calculate how many users are impacted following an incident than when calculating the number of mobile users as mobile services can be used for many purposes (not only for phone calls). Furthermore, mobile users are constantly “on the move”. Therefore if for example one were to measure the number of mobile users impacted based on geographical area, this would be different based on individual factors (the user may work in a different area to where he lives). There are many other individual factors as such, making the process complicated. Similarly it is difficult to make correlations between users who use fixed and mobile services simultaneously.

text: Time/duration related indicators

text: The gravity of the incident and the type of resources it affects will determine the amount of time and resources needed to recover from that incident. An incident may require far more resources to recover from than what the organization can handle. Those resolving an incident should consider the effort necessary to actually recover from an incident and carefully weigh that against the value of the recovery effort. As with number of users affected, this indicator can be measured in different ways. The commonly identified methods for NRAs and providers were to assess the impact of the former based on the time duration of an incident but also the time to recover from an incident.

text: For example, “time duration” and “time to recovery” were used as indicators by 100% of NRAs who took part in the survey. Indicators such as “fluctuating time period”, was used on average by 31% and “specific time” at which an incident occurred was only used by 15% of the NRAs.

text: Similarly providers were more commonly using the same two indicators (“time duration” and “time to recovery”) in comparison to the other time related indicators too. 74% replied that they were measuring the duration of an incident and time to recover from an incident for fixed telephony and internet technologies. The same is also true for mobile technologies which were used by 48% for “duration of an incident” and 52% for “time to recovery”.

text: 3.2.1 Duration of an incident

text: The time duration of an incident can be assessed from the moment at which the incident is identified until full recovery of the connections (i.e. up until the last connection is restored on the last based station affected).

text: However, the challenges that some providers have expressed is that there is a lapse of time during which service starts to degrade but is not reported by the providers. Therefore the incident may not fully reflect the total duration of an incident interruption. Another way to calculate duration of an incident is to break it down into several sections including when the incident started, when it is estimated to be over, and when it is actually over. This helps in determining how long the interruption lasted. Determining the time duration of an incident is a beneficial approach as it is possible to see how long it took to recover from the incident and thus measure how efficient the provider was in resolving it. In some cases, duration of an incident is only measured after the incident has been resolved, not during resolution.

text: 3.2.2 Specific time at which an incident occurred

text: Incidents can also be measured based on the specific time at which they occurred. For example, an incident that happened on the weekend can have a different impact from that which took place during a weekday (and that too during working hours). This is because, the chances that more people were making use of telecommunications services during that time are greater, and thus the impact of the incident as well as its consequences, would have also been greater. So, this indicator would be addressed if the number of affected users was calculated based on statistic usage curves (which vary throughout the day) and not based on provisioned customers.

text: 3.2.3 Time to recover

text: Time to recover can often be confused with duration of an incident. Therefore it is necessary to create a distinction between the two. Where duration of an incident reflects the action taken from beginning to end of an incident, time to recover is calculated from the moment the restoration work has started on the incident and ends with full recovery. Time to recover from an incident (i.e. hours or minutes etc.), often takes less time than the actual duration of the incident.

text: That being said, there are also some additional complexities with measuring this indicator. This is because it is often hard to differentiate between when an incident actually occurred, in comparison to when it was realized or acknowledged. Furthermore, in terms of calculating the time to recover from an incident, NRAs and providers also expressed that they found it a bit challenging to know on what basis to measure the recovery of an incident. For example, does this depend on the time it took for all services and users to fully recover from an impact of an incident, or did this depend on when the number of users and services affected, were within a certain range (i.e. below the defined threshold).

text: 3.2.4 Fluctuating time period of an incident

text: Measuring the fluctuating time period of an incident might prove challenging because the quality of the offered service usually will not degrade instantly. There may be some fluctuations in the services’ functionality which could still be rather volatile, for a certain amount of time, before the service eventually results in total failure. Focus should be placed on service degradation.

text: That being said, when providers report on time/duration related indicators, they need to be specific on what basis they are measuring “time”. This is because if one provider has calculated the impact based on time to recover from the incident, whereas the other has done the same based on time duration of the incident, the end information reported to the NRAs cannot be accurately measured nor compared between one another.

text: Geographical area related indicators

text: Measuring security incidents which affect certain geographical areas, allows NRAs and providers to identify which region is most affected by an incident. The benefit of measuring security incidents using this approach is that, through this, it is possible to determine whether or not the geographic landscape of a location plays a role in the frequency at which an incident takes place and in determining the type of incident that has occurred. Geographic impact, on both fixed line and mobile services, is calculated by almost all respondents of the survey.

text: Calculating the geographic spread of an incident is especially important to consider in countries with diverse surroundings. For example, places that are urbanised and better connected would experience less disruptions, but the disruptions would affect more people than when compared to areas that are less urbanised. That being said, the impact of an incident affecting an island near the sea would be

text: greater due to poor network coverage. Furthermore, the chance of an incident occurring in remote areas is greater due to national hazards such as storms and bad weather.

text: 3.3.1 Impact based on location

text: Security incidents can be measured as per the number of square km affected. For instance, in countries where access to certain areas is rather remote (i.e. mountains, isolated islands, forests etc.), an outage can impact either half the city, or nobody at all (which should be taken into consideration).

text: Furthermore, depending on the size of the geographic area and the impact which the incident caused, the incident is either requested to be reported to the NRA, or not reported at all. Whether or not to report an incident depends on the thresholds set by the NRA. In most cases, the NRAs were found to have followed the guidelines provided by ENISA when defining such thresholds.

text: For some NRAs, it’s difficult to assess the location of an incident as one incident can affect several locations. Furthermore, if there is an area with only one operator and this operator experienced an interruption of services, there is an increased chance that the consequences of the incident will be more severe.

text: 3.3.2 Geographic area based on infrastructure coverage

text: Another way to measure incidents’ impact can be based on the geographical coverage of the networks. Some NRAs did not use a specific geographic area to measure this indicator as one base station can cover many areas. Instead, this indicator can be measured by calculating the number of base stations that are affected. By identifying the number of base stations impacted, it is possible to identify each of the different areas that are associated with that base station and hence, calculate the geographic spread.

text: 3.3.3 Geographic area based on number of customers affected

text: Measuring the geographic impact of an incident can also be calculated by combining the geographic spread with the number of customers affected. That being said, regardless of the area which was impacted, the total number of customers can be measured first. Based on the total number of customers that are impacted, it was later easy to identify the area in which the incident took place.

text: 3.3.4 Geographic area based on number of services affected

text: Similarly, geographic impact can also be calculated based on the number of services affected and the areas where they are provided. Services such fixed or mobile telephony, voice and internet access services are included in this calculation. Taking the example of fixed line services, if 500 or more customers are affected, then the incident is considered as significant (depending on the thresholds set by the country), and as such should be reported to the NRA. Determining the number of services affected ant the area where the services are provided, can help determine and locate the geographic spread of the incident.

text: Indicators related to affected infrastructure and services

text: Providers are requested to assess the types of infrastructures and services that are affected following an incident. Assessing the types of infrastructure and services affected can enable an overview of the ones that are most vulnerable to threats. Precaution can therefore be taken on those assets and investment in newer infrastructures can also be made if needed.

text: 3.4.1 Network infrastructure and assets

text: Using indicators to measure security incidents on network infrastructure and assets is generally quite straightforward. Providers are requested to give information to their NRAs on the types of infrastructures that have been impacted. This includes information such as the providers’ internal networks (core networks and active networks) that are affected as well as on the type of services that were affected (TV services, voicemail services etc.). However, it is not always clear on how to measure this aspect. The main question that needs to be asked is does this calculation depend on the number of services and assets that have resulted in total shut down, or does this instead refer to a percentage in the number of service that are impacted (20% of services for example).

text: Impact on emergency services

text: Ensuring that security incidents do not impact emergency services is highly important in the telecommunications sector. In case of any emergency, people expect to be able to reach the right services and on time. This includes dialling numbers such as 112, to report events such as accidents, burglary, fires etc. Emergency functions should be working at all times in order to protect the general safety of the public.

text: Analysing the impact of incidents on the emergency services was measured for all technology types such as fixed telephony, fixed internet access, mobile telephony and mobile internet access. This is especially true for NRAs wherein 92% were measuring the former for fixed technologies and 85% for mobile.

text: That being said, measurement methods can be complicated because for example, if an incident affects a phone service, it can potentially affect emergency service calls as well. In some cases, only the emergency services are affected while other telecommunications services are still functioning. The numbers of affected users are counted differently here as users may still have their services working while being unable to reach the emergency services. In this situation, the number of users is the number of users that cannot connect to an emergency number. As stated by most of the respondents, for determining the impact on emergency services, it is often the case that NRAs and providers rely on estimates (such as previous statistics etc.).

text: Root causes dependent indicators

text: Determining the impact of an incident based on the analysis of the root cause, can be a good approach. Determining what went wrong, and what led to the incident to occur in the first place, can minimize or monitor similar patterns of the incident should such an incident occur in the future.

text: However, some issues that providers can be faced with, is that although they were required to report the root cause of the incident, the form which they had to complete, was structured in a way that made it difficult to understand. The form was sometimes too technical and those completing it were not able to provide enough precision on the specific reason for the incident. Without enough precision on the root cause of the incident, it makes it difficult for NRAs and even providers to clearly analyse what went wrong and assess the points of improvement.

text: Malicious attacks also falls under this category but has been explained more in detail in section 3.8 of the report

text: 3.6.1 Human errors

text: Using “human errors” as a measurement to assess security incidents can help to assess whether the incidents have taken place due to negligence and/or lack of awareness of people. Human error incidents are usually caused during the operation of equipment or facilities, the use of tools and the executions of procedures, etc. In order to avoid this kind of error in the future, trainings can be conducted for the relevant parties involved. That being said, human error also includes things such as misconfigurations. If a system is not properly configured, it can lead to security incidents.

text: 3.6.2 System failures

text: Measuring the impact of security incidents as a result of system failures, such as hardware and software failure, can encourage the NRAs but mostly providers to put in place rigorous monitoring mechanisms of their systems. Furthermore, in case there are a lot of incidents that have occurred due to system failures, providers can look to invest in newer infrastructures. Nevertheless, this process can get quite technical. Therefore, by analysing the reason for which the incident resulted in system failure and thereafter providing high-level information on this can help the necessary people concerned to determine whether or not to spend on new infrastructure.

text: 3.6.3 Natural phenomena

text: The indicator “natural phenomena” includes measuring incidents that are caused by severe weather, earthquakes, floods, wildfires, and so on. After measuring the probability of an incident occurring due to natural hazards, providers can build stronger infrastructure or backup systems in order to ensure continued availability. That being said, after identifying the areas which are most likely to be affected by natural phenomenon and/or measuring the probability of natural phenomenon occurrence, means that providers can be better prepared for such incidents and take the necessary precautions accordingly.

text: 3.6.4 Internal vs. Third party failure

text: Third party failures are an important variable when measuring security incidents. If the incident was due to an outside vendor technical issues, is an important thing to consider, as outsourcing some of the services to outside parties is a common approach of providers nowadays. Determining if an incident is internal or external can assist providers in the identification of whom the incident will impact and areas to approach in order to mitigate the incident. 16% of all incidents reported in 2014 to ENISA, were third party failures.

text: 3.6.5 Malicious actions / cyberattacks

text: Incidents related to cyber-attacks can also have an impact on telecommunications services. As such NRAs and providers should also consider measuring security incidents from this point of view. This is especially true for internet services (both fixed and mobile). For example, denial of service attacks (DOS), or attacks on the network (related to cyber-attacks) can result in downtime. As with the telecommunications

text: sector, cyber-security related incidents can affect a varied range of industries. There is plenty of information on how to calculate the impact of an attack, detailing the use of algorithms etc.

text: Source/destination of “attacks”

text: Measuring the source or destination of an incident can help NRAs and providers to identify the cause of an attack as well as its target. This also helps in identifying incident’s criticality as well as the severity of its impact. Understanding the severity of the impact can be derived by studying who the former will affect: the customers, the network or the whole country. Necessary measures to mitigate the risks depending on the destination of the attack can thereafter be put in place. That being said, identifying the source of an attack helps in determining if or not the incident was intentional. The need to identify “attack” incident types is becoming more and more common as such incidents are being encountered on an often basis. NRAs and providers should be prepared to handle such situations. Furthermore, there are different types of cyber security attacks and the necessary measures to address them, may differ from one another.

text: Vulnerabilities within networks and services

text: Taking into account network and services vulnerabilities can also be another approach to take. As a provider, knowing and understanding vulnerabilities within the network can be an advantage when addressing incidents. For example, if one segment of the network can be made unavailable due to a serious vulnerability, that may be also present in other segments, knowing the situation and how to address it can help the provider in preparing the response to the incident and can help him prevent similar incidents in the future.

text: Economic impact related indicators

text: Though it was expressed by both the NRAs and providers that cost of damage was an indicator which was not used commonly, assessing security related incidents from this viewpoint can lead to some benefits. Providers were using cost to recover from an incident more frequently to measure the impact of recovery of an incident (50%) than when compared with measuring the actual loss of revenue following an incident (38%) or measuring the loss of market share (14%).

text: Examining economic impact related indicators can be used by the providers to assess the overall damage of an incident.

text: Impact on confidentiality, availability and integrity (CIA)

text: Evaluating the impact an incident has on availability, is the most pertinent factor in the telecommunications sector and the majority of providers were measuring impact of an incident based this metric (40% agreed). Nevertheless, security incidents can also negatively affect other areas too. If information is disclosed to unauthorized individuals, or if, for example, the information is not accurately available, businesses may suffer undesirable consequences. This could lead to a loss in customer confidence, contractual damages and can even result in financial losses

text: Measuring the impact of an incident based on confidentiality, integrity and availability

text: As a general remark, the CIA triad (confidentiality, integrity and availability) is designed to guide policies for information security within an organization. The triad can be adapted to telecommunications without only emphasising on availability. At the moment only 30% of providers are measuring incidents based on confidentially and integrity.

text: Similarly, availability was the most commonly assessed indicator among NRAs, with 55% using this indicator to measure the impact of security incidents (mainly due to Art. 13a specific requirements). Integrity was measured among 27% and 18% focused on the confidentiality aspect to measure security incidents.

text: Protecting sensitive information is becoming a key topic, not only in telecoms but in information security as well and by measuring the impact an incident has on sensitive information and on the integrity of such information can be another angle by which to assess security related repercussions.

text: Indicators used per technology

text: A point which needs to be highlighted is that for some of the questions in the survey, respondents were allowed to select more than one technology when specifying the indicators used to measure security incidents (as seen in sections 3.1 to 3.9). For example, for the indicator “number of users impacted”, providers answered that they used this indicator to measure the impact of an incident for “fixed telephony”, “mobile telephony” and “fixed internet”, “mobile internet” etc. To identify the percentage of indicators used per each individual technology, the same data used in the sections above can be presented in a different way. Using this method it is possible to identify why some indicators were used a lot less or a lot more for some technologies than when compared to others.

text: Indicators used per technology: Fixed internet

text: From the analysis it can be said that indicators used in the context of fixed technologies (both telephony and internet) were used in general more than when compared to mobile technologies, among both NRAs and providers.

text: To add to this point, it can be said that NRAs seem to be using the indicators more “effectively” than providers. For example, indicators such as “number of users impacted” and “duration of incidents”, is used by all NRAs who answered to the survey, for all technologies (100%). Furthermore, at the NRA level, the answers were very similar from one technology to the other. On the other hand, this was not the case at the provider level. The extent to which each indicator was used, and that too for each technology depended a lot on the provider’s level of maturity and also to a big extent on the provider’s relationship with its NRA. On average, indicators were used quite well for fixed telephony technologies. However, the percentage of providers who were using the indicators (mentioned above), to measure the impact on mobile internet, was considerably less (almost half as less), for mobile technologies. The reason for this relates once again to the fact that knowing how to evaluate the impact of a fixed technology is more straightforward than that of mobile. As is described at the start of this section, there are many additional complexities involved in mobile telephony and mobile internet services that both NRAs and providers should consider.

text: 4. Particularities regarding measuring the impact of incidents Defining significance of an incident

text: Defining the significance of an incident is an important aspect for measuring its impact. That being said, some indicators are definitely more pertinent than others. The

text: Article 13

text: a group along with ENISA have established some informal indicators along with the corresponding thresholds, to be used by NRAs and providers in the EU. These are not imposed, however they can be used to measure the significance or severity of an incident and can also encourage NRAs and providers to use their own thresholds. Based on those thresholds and indicators, they could be adopted as is by the involved stakeholders or modified accordingly, based on their necessity. This was also the case of most NRAs that used the indicators and thresholds as an inspirations, but followed a national approach based on their particularities.

text: According to the data gathered during this study, significance is mostly measured based on two key indicators: the number of customers impacted, and the duration of an incident. An additional indicator which is also commonly used is the number of services affected. To derive appropriate results, NRAs and providers can use a combination of these indicators in their calculation methods. For example, significance can either be calculated based on the number of users impacted vs. duration of an incident, or the number of services affected vs. duration of an incident.

text: If an incident results in downtime, but the time taken to recover from the impact is below the threshold defined by the NRA, or the number of users that were affected by the incident is below a certain threshold, then providers are not obliged to report the incident to their NRAs, as the incident is not of a significant impact. As per the informal guidelines of the

text: Article 13

text: a group, incidents should only be reported if the incident reaches the red areas from the table below, as regards to the number of affected users and the duration:

text: Using scales of criticality for assessing the impact

text: Another approach to take when measuring the significance is to measure the criticality of the incident.

text: Using scales of criticality to assess a security incident, to a certain extent coincides with measuring the significance of an incident. NRAs and/or providers can categorize indicators into criticality levels such as: “red”, “amber” and “green”. If for example, several incidents took place within the same time frame, making use of the criticality levels can help in prioritizing which incident to resolve first.

text: That being said, in cases where criticality levels were being considered, for the most part almost all relied on the ENISA metrics. 83% of respondents revealed that they were using this to categorize the impact of the incidents. One provider mentioned that the number of resources required to solve an incident also depended on its criticality. For them, the general rule they were using depending on the criticality of an incident is that a serious incident would be addressed by a specific department within the organization, a very serious incident, would require the help of several departments and finally, if the incident resulted in a crisis, the incident would be taken care of at operator level.

text: Using combination of indicators

text: To assess the severity of an incident and to also be able to know the level of resources required to solve the incident, the provider was relying on certain thresholds. The impact of an incident depended on the time duration as well as the number of users affected for each service category. Measuring security incidents using a combination of indicators is an interesting approach to take. Many providers and NRAs are already using this method and the main advantage is that they can observe the impact of a security incident from various perspectives.

text: Furthermore, combining indicators can help gather more precise information on the impact of an incident. For example, it can allow NRAs and providers to understand the correlation between the number of user vs. the number of services impacted, and the number of users vs. time duration etc. That being said, the indicators which can be use are countless, and it is up to the specific NRA and provider to

text: decide how the want to use the indicators, and on what basis they want to calculate the impact of an incident.

text: Among providers, the most commonly used indicators were the following: number of users affected combined with duration of an incident (83%) and number of users affected combined with geographic spread (74%). Furthermore, providers were also calculating number of users affected with geographic spread (70%). The same is also true for NRAs wherein number of users impacted was often calculated in combination with the duration of the incident (for 85%).

text: Assessing impact of incidents based on customer “importance”

text: Assessing the impact of an incident based on customer “importance” is another methodology which can be used mostly by providers. From the viewpoint of the NRA, measuring incidents based on customer importance can service to the needs of high profile clients, or critical infrastructure clients who deal with sensitive information, such as banks, special government institutions, hospitals etc. Focusing on these clients can also help ensure greater security overall. From the viewpoint of the provider, measuring security incidents based on customer importance has other merits, since this can help towards gaining customer loyalty and trust. Furthermore, doing so can improve the customer’s perception of reliability with regards to the providers, thus resulting in competitive advantage and customer retention.

text: However, measuring incidents based on customer importance is not so widely used among NRAs and providers. All 100% of the NRAs stated that a distinction between important and standard users was not made. Nevertheless, during interviews it was identified that most providers had a special group of customers, with whom either they held extra SLAs or who they knew is critical for them. For those customers, most providers, kept a more close watch and notified them should anything happen. This is especially true for critical infrastructures (banks, hospitals, etc.)

text: Assessing impact of incidents based on quality and service degradation

text: There are quite a lot of differing opinions on how to measure the impact of an incident based on quality and the degradation level of services. While at the moment, this indicator is not being used, the concept of service degradation and assessing the quality of a service remains important. That being said, clarification needs to be made surrounding the concept of what is considered “service degradation”. For example, does this refer to the moment at which providers experience a loss in service to some of their systems (while others are still functioning), or is this the moment at which all services are completely unavailable. Furthermore, since service degradation can fluctuate, is this indicator calculated during the time period at which systems are facing service degradation or will this be later measured after the incident has occurred. Using the correct terminology can help providers and NRAs achieve more accuracy when measuring the impact of a security incident based on this indicator. This includes knowing how to differentiate between concepts such as “services failing”, “services going down”, “services affected”, “total failure” and “services that are not working correctly” etc.

text: To summarize this point however, it is important to emphasise that technology is always changing, meaning that measurement methods which could seem appropriate at one point in time may not necessarily be so at another. The same is also true for service degradation in particular where in the future due to more advanced technologies, it may be the case that systems would not face total or complete loss of service, but rather periods of service degradation instead.

text: End-user requirements

text: End-users can play a vital role in the security incident reporting process as they are the ones who essentially make use of these infrastructures and services. Furthermore, bringing more transparency

text: into the process by informing end-users of the indicators used for measuring security incidents and also informing them of the incidents that have taken place, can be a beneficial approach. Empowering endusers with the necessary skills on incident identification and reporting could be an approach to be adopted, as suggested by some respondents.

text: So far, almost all providers report incidents to their NRAs only, while end-users do not have access to such information. To add to this point, end-users have to either specifically request for this information if they need it, or wait for the yearly reports from the NRAs or ENISA. However, the information that they receive will most likely be of past incidents that have occurred in the year, and therefore may not be so relevant to them anymore. Similarly, end-users many only find out about the incidents through the media.

text: Only one provider, who was interviewed, mentioned that apart from the yearly reports sent to the NRA, they already had in place another reporting process which was being used in parallel to ensure that endusers were also made aware of the security incidents. The majority however, stated that they had no direct interaction with their end-users whatsoever. Thus, the former can be seen as a potential area for further improvement and/or development. Working more closely with end-users can lead to gaining the trust of the customer and at the same time, further result in greater competitive advantage which is a key priority from the view point of the providers. However, as a provider, going public with details regarding incidents can be disadvantageous in certain situations (e.g. exposing a service/technology/infrastructure vulnerability which increases the appetite for attacks).

text: That being said, some end-users require more attention than others. These end-users include organizations such as banks that not only need to be informed immediately of the incidents but also have to have a contractual agreement with the providers, such as an SLA with which they may rely on the measurement of certain indicators like uptime etc. This is especially needed in the banking sector.

text: To summarize on this, further information has not been collected from the NRAs or from providers regarding the requests/satisfaction of end users on the specific matter of indicators.

text: 5. Conclusions

text: The overall purpose of this document is to provide guidelines to national regulatory authorities (NRAs) and telecommunications providers within EU member states and to assist them in the process of measuring the impact of security incidents affecting the availability of electronic communication services. Therefore, interested stakeholders have at their disposal a catalogue of indicators to be used to tailor impact assessment and design the corresponding solutions.

text: A milestone in measuring the significance of incidents within the telecommunications sector in EU was set up along with the adoption of the Framework Directive (Directive 2009/140 EC) within the 2009 Telecom Package, which included Art. 13a. This article aims at ensuring the security and integrity of electronic communication networks and services in EU. This is partially achieved through requiring telecommunication service providers to take the appropriate technical and organizational measures to manage the risks posed to security of networks and services, guarantee the integrity of their networks (ensure the continuity of supply of services provided over those networks) and notify the competent national regulatory authority (NRA) of a breach of security or loss of integrity that has had a significant impact on the operation of networks or services.

text: Measuring the significance of telecommunications incidents became a necessity within the EU, and led to numerous debates between involved stakeholders. A lot of work has been done in this area both by ENISA, along with the Art. 13a expert group, and also by private or independent bodies, as some work is publicly available online. This report comes as a practical approach, and contains the view of both NRAs and providers within the EU, on real indicators used for measuring significance of incidents affecting the availability of telecommunication network and services.

text: Having said this, is it advised that making use of a standardized approach among NRAs and providers can help derive more precise results in the incident reporting process. So, the key recommendation from the study is that NRAs and providers should further increase the level of harmonisations in the approach taken to measure security incidents. ENISA will continue to support the community through the Art. 13a group, and continue to provide best practices, guidelines and support for all stakeholders in members states.

text: ENISA

text: European Union Agency for Network and Information Security Science and Technology Park of Crete (ITE) Vassilika Vouton, 700 13, Heraklion, Greece

text: Athens Office

text: 1 Vass. Sofias & Meg. Alexandrou Marousi 151 24, Athens, Greece Catalogue Number TP-04-15-917-EN-N

text: PO Box 1309, 710 01 Heraklion, Greece ISBN: 978-92-9204-156-4 DOI: 10.2824/887699 Tel: +30 28 14 40 9710 info@enisa.europa.eu www.enisa.europa.eu