text: “The Economics of Information Security Investment “, Lawrence Gordon and Martin Loeb, http://ns1.geoip.clamav.net/~mfelegyhazi/courses/BMEVIHIAV15/readings/04_GordonL02economics_security_investment.pdf

text: See detailed Cost of ICT incident calculation exercise, “CERT exercise handbook”, ENISA, 2012

text: Imagine you calculate the cost –

text: reputational costs, loss of customers,

text: etc. – of having your company's name in the newspaper after an embarrassing cybersecurity event to 4.1 The drawback of estimation

text: be $20 million. Also assume that the

text: happening in any one year. ALE says

text: you should spend no more than

text: $2,000 mitigating that risk.

text: So far, so good. But maybe your CFO

text: thinks an incident would cost only

text: $10 million. You can't argue, since

text: your security budget in half. A vendor decision rather than enlighten it. trying to sell you a product finds a Web analysis claiming that the odds

text: of this happening are actually 1 in

text: 1,000. Accept this new number, and

text: It gets worse when you deal with even more rare and expensive Trust your experience events. Imagine you're in charge of terrorism mitigation at a chlorine

text: plant. What's the cost to your

text: company, in money and reputation,

text: of a large and very deadly explosion?

text: thousand, 1 in a million, 1 in 10 percentage related in any study. million? Depending on how you answer those two questions -- and any answer is really just a guess -you can justify spending anywhere from $10 to $100,000 annually to 4.2 Gordon & Loeb Model mitigate that risk.

text: “On the Gordon&Loeb model for Information Security Investment”, 2006, Jan Willemson, Universtity of Tartu,

text: Cyber-attacks can get costly if not resolved quickly. Results show a positive relationship between the time to contain an attack and organisational cost. The average time to resolve a cyber-attack was 24 days, with an average cost to participating organisations of £135,744 over this 24-day period. Results show that malicious insider attacks can take more than 50 days on average to contain.

text: Source Ponemon Study, Oct. 2012 – Cost of cybercrime UK

text: ENISA annual incident report http://www.enisa.europa.eu/activities/Resilience-and- CIIP/Incidents-reporting/annual-reports/annual-incidentreports-2011 CSI Computer Crime and Security Survey http://gocsi.com/survey

text: CSIRT CZ https://www.csirt.cz/files/csirt/statistics/stats.html US Department of Justice: PRO IP Act, http://www.justice.gov/dag/iptaskforce/proipact/doj-proip-rpt2011.pdf

text: US Bureau of Justice Statistics: http://bjs.ojp.usdoj.gov/content/pub/pdf/cb05.pdf

text: 2011GlobalSecurityStatsAndTrends.pdf Trustwave Global Security Report https://www.trustwave.com/global-security-report Symantec Internet Security Threat http://www.symantec.com/threatreport/

text: Verizon Data Breach Investigations http://www.verizonbusiness.com/about/events/2012dbir/ http://public.tableausoftware.com/views/VERISCommunity

text: /DemographicsandAgent http://www.veriscommunity.net/

text: See Waldo Rocha Flores et al., Assessing Future Value of Investments in Security-Related IT Governance Control Objectives – Surveying IT Professionals, http://www.ejise.com/issue/download.html?idArticle=773

text: See http://www.first.org/global/sigs/metrics

identifier: Introduction to Return on Security Investment

publisher: Europeiska unionens cybersäkerhetsbyrå

sprak: en

title: Introduction to Return on Security Investment

serie: rapporter

text: Helping CERTs assessing the cost of (lack of) security [Deliverable – December 2012]

type: rubrik

text: Introduction to Return on Security Investment I Helping CERTs assessing the cost of (lack of) security

type: stycke

text: About ENISA

text: The European Network and Information Security Agency (ENISA) is a centre of network and information security expertise for the EU, its Member States, the private sector and Europe’s citizens. ENISA works with these groups to develop advice and recommendations on good practice in information security. It assists EU Member States in implementing relevant EU legislation and works to improve the resilience of Europe’s critical information infrastructure and networks. ENISA seeks to enhance existing expertise in EU Member States by supporting the development of cross-border communities committed to improving network and information security throughout the EU. More information about ENISA and its work can be found at www.enisa.europa.eu Follow us on Facebook Twitter LinkedIn YouTube & RSS feeds

text: Contact details

text: To contact ENISA for this report please use the following details:  E-mail: opsec@enisa.europa.eu  Internet: http://www.enisa.europa.eu

text: Legal notice

text: Notice must be taken that this publication represents the views and interpretations of the authors and editors, unless stated otherwise. This publication should not be construed to be an action of ENISA or the ENISA bodies unless adopted pursuant to the ENISA

text: Regulation (EC) No 460/2004

text: as amended by

text: Regulation (EC) No 1007/2008

text: . This publication does not necessarily represent state-of the-art and it might be updated from time to time. Third-party sources are quoted as appropriate. ENISA is not responsible for the content of the external sources including external websites referenced in this publication. Member States are not responsible for the outcomes of the study. This publication is intended for educational and information purposes only. Neither ENISA nor any person acting on its behalf is responsible for the use that might be made of the information contained in this publication. Reproduction is authorised provided the source is acknowledged. © European Network and Information Security Agency (ENISA) 2012

text: Introduction to Return on Security Investment 1 Helping CERTs assessing the cost of (lack of) security

text: 1 Executive Summary

text: As for any organization, CERTs need to measure their cost-effectiveness, to justify their budget usage and provide supportive arguments for their next budget claim. But organizations often have difficulties to accurately measure the effectiveness and the cost of their information security activities. The reason for that is that security is not usually an investment that provides profit but loss prevention. So what is the right amount an organization should invest in protecting information? The aim of this document is to initiate a discussion among CERTs to create basic tools and best practices to calculate their Return on Security Investment (ROSI). This key notion is essential when justifying costs engagement and budgets for those entities that deal with security on a regular basis (security departments, CERTs, etc.). Although the methods outlined here are straightforward, their application to the real world should take into account a general tendency to misevaluate the actual cost of an incident, a central notion of the ROSI calculation. While being controversial, the Gordon & Loeb Model is an attempt to ease the finding of the optimal level of investment to protect a given asset. Due to the diversity of their nature, funding models and capabilities, calculating the return on investment of CERTs has to go beyond a single ROSI calculation. In fact, assessing the costeffectiveness of CERTs should take into account the beneficial actions that CERTs achieve by contributing to detect, handle, recover from and deter incidents early and efficiently. And, the earlier an incident is handled, the less expensive is its mitigation. The profitability of a CERT is therefore assessed by determining the difference of incident handing costs with the help of CERT versus not having a CERT.

text: Introduction to Return on Security Investment 2 Helping CERTs assessing the cost of (lack of) security

text: 2 The need for ROSI calculation 2.1 Answers to important questions

text: Return on investment

text: In every public or private organisation, each budget investment has to be justified and its effectiveness is often evaluated afterward. In finance, this evaluation is called the Return on investment or rate of return. The ROI is calculated as follow:

text: Example of ROI calculation:

text: Alice would like to run a lemonade business for summer. She needs money for setting up the business. Bob gives her 200€ to start her business. In return, Alice agrees to give Bob 50% of the benefits. At the end of summer, Alice made 1000€ of benefits. Bob gets 500€. Bob’s Return on Investment is calculated as follow:

text: Return on security investment

text: The concept of the ROI calculation applies to every investment. Security is no exception. As stated in ENISA’s work program 2012, “executive decision-makers want to know the impact security is having on the bottom line. In order to know how much they should spend on security, they need to know how much is the lack of security costing to the business and what are the most cost-effective solutions.” Applied to security, a Return On Security Investment (ROSI) calculation can provide quantitative answers to essential financial questions:

text: - Is an organization paying too much for its security? - What financial impact on productivity could have lack of security? - When is the security investment enough? - Is this security product/organisation beneficial?

text: 2.2 The false notion of security investment

text: The classical financial approach for ROI calculation is not particularly appropriate for measuring security-related initiatives: Security is not generally an investment that results in a profit. Security is more about loss prevention. In other terms, when you invest in security, you don’t expect benefits; you expect to reduce the risks threatening your assets. With this

text: Introduction to Return on Security Investment 3 Helping CERTs assessing the cost of (lack of) security

text: approach, the quantitative assessment the Return on Security Investment is done by calculating how much loss you avoided thanks to your investment.

text: Security Investment Loss reduction

text: The aim of cost-effective security.

text: Introduction to Return on Security Investment 4 Helping CERTs assessing the cost of (lack of) security

text: 3 Methodology for ROSI calculation

text: Assessing security investment involves evaluating how much potential loss could be saved by an investment. Therefore, the monetary value of the investment has to be compared with the monetary value of the risk reduction. This monetary value of risk can be estimated by a quantitative risk assessment.

text: 3.1 Basic concepts of risk assessment

text: Quantitative risk assessment is achieved by determining several components of a risk. The following notions need to be defined:

text: Single Loss Expectancy (SLE)

text: The SLE is the expected amount of money that will be lost when a risk occurs. In this approach, SLE can be considered as the total cost of an incident assuming its single occurrence. Due to the specific nature of cyber incident, the major complexity is to take into account all the assets this incident has an impact on. For instance, a stolen laptop will not only cost the replacement of the laptop itself but will also imply productivity loss, reputation loss, IT support time and, possibly, cost of intellectual property loss. The total cost of an incident should include the cost of direct losses (website downtime, hardware replacement, data loss replacement, etc.) and the cost of indirect losses (investigation time, loss of reputation, impact on image, etc.). There are no universal values for SLEs. What will be included in the calculation of the SLE of a specific threat will depend on the business objectives, cultural values and existing security measures. In the end, one entity could estimate the SLE of a stolen laptop to the value of the laptop itself (i.e. 2.000€) while another organisation dealing with highly-sensitive information would value this loss to 100.000 € as it would affect its image, its potential contracts and its competitive advantage. Although the SLE can be evaluated in different ways, the ROSI calculation often implies comparison of different SLEs. Therefore, it is important to be consistent in the way it is calculated.

text: Annual Rate of Occurrence (ARO)

text: The ARO is a measure of the probability that a risk occurs in a year. Again, this data is an approximation and can depend on many factors: the ARO of a flood will depend on geographic factors, the ARO of a disk failure is influenced by the operating temperature, the ARO of a burglary will depend on the location of the asset, etc. And, of course, the ARO is also

text: Introduction to Return on Security Investment 5 Helping CERTs assessing the cost of (lack of) security

text: depending on the existing security measures: the ARO of a successful malicious code attack will decrease significantly after implementing an effective anti-virus.

text: Annual Loss Expectancy (ALE)

text: The ALE is the annual monetary loss that can be expected from a specific risk on a specific asset. It is calculated as follow:

text: 3.2 ROSI calculation

text: The ROSI calculation combines the quantitative risk assessment and the cost of implementing security counter measures for this risk. In the end, it compares the ALE with the expected loss saving.

text: Return on Security Investment (ROSI)

text: Following the ROI definition, the ROSI is defined as below:

text: Implementing an effective security solution lowers the ALE: the more a solution is effective, the more reduced is the ALE. This monetary loss reduction can be defined by the difference of the ALE without the security solution versus the modified ALE (mALE) implementing the security solution.

text: Which also equals to the mitigation ratio of the solution applied to the ALE:

text: Example 1: The Acme Corp. is considering investing in an anti-virus solution. Each year, Acme suffers 5 virus attacks (ARO=5). The CSO estimates that each attacks cost approximately 15.000 € in loss of data and productivity (SLE=15.000). The anti-virus solution is expected to block 80% of the attacks (Mitigation ratio=80%) and costs 25.000€ per year (License fees 15.000€ + 10.000€ for trainings, installation, maintenance etc.). The Return on security investment for this solution is then calculated as follow: ( )

text: According to this ROSI calculation, this anti-virus solution is a cost-effective solution.

text: Introduction to Return on Security Investment 6 Helping CERTs assessing the cost of (lack of) security

text: In the end, ROSI calculation is based on 3 variables: estimated potential loss (ALE), estimated risk mitigation, and cost of the solution. If the cost of the solution is easier to predict – provided all indirect costs are considered – the two other variables are estimations that makes ROSI more approximate.

text: Introduction to Return on Security Investment 7 Helping CERTs assessing the cost of (lack of) security

text: 4 The limits of ROSI

text: The data imperative

text: Estimating the amount of money saved from losses that may

text: never happen is a hard task that, in the real world, requires

text: more than straightforward application of simple formulas.

text: odds are 1 in 10,000 of that The ROSI calculation is the result of many approximations.

text: The cost of cyber security incidents and annual rate of

text: occurrence are hard to estimate and the resulting numbers

text: can very highly from one environment to another. These

text: approximations are often biased by our perception of the risk

text: and the ROSI calculation can be easily manipulated (See ‘The

text: we're just estimating. But he just cut data imperative’) to serve the user's interest or to justify a

text: The accuracy of statistical data used in the ROSI calculation is

text: therefore essential. However, actuarial data on security

text: suddenly a product costing 10 times incidents are hard to find as companies are often reluctant to as much is still a good investment. provide data on their security incidents.

text: It's often a better practice to extrapolate from the

text: organisation's historical data on incidents than to rely on the

text: study of a vendor. In practical terms, if, in the past 5 years, a

text: $100 million? $1 billion? $10 billion? website has been the target of a denial-of-service attack 6 And the odds: 1 in a hundred times then an ARO of 6/5 would be more accurate then a

text: Lawrence Gordon and Martin Loeb are economists at the Source: Bruce Schneier, Security ROI, University of Maryland. Their study, published in 2002, “The http://www.schneier.com/blog/archives/ Economics of Information Security Investment” is well

text: known and often cited (552 references according to Scholar Google).

text: Introduction to Return on Security Investment 8 Helping CERTs assessing the cost of (lack of) security

text: In their study, the authors state that, contrary to the basics of risk assessments, an asset of greater value should not necessarily benefit from a greater investment to protect it. The optimal information security investment does not always increase proportionately to increases in vulnerability; there is a point at which it is not in the best interest of a firm to make increasingly larger investments in information security.

text: According to this study, “the optimal amount to spend on information security never exceeds 37% of the expected loss resulting from a security breach (and is typically much less that 37%). Hence, the optimal amount to spend on information security would typically be far less than even the expected loss from a security breach”.

text: The Gordon & Loeb model has been questioned by another study showing that there was possibly no fixed percentage for optimal investment.

text: These conflicting studies show that ROSI calculation remains an approximate model and that the resulting numbers should be regarded with care. Organisations should consider the results as guidelines rather than strict rules to follow. ROSI calculation will never be perfectly accurate.

text: Introduction to Return on Security Investment 9 Helping CERTs assessing the cost of (lack of) security

text: 5 Assessing the cost-effectiveness of CERTs

text: CERTs internally providing services to an entity are “non-profit” organisations; their goal is not to make money but to prevent losses by avoiding, containing and recovering from an incident in a quick and efficient way. Therefore, the cost-effectiveness of CERTs has to be regarded as security investment: their returns on investments are the savings they provide.

text: A factual approach is advised here: ALE is often easier to calculate a posteriori, from more accurate historical data. Therefore, assessing the cost-effectiveness of CERTs can be approximated by assessing the difference of past incident response cost done with CERTs versus what would have been the incident response cost without CERTs.

text: As a rule of thumb, the quicker an incident is detected, the less expensive it is to recover from it. Depending on the type of incident, damages can grow exponentially over time. Therefore the time-saving provided by CERTs activities in incident eradication represents a financial saving in terms of damage and downtime reduction. The actual savings provided by a CERT can then be estimated by summing all the savings provided to its constituency.

text: t 18 days without en CERT d ci in f o 2 days with Cost CERT Savings provided by CERT

text: Days to eradication

text: CERT leads to faster response which leads to savings

text: Introduction to Return on Security Investment 10 Helping CERTs assessing the cost of (lack of) security

text: Obviously, to estimate the net savings of CERT, the cost of operating a CERT has to be deduced from the overall savings. In that matter, the cost of logistics such as building, trainings, administrative, materials, etc. will have to be deduced from the savings a CERT provides and the resulting number will be the actual savings of a CERT.

text: Introduction to Return on Security Investment 11 Helping CERTs assessing the cost of (lack of) security

text: 6 Remaining issues and further reading

text: This introductory study offers a quick overview on how cost-effectiveness of security can be approached. ROSI is a complex topic and despite the numerous studies on this topic, a lot of aspects remain unresolved.

text: Gathering statistical data

text: Data accuracy is essential in ROSI calculations. Unfortunately, the threats move quickly and companies are often reluctant to reveal data on their security incidents. Therefore, little statistical information exists on the occurrence and cost of incident and effectiveness of security measure. Some CERTs regularly produce activity reports and incident statistics. These are valuable information to better estimate the Annual Loss Expectancy of a threat. Some interesting figures:

text: Title URL

text: Annual report

text: Cybercrime against Businesses, 2005

text: US Cert, 2011 Global Security Statistics http://buildsecurityin.uscert.gov/swa/presentations_032011/CharlesHenderson-

text: and Trends

text: Report

text: Report

text: Introduction to Return on Security Investment 12 Helping CERTs assessing the cost of (lack of) security

text: Other models

text: The ALE model presented here is a classic approach to calculate ROSI. More complex models exist. For instance, the Net Present Value model takes into consideration the decreasing value of saved income. The Australian Department of Finance and Services introduced a hybrid ROSI calculation model combining ALE and an Australian risk assessment method called Threat and Risk Assessment (TRA). This method is based on tables covering the possible threats and their counter measures. In this bottom-up approach, each risk and its associated counter measure are evaluated resulting in a global ROSI calculation for an entity. This model is detailed in the Guide for Government Agencies – Calculating Return on Security Investment .

text: ROSI Calculator

text: Tools exist to help the calculation of ROSI. Although they can present a simplified and partial view of this complex task, they are useful to support the workflow and calculations involved in this process: http://www.iso27001standard.com/en/rosi/return-on-security-investment

text: Introduction to Return on Security Investment 13 Helping CERTs assessing the cost of (lack of) security

text: 7 Conclusion

text: This introductory paper presents the basis of Return on Security Investment calculation and how it can help CERTs in assessing their cost effectiveness. ROSI is a complex topic and this first attempt to introduce this topic has to be further developed to address remaining issues on CERTs and ROSI calculation: Which model best applies to CERTs? What to include in the cost of an incident? How to measure the added value of CERT teams in incident handling? How can CERTs estimate the value of assets they protect indirectly? How this valuable information could be shared among CERTs and benefits to all the community? The FIRST Metrics SIG is working to better the metrics and evaluation methods for internal evaluation of CERTs. As part of this work, the Metrics SIG is addressing the topic of cost of incidents and return on security investment. The results of this research will help CERTs in assessing their profitability.

text: Introduction to Return on Security Investment 14 Helping CERTs assessing the cost of (lack of) security

text: 8 Annex I: References

text:  Return on Security Investment (ROSI): A Practical Quantitative Model, no date, http://www.ra.cs.uni-tuebingen.de/lehre/uebungen/ss09/introsec/ROSI- Practical_Model.pdf,  K. J. Soo Hoo. How much is enough? A risk management approach to computer security. http://iis-db.stanford.edu/pubs/11900/soohoo.pdf, June 2000.  CERT podcast: Security for business leader – The ROI of security http://www.cert.org/podcast/notes/2roi.html  Tom Campbell, SANS Institute - An Introduction to the CSIRT Set-Up and Operational Considerations, 2004, http://cyberdefense.sans.org/resources/papers/gsec/introduction-computer-security-incidentresponse-106281  ISACA - IS Auditing Guideline: G41 Return on Security Investment (ROSI), 2010, http://www.isaca.org/Knowledge-Center/Standards/Documents/G41-ROSI-5Feb10.pdf  Christian Locher, Methodologies for evaluating information security investments, 2005, http://csrc.lse.ac.uk/asp/aspecis/20050136.pdf

text: Introduction to Return on Security Investment 15 Helping CERTs assessing the cost of (lack of) security

text: P.O. Box 1309, 71001 Heraklion, Greece www.enisa.europa.eu