text: November 2019

text: November 2019

text: November 2019

text: November 2019

text: November 2019

text: November 2019

text: https://www.pcisecuritystandards.org/ 3 https://www.isa.org/intech/201810standards/

text: November 2019

text: https://www.commoncriteriaportal.org/ 5 https://www.owasp.org/index.php/Category:OWASP_Application_Security_Verification_Standard_Project

text: November 2019

text: https://www.bsimm.com/ 7 http://tsfdn.org/wp-content/uploads/2016/03/TS502-0-TS-Essentials-Specification-Issue-1.2-WHITE.pdf 8 https://www.owasp.org/index.php/OWASP_SAMM_Project

text: November 2019

text: https://securesoftwarealliance.org/framework-secure-software/ http://www.safecode.org/ 11 https://safecode.org/wp-content/uploads/2018/02/SAFECode_Perspective_on_Cybersecurity_Certification.pdf

text: November 2019

text: November 2019

text: November 2019

text: November 2019

text: November 2019

identifier: Advancing Software Security in the EU

title: Advancing Software Security in the EU

text: ABOUT ENISA

text: The mission of the European Union Agency for Cybersecurity (ENISA) is to achieve a high common level of cybersecurity across the Union, by actively supporting Member States, Union institutions, bodies, offices and agencies in improving cybersecurity. We contribute to policy development and implementation, support capacity building and preparedness, facilitate operational cooperation at Union level, enhance the trustworthiness of ICT products, services and processes by rolling out cybersecurity certification schemes, enable knowledge sharing, research, innovation and awareness building, whilst developing cross-border communities. Our goal is to strengthen trust in the connected economy, boost resilience of the Union’s infrastructure and services and keep our society cyber secure. More information about ENISA and its work can be found at www.enisa.europa.eu.

text: CONTACT

text: For contacting the authors please use isdp@enisa.europa.eu. For media enquiries about this paper, please use press@enisa.europa.eu.

text: CONTRIBUTORS

text: Rob van der Veer (SIG), Menno Valkema (SIG), Fabio Guasconi (Bl4ckswan), Prokopios Drogkaris (ENISA)

text: EDITORS

text: Prokopios Drogkaris (ENISA) and Athena Bourka (ENISA)

text: ACKNOWLEDGEMENTS

text: We would like to thank Apostolos Malatras (ENISA), Philippe Blot (ENISA) and Eric Vetillard (ENISA) for their valuable comments and reviews while preparing this study.

text: LEGAL NOTICE

text: Notice must be taken that this publication represents the views and interpretations of ENISA, unless stated otherwise. This publication should not be construed to be a legal action of ENISA or the ENISA bodies unless adopted pursuant to the

text: Regulation (EU) No 2019/881

text: . This publication does not necessarily represent state-of the-art and ENISA may update it from time to time. Third-party sources are quoted as appropriate. ENISA is not responsible for the content of the external sources including external websites referenced in this publication.

text: This publication is intended for information purposes only. It must be accessible free of charge. Neither ENISA nor any person acting on its behalf is responsible for the use that might be made of the information contained in this publication.

text: COPYRIGHT NOTICE

text: © European Union Agency for Cybersecurity (ENISA), 2020 Reproduction is authorised provided the source is acknowledged.

text: For any use or reproduction of photos or other material that is not under the ENISA copyright, permission must be sought directly from the copyright holders. ISBN 978-92-9204-343-8, DOI 10.2824/81950

text: ADVANCING SOFTWARE SECURITY IN THE EU

text: TABLE OF CONTENTS

text: 1. INTRODUCTION 4

text: 1.1 SCOPE – OBJECTIVES 4

text: 1.2 STRUCTURE OF THE DOCUMENT 5

text: 2. TOWARDS IMPROVED SOFTWARE SECURITY 6

text: 2.1 SECURITY REQUIREMENTS 6

text: 2.2 SECURE SOFTWARE ENGINEERING 6

text: 2.3 SECURE DEVELOPMENT LIFECYCLE 7

text: 2.4 EXISTING STANDARDS AND GOOD PRACTICES 7

text: 3. WHAT IS LACKING IN SOFTWARE SECURITY 10

text: 3.1 CLEAR GUIDANCE 10

text: 3.2 QUALITY ASSURANCE 10

text: 3.3 SUSTAINED TRUST ISSUES 11

text: 3.4 ASSURANCE CLARITY BETWEEN PROCESS AND PRODUCT 11

text: 4. DRIVING SOFTWARE SECURITY IN THE EU 12

text: 4.1 DEVELOP A COMMON REPOSITORY FOR SHARED SECURITY MEASURES 12

text: 4.2 IMPROVE TECHNICAL STANDARDS LANDSCAPE 12

text: 4.3 PROVIDE ASSURANCE IN THE ENGINEERING PROCESS 12

text: 4.4 INCREASED SCHEMES APPLICABILITY AND CLEARLY COMMUNICATED ASSUMPTIONS 13

text: 5. REFERENCES 14 ADVANCING SOFTWARE SECURITY IN THE EU

text: EXECUTIVE SUMMARY

text: Secure software development and maintenance is attracting a lot of attention lately, due to the rapidly increased dependency of everyday products, services and process to the underlying software. Quite often, weaknesses behind reported security incidents and/or breaches are being materialized due to the lack of adherence on fundamental security principles and techniques. In order to promote further the assurance on the level of security or even the mitigated security threats, software development and maintenance is becoming increasingly subject to evaluation, and eventually certification, of ICT products, services and processes. Based on this, as part of ENISA activities in the area of supporting the preparatory policy discussions in the area of certification of products, services and processes, this study aims to touch upon the aspects to be considered in EU cybersecurity certification schemes (relevant to software development and maintenance).

text: This study discusses some key elements of software security and provides a concise overview of the most relevant existing approaches and standards while identifying shortcomings associated with the secure software development landscape, related to different inherent aspects of the process. Lastly, it provides a number of practical considerations relevant to the different aspects of software development within the newly established EU cybersecurity certification framework and the EU cybersecurity certification schemes. These considerations are listed below:

text:  Manufacturer(s) or provider(s) of certified ICT products, ICT services or ICT processes, should explore the deployment and maintenance of repositories not only for publicly disclosed vulnerabilities but also for shared security aspects of certified products, services and processes towards aligning on requirement commonalities and ways to mitigate common security risks.  Following the publication of the Union Rolling Work Programme, European Standards Organizations (ESOs) and Standards Developing Organization (SDOs) should coordinate on the priority areas they can support, put forward standardization activities to benefit the future developed schemes and communicate periodically such planning to the EC and relevant CSA stakeholders.  EU cybersecurity certification schemes for products, services and process should include, to the extent possible, not only requirements for the end product/service/process but also assurance for the engineering process, by setting process guidelines for software development, maintenance and operation.  During the development of EU cybersecurity certification schemes, lightweight conformity assessment methods for the basic assurance level should be considered as a response to the existing fragmented landscape of software development and maintenance.  Software developers and product manufacturers should put forward their experience and expertise and promote the uptake of EU cybersecurity certification schemes.

text: ADVANCING SOFTWARE SECURITY IN THE EU

text: 1. INTRODUCTION

text: Software is becoming increasingly important in everyday activities, as the world is digitizing rapidly and unfortunately not without any problems [1]. Security breaches are increasing in number and in severity [2]. Quite often, the origin of security breaches is identified in omissions and errors that took place during software development or maintenance. Vulnerabilities like Heartbleed demonstrate how minor decisions taken by a developer can result in major hidden vulnerabilities that go undetected by tools and tests for years on end, with disastrous effects.

text: When looking at the weaknesses behind reported security incidents, it is striking how

text: Fundamental

text: fundamental security principles and techniques are often overlooked: clear text password

text: security

text: storage, SQL injection vulnerabilities, missing authorisation checks, insufficient logging etc. Security should be built in to the end product or service and software plays an increasingly principles are bigger role towards materializing this principle. Much of the security of systems is determined by often overlooked software, since much of the behaviour and inner working of applications and devices are

text: during software

text: specified in source code – even though that source code runs on hardware. For example: of all

text: development.

text: the technical requirements in the new ETSI TS 103 645 v.1.1.1 technical specifications “Cyber Security for Consumer Internet of Things for IoT devices” [3], the majority of Cyber security provisions relate to the software implementation. Further to that, 60% of security breaches in 2019 involved known but unpatched vulnerabilities [4]

text: The Cybersecurity Act [5], under Title III, establishes the European cybersecurity certification framework for products, services and processes. The EU cybersecurity certification framework foresees four main phases, namely i) the creation and publication of the Union Rolling Working Programme (URWP), ii) the preparation of a candidate cybersecurity certification scheme, iii) the enforcement by legislation of the accepted candidate scheme and iv) the implementation of schemes by Member States. Throughout these phases, multiple stakeholders are involved, with the European Commission, ENISA, the European Cybersecurity Certification Group (ECCG) and the Stakeholder Cybersecurity Certification Group (SCCG) having a pivotal role in the governance of the framework. The certificates issued under the schemes will be valid in all EU Member States. Depending on the assurance level (and risks involved), the certification may entail self-assessment by the manufacturer or provider of ICT products, services or processes or involve either a national cybersecurity certification authority or a conformity assessment body.

text: 1.1 SCOPE – OBJECTIVES

text: Secure software development and maintenance is attracting a lot of attention lately, due to rapidly increased dependency of everyday products, services and process to the underlying software [2], [6], [7] & [8]. As such, software development and maintenance is expected to be subject to evaluation, and eventually certification, of ICT products, services and processes. Based on this, as part of ENISA activities in the area of supporting the preparatory policy discussions in the area of certification of products, services and processes, this study aims to provide:

text:  a starting point for exploring the concept of secure software development and maintenance and  aspects to be considered in EU cybersecurity certification schemes (relevant to software development and maintenance).

text: ADVANCING SOFTWARE SECURITY IN THE EU

text: This study is envisioned as a reference document that complements similar ongoing initiatives at National level, while drafting candidate cybersecurity certification schemes and also as a nonbinding guidance document for EU cybersecurity certification framework stakeholders during the promulgation and maintenance of adopted EU cybersecurity certification schemes.

text: A more detailed analysis in the area of IoT security, with a particular focus on software development guidelines for secure IoT products and services throughout their lifetime, is also available by ENISA [7]. The focus of the aforementioned publication was to define a set of good practices and guidelines to be applied in the different phases of the secure SDLC of IoT solutions.

text: 1.2 STRUCTURE OF THE DOCUMENT

text: The rest of the document is structured as follows:

text:  Section 2 discusses some key elements of software security in order to allow a better comprehension of the document’s direction, being completed with an overview of the most relevant existing approaches and standards.  Section 3 provides an overview of the shortcomings inside the software security landscape, related with different inherent aspects of the process, of the products and of the concepts surrounding the software security concept itself.  Section 4 provides a number of practical considerations that can be considered and adopted with regards to the different aspects of software development within the newly established EU cybersecurity certification framework and the EU cybersecurity certification schemes.

text: ADVANCING SOFTWARE SECURITY IN THE EU

text: 2. TOWARDS IMPROVED SOFTWARE SECURITY

text: 2.1 SECURITY REQUIREMENTS

text: Functional requirements are about behaviour of the system towards the outside world (e.g. a user), whereas non-functional requirements are mainly about the internal mechanisms. Many of the security requirements are non-functional; for example on how to store passwords in a database. Security requirements originate from different sources, such as

text:  explicit functional and non-functional requests from user(s),  requirements and obligations originating from the underlying legal framework  requirements that are considered as best practices, company policies, in widely accepted guidelines, from threat assessment but also, from the experience of a developer, e.g. “I always make sure my error messages don’t contain any personal information”.

text: In some sectors, a specific security standard and/or relevant certifications have become common practise, such as the ones produced by the Payment Card Industry Security Standards Council (PCI SSC) baselining security requirements in the payment card processing domain. Compliance to PCI SSC standards is not a legal requirement (with some small exceptions), yet it is typically contractually transmitted down the chain to vendors and service providers. As such, in this area, PCI SSC standards have become the norm on duty of care and play a role in liability of service providers since not meeting the requirements could be regarded as negligence. Such implications make organisations treat PCI SSC standards almost as a de facto set of requirements. Another example is the ISA/IEC 62443 series of standards, which is receiving growing interest in the domain of industrial control systems.

text: 2.2 SECURE SOFTWARE ENGINEERING

text: The ISO/IEC 27000:2018 [9] definition of information security mentions “preservation of confidentiality, integrity and availability of information” while noting that additional “properties such as authenticity, accountability, non-repudiation and reliability can also be involved”. Software security typically involves reducing the probability and impact of unauthorized/inappropriate access, use, disclosure, disruption, deletion/destruction, corruption, modification, inspection, recording or devaluation of data and functions managed by software. Technically, software security is achieved with an ongoing secure software development and deployment process, or secure software engineering where security is built in, and provided, involving people, tools and practices.

text: Secure software engineering can be seen as providing countermeasures against applicable security threats. Therefore it is important to identify the necessary mitigating measures, deploy them properly and verify their implementation as a safety net for unwillingly introduced vulnerabilities. Verification can be performed through automated or manual tests and manual review. Therefore secure software engineering is not just about finding security weaknesses but also about preventing the mistakes that cause them, through training, instruction, technology choices, security by design, etc. In that sense, secure software engineering can be seen as an important part of the practice of preventing and mitigating security risks.

text: ADVANCING SOFTWARE SECURITY IN THE EU

text: Key elements of secure software engineering are: threat modelling, risk analysis, guidelines, education, requirements, design analysis, issue management, compliance, verification, defect management

text: hardening threat modelling 1) requirements, 2) compliance, 3) education, 4) guidelines, 5) risk analysis, 6), 7) defect management, 8) verification, 9) design analysis, 10) issue management and 11) hardening. Software maintenance is typically defined as the engineering phase that happens after the software is released. On average, this phase accounts for 80% of the total development effort [10], which means that software quality aspects during maintenance should remain important attention points. Therefore, within the scope of this document, software development includes maintenance.

text: To this respect, Security Maturity Models (SMM) are a very useful tool since they guide organisations in defining their level of security in accordance with the requirements they wish to fulfil [7]. The maturity of security assesses the understanding of the current level of security, its needs, benefits, and the cost of its support. This assessment takes into account specific threats to the regulatory and compliance requirements of an organisation's sector, the unique risks present in an environment, and the organisation's threat profile.

text: 2.3 SECURE DEVELOPMENT LIFECYCLE

text: The Secure Development Lifecycle (SDL) is the introduction of security activities into processes involved in application management, application provisioning and operation, infrastructure management and application audit, collectively representing the Software Development Lifecycle (SDLC) [11]. SDLC is the process of the development of a software that includes planning, analysis, design, testing, and implementation. Software developers may use different models with regards to development [12], such as Waterfall, Iterative or Agile. However, all models usually feature the following activities i) requirements elicitation, ii) software design, iii) development/implementation, iv) testing and acceptance, v) deployment and integration and vi) maintenance and disposal. Within the SDL, the principle of security by design comes into play as security should be built into the product/software from its inception and be constantly reviewed until it is ready for release to customers.

text: 2.4 EXISTING STANDARDS AND GOOD PRACTICES

text: There are several standards and good practices focusing on software security. Most notably:

text:  Common Criteria is a consolidated and widely recognized framework for product (often times meaning software) security evaluations. The evaluation process, in essence, pertains to the assessment against pre-defined security functional and assurance requirements. Depending on the assurance level to be reached (ranging from the minimum level 1 to the maximum level 7) a different set of security assurance requirements is applicable.  The OWASP ASVS (Application Security Verification Standard) is a community developed verification framework focusing on technical security controls verifiable in the software product and in the development process. It distinguishes maturity levels that reflect a system’s security profile; this suggests which security requirements must or may apply. Apart from this, ASVS foresees use cases that include developer training, developer/architect reference, supplier governance, guidance on: planning, documentation, threat modeling and coding guidelines, pipeline automation, test automation and dependency management.

text: ADVANCING SOFTWARE SECURITY IN THE EU

text:  BSIMM (latest version 9, 2018) is a commercial initiative, based on a bottom-up approach that starts with listing activities of large successful software companies. This benchmark shows which controls can be considered in maturity terms, in which higher maturity levels are typically a step-up from lower maturity levels. Its four categories of controls map largely to the categories in OWASP SAMM.  BSI PAS 754 “Software Trustworthiness. Governance and management. Specification” is a standard developed by the British Standards Institution encompassing both security and reliability concerns (safety, reliability, availability, resilience and security). It describes a wide applicable approach for organizations aiming to adopt system trustworthiness practices and is based on the concepts of governance, risk management, control applications and compliance.  ISA 99 / IEC 62443 provides a set of standards aimed at industrial control systems and provides a flexible framework to address and mitigate current and future security vulnerabilities in industrial automation and control systems (IACSs). It also includes a standard for secure software development (62443-4-1) which specifies process requirements for the secure development of products used in industrial automation and control systems. Requirements include verification of formal processes being used and visible in the product. Thereby this standard verifies process requirements partly by checking characteristics of the product, which underlines the solid relationship between the two.  ISO/IEC 27034 is a multipart, guidance international standard focusing on application security. Each of its numerous parts goes down in deep details on how software security should be achieved. The most relevant ones are: o 27034-3: application security management process; o 27034-4: validation and verification (still to be published); o 27034-5: protocols and application security controls data structure; o 27034-7: assurance prediction framework.

text: Its development has been strongly backed by large software enterprises in order to fill the gap in international standardization on this topic.  ISO/IEC 62304 is a certifiable standard in the field of medical (device) software focusing on life cycle requirements for the development of medical software and software within medical devices. It distinguishes three classes of safety requirements which are similar to a level of assurance/criticality of system security. It is divided into separate concerns of software maintenance, (incident) response, configuration management and risk governance. The different categories follow common steps in a software development process (from design/planning to implementation/verification, release/anomaly response).  PCI SSC has been relying for the last decade on the PA-DSS standard for payment applications certification. More than 5000 software products have been certified during this period of time relying on a structured mechanism including accreditation, demanding third-party audits and incremental certification for new versions. Currently PCI SSC is in a transition for a new framework for software security which will be in full effect in 2022 and will leverage two different standards: one addressing the SDL and one the security of the product itself.  OWASP Software Assurance Maturity Model (SAMM) is a community developed framework to help organizations formulate and implement a strategy for software security. The framework covers four core business functions of software development, namely i) governance, ii) construction, iii) verification and iv) operations, with security

text: ADVANCING SOFTWARE SECURITY IN THE EU

text: practices tied to each one of them. The building blocks of the model are the three maturity levels defined for each of the twelve security practices. These define a wide variety of activities in which an organization could engage to reduce security risks and increase software assurance.  The Microsoft SDL is considered a classical model in secure development frameworks. It distinguishes Training (a prerequisite), Requirements, Design, Implementation, Verification, Release, and Response (to external or unexpected events).  The Dutch SSA (Secure Software Alliance) has defined a framework for secure software development intending to conform to all phases of the SDLC. It focuses on threat modeling as a prerequisite for secure software development.  Safecode.org is an initiative to identify and promote best practices for secure software development. Their “Fundamental Practices for Secure Software Development” follows steps in the development process from governance to design, coding, testing, and vulnerability response. Relevant to the present work, Safecode has published a short document with its vision on cybersecurity certification in which they note the following: o Security certification should encompass the complete product (including configurations in its deployment operation), not just security features. o Certification should be possible on an unfinished product. Development process has more predictive value of a product’s security than testing it afterwards. o Certification levels should be linked to product classes, to lessen the assumption that the highest level is always preferred.

text: Further to the certifications that might accompany some of the aforementioned standards, some lightweight certification schemes have also been developed such as the CSPN (France), the BSZ (Germany), LINCE (Spain) and BSPA (Netherlands).

text: ADVANCING SOFTWARE SECURITY IN THE EU

text: 3. WHAT IS LACKING IN SOFTWARE SECURITY

text: 3.1 CLEAR GUIDANCE

text: While, as presented earlier, many software security related standards exist, their requirements largely overlap. This overlap demonstrates that software security is mainly a generic problem and both Standards Developing Organizations (SDOs) and European Standards Organizations (ESOs) or good practice producers are often working without proper coordination and effective liaisons. Further to that and as already acknowledged by relevant ENISA work in the area [13], traditional standardisation processes can be time-intensive, potentially causing delays in the application of necessary standards and interoperability. Considering the number of existing approaches, there are no widely used standardized ways for assessing horizontally the security of software products, apart from some methods to perform a penetration test or a code review. The situation is even worse if we consider software security certification as only a few schemes from the ones listed in Section 2.4 support it, with limited exposure and acceptance outside the geographical area of the issuing organizations.

text: 3.2 QUALITY ASSURANCE

text: In today’s software development landscape, organisations and individuals have difficulty in identifying the level of security of software products. Such information is important to gain trust but cannot be easily acknowledged from the outside. Some security functions (e.g. authentication methods) are visible for everybody, but that is only a really small part of the total set of security functionalities that a software may need to provide. For example, from the 286 controls of the OWASP Application Security Verification Standard Project [14], only 22 can be evaluated by a non-security expert. There seems to be information asymmetry between producers and consumers of software. Consumers cannot identify immediately the level of security of the software products they buy. Even worse, in commercial procurement procedures, (e.g. a mobile operator procures the development of a mobile application) organisations have difficulty firstly in setting the right requirements and secondly in evaluating the level of security of the software product they acquired.

text: The same information asymmetry causes assurance unclarity: it is often not clear to a nonexpert what a certificate or an assessment report actually means with regards to the level of security or even the security risks mitigated for a certain software product. Secondly, because of the intricate supply chain we have today, assurance suffers from the complexity of the whole system. Software systems can consist of multiple subsystems, contain components from third parties, or even open source (that are in some cases publicly available). Lastly, the operational environment is also part of the security scope. That environment can be controlled by an external party if the software is (partly) provided as a service. These scope issues are not always apparent to non-experts, resulting in a mistaken perception on the scope of evaluated software components. For example, suppose a mobile app has been evaluated as compliant to a mobile application security standard, while at the same time the following parts were out of scope:

text:  The part of the application that runs on a webserver (which is often the case for apps).  A third-party component because the supplier did not provide access to the source code.  The operational environment in which this webserver is deployed (network, firewall, patching of the server etc.).

text: ADVANCING SOFTWARE SECURITY IN THE EU 3.3 SUSTAINED TRUST ISSUES

text: Maintaining confidence in the security level of a software product over time is not directly addressed by existing assurance/certifications initiatives. Such challenges in sustained trust are there for three reasons:

text:  Software changes after an evaluation/certification may introduce new vulnerabilities (point in time issue).  Even without software updates/changes, newly discovered vulnerabilities can be found at the software, or in its (open source/third-party) components/libraries.  The software’s operational environment in not covered directly by the same evaluation process and is also subject to changes/updates or newly discovered vulnerabilities.

text: Under some of the existing certification schemes, issued certificates become invalid after (significant) changes are performed to the evaluated product/software. A side effect is that, sometimes, vendors refrain from updating their software in order not to have to undergo an evaluation/certification process again, due to aforementioned updates. In such cases, the security certification can ironically be considered as making systems likely less secure.

text: 3.4 ASSURANCE CLARITY BETWEEN PROCESS AND PRODUCT

text: Software development process assessment can provide a good indication on whether a software developing organization (or the corresponding supply chain) is able to develop and to maintain secure software, even though it has severe assurance limitations. This type of process assessment is often considered as a way to provide assurance that software remains secure, change upon change. This is an attractive idea, as it would reduce the effort in re-assessment or re-certification substantially. However, no matter how good a process is, or seems to be, mistakes can always be made. In other words: software process assessment is helpful but cannot provide the same level of assurance that products assessments can – unless the process features trustworthy product assessment of course.

text: Software product assessment, on the other side, provides insights only into the current level of security, with little assurance about the future. It is only a snapshot. This can be a reason for assessing the software development process as well. Let’s look conceptually at what a development process entails. It is about having procedures, policies, culture and tools in place so that security is being built in from the start. It is not the total set of programming steps that developers take to write source code with security built in. If that would need to be assessed, then the best subject to assess would be the source code itself (the product). In other words: secure software process assessment is about investigating the developers’ organisation.

text: Unfortunately, assessment of security in a software development process is hard to perform in a reliable way, as:

text:  A large part of the process effectiveness is determined and driven to a certain extent by the skills, and the knowledge of the people, and the actual priority that security gets in software development: aspects that are hard to measure and are not directly included in the popular secure SDLC standards. The resources dedicated to software development and quality are an additional driving factor.  It is relatively easy to make a complex process like software development look good. There is a big difference between having a procedure and actually following it in practice and perform it well. For example: one can argue that peer review is being performed on source code, but how skilled are the reviewers, how much time do they have and is there a culture where honest feedback can be provided? These are aspects that are hard to measure and easy to claim. There is also a big difference between having a tool, using it, and using it well. Furthermore, there are vast differences between security tools when it comes to coverage and quality of results.

text: ADVANCING SOFTWARE SECURITY IN THE EU

text: 4. DRIVING SOFTWARE SECURITY IN THE EU

text: 4.1 DEVELOP A COMMON REPOSITORY FOR SHARED SECURITY MEASURES

text: Aligning commonalities of requirements across different schemes prevents proliferation and fragmentation, while also making drafting and maintaining a scheme more efficient in terms of mitigating the risks. As such, the commonalities evolve in quality and efficiency of mitigating the risks over time. For such common ground in harmonization to be maintained and to be used, a mapping governance, documenting the overlaps, could greatly support scheme development and maintenance. Such a mapping could result in the definition of a common repository for shared security aspects (access control, authorization, encryption etc), threat models and approaches against known adversary tactics, over different schemes as part of obligations introduced by Cybersecurity Act

text: Article 55

text: on Supplementary cybersecurity information. Such a repository could be extended to become a knowledge sharing platform which would greatly help to reach the mentioned goals. Next to the technical requirements, this repository could also keep metadata such as: mapping to existing standards, revision history, related threats, and implementation guides for different technologies, or references to those.

text: Manufacturer(s) or provider(s) of certified ICT products, ICT services or ICT processes, should consider the deployment and maintenance of repositories not only for publicly disclosed vulnerabilities but also for shared security aspects of certified products, services and processes towards aligning on requirement commonalities and ways to mitigate common security risks.

text: 4.2 IMPROVE TECHNICAL STANDARDS LANDSCAPE

text: The Cybersecurity Act reinforces the need for referencing international, European or national standards within the EU cybersecurity certifications schemes. Picking the best source requires taking into account many factors: comprehensiveness, clarity and fitness to the scope of the cybersecurity certification scheme. Despite the ongoing standardization efforts, gaps in standardization towards the Cybersecurity Act goals still exist [13] [15]. Such gaps present risks that additional standardization efforts may mitigate, but overlaps in standardization efforts present risks that may be mitigated only with a coordinated approach. A good starting point could be the Union Rolling Work Programme (CSA Art. 47), the first edition of which is expected to be published by mid-2020.

text: Following the publication of URWP, SDOs and ESOs should coordinate on the priority areas they can support, put forward standardization activities to benefit the future developed schemes and communicate periodically such planning to the EC and relevant CSA stakeholders.

text: 4.3 PROVIDE ASSURANCE IN THE ENGINEERING PROCESS

text: When analysing available SDLC (Software Development Lifecycle) standards it becomes clear that there is consensus on what activities should be included. This consensus centers on the key activities that can be distinguished in the development lifecycle like: training, requirements, coding guidelines, design, design review, threat modelling, secure verification (automated testing, static analysis tools, test tools, manual code review and penetration testing), dependency management, incident management, vulnerability management and environment hardening.

text: ADVANCING SOFTWARE SECURITY IN THE EU

text: Many of the existing process standards include maturity levels, supporting organizations to assess their own maturity and to get guidance on improving that maturity, and also appreciating that there is no line to draw when it comes to engineering. As described earlier, process assurance provides limited certainty on the security of the software that comes out of the process. One way to address this is to include provisions, depending also on the level of assurance and the scope of the scheme, on the verification of the software and the process used. Such an assurance on the process would not only improve the end product but could also benefit the whole supply chain of the aforementioned software product or the supply chain that the software product is involved. Lastly, assurance on the process could also be considered as a weighting factor, similar to certification, with regards to mitigating (possible) liabilities or as a criterion on selection of supplier(s).

text: EU cybersecurity certification schemes for products, services and processes should include, to the extent possible, not only requirements for the end product/service/process but also assurance for the engineering process, by setting process guidelines for software development, maintenance and operation.

text: 4.4 INCREASED SCHEMES APPLICABILITY AND CLEARLY COMMUNICATED ASSUMPTIONS

text: In order to constitute the evaluation and (eventually) certification processes appealing to software developers, it is important to provide assurance levels that are appropriate for the different levels of risk given the context and the expected domain of use. Since the EU cybersecurity certification framework envisions three assurance levels, one of which allows for self-conformity assessment, it seems beneficial to retain the notion of lightweight conformity assessment.. By providing lightweight approaches, the cost of re-certification (through third parties) will no longer prevent companies to release updated software. Further to that, mechanisms can be put into place to base the conformity assessment on a change analysis of the software, similar to existing national approaches. Additionally, certification of the secure software development process is also a way to provide more sustained trust. Such a certificate delivers meta-assurance that sets requirements on the level of assurance taking place in the development process.

text: Given the magnitude of the context and the domain(s) in which software products can be used, the aforementioned certification schemes should be applicable horizontally, to the widest extent possible. In cases where this is not feasible, they could be considered as intuitive guidelines and best practices by providing a clear indication of the risks they mitigate and the assumptions made within the scope of the scheme.

text: During the development of EU cybersecurity certification schemes, lightweight conformity assessment for assurance level basic should be considered as a partial response to the existing fragmented landscape of software development and maintenance.

text: Software and product manufacturers should put forward their experience and expertise and promote the uptake of EU cybersecurity certification schemes, including the selfassessment components.

text: During the identification of strategic priority areas, under the URWP, and the development of the EU cybersecurity certification schemes, EC, ENISA, SCCG and ECCG should ensure that assumptions on the scope, application area, mitigated threats and achieved security characteristics achieved are clearly communicated to the end users of the framework.

text: ADVANCING SOFTWARE SECURITY IN THE EU

text: 5. REFERENCES

text: 1. Verizon: 2019 Data Breach Investigations Report. (2019) 2. THALES: 2019 Thales Data Threat Report – Global Edition. (2019) 3. ETSI: ETSI TS 103 645 V1.1.1 (2019-02)- Cyber Security for Consumer Internet of Things. (2019) 4. Security Boulevard Available at: https://securityboulevard.com/2019/10/60-of-breachesin-2019-involved-unpatched-vulnerabilities/ 5. European Union:

text: Regulation (EU) 2019/881

text: of the European Parliament and of the Council of 17 April 2019 on ENISA and on information and communications technology cybersecurity certification and repealing

text: Regulation (EU) No 526/2013

text: (Cybersecurity Act). (2019)

text: 6. Australian Cyber Security Centre: Guidelines for Software Development. (2019)

text: 7. ENISA: Good Practices for Security of IoT - Secure Software Development Lifecycle. (2019)

text: 8. SafeCode: Fundamental Practices for Secure Software Development. (2018)

text: 9. ISO/IEC: ISO/IEC 27000:2018 Information technology — Security techniques — Information security management systems — Overview and vocabulary. (2018)

text: 10. Pigoski, T.: Software Maintenance. (2001)

text: 11. ISO/IEC: ISO/IEC 27034-1:2011 Information technology — Security techniques — Application security — Part 1: Overview and concepts. (2011)

text: 12. Shiklo, B.: 8 Software Development Models: Sliced, Diced and Organized in Charts. (Accessed November 2019) Available at: https://www.scnsoft.com/blog/softwaredevelopment-models

text: 13. ENISA: ICT security certification opportunities in the healthcare sector. (2019)

text: 14. OWASP: Application Security Verification Standard 4.0. (2019)

text: 15. ENISA: Guidance and gaps analysis for European standardisation. (2019)

text: 16. PCI Security Standards Council: PCI Mobile Payment Acceptance Security Guidelines for Merchants as End-Users. (2014)

text: -N -EN -206 -20 -03 TP

text: ABOUT ENIS A

text: The mission of the European Union Agency for Cybersecurity (ENISA) is to achieve a high common level of cybersecurity across the Union, by actively supporting Member States, Union institutions, bodies, offices and agencies in improving cybersecurity. We contribute to policy development and implementation, support capacity building and preparedness, facilitate operational cooperation at Union level, enhance the trustworthiness of ICT products, services and processes by rolling out cybersecurity certification schemes, enable knowledge sharing, research, innovation and awareness building, whilst developing crossborder communities. Our goal is to strengthen trust in the connected economy, boost resilience of the Union’s infrastructure and services and keep our society cyber secure. More information about ENISA and its work can be found at www.enisa.europa.eu.

text: ISBN: 978-92-9204-343-8 DOI: 10.2824/81950


text: September 2019

text: September 2019

text: September 2019

text: September 2019

text: September 2019

text: ENISA Work Program 2019 Outputs 4.1.3 and 4.1.4

text: September 2019

text: September 2019

text: https://www.nato.int/cps/en/natohq/official_texts_133169.htm 3 https://ccdcoe.org/research/tallinn-manual/

text: September 2019

text: September 2019

text: September 2019

text: September 2019

text: rd The aim of the 3 Panel was to discuss the technical challenges for the OpenCSAM project and identify future work for the next cycles of development. The panel was comprised of participants with diverse backgrounds, working in EU institutions, the private sector as well as in academia. More specifically, there were representatives from SDL (Mr George Bara), the University of Oxford (Dr Jassim Happa), ESDC (Dr. Gregor Schaffrath), ENISA’s contractor for OpenCSAM - Eau de Web ( Mr V.Posea & Mr T.Ichim), and the BCU School of computing and digital technology (Dr. Syed Naqvi). The panel was moderated by ENISA (Mr Cosmin Ciobanu). The panel discussion kicked off with the background and goals for developing the OpenCSAM tool. Being a tool developed for the blueprint, the decision makers require tailored situational awareness reports that provide context about ongoing developments during a cybersecurity incident.

text: September 2019

text: September 2019

text: OT (Operational Technology) refers to the hardware, firmware and software that either monitor or control processes and activities in the industrial sector.

text: September 2019

text: September 2019

identifier: AI an opportunity for the EU cyber crisis blueprint - Report

amnesord: EU incident response and cyber crisis management

publisher: Europeiska unionens cybersäkerhetsbyrå

sprak: en

title: AI an opportunity for the EU cyber crisis blueprint - Report

serie: rapporter

text: ARTIFICIAL INTELLIGENCE

type: rubrik

text: Conference Report SEPTEMBER 2019 ARTIFICIAL INTELLIGENCE

text: ABOUT ENISA

text: The mission of the European Union Agency for Cybersecurity (ENISA) is to achieve a high common level of cybersecurity across the Union, by actively supporting Member States, Union institutions, bodies, offices and agencies in improving cybersecurity. We contribute to policy development and implementation, support capacity building and preparedness, facilitate operational cooperation at Union level, enhance the trustworthiness of ICT products, services and processes by rolling out cybersecurity certification schemes, enable knowledge sharing, research, innovation and awareness building, whilst developing cross-border communities. Our goal is to strengthen trust in the connected economy, boost resilience of the Union’s infrastructure and services and keep our society cyber secure. More information about ENISA and its work can be found at www.enisa.europa.eu.

type: stycke

text: CONTACT

text: For contacting the authors please use COD3@enisa.europa.eu. For media enquiries about this paper, please use press@enisa.europa.eu.

text: AUTHORS

text: Georgios Chatzichristos, Fabio di Franco, Ioannis Agrafiotis, Cosmin Ciobanu

text: ACKNOWLEDGEMENTS

text: ENISA would like to thank all speakers, panellists and contributors to this conference for their valuable inputs that this report summarizes.

text: LEGAL NOTICE

text: Notice must be taken that this publication represents the views and interpretations of ENISA, unless stated otherwise. This publication should not be construed to be a legal action of ENISA or the ENISA bodies unless adopted pursuant to the

text: Regulation (EU) No 2019/881

text: . This publication does not necessarily represent state-of the-art and ENISA may update it from time to time.

text: Third-party sources are quoted as appropriate. ENISA is not responsible for the content of the external sources including external websites referenced in this publication.

text: This publication is intended for information purposes only. It must be accessible free of charge. Neither ENISA nor any person acting on its behalf is responsible for the use that might be made of the information contained in this publication.

text: COPYRIGHT NOTICE

text: © European Union Agency for Cybersecurity (ENISA), 2020 Reproduction is authorised provided the source is acknowledged.

text: For any use or reproduction of photos or other material that is not under the ENISA copyright, permission must be sought directly from the copyright holders.

text: ARTIFICIAL INTELLIGENCE

text: TABLE OF CONTENTS

text: 1. CONFERENCE ORGANIZATION 5

text: 1.1 SCOPE OF THE CONFERENCE 5

text: 1.2 PARTICIPATION 5

text: 1.3 THEMATIC AREAS 5

text: 2. AREA 1 – THE FUTURE OF THE BLUEPRINT 6

text: 2.1 GAPS AND TRENDS FOR THE BLUEPRINT FUTURE 6

text: 2.2 AN EFFECTIVE AND HOLISTIC APPROACH 6

text: 2.3 BUILDING TRUST 6

text: 2.4 CIV-MIL COLLABORATION 7

text: 2.5 CONCLUDING REMARKS 7

text: 3. AREA 2 – HOW CAN AI HELP THE BLUEPRINT 8

text: 3.1 THE NEED FOR AI IN ORGANIZATIONS AND ITS PERILS 8

text: 3.2 AI IN THE CONTEXT OF CYBERSECURITY AND THE BLUEPRINT 8

text: 3.3 ACCOUNTABILITY OF AI AND ASSURANCE OF QUALITY 9

text: 3.4 FAKE NEWS AND THE BLUEPRINT 9

text: 3.5 CONCLUDING REMARKS 10

text: 4. AREA 3 – IMPROVING OPENCSAM 11

text: 4.1 EXTRACTION OF INFORMATION 11

text: 4.2 TEXT, TOPIC, LANGUAGE AND SENTIMENT CLASSIFICATION 11

text: 4.3 COMPANY-ENTITY EXTRACTION 11

text: 4.4 EVENT TIMELINE EXTRACTION 12

text: 4.5 TEXT SUMMARIZATION 12

text: 4.6 CONCLUDING REMARKS 12

text: ARTIFICIAL INTELLIGENCE

text: 5. AREA 4- CYBER AUTONOMOUS RESPONSE, THREAT DETECTION AND

text: SECURITY AUTOMATION 13

text: 5.1 CONCLUDING REMARKS 14

text: 6. CONCLUSIONS – FUTURE WORK 15 ARTIFICIAL INTELLIGENCE

text: EXECUTIVE SUMMARY

text: On the 3rd and 4th of June 2019, ENISA organized in Athens, Greece, a conference on Artificial Intelligence in the EU Cyber crisis blueprint context under the title ‘ Artificial Intelligence-An opportunity for the EU cyber-crisis management’.

text: Artificial Intelligence (AI) is not something new. What is new is the pace that information is disseminated in today’s society and the amount of data produced that renders the use of advanced technologies like AI essential for organisations. AI technology, if properly applied, has the potential to create a competitive advantage over traditional methods of data classification. An increasing number of public and private organisations are using now AI, both at operational and technical levels. AI systems are utilised to provide intelligence on the services which organisations offer, to distinguish behavioural aspects of systems and networks, and to help humans understand complex relationships between different entities of their working environment. In cybersecurity, AI systems have the ability to highlight anomalies on network traffic identifying invisible “unknown unknown things” in the systems, but also as an efficient classifier of vast amounts of data like in the case of threat intelligence.

text: In the EU cyber crisis cooperation context, most commonly referred as ‘the Blueprint’, AI uses are just beginning to emerge, mainly at the Open Source intelligence domain. The need for situational awareness, one of the Blueprint’s main pillars, has driven ENISA to initiate the development of a project under the name ‘Open Cyber Situational Awareness Machine – OpenCSAM) that attempts to address the need for accurate aggregation of relevant information and reporting. The project is using supervised learning and natural language processing to facilitate incident responders at all levels of administration in the drafting of situational awareness reports for the Blueprint.

text: This report summarizes the main takeaways from the two days of the conference and the discussions of the four (4) thematic panels from the Political to the Technical level. The presentations that were delivered in the context of the four (4) thematic areas can be found here: https://www.enisa.europa.eu/events/artificial-intelligence-an-opportunity-for-the-eu-cybercrisis-management/workshop-presentations/event-presentations

text: ARTIFICIAL INTELLIGENCE

text: 1. CONFERENCE ORGANIZATION

text: 1.1 SCOPE OF THE CONFERENCE

text: The conference aimed at supporting ENISA’s activities for cyber-crisis management . Another important driver has been the investigation of ways that Artificial Intelligence can be used in the Blueprint context to improve cyber security and to assist decision making.

text: The event was organized in two days and in four different sessions with corresponding discussion panels. In the first day, the centre of gravity was on the Political and Operational collaboration and the possible uses of Artificial Intelligence in these two levels of governance. In the following day the focus shifted to technical level discussions, AI solutions and applications already in use, challenges and future trends. There has been also a dedicated session to ENISA’s Open Cyber Situational Awareness Machine (OpenCSAM), which is a research and development project that relies heavily on artificial intelligence and aims at supporting the reporting functions of the Blueprint.

text: 1.2 PARTICIPATION

text: ENISA attempted with this event to bring together cyber and AI stakeholders across different sectors and domains. In the speakers’ list, there were representatives from EU Institutions, the European Commission, The European External Action Service, EDA, Europol EC3 and Cert EU while among the audience there were representatives from the European Council, Eurocontrol and the ESDC. Academia had also a very strong presence with speakers from Academic Institutions well known for their research in cyber security and artificial intelligence. Finally yet importantly, public authorities and private companies delivered their inputs and presented solutions for the autonomous cyber response area.

text: Overall, over 120 participants from 12 Member States were present during the event. It is important to mention that almost half of the participants were either lawyers or related to legal and legislative activities in the public and in the private sector.

text: 1.3 THEMATIC AREAS

text: The conference covered four thematic sections. Sections 1 and 2 during the first day focused on the Blueprint, Section 3 on the second day focused on ENISA’s OpenCSAM project and the last section focused on autonomous cyber response at the technical level.

text: ARTIFICIAL INTELLIGENCE

text: 2. AREA 1 – THE FUTURE OF THE BLUEPRINT

text: The aim of the first panel was to verify the current status of cooperation between the key EU cyber stakeholders and discuss on the enhancement and possible expansion of this collaboration. In the panel there were representatives from the European Commission/DG CNECT (Dr. Ioannis Askoxylakis), the European External Action Service (Ms Agniezka Wierzbicka), The European Defence Agency (Mr Mario Beccia), Cert EU (Mr Georgios Psykakos), Europol EC3 (Ms Aglika Klayn) and the discussion was moderated by ENISA (Mr Georgios Chatzichristos).

text: 2.1 GAPS AND TRENDS FOR THE BLUEPRINT FUTURE

text: The Commission mentioned that there is a big push for further Operationalization of the Blueprint. EU cyber exercises in the last years are revealing the need. From the Member States side, the NIS Cooperation Group established under the NIS Directive is active working with the Commission on the implementation of the Blueprint at the Operational level of the Member States’ mechanisms. The four (4) MoU stakeholders (ENISA, EUROPOL EC3, EDA and CertEU) are the ‘right’ players as they represent all critical functions of cyber security in the EU ecosystem. However as the impact of cyber-attacks is not only on those players, all other EU Institutions and bodies should not only have a certain level of preparedness but also a role in the Blueprint as they could be called to exchange information, produce impact assessments, and respond to incidents on their domain of activities. In addition, due to the borderless nature of cyber space, the need for cooperation with non EU Countries that share the same core values is of key importance for the EU.

text: 2.2 AN EFFECTIVE AND HOLISTIC APPROACH

text: Europe should avoid building new mechanisms, policies and capabilities without taking into account two important factors.

text: Utilize existing crisis management mechanisms, protocols, policies and experience: The Commission said that the Blueprint goal is to align current crisis management mechanisms and use good practices from other Sectors (eg. Banking Sector or Aviation Sector) regarding crisis management. EUROPOL EC3 added that we should borrow experience from existing work, such as the LE ERP protocol.

text: Ensure effectiveness: The measurement of effectiveness has been identified as of crucial importance for the Blueprint. EDA emphasized this fact, agreeing with the University of Oxford that this should be an area of research for the EU.

text: 2.3 BUILDING TRUST

text: Cyber collaboration needs to address also sensitivities and reluctance in collaborating from some National Authorities. EDA gave some analogies in the capability development from its own experience in the Common Security and Defence Policy (CSDP) domain. Some Member States might be willing more to collaborate (mainly smaller Member States) while others are more reluctant. The EEAS added that in their experience for overcoming such problems, capacity building is important. This will lead to an increase of the maturity level and will lead to better collaboration. EUROPOL EC3 strongly agreed on this. This expands even beyond Member States, into third countries. Trust building is also of great importance. Trust is a continuous process that requires continuous efforts to build. We do not trust Organizations and

text: ARTIFICIAL INTELLIGENCE

text: mechanisms but we do trust the humans that make them work. EDA said that the Banking sector, for example, uses many circles or communities of people who come often together, get to know each other and build trust. Before having protocols, capabilities and information exchange networks, we should invest in the human factor first. CERT EU complemented that day-to-day Operations also lead to human interaction with different entities and this help to build trust. The ESDC presented another aspect of trust by connecting it to the attribution issue. Attribution, very often relies on classified Intelligence information that cannot be disclosed to all parties of the incident response mechanisms or to the public. In that way, incident response mechanisms are vulnerable to targeted propaganda attacks. EUROPOL EC3 emphasized the need of transparency in the processes. EEAS said that attribution is primarily a responsibility of the Member States. Attribution based on National Intelligence sources should be very carefully handled, especially regarding the communication of public messages, balancing between the protection of National Security interests and the need for transparency, a crucial factor for trust. ENISA said that for trust building in incident response we need to take into account three elements. ‘Who am I, Who are you and what will you do with the data that I am giving you’. A balance in the information exchange is also critical in the sense that ‘I will give you something that you need and you will give me something that I need.’

text: 2.4 CIV-MIL COLLABORATION

text: Today’s APTs and hybrid threats prove that that the Cyber domain is increasingly used for complicated cyber operations that target both civilian and military targets. In 2016, NATO declared the cyber domain as a domain of military operations . This brought a revision in operational concepts and planning and introduced new legal concerns . The cyber domain differs from the traditional Operational domains (Air, Sea, Land and Space) in the sense that is being depended and operated by the industry (i.e. telecom companies, IT manufacturers, IT service providers, media etc.). Because cyber domain’s interdependencies are so complex and borders between military and civilian uses are blurry or non-existent, the effective civilian – military collaboration is so important. The Blueprint should act as a bridge that will bring closer the Civilian and the CSDP worlds for the better mitigation of common cyber threats. Both EEAS and EDA agreed the civil – military cooperation has always been a very complicated case. It is therefore next to impossible for the military not to take into account the civilian world when it comes to cyber security. Joint exercises is a good first step towards this collaboration.

text: 2.5 CONCLUDING REMARKS

text:  Foresee roles and procedures for all EU Institutions in the Blueprint context.  Need for continuous operations of certain blueprint functions, like Situational Awareness and Horizon scanning  Build trust by developing information exchange networks, personnel exchange and frequent exercises between Blueprint stakeholders at the Operational level.  Identify synergies between the Civil and Military domains in information exchange and capabilities.

text: ARTIFICIAL INTELLIGENCE

text: 3. AREA 2 – HOW CAN AI HELP THE BLUEPRINT

text: The aim of the second panel was to discuss developments in the field of Artificial Intelligence and explore how advances in this area can be utilised in the implementation of the Blueprint. The panel was comprised of participants with diverse backgrounds, working in public and private institutions as well as in academia. More specifically, there were representatives from Ministry of Defense (MoD) of Bulgaria (Dr George Sharkov), the European Defense Agency (Mr Mario Beccia), Microsoft (Mr Maciej Surowiec), University of Oxford (Dr Jassim Happa) and Europol (Mr Dimitrios Zacharis). The panel was moderated by ENISA (Dr Ioannis Agrafiotis).

text: 3.1 THE NEED FOR AI IN ORGANIZATIONS AND ITS PERILS

text: The pace with which information is disseminated in today’s society and the amount of data produced renders the use of advanced technologies essential for organisations. AI systems have the potential to create competitive advantage and have been adopted by organisations both at operational and technical levels. Such systems are utilised to provide intelligence on the services, which organisations offer, to distinguish behavioural aspects of systems and networks, and to help humans understand complex relationships between different entities of their working environment. In cybersecurity, AI systems have the ability to highlight anomalies on network traffic, indicating possible cyber-attacks and facilitate analysis to identify invisible “unknown unknown things” in the systems.

text: The adoption of AI systems, however, has its perils. Participants concurred that recent advances in networking and storage infrastructure have enabled the implementation of AI algorithms that were invented in the 1970s. We have not yet realised the full potential of AI systems and the effects that these could have on our society. Concerns were raised on how AI systems are trained, since there is evidence to suggest that data poisoning of training datasets can manipulate the performance of AI systems. Furthermore, participants explained that widely applied algorithms, such as neural networks, act as black boxes, reducing transparency in how their outcome is produced. Such opaque behaviour can further introduce ethical dilemmas and biased decisions. Finally, participants explained that AI systems can be utilised by adversaries to design malware.

text: 3.2 AI IN THE CONTEXT OF CYBERSECURITY AND THE BLUEPRINT

text: Participants suggested that organisations mainly use AI techniques in cybersecurity to detect anomalies in their network environments. A possible explanation given by the participants for such wide adoption of anomaly detection systems is that “it is more costly to deploy systems that protect against attacks rather than implement solutions which detect or mitigate successful attacks”. There is a plethora of products offering behavioural analysis, however, such solutions operate in silos and it is becoming increasingly difficult to correlate data from such sources for better situational awareness. It was further mentioned, that solutions claim to utilise AI to detect 0-day attacks, however, participants questioned the validity of such tools especially when you need a 0-day attack deployed in a network to test the accuracy of such systems.

text: Power asymmetries between attackers and defenders was another topic of discussion where AI can potentially change the balance. Attackers have no ethical constraints and with risk appetite to try novel technologies. On the other hand, people who defend networks for organisations abide to specific regulations and new technologies have to be fully tested, justified and economically efficient before being deployed. Participants deemed that with AI systems,

text: ARTIFICIAL INTELLIGENCE

text: defenders can process vast amount of data, unveil correlations and make informed decisions faster, all of which can help to bridge the gap in power asymmetries and potentially reduce significantly the amount of time needed to detect Advanced Persistent Threats (APTs) (according to panellists, organisations in average require 150 days to identify APTs in their networks).

text: In order to deploy AI systems successfully, participants explained that organisations should expose these in a controlled environment first and gradually release them on the rest of the network. It is of paramount importance to have analysts training these systems to ensure that data from potentially compromised networks will be properly handled and will not compromise the output of AI systems. Participants further mentioned that the notion of redundancy is important when such systems are adopted and organisations should consider and combine inputs from different AI algorithms for better-informed decisions. Finally, benchmarking of such tools is another step that the security community should work towards, since most systems are treated as black boxes due to Intellectual Property (IP) issues (manufacturers of AI systems are not transparent in the algorithms as well as the features these systems use). Therefore, benchmarking can help organisations to identify which AI systems in which context are more accurate. Participants suggested that these are all valuable lessons which should be reflected in the AI system for Blueprint.

text: 3.3 ACCOUNTABILITY OF AI AND ASSURANCE OF QUALITY

text: To address problematic situations from the use of AI, participants discussed the issue of accountability and quality assurance for AI systems. AI systems in essence utilise optimisation techniques to classify events and detect anomalies. Therefore, such systems cannot guarantee the quality of the outcome, especially those systems that depend on neural network techniques, which are not transparent in how decisions are made.

text: Participants suggested that we need to systematically test AI systems and learn how these perform in different contexts and how reliable their results can be. For black box systems, we need to obtain visibility in which neurons are ‘excited’ on given information and start building a picture on which features influence decisions. Furthermore, emphasis should be given on standardising the datasets based on which systems are trained. Ethical guidelines and potential biases should be taken into account when constructing such datasets. Finally, participants concurred that as with any other technology, organisations will adapt to the use of AI systems in cybersecurity and trust in such systems will be fostered through experience.

text: 3.4 FAKE NEWS AND THE BLUEPRINT

text: The system that will support the implementation of the Blueprint will consume information from different sources with varying degrees of trustworthiness. Therefore, AI techniques could be utilised to help users fuse all this information appropriately. Particular emphasis was given on how misinformation and disinformation attacks, which can compromise the trustworthiness of the intelligence created by combining information from different sources, can be detected and mitigated.

text: Participants suggested that AI algorithms can cluster automatically trustworthy sources by identifying how information is disseminated, from which sources, which rumours are corroborated by which sources and what is the network topology of rumours. AI systems can then provide warnings of potentially untrusted sources to users before fusing information. Finally, participants suggested that in order to tackle the problem of fake news holistically, we need to compliment AI systems with a cultural change in users’ mindset. Therefore, they suggested that it is of paramount importance to educate users to think critically when reading and reproduce information.

text: ARTIFICIAL INTELLIGENCE 3.5 CONCLUDING REMARKS

text:  Gradually expand the implementation of AI systems, from a controlled environment to the rest of the network.  Carefully select the training dataset for the algorithms.  Involve humans in the training of AI systems to ensure that quality of outputs is acceptable.  Educate users on critical thinking and on evaluating the trustworthiness of sources.  Collaborate with other stakeholders like the European Commission’s high level expert group on AI on the development of guidelines and assessment tools on trustworthy artificial intelligence

text: ARTIFICIAL INTELLIGENCE

text: 4. AREA 3 – IMPROVING OPENCSAM

text: 4.1 EXTRACTION OF INFORMATION

text: One major technical difficulty in the development is the extraction of information (text) from the sources that OpenCSAM is using. What is the state-of-the-art on this matter, how it can be improved, to what extent you can automate the scraping and text extraction?

text: To this question the experts agreed that indeed this is a complex problem, incurring high costs for scaling, technically complicated and challenging due to heterogeneous web technologies. There are some platforms that can do the job with reasonable success. Most of the websites are dynamic nowadays and this might create difficulties. The best way to scrape text from websites is by using computer vision/image & character recognition techniques. We don't need to store everything (all the text from all the websites) we are interested in the cyber security related topics which could be simplify the problem. One essential aspect noted by the panellists is "from where should we scrape data?", from which sources, which are the "best/optimal" sources. Where are the pockets of information that we are really interested in and what metrics we can define for this purpose?. The knowledge graph is a curated ontology that helps in identify the right kind of information.

text: 4.2 TEXT, TOPIC, LANGUAGE AND SENTIMENT CLASSIFICATION

text: The next topic of discussion was on text, topics, language and sentiment classification. Again here, the question posed was what is the state-of-the-art on this matter. It was mentioned that Google has released a generic model for text classification highly dependent on computing power. This can be fine-tuned for specific purposes, and it can be trained for specific use cases. The biggest challenge is having labelled data, essentially batches of text that is "known" to belong to a certain category. Another issue highlighted by the panellists relates to lack of labelled data for different languages other than English. Uncertainty and Trustworthiness were raised as a potential issue - to what degree the information collected reflects the reality. Coupled with this problem is also the issue of semantics and meaning of specific terms which could interpreted differently nowadays (different connotations) as compared to their initial meaning in the past with impact false positives rates. The meaning of a term should always be kept within the context of the article or resource.

text: 4.3 COMPANY-ENTITY EXTRACTION

text: The next challenge that was discussed was the extent to which it is possible to extract the different entities from a text. Assuming that we have a cyber-attack article for example, can we

text: ARTIFICIAL INTELLIGENCE

text: identify the “parties” involved in the story? This is a classical task in Natural Language Processing (NLP) code-named ‘entity recognition’, and there are some quite advanced tools that can be used together with specific datasets with good results. Specific entities need to be defined, using open-source tool ‘Prat’. The analyst will have the task of training the algorithms and build upon a corpus of specific entities relevant to the specific domain. Besides algorithm training but also training the users in understanding the probabilistic nature of the systems that they are using is important. An interesting point raised was if we know what is the percentage of the data and how accurately a text is translated by for ex. Google Translate? It was also noted that it is not possible to automatically discover threats that haven't in the wild for a while and did not have had sufficient time to propagate. The goal in OpenCSAM is to take feedback from the user and simplify the problem.

text: 4.4 EVENT TIMELINE EXTRACTION

text: Next topic in line was the question whether it is possible to extract the temporal characteristics of an event. When did a particular event happen? This question can be considered in the context of the previous assumption, that is whether we can extract from an article the date/time that an event took place? The conclusion here was that there is no way to tell when a certain article was published on the internet. One can only crawl at regular time intervals such as every couple of minutes. In some cases some articles were published in the "future" "See election result.". It is a very difficult task in general, we can only rely on the declared published date of the article and the date that the crawlers discovered the article.

text: 4.5 TEXT SUMMARIZATION

text: The last topic investigated was the state-of-the-art on text summarization and how it can be improved. Can we reduce the dimensions of a text significantly while preserving the core meanings and statements? In the discussion it was mentioned that in general there are two types of text summarization: Abstractive vs extractive. Abstractive approach is the human approach as one would write an executive summary of a document, as for the extractive approach relates more to extract the most common words of phrases from the text. The dealing with this is to use existing NLP algorithms and train neural network for our corpus to show similar terms or principal terms.

text: 4.6 CONCLUDING REMARKS

text:  Continue the development of OpenCSAM in annual spirals focusing on the key challenges mentioned above.  Build a community around OpenCSAM from EU Institutions, the Academia, Member States public authorities and the private sector.

text: ARTIFICIAL INTELLIGENCE

text: 5. AREA 4- CYBER AUTONOMOUS RESPONSE, THREAT DETECTION AND SECURITY AUTOMATION

text: The aim of the fourth panel was to discuss developments in the field of Cyber autonomous response and threat detection in operational activities. The panel was comprised of participants working in academia and industries. More specifically, there were representatives from IBM Security (Mr. Domenico Raguseo), from S2GRUPO (Dr Luis Búrdalo & Dr. Miguel A. Juan), from University of Milan (Prof. Ernesto Damiani & Prof. Claudio Ardagna) and from Strathclyde University (Prof. Ivan Andonovic). The panel was moderated by ENISA (Dr Fabio Di Franco).

text: The moderator introduced the session based on the results of the previous sessions and which capability an AI system should have: an AI system should have the capability to learn, reason and think and then take action in response to what it is sensed and the planned objectives. However seems that the term AI has become a buzzword used for marketing purposes.

text: The terms AI might be interpreted in different way and the following schema might help in building an AI maturity model:

text:  Automated Intelligence: apply automation to routine processes that requires no human judgment  Assisted Intelligence: assist people in accomplish their task providing aggregated information and automated functionality  Augmented Intelligence: assist human judgement with a set of information which help people to make better decision (strategic and tactical/operational level)  Autonomous Intelligence: Automating decision making processes without human intervention

text: As more organizations adopt a policy of continuous monitoring, security teams find themselves with voluminous quantities of monitoring and limited number of resources for the operation of the SOC. It is true that computing power and data science has progressed to the point where we can use machines to analyse the data to detect patterns and then use the patterns to create predictions and models, which are going to be tested. However, the network monitoring and detection fields have not progressed as fast as other fields (e.g. image recognition). A key point to consider is that quantity of labelled data that are available in image recognition in order to train the model is enormous and it is not available in other disciplines.

text: It is clear from the panel that we are still in the early stage of using an Artificial intelligence to automate decisions without human intervention in the network monitoring. Machines might assist humans in accomplishing their task providing aggregated information and automated functionality and reduce the time spent in boring and repetitive tasks.

text: In this respect, S2grupo shows that anomaly detection might be used in OT since it is easier to train the model as the expected behaviours are more repeatable. The research they conducted

text: ARTIFICIAL INTELLIGENCE

text: in OT will be used soon to reduce the amount of low complexity alerts in order for SOC teams to focus on more high valuable tasks and to reduce the errors due to fatigue.

text: Another take-away from the discussion is that network environments are very different and heterogeneous. The adaptation of a machine learning model trained in one environment to another environment is not a trivial task as the University of Milan pointed out. The adaptation phases might take long: in that phase, analysts will need to be assigned to validate what is a normal behaviour or an unusual activity in that context until an Acceptable Levels of False Alarms is reached. Moreover, the evasive adversaries make the task harder: it is very hard to detect a threat and it is even harder to predict a future behaviour in order to train a model.

text: Examples of Machine Learning and big Data analytics used today in the detection of attacks use information from threat intelligence combined with specialized algorithms applied at different level of the kill chain . In fact, machine learning algorithms might be chosen based on specific problem to solve and each stage of the kill chain (starting from the Initial exploit, through the stages of Gain persistence and Local network discovery until data exfiltration) will need to be analysed separately with specialized algorithms based on specific training data. Finding a highly specialized algorithm which might solves that specific problem is already a technical challenge based on trials and errors. Another complexity to add to the puzzle is that not all the threats will use the same attack vectors or the same tactics. Therefore, only by using a cognitive model based on the combined information from specialized algorithms and from the threat intelligence of the attack behaviours, an attack might be detected and stopped.

text: IBM notices that Automated Threat intelligence might be able to stop an attack before the initial access has been performed or during the execution phase. The future of Incident Analysis in Cyber Security is to automatically investigate incidents and anomalies based on insights from millions of external sources and cognitive reasoning. The role of the incident analysts will evolve and an Augmented Intelligence will provide fast and aggregated information which will help analysts to make better decision.

text: 5.1 CONCLUDING REMARKS

text:  Machine Learning can provide fast and aggregated information that helps analysts to make better decision and reduce the time spent in boring and repetitive tasks.  More research is needed for an automatic response without human interventions  Build collaboration with research community

text: ARTIFICIAL INTELLIGENCE

text: 6. CONCLUSIONS – FUTURE WORK

text: The event was a success given the feedback received from participants. Most of them would like to see events like this happening more often. Some representatives from Member States, like Portugal, have even volunteered to host follow up events in the future. Given this demand, it is suggested to bi-annually organise an event around the Blueprint in order to periodically evaluate progress, identify new challenges and bring together relevant communities from EU Institutions, the private sector and academia.

text: An important takeaway of the event has been that Artificial Intelligence and Operational collaboration in the Blueprint context are issues of particular interest by the legal world. Almost half of the audience were professionals dealing with law and its extensions into the cyber domain.

text: Regarding the Operational collaboration for the Blueprint, a number of EU Institutions showed interest and attended, like DG HOME, Eurocontrol and the European Council. The discussions that took place between these stakeholders revealed the need for extending the Blueprint to cover the whole of the EU Institutions and also to investigate success stories in Operational collaboration of other sectors, like the Banking and the Aviation sector in order to benefit from their experience in crisis management.

text: More research is required on information fusion and the produce of advice at the Operational level. The impact of large scale cyber-attacks cannot be seen as just technical. There are social, nd rd political, reputational and economic aspects as well as 2 and 3 order impacts that need to be accounted for. This is a topic of ongoing research and EU and ENISA should put more efforts towards this direction.

text: ENISA’s Open Source Cyber Situational Awareness Machine draw a lot of attention, especially from EU Institutions. It has been recognised as an innovative research and development project. Many participants expressed interest to participate to the development as beta testers and evaluators while some even expressed interest to contribute in the development process. ENISA should continue developing OpenCSAM in spirals and perhaps collaborate with researchers and the private sector for building a permanent research and development capability for Artificial Intelligence.

text: On the technical side, ENISA is planning to prepare a report on Artificial Intelligence and machine Learning in Operational Activities. In particular, this report will navigate on how Natural Language Processing based on OSINT and internet feeds might help in prioritizing response in a SOC and how an assisted intelligence might help to detect and monitor the network and autonomously respond to it.

text: -X -XX -000 -00 -00 XX

text: ABOUT ENIS A

text: ISBN 000-00-0000-000-0 doi: 0000.0000/000000