text: www.enisa.europa.eu European Union Agency For Network and Information Security

text: Overview of ICT certification laboratories

text: Final | v1.1 | January 2018

text: Final | v1.1 | January 2018

text: Final | v1.1 | January 2018

text: Final | v1.1 | January 2018

text: Final | v1.1 | January 2018

text: https://ec.europa.eu/digital-single-market/en/network-and-information-security-nis-directive https://ec.europa.eu/digital-single-market/en/news/commission-staff-working-document-cppp-and-accompanyingmeasures https://ec.europa.eu/info/law/better-regulation/initiatives/com-2017-477_en

text: Final | v1.1 | January 2018

text: Final | v1.1 | January 2018

text: TERM DEFINITION

text: demonstration that specified requirements relating to a product, process, system, person or

text: conformity assessment body are fulfilled

text: need or expectation that is stated specified requirements Note 1 to entry: Specified requirements may be stated in normative documents such as regulations, standards and technical specifications

text: accreditation body authoritative body that performs accreditation conformity assessment

text: body that performs conformity assessment services

text: body conformity assessment

text: rules, procedures, and management for carrying out conformity assessment

text: conformity assessment conformity assessment system related to specified objects of conformity assessment, to scheme which the same specified requirements, specific rules and procedures apply

text: agreement group bodies that are signatories to the agreement on which an arrangement is based

text: assessment of a body against specified requirements by representatives of other bodies in, or

text: peer assessment candidates for, an agreement group

text: Table 1: General normative context for conformity assessment based on the ISO 17000

text: Final | v1.1 | January 2018

text: TERM DEFINITION SOURCE

text: issue of a statement, based on a decision following review, that fulfilment of specified requirements has been demonstrated

text: attestation Note 1 to entry: The resulting statement, referred to in this International ISO/IEC 17000 Standard as a “statement of conformity”, conveys the assurance that the specified requirements have been fulfilled. (..)

text: third-party conformity assessment body operating certification schemes

text: certification body Note 1 to entry: A certification body can be non-governmental or governmental ISO/IEC 17065 (with or without regulatory authority).

text: certification system related to specified products, to which the same specified requirements, specific rules and procedures apply

text: certification scheme ISO/IEC 17065 Note 2 to entry: A “certification system” is a “conformity assessment system”, which is defined in ISO/IEC 17000:2004.

text: body that performs one or more of the following activities:  calibration laboratory ISO/IEC 17025  testing  sampling, associated with subsequent calibration or testing

text: third-party attestation related to a conformity assessment body conveying

text: accreditation formal demonstration of its competence to carry out specific conformity ISO/IEC 17000

text: assessment tasks

text: certification third-party attestation related to products, processes, systems or persons ISO/IEC 17000

text: combination of the selection and determination functions of conformity

text: evaluation ISO/IEC 17000

text: assessment activities

text: selection and determination functions: ISO/IEC 17000

text: provision of a sample of the object of conformity assessment, according to a sampling ISO/IEC 17000 procedure

text: determination of one or more characteristics of an object of conformity testing ISO/IEC 17000 assessment, according to a procedure

text: examination of a product design, product, process or installation and inspection determination of its conformity with specific requirements or, on the basis of ISO/IEC 17000 professional judgement, with general requirements

text: verification of the suitability, adequacy and effectiveness of selection and determination activities, and review the results of these activities, with regard to fulfillment ISO/IEC 17000 of specified requirements by an object of conformity assessment

text: Table 2: Attestation concept application in the general normative context

text: Final | v1.1 | January 2018

text: TERM DEFINITION

text: process demonstrating whether specified requirements relating to a product, process, conformity assessment service, system, person or body have been fulfilled

text: sole body in a Member State that performs accreditation with authority

text: national accreditation body derived from the State

text: body that performs conformity assessment activities including calibration, testing,

text: conformity assessment body certification and inspection

text: attestation by a national accreditation body that a conformity assessment body meets the

text: requirements set by harmonized standards and, where applicable, any additional accreditation requirements including those set out in relevant sectoral schemes, to carry out a specific

text: conformity assessment activity

text: process for the assessment of a national accreditation body by other national

text: peer evaluation accreditation bodies, carried out in accordance with the requirements

text: (EC) No 765/2008

text: , and, where applicable, additional sectoral technical specifications

text: Table 3: Terms of definitions establishing the legal context according to Regulation 765/2008

text: Currently, the whole series of ISO/IEC 15408 and ISO/IEC 18045 are subject to revision process. All terms and definitions are from official published editions dated to 2009

text: Final | v1.1 | January 2018

text: TERM DEFINITION SOURCE

text: evaluate assessment of a PP, an ST or a TOE, against defined criteria ISO/IEC 15408-1

text: subject of evaluation:

text: Target of Evaluation (TOE) ISO/IEC 15408-1

text: implementation-independent statement of security needs for a TOE Protection Profile (PP) ISO/IEC 15408-1 type

text: implementation-dependent statement of security needs for a specific Security Target (ST) ISO/IEC 15408-1 identified TOE

text: body that sets the standards and monitors the quality of evaluations evaluation authority conducted by bodies within a specific community and implements ISO/IEC 15408-1

text: ISO/IEC 15408 for that community by means of an evaluation scheme

text: administrative and regulatory framework under which ISO/IEC 15408 evaluation scheme ISO/IEC 15408-1

text: is applied by an evaluation authority within a specific community

text: organization with a management system providing evaluation and or testing work in accordance with a

text: defined set of policies and procedures and utilizing a defined

text: methodology for testing or evaluating the

text: security functionality of IT products laboratory ISO/IEC 19896-1 Note 1 to entry: These organizations are often given alternative names by various approval authorities. For example, IT Security Evaluation Facility (ITSEF), Common Criteria Testing Laboratory (CCTL), Commercial Evaluation Facility (CLEF).

text: report that documents the overall verdict and its justification, evaluation technical report ISO/IEC TR 18045

text: produced by the evaluator and submitted to an evaluation authority

text: generate a verdict by a simple comparison

text: check NOTE Evaluator expertise is not required. The statement that uses this ISO/IEC TR 18045 verb describes what is mapped.

text: declare that something has been reviewed in detail with an confirm ISO/IEC 15408-1 independent determination of sufficiency

text: The term ‘laboratory’ does not appear in officially published version of Common Criteria standards, however it is introduced in a standard ISO/IEC FDIS 19896-1, and has been incorporated into revised ISO/IEC 15408-1 (currently under development)

text: Final | v1.1 | January 2018

text: provide a conclusion gained by an analysis which is less rigorous than demonstrate ISO/IEC 15408-1 a “proof”

text: affirm a particular conclusion based on independent analysis with the determine ISO/IEC 15408-1 objective of reaching a particular conclusion

text: prove show correspondence by formal analysis in its mathematical sense ISO/IEC 15408-1

text: rigorously review in detail with an independent determination of sufficiency verify Note 1 to entry: Also see “confirm”. This term has more rigorous ISO/IEC 15408-1 connotations. The term “verify” is used in the context of evaluator actions where an independent effort is required of the evaluator.

text: Table 4: Specific normative concept of conformity assessment based on the Common Criteria approach

text: TERMS DEFINITION

text: assessment of an IT product or a protection profile against the IT security evaluation

text: evaluation: criteria and IT security evaluation methods to determine whether or not the claims made are justified certification: process carried out by a CB leading to the issuing of a certificate

text: participant A signatory to this Agreement. independent organization responsible for assessing the performance of other accreditation body organizations against a recognized standard, and for formally confirming the status of those that meet the requirements of the standard

text: https://www.commoncriteriaportal.org/ccra/ https://www.sogis.org/

text: Final | v1.1 | January 2018

text: organization responsible for carrying out certification and for overseeing the day-to-day certification body (CB)

text: operation of an evaluation and certification scheme

text: organization which carries out evaluations, independently of the developers of the IT

text: evaluation facility products or protection profiles evaluated

text: IT Security Evaluation Facility accredited Evaluation Facility, licensed or approved to perform evaluations within the (ITSEF) context of a particular IT Security Evaluation and Certification Scheme

text: systematic organization of the functions of evaluation and certification under the evaluation and certification

text: authority CB in order to ensure that high standards of competence and impartiality are

text: scheme maintained and consistency is achieved

text: relationships imposed by the arrangement:

text: formally confirmed by an accreditation body as meeting a predetermined standard of

text: accredited impartiality and general technical, methodological and procedural competence

text: assessed by a CB as technically competent in the specific IT technical domain and field

text: of security evaluation and formally authorized to carry out evaluations within the

text: licensed context of a particular within the context of a particular evaluation and certification scheme

text: public document issued by a compliant CB and authorized by a Participant which

text: conformant certificate confirms that a specific IT product or protection profile has successfully completed

text: evaluation by an ITSEF

text: procedure by which representatives of a CB observe evaluations in progress or review monitoring of evaluation completed evaluations in order to satisfy themselves that an ITSEF is carrying out its functions in a proper and professional manner.

text: assessment of compliant CBs (term not defined although described in the

text: voluntarily periodic assessment arrangement)

text: Table 5: Concept of evaluation and certification based on provisions of the SOGIS MRA

text: GENERAL NORMATIVE LEGAL SPECIFIC NORMATIVE ARRANGED conformity assessment conformity assessment evaluation evaluation certification no equivalent no equivalent certification accreditation accreditation not applicable accreditation

text: accreditation body national accreditation body not applicable accreditation body

text: certification body conformity assessment body evaluation authority certification body (CB)

text: Table 6: Equivalent terms

text: Final | v1.1 | January 2018

text: GENERAL NORMATIVE LEGAL SPECIFIC NORMATIVE ARRANGED conformity assessment evaluation and scheme no equivalent evaluation scheme certification scheme certification scheme

text: IT Security evaluation laboratory conformity assessment body laboratory 8 facility (ITSEF)

text: participants of the agreement group no equivalent not applicable arrangement

text: accredited certified attestation no equivalent no equivalent licensed conformant certificate

text: voluntarily periodic peer assessment no equivalent no equivalent assessment

text: Table 7: Terms with similar meanings

text: GENERAL NORMATIVE LEGAL SPECIFIC NORMATIVE ARRANGED peer assessment - used in peer evaluation -used in voluntarily periodic general context of relation to national not applicable assessment - used in conformity assessment accreditation bodies relation to CBs

text: conformity assessment evaluation activities: activities - selection and check, confirm, follows 'Specific determination functions: no equivalent demonstrate, normative' sampling, testing, determine, prove, verify inspection, review

text: Table 8: Differences in terms

text: A requirement for a licencing is significant attribute comparing to a general definition of laboratory. Additionally, considering differences in descriptions of evaluation activities vs. selection and determining function performance similarity criterion applies only to ITSEF which is defined by an accreditation requirement, not to an evaluation facility itself

text: Final | v1.1 | January 2018

text: European European colevel operation for Accreditation (EA)

text: EC Regulation 765/2008

text: National National Accreditation Accreditation EN ISO/IEC Body Body National 17065/EN level ISO/IEC 17025 Accreditation

text: Conformity Conformity Assessment Body/ Assessment Body/

text: Multilateral SOGIS arrangement MRA/ Certification Body ITSEF (laboratory) level CCRA

text: *in some schemes licensing can be done in parallel with accreditation or as a part of accreditation

text: Figure 1: Overview of requirements

text: Final | v1.1 | January 2018

text: European EN ISO/IEC EN ISO/IEC harmonized 17065 17025 standards

text: Certification Body ITSEF (laboratory)

text: ISO/IEC ISO/IEC International 15408 18045 standards

text: CC CEM JIL Docs SOGIS MRA Other normative docs

text: CC CEM CCDB Docs CCRA

text: Figure 2: Sources of requirements for laboratories and Certification Bodies

text: Final | v1.1 | January 2018

text: Final | v1.1 | January 2018

text: Final | v1.1 | January 2018

text: Basically, Common Criteria consist of: 1) 3-part standard containing: concepts and models (ISO/IEC 15408- 1), security functional requirements (ISO/IEC 15408 2), security assurance requirements (ISO/IEC 15408-3) and 1-part standard containing evaluation methodology description (ISO/IEC 18045 “Common Methodology for Information Technology Security Evaluation” (CEM)). Further details are provided in Chapter 5. ISO/IEC 19790 deals with Security requirements for cryptographic modules. It is the ISO pendant of the FIPS 140-2 requirements.

text: Final | v1.1 | January 2018

text: Final | v1.1 | January 2018

text: Final | v1.1 | January 2018

text: https://www.commoncriteriaportal.org/labs/ https://www.sogis.org/uk/status_participant_en.html

text: Final | v1.1 | January 2018

text: Table 9: Number of licensed laboratories under different schemes

text: NUMBER OF TRADED PART OF A PRIVATE RESEARCH GOVERNMENT COUNTRY LICENSED PRIVATE MULTINATIONAL COMPANY INSTITUTE AGENCY LABS COMPANY CORPORATION

text: Australia 2 2 Canada 4 1 3 France 6 4 1 1 Germany 8 7 1 India 1 1 Italy 6 3 2 1 Japan 5 1 1 3 Korea 6 2 2 2 Malaysia 2 1 1 Netherlands 2 2 Norway 4 4 Spain 3 1 1 1 Sweden 2 2 Turkey 5 1 1 2 1 UK 2 2 USA 9 5 4 Total 67 33 4 17 7 6

text: Table 10: Licensed laboratories operating under CCRA SOGIS schemes - legal forms

text: Final | v1.1 | January 2018

text: NAME COUNTRY OF ORIGIN SCHEME(S)

text: atsec Germany Germany Germany, Italy

text: Systems Software Laboratory CCLAB Hungary Italy the Netherlands, Japan, Norway, Brightsight the Netherlands Turkey TUV GmbH Evaluation Body for IT Sec Germany Germany, Japan Advance Data Security USA Norway Epoche and Espri Spain Spain, Turkey CygnaCom Solutions USA USA, Turkey

text: Table 11: Country of origin and scheme assignment for laboratories

text: Final | v1.1 | January 2018

text: TYPE OF BUSINESS

text: COUNTRY certification IT security core minimal Total oriented oriented Australia 2 2 Canada 1 3 4 France 3 3 6 Germany 3 3 2 8 India 1 1 Italy 3 1 2 6 Japan 3 1 1 5 Korea 5 1 6 Malaysia 1 1 2 Netherlands 2 2 Norway 3 1 4 Spain 1 1 1 3 Sweden 1 1 2 Turkey 4 1 5 UK 2 2 USA 3 6 9 Total 30 3 8 26 67

text: Table 12: Type of business of laboratories per scheme

text: Final | v1.1 | January 2018

text: Figure 3: Licensed laboratories operating under CCRA scheme

text: Figure 4: Licensed laboratories operating under SOGIS MRA scheme

text: Final | v1.1 | January 2018

text: Minimal Core 35% 46%

text: Certification oriented 6% IT security oriented 13%

text: Figure 5: Importance of CC evaluation for the companies' business model under the CCRA

text: Core Minimal 36% 36%

text: Certification oriented IT security 7% oriented 21%

text: Figure 6: Importance of CC evaluation for the companies' business model under the SOGIS MRA

text: Final | v1.1 | January 2018

text: APPLICATION PROCEDURE

text: Accreditation against Test evaluation performed by No. Country Fulfilling technical requirements ISO/IEC 17025 as a the applicant prerequisite 1 UK Provisional Appointment Permanent Appointment no

text: Australia & New license granted after assessment 2 not applicable no Zealand and upon a licensing agreement

text: part of the assessment 3 Sweden license granted no procedure part of the assessment 4 Canada license granted yes procedure part of the assessment 5 Germany license granted yes procedure

text: both, accreditation and part of the assessment 6 The Netherlands licensing based on successful no procedure trial evaluation

text: license granted after assessment 7 USA not applicable yes and upon a licensing agreement

text: Malaysia license granted after assessment not applicable no

text: part of the assessment 9 Japan license granted yes procedure

text: Table 13: Application procedure differences among schemes

text: Final | v1.1 | January 2018

text: COUNTRY LICENSE FEES

text: Introductory fee Subscription or membership

text: yes (applicable to Provision Applicants as 1 UK no (see next column) well)

text: Australia & New 2 no no Zealand 1) Application 3 Sweden yes (annual) 2) License

text: Germany Dependent on the inspection field renewal - every 2 years

text: USA no no Annual subscription for 3 consecutive years) 8 Malaysia Application Renewal - every 3 years 1) Application 9 Japan not identified 2) Evaluator Qualification

text: Table 14: Licence fee models for laboratories

text: Final | v1.1 | January 2018

text: COUNTRY RULES FOR THE EVALUATION INDEPENDENCE

text:  autonomous in operational and administrative;  evaluation independent of TOE development;  no evaluation of the work of any group or division within the parent company (the rule may 1 UK be relaxed at the discretion of the CB);  consultancy can be provided unless it does not impair the independence of the evaluation i.e. an individual cannot evaluate his or her own work.

text:  functional separation from the lab's parent organization must be maintained;  evaluating a product of parent company or within the group is not allowed; Australia & New 2  evaluation support consultancy and evaluation services are permitted, however the Zealand laboratory must be able to demonstrate the separation of these activities from evaluation activities.

text:  establishing and implementing documented procedures for identifying conflicts of interest and for ensuring that such conflicts do not adversely influence the quality of the evaluations is required;  pre-evaluation consulting may be provided by the laboratory providing proper separation 3 Sweden of evaluation and consultancy work;  personal involvement of the laboratory personnel with the supplier of a product under evaluation within the preceding two years, either in design of the product or consultancy services to the supplier regarding methods of dealing with matters that are barriers to the product being certified, is prohibited.

text:  sufficient separation of control with the parent company shall be demonstrated; 4 Canada procedures to ensure no conflict of interest between personnel advising and evaluating shall be established and implemented.

text:  conflict of interests with regards to the outcome of an evaluation for the laboratory , its parent corporation and individual staff members shall be identified and removed;  labs have to identify potential conflicts of interest in the beginning of an evaluation and 5 Germany report to the certification authority;  any person active in the evaluation process cannot have been involved in the TOE development.  an individual cannot be hired by the Sponsor for consultancy within 2 years of termination 6 The Netherlands of their employment at the laboratory performing the evaluation.

text:  conflict of interest with regards to the outcome of an evaluation for the laboratory , its parent corporation and individual staff members shall be identified and removed ; 7 USA  consulting and evaluation activities shall be separated within the laboratory (ie. different people shall be assigned to the two).

text:  any person active in the evaluation process cannot have been involved in the TOE 8 Malaysia development, consultancy to the Sponsor within the last 2 years.  laboratories are prohibited from providing consulting services in parallel to evaluation one with regards to a given evaluation; 9 Japan  in case of divisions of larger companies labs have to be operational independent, and operating on own budget.

text: Table 15: Independency requirements

text: Final | v1.1 | January 2018

text: COUNTRY CB'S ROLE IN THE EVALUATION PROCESS

text:  determining whether a TOE will be certifiable in principle;  monitoring all evaluations conducted under the Scheme; 1 UK  assessing all evaluations’ results and issuing certificates;  (optionally) conducting Evaluation Progress Reviews.  accepting evaluation tasks and plans to conduct evaluation;  allocating two certifiers allocated to an evaluation task (certifiers conduct oversight Australia & New 2 through meetings, discussing technical details, reviewing reports, maintaining certification Zealand records, monthly reports on progress);  reviewing draft Evaluation Technical Report.

text:  deciding whether to undertake or decline the Certification;  evaluating partially the result by single evaluation reports (SER), each submitted to the certifier for acceptance, and full evaluation report acceptance; 3 Sweden  creating technical oversight reports;  possible direct supervising the Developer Sites or evaluator's independent testing by evaluator.

text:  determining whether the Target of Evaluation is suitable for evaluation;  providing technical oversight (allocated personnel, i.e. independently performing a subset 4 Canada of evaluation activities and comparing the results, directly observing activities in progress, reviewing reports);  accepting Evaluation Technical Report (ETR).

text:  determining whether a TOE will be certifiable in principle;  oversighting the process; 5 Germany  monitoring certain activities of the evaluation facility, such as the execution of tests/penetration tests or the execution of site audits at the developer in each case on site,  partial reports are only "reviewed and commented on" only final ETR is subject to approval.

text:  accepting applications for certificates;  supervising every evaluation (by appointing Certifier who accepts reports and creates 6 The Netherlands Certification Report);  executing the evaluator obligations to submit monthly status reports.

text: Final | v1.1 | January 2018

text:  implementing active involvement in the evaluation process:  government evaluators may be assigned as members of a laboratory evaluation team;  each evaluation gets a Validator, as a liaison between certification body and a laboratory (they decide what is the scope of their involvement), mostly for technical oversight; 7 USA observation reports;  Technical Rapid Response Team is assigned for each technology type and is expected to provide timely response to questions;  Evaluation Consistency Review is to ensure the LT technical consistency, in conjunction with TRRT addresses issues across multiple evaluations

text:  accepting the evaluation;  supervising (at least 2 certifiers assigned) the evaluations by:  technical review of evaluators' work at predefined key points in the evaluation; 8 Malaysia monthly progress reports;  review and approve test plans;  prepare certification report.  accepting the evaluation; 9 Japan  agreeing on evaluation work plan.  implementing oversight activities: as part of inspections may be in place, certifiers may be 10 Norway present during testing;  approving the ETR.  introducing 3 levels of monitoring of an evaluation:  basic level (low complexity, not long, only ETR required),  medium level (default: partial reports corresponding to the evaluation activities, which 11 Spain in turn correspond with the evaluation of the security assurance requirement classes in the standard),  monitoring level (exceptionally to evaluations having a larger than normal number of nonconformities, special technological challenges - at discretion of the CB).  oversighting the process; 12 France  approving the ETR.

text: Table 16: Supervising the evaluation in relevant schemes

text: Final | v1.1 | January 2018

text: COUNTRY AUDIT OR MONITORING PROCEDURE

text:  monitoring the performance of a laboratory can be activated at any point during the laboratory membership of the scheme; 1 UK  as a minimum: annual audit to co-incide with a laboratory annual contract extension (in case, an existing laboratory wishes to extend their scope of operation).

text: Australia & New 2  obligatory reporting to the CB of future evaluation tasks, changes to staff every 3 months. Zealand  yearly assessment resulting in automatically renewal of the license; surveillance of the 3 Sweden laboratory operation by continuous certification oversight.  Observation Reports generated in response to issues that require corrective action by the 4 Canada laboratory;  assessment of personal changes. 5 Germany  annual audits  submitting a list of the qualified evaluators with proofs of competence on annual base is 6 The Netherlands required;  annual formal technical assessment of the laboratory. 7 USA  new audit process "Check-In/Check-Out" is currently being implemented*

text:  submitting an annual business report with key performance measures: time to complete; 8 Malaysia effort spent; vulnerabilities discovered; consumer satisfaction etc;  reviewing of the laboratory operations at the discretion of CB.

text:  periodic assessments by the CB (not if lab is periodically assessed by the Japanese 9 Japan Accreditation Body).  licensing audit performed every 2 years; 10 France  checking e if the laboratory meets the licensing criteria - at any time.

text: *CB conducts periodic meetings in which those involved in the evaluation process will track progress and discuss issued within a specific evaluation

text: Table 17: Auditing or monitoring the licensed laboratories

text: Final | v1.1 | January 2018

text: Final | v1.1 | January 2018

text: Table 18: AVA_VAN.1 assurance component presentation

text: Final | v1.1 | January 2018

text: Figure 7: Structural relationships in the evaluation process (source: ISO/IEC 18045)

text: ‘Evaluation authority’ can be called ‘certification body in specific contexts, see Chapter 2 for further details.

text: Final | v1.1 | January 2018

text: Figure 8: The evaluation process according to ISO/IEC TR 18045

text: Final | v1.1 | January 2018

text: https://www.commoncriteriaportal.org/files/CCRA%20-%20July%202,%202014%20- %20Ratified%20September%208%202014.pdf namely, Annex B.3 Accreditation and Licensing of Evaluation Facilities CCDB – Common Criteria Development Board manages the technical work program for the maintenance and ongoing development of the CC and CEM and reach agreement on the application of the CC and CEM to evaluations. https://www.commoncriteriaportal.org/cc/#supporting https://www.sogis.org/uk/supporting_doc_en.html This document is not publicly available

text: Final | v1.1 | January 2018

text: Final | v1.1 | January 2018

text: NAME OF THE SOGIS MRA DOCUMENT

text: Application of Attack Potential to HW Devices with Security Boxes (for trial use) 2. Application of Attack Potential to POIs 3. Application of Attack Potential to Smartcards 4. Application of CC to Integrated Circuits 5. Attack Methods for HW Devices with Security Boxes 6. Attack Methods for POIs

text: Attack Methods for Smartcards and Similar Devices

text: Collection of Developer Evidence

text: Composite product evaluation for Smart Cards and similar devices

text: ETR for composite evaluation template

text: Guidance for Smartcard evaluation

text: CEM Refinements for POI Evaluation

text: Security Architecture requirements (ADV_ARC) for Smart Cards and similar devices – Appendix 1

text: Security Evaluation and Certification of Digital Tachographs

text: Certification of "open" smart card products

text: Table 199: Review of the SOGIS MRA documents which supports security evaluations

text: Final | v1.1 | January 2018

text: STANDARD NUMBER STANDARD TITLE Guidance for developing security and privacy functional requirements based on ISO/IEC 19608 ISO/IEC 15408 ISO/IEC TR 20004:2015 Refining software vulnerability analysis under ISO/IEC 15408 and ISO/IEC 18045 ISO/IEC TS 30104:2015 Physical security attacks, mitigation techniques and security requirements ISO/IEC 19790:2012 Security requirements for cryptographic modules ISO/IEC 19792:2009 Security evaluation of biometrics Testing methods for the mitigation of non-invasive attack classes against ISO/IEC 17825:2016 cryptographic modules ISO/IEC 18367:2016 Cryptographic algorithms and security conformance testing ISO/IEC 20540* Guidelines for testing cryptographic modules in their operational environment ISO/IEC 24759:2015 Test requirements for cryptographic modules ISO/IEC 29128:2011 Verification of cryptographic protocols ISO/IEC 29147:2014 Vulnerability Disclosure ISO/IEC 30111:2011 Vulnerability handling processes ISO/IEC 30107-3:2017 Biometric presentation attack detection -- Part 3: Testing and reporting *awaiting publication

text: Table 20: Review of International standards related to security evaluations of various types of ICT products

text: Based on the ISO/IEC JTC1/SC27 Programme of Work, document not publicly available http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=uriserv%3AOJ.L_.2014.257.01.0073.01.ENG http://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32016D0650

text: Final | v1.1 | January 2018

text: EN 419211 PART EUROPEAN STANDARD TITLE

text: EN 419211-1:2014 Overview EN 419211-2:2013 Device with key generation EN 419211-3:2013 Device with key import Extension for device with key generation and trusted channel to certificate generation

text: EN 419211-4:2013

text: application Extension for device with key generation and trusted channel to signature creation

text: EN 419211-5:2013

text: application Extension for device with key import and trusted channel to signature creation

text: EN 419211-6:2014

text: Table 221: European Standards referenced in the Commission Implementation Decision no 650/2016

text: EN NO STANDARD TITLE

text: EN 419212 Application Interface for smart cards used as Secure Signature Creation Devices EN 419212-1:2014 Basic services EN 419212-2:2014 Additional services EN 419251 Security requirements for device for authentication EN 419251-1:2013 Protection profile for core functionality EN 419251-2:2013 Protection profile for extension for trusted channel to certificate generation application EN 419251-3:2013 Additional functionality for security targets

text: Table 22: European Standards related to the ISO/IEC 15408 and ISO/IEC 18045

text: Final | v1.1 | January 2018

text: Final | v1.1 | January 2018

text: Final | v1.1 | January 2018

text: Final | v1.1 | January 2018

text: Germany 62 46 11 878 Spain 7 7 8 80 France 75 57 57 777 Italy 1 8 3 21 thNetherlands 5 12 45 Sweden 6 7 3 21 UK 7 10 1 41

text: Table 23: Number of certificates per country 2014-2016

text: Figure 9: Number of certificates by category

text: Final | v1.1 | January 2018

text: Figure 10: Numbers of certificates by category w/o smart cards

text: EVALUATION CERTIFICATION BODY PRIVATE COUNTRY FACILITY (EF) REQUIREMENTS FOR EVALUATORS (CB) COMPANY NAME

text: levels of qualification: Trainee evaluator (successful training, which is APPROVED by the CB and CONDUCTED by a Qualified Evaluator [or Specialist if appropriate field]); Commercial Commercial Qualified evaluator (a Trainee Evaluator who National Cyber Security Evaluation UK Evaluation has been assessed by the CB to be capable of Centre (NCSC) Facility Facility (CLEF) evaluation work w/o supervision; Specialist (CLEF) evaluator (assessed as Qualified for some subset of the CC Assurance Classess, for others he/she is equivalent to a Trainee). Assessment for Qualified and Specialist based

text: Final | v1.1 | January 2018

text: on a positive recommendation by the CLEF

text: managment and written reports produced by

text: the Trainee Evaluator [inc trail evaluation]

text: Australasia IT Australasia IT Australia & Australasian Evaluation every evaluator has to be approved by ACA Evaluation New Zealand Certification Authority Facility (CV for approval); AISEF CC training Facility (AISEF) (AISEF)

text: two levels: Evaluator (works under Qualified

text: Evaluator's supervision) and Qualified

text: Evaluator (Evaluator assessed by the CB and

text: meets the requirements); the head of ITSEF

text: applies for a staff member to be Evaluator or

text: Qualified Evaluator (with CV and declaration

text: IT Security of competence); candidate for Evalutor status Swedish Certification IT Security Evaluation shall complete the CC training offered by the Sweden Body for IT Security Evaluation Facilities CB and pass the CC/ Scheme examination, for (CSEC) Facilities (ITSEF) (ITSEF) a Qualified Evaluator: Evaluator's progress is

text: monitored by the CB, the Evaluator shall

text: demonstrate experience in planning and

text: conduct evaluation activities and at least once

text: have independently written Evaluator results

text: for all Evaluator actions in each assurance

text: family at EAL4 or higher

text: Common Common staff members of the Company shall possess a Communications Criteria Criteria Certificate of Evaluator Approval, issued by Canada Security Establishment Evaluation Evaluation the CB (conditioned on positive assessment of (CSE) Facilities Facilities (CCEF) the CV and passing of the CC Evaluator Exam) (CCEF)

text: each of individual evaluators is recognized by Bundesamt fuer CB, after assessment of their competence and Sicherheit in der CC- Germany CC-Pruefstellen completion of a BSI-conducted training, all Informationstechnik Pruefstellen evaluators must participate in a trial

text: evaluation (on a fictional case)

text: TUV Rheinland CC training course (either by the CB or

text: The Nederland and the approved by the CB) concluded with an ITSEF ITSEF Netherlands Ministry of the Interior examination. Passing the exam results in a

text: and Kingdom Relations licesed Evaluator status

text: Common No NIAP requirements in terms of personnel; Common National Information Criteria NIST (NCLAP) accreditation lists requirements Criteria Testing USA Assurance Partnership Testing for the personnel, including education, skills, Laboratories (NIAP) Laboratories training - all responsibility of the laboratory, (CCTL) (CCTL) assessed by NIST during accreditation reviews

text: Senior MySEF evaluator: at least 2 years of CC

text: evaluation or certification expierenece, MS

text: Malaysian ISO/IEC 17025 and its application to MySEF Malaysian Malaysian Common Security operations, recognised as an Authorised Security Malaysia Criteria Certificaton Evaluation signatory for Department of Standards Evaluation Body (MyCB) Policy Malaysia or any Accredited Body; MySEF Policy (MySEF) (MySEF) evaluator must show pre-requisite knowledge

text: (by tertiary qualifications, professional

text: certifications or equivalent expierence)

text: Commercial Commercial IT Promotion Agency qualified evaluator (at least one) (qualification Japan Evaluation Evaluation Japan (IPA) by means of a trial evaluation), training Facility (CEF) Facility (CEF)

text: Final | v1.1 | January 2018

text: provided by the CEF with a certificate of completion one of the tasks of the SERTIT is "keeping an Norway SERTIT ITSEF ITSEF overview of the professional status of the employees of an ITSEF" ANSSI is responsible for the assessment of the capability of the evaluation facility relating to France Anssi ITSEF ITSEF the scope of its license. At the request of the CB ITSEF must present evidence that its staff skills meet the scope

text: Table 22: Requirements on Personnel

text: Final | v1.1 | January 2018

text: Drejtoria e Pergjithshme e Akreditimit Str Sami Frashri, No.33 1001 Tirana Phone: +355 4 22 69 325 Fax: - Website: www.dpa.gov.al E-mail: armond.halebi@dpa.gov.al

text: Re-evalutation within 2 years after initial evaluation

text: Federal Ministry of Science, Research and Economy Division I/12 Stubenring 1 A 1010 VIENNA Phone: +43 1 71 100 805411 Fax: +43 1 71 100 8045411 Website: www.bmwfw.gv.at/akkreditierung E-mail: akkreditierung@bmwfw.gv.at

text: Federal Public Service Economy - Division Accreditation 16, Boulevard du Roi Albert II 2nd floor B-1000 Brussels Phone: +32.2 27 75 434 Fax: +32.2.27 75 441 Website: www.belac.fgov.be E-mail: belac@economie.fgov.be

text: 52A Dr. G.M. Dimitrov blvd. 1797 Sofia Phone: +359 2 873 53 02 Fax: +359 2 873 53 03 Website: www.nab-bas.bg E-mail: office@nab-bas.bg

text: Ulica grada Vukovara 78 10000 ZAGREB Phone: + 385 1 610 6322 Fax: +385 1 610 9322

text: European co-operation for Accreditation, http://www.european-accreditation.org

text: Final | v1.1 | January 2018

text: Website: www.akreditacija.hr E-mail: akreditacija@akreditacija.hr

text: Ministry of Energy, Commerce, Industry and Tourism 13-15 A. Araouzos Str. 1421 NICOSIA Phone: +357 22 409 353 or 357 22 409 310 Fax: +357 22 754 103 Website: www.cys.mcit.gov.cy E-mail: aioannou@cys.mcit.gov.cy

text: Olsanska 54/3 CZ-130 00 PRAGUE 3 Phone: +420 272 096 222 Fax: +420 272 096 221 Website: www.cai.cz E-mail: mail@cai.cz

text: Signed PTP for the 1st time in 04/2017

text: Dyregaardsvej 5B 2740 Skovlunde Phone: +45 77 33 95 00 Fax: +45 77 33 95 01 Website: www.danak.org E-mail: danak@danak.dk

text: Signed PTP for the 1st time in 04/2017

text: Mäealuse 2/1 12618 Tallinn Phone: + 372 6 021 801 Website: www.eak.ee E-mail: info@eak.ee

text: P.O. Box 66 Opastinsilta 12 B 00521 HELSINKI Phone: + 358 29 5052 000 Website: www.finas.fi E-mail: akkreditointi@finas.fi

text: Signed PTP for the 1st time in 04/2017

text: rue Jacques Hillairet 75012 PARIS Phone: (33) 01.44.68.82.20 Fax: (33) 01.44.68.82.21 Website: www.cofrac.fr

text: Final | v1.1 | January 2018

text: E-mail: information@cofrac.fr

text: Signed PTP for the 1st time in 04/2017

text: Thisseos str 17676 Kallithea ATHENS Phone: + 30 210 7204 502 Fax: + 30 210 7204 501 Website: www.esyd.gr E-mail: esyd@esyd.gr

text: Signed PTP for the 1st time in 04/2017

text: 04/2017 for inspection

text: Einkaleyfastofan Engjateigur 3 IS-150 REYKJAVIK Phone: +354 580 9400 Fax: +354 580 9401 Website: www.isac.is E-mail: isac@isac.is

text: Metropolitan Building James Joyce Street Dublin 1 Dublin Phone: 00 353 1 6147152 Website: www.inab.ie E-mail: inab@inab.ie

text: Signed PTP for the 1st time in 04/2017

text: Via Guglielmo Saliceto, 7/9 00161 Roma

text: Final | v1.1 | January 2018

text: Phone: +39 06 8440991 Fax: +39 06 8841199 Website: www.accredia.it E-mail: trifil@accredia.it

text: Signed PTP for the 1st time in 04/2017

text: 157, kr.Valdemara Str. LV - 1013 RIGA Phone: + 371 67373051 Fax: + 371 67362990 Website: www.latak.lv E-mail: administracija@latak.lv

text: T. Kosciuskos st. 30 01100 Vilnius Phone: +370 706 65173 Fax: +370 706 64602 Website: www.nab.lt E-mail: info@nab.lt

text: 1, avenue du Swing L-4367 BELVAUX Phone: +352 24 77 43 00 Fax: +352 24 77 93 10 Website: www.ilnas.public.lu E-mail: dominique.ferrand@ilnas.etat.lu

text: Mizzi House National Road HMR9010 Blata l-Bajda Phone: + 356 21 255548 Fax: + 356 21 242406 Website: www.nabmalta.org.mt E-mail: claudio.boffa@nabmalta.org.mt

text: Ul. Dzordza Vasingtona 51 20000 Podgorica Phone: 382 81 246 279 Fax: 382 81 246 283 Website: www.atcg.co.me E-mail: atcg@co.me

text: Skedsmogata 5 NO - 2000 LILLESTRØM Phone: + 47 64 84 86 00 Website: www.akkreditert.no

text: Final | v1.1 | January 2018

text: E-mail: akkreditert@akkreditert.no

text: Signed PTP for the 1st time in 04/2017

text: ul. Szczotkarska 42 01-382 Warszawa Phone: +48 22 355 70 00 Fax: +48 22 355 70 18 Website: www.pca.gov.pl E-mail: sekretariat@pca.gov.pl

text: Signed PTP for the 1st time in 04/2017

text: Rua António Gião, 2 - 4º 2829-513 Caparica Phone: +351 212 948 201 Fax: +351 212 948 202 Website: www.ipac.pt E-mail: acredita@ipac.pt

text: 242, Calea Vitan sector 3 031301 Bucharest Phone: + 40 21 402 04 71 Fax: + 40 21 402 04 89 Website: www.renar.ro E-mail: renar@renar.ro

text: Vlajkoviceva 3 11000 BEOGRAD Phone: + 381 11 313 03 73 Fax: + 381 11 313 03 74 Website: www.ats.rs E-mail: office@ats.rs

text: P.O. Box 74 Karloveska 63 SK 840 00 BRATISLAVA Phone: + 421 948 349 517 Website: www.snas.sk E-mail: snas@snas.sk

text: Final | v1.1 | January 2018

text: Website: www.slo-akreditacija.si E-mail: dejana.robic@slo-akreditacija.si

text: Serrano, 240 28016 MADRID Phone: + 34 91 457 3289 Fax: + 34 91 458 6280 Website: www.enac.es E-mail: enac@enac.es

text: Signed PTP for the 1st time in 04/2017

text: Box 878 SE - 501 15 BORAS Phone: + 46 33 17 77 00 Fax: + 46 33 10 13 92 Website: www.swedac.se E-mail: merih.malmqvist@swedac.se

text: State Secretariat for Economic Affairs SECO Holzikofenweg 36 3003 BERN Phone: + 41 58 463 35 11 Website: www.sas.admin.ch E-mail: info@sas.ch

text: Republic of Kej Dimitar Vlahov Macedonia No.4, Building 2, floor 3 1000 Skopje Phone: +389 (0)2 3293 080 Fax: +389 (0)2 3293 089 Website: www.iarm.gov.mk E-mail: vesna.georgievska@iarm.gov.mk

text: Daalseplein 101 PO Box 2768 NL-3500 GT UTRECHT Phone: + 31 30 239 4500 Website: www.rva.nl E-mail: jan.vander.poel@rva.nl

text: Signed PTP for the 1st time in 04/2017

text: Mustafa Kemal Mahallesi 2125 Sokak No:1 06520 Çankaya/Ankara Phone: 00 90 312 410 8200 Fax: 00 90 312 410 8300 Website: www.turkak.org.tr

text: Final | v1.1 | January 2018

text: E-mail: uim@turkak.org.tr

text: Signed PTP for the 1st time in 04/2017

text: Pine Trees Chertsey Lane TW18 3HR STAINES-UPON-THAMES Phone: + 44 17 84 42 9000 Website: www.ukas.com E-mail: info@ukas.com

text: Signed PTP for the 1st time in 04/2017

text: Final | v1.1 | January 2018

identifier: Overview of the practices of ICT Certification Laboratories in Europe

title: Overview of the practices of ICT Certification Laboratories in Europe

text: FINAL V1.1 JANUARY 2018 About ENISA

text: The European Union Agency for Network and Information Security (ENISA) is a centre of network and information security expertise for the EU, its member states, the private sector and EU citizens. ENISA works with these groups to develop advice and recommendations on good practice in information security. It assists member states in implementing relevant EU legislation and works to improve the resilience of Europe’s critical information infrastructure and networks. ENISA seeks to enhance existing expertise in member states by supporting the development of cross-border communities committed to improving network and information security throughout the EU. More information about ENISA and its work can be found at www.enisa.europa.eu.

text: Contact

text: For queries in relation to this paper, please use isdp@enisa.europa.eu For media enquires about this paper, please use press@enisa.europa.eu.

text: Legal notice

text: Notice must be taken that this publication represents the views and interpretations of ENISA, unless stated otherwise. This publication should not be construed to be a legal action of ENISA or the ENISA bodies unless adopted pursuant to the

text: Regulation (EU) No 526/2013

text: . This publication does not necessarily represent state-of the-art and ENISA may update it from time to time.

text: Third-party sources are quoted as appropriate. ENISA is not responsible for the content of the external sources including external websites referenced in this publication.

text: This publication is intended for information purposes only. It must be accessible free of charge. Neither ENISA nor any person acting on its behalf is responsible for the use that might be made of the information contained in this publication.

text: Copyright Notice

text: © European Union Agency for Network and Information Security (ENISA), 2018 Reproduction is authorised provided the source is acknowledged.

text: ISBN: 978-92-9204-248-6 DOI: 10.2824/35439

text: Table of Contents

text: Executive Summary 5

text: 1. Introduction and scope 6

text: 2. Terms and definitions 7

text: Introduction 7

text: General normative context 7

text: Legal context 10

text: Specific normative context 10

text: Arranged context 12

text: Comparison of terms use in different contexts 13

text: 2.6.1 Equivalents 13 2.6.2 Similarities 14 2.6.3 Differences 14

text: 3. Legal framework and regulations for evaluation laboratories 15

text: Regulation No 765/2008 16

text: National level 17

text: Standard Level 18

text: Requirements from international arrangements 19

text: 3.4.1 General considerations 19 3.4.2 CCRA 19 3.4.3 SOG-IS 20

text: 4. Organisation of laboratories 22

text: Legal forms of laboratories 22

text: 4.1.1 Introduction 22 4.1.2 Research methodology 22 4.1.3 Presentation of results 22 4.1.4 Differences between European laboratories and others 25

text: Licensing and supervising 27

text: 4.2.1 Introduction 27 4.2.2 Licensing 28 4.2.3 Supervision 31

text: 5. Standards used in the evaluation process 34

text: Assurance paradigm in ISO/IEC 15408-3 34

text: Evaluation process according to ISO/IEC TR 18045 35

text: Requirements for laboratories operating in international schemes 38

text: 5.3.1 CCRA 38 5.3.2 SOGIS-MRA 38

text: Standards supporting specific areas of evaluation 40

text: 5.4.1 Standards in support of evaluation methods and techniques 40 5.4.2 European standards supporting security evaluations 41

text: 6. Practices of laboratories 43

text: Typical processes and timeframes for evaluation 43

text: Operational procedures 44

text: Capacity and capabilities 45

text: Personnel 47

text: Annex A: List of full members of EA 50

text: Bibliography 57

text: Executive Summary

text: Certification plays an important role in raising the level of trust and security in ICT products and services. This is also valid for new systems that make extensive use of digital technologies and which require a high level of security. National initiatives have been emerging to set high-level cybersecurity requirements for ICT components on traditional infrastructure, including certification requirements. Important as they may be, they may nurture risks such as market fragmentation and challenges to interoperability.

text: This study seeks to identify and analyse the current landscape of ICT security certification laboratories in EU Member States, comparing them also with third countries practices. The findings of this study will constitute the basis for the Agency’s proposal towards an EU wide ICT products and services certification framework.

text: Terms and their definitions are discussed in order to enable the identification of certification and/or evaluation models, and their instances usually called ‘certification (evaluation) schemes’ where respective entities can play different roles with different relations among them. To recognize equivalents, similarities and differences among terms in use, the following contexts are discussed: general normative, legal, specific normative and arranged.

text: The legal framework in the context of the certification of products can be seen on different levels as follows:

text: 1. General requirements as set up by

text: Regulation (EC) No 765/2008

text: of the European Parliament, 2. National level requirements from relevant accreditation bodies, 3. Certification or evaluation requirements from various standards, 4. Requirements resulting from international arrangements.

text: It has been found that in general, laboratories operate under respective national schemes. Although all are providing services of evaluating the security of ICT products based on an approved and unified methodology, their legal and business context varies, reflecting characteristics of local economies and policies of the Certification Body. By researching all licensed laboratories world-wide this report identifies relevant patterns, similarities and differences. The research will focus on the important implication from the European Union’s perspective.

text: Standards used in the evaluation process include mainly ISO/IEC 15408-3 and ISO/IEC TR 18045. The main use of ISO/IEC 15408 is to assess the security of IT products. There are direct relationships between ISO/IEC 15408-3 assurance structure and the structure of evaluation process as described in ISO/IEC TR 18045. The ISO/IEC TR 18045 provides a description of evaluation process in terms of: roles and responsibilities and general evaluation model.

text: The evaluation according to Common Criteria usually happens in a typical ping-pong run. Evaluations are organized by examining several assurance classes separately. By the time this report has been prepared (November 2017) a total of 1864 certificates have been reported under www.commoncriteriaportal.org (main portal for CC certification) by European laboratories (around 150 per year). It falls into the responsibility of the certification body to ensure that evaluation labs meet the requirements. This includes competence requirements for evaluators. In this context, some certification authorities also have documented criteria that summarize the requirements.

text: 1. Introduction and scope

text: Today’s societies and economies are based on Information and Communication Technologies (ICT). Digitalisation increases cyber security risk across many sectors. There is a need to minimize the risk inherent in the use of ICT in society and the economy. One of the ways to achieve this is through ICT product certification. The certification obligations are implied by existing EU legislations; by the market needs, the industry, and ICT risk owners’ expectations.

text: Taking due account of recent legislative and policy developments, such as the adoption of the NIS 1 2 directive , publication of the European Commission (EC) position on the cPPP , and most importantly, the draft proposal of the EU Cybersecurity Act ENISA continues to support the EC and the Member States in developing a certification framework for ICT security products and services, which on one hand will boost competition, and on the other promote mutual recognition of certificates and harmonisation of certification practices up to defined levels.

text: This study identifies and analyses the current landscape of ICT security certification laboratories in EU Member States, comparing them also with third countries’ practices. The findings of this study will constitute the basis for the Agency’s proposal towards an EU wide ICT products and services certification framework.

text: In the following sections, several aspects of the functioning of certification laboratories for ICT products will be described and analysed:

text:  Terms and definitions  Legal framework  Organisation of laboratories  Standards used in the evaluation process  Practices of laboratories

text: 2. Terms and definitions

text: Introduction

text: Discussing terms and their definition enables the identification of certification and/or evaluation models, and their instances usually called ‘certification (evaluation) schemes’ where respective entities can play different roles with different relations among them. Furthermore, it helps to recognize equivalents, similarities and differences among the terms in use depending on various contexts.

text: To recognize:

text: A. equivalents – i.e. terms having the same meaning regardless the context, B. similarities –i.e. terms relevant for a particular scheme while their meanings can be considered as narrowing the context, and C. differences – i.e. terms relevant for a particular context only, or the same terms of different meanings,

text: this chapter discusses the following context:

text: 1. general normative – introduced by the ISO harmonized standards specifying broad context for conformity assessment activities performed by organizations with defined roles and responsibilities, 2. legal - as introduced by the Regulation 765/2008 as a conformity assessment framework, 3. specific normative – introduced by international standards i.e. ISO/IEC 15408 series and ISO/IEC 18045 (called hereinafter Common Criteria or CC), with regards to security evaluation and certification of IT products, 4. arranged –introduced in international arrangements i.e. CCRA and SOGIS-MRA, containing more detailed requirements, as agreed among the parties of these arrangements.

text: Set of relevant terms and definitions followed by illustration of relevant models are presented in clauses 2.2 -2.5. Then, clause 2.6 of this chapter will identify equivalents, similarities and differences between relevant terms.

text: General normative context

text: The primary concept of conformity assessment has been established in ISO’s series of international standards, in particular, EN ISO/IEC 17000:2004, EN ISO/IEC 17065:2012 and EN ISO/IEC 17025:2017 (mentioning of a publication is a designation of the most recent version of a standard).

text: The defined approach - which is called hereafter ‘General normative context’ - refers to a concept of conformity assessment to specific requirements. It is described further by attestation activities which lead to a clear statement of conformity for a given product or service. Such a result could be achieved by using organizations, rules, procedures and management.

text: Terms and respective definitions for this general normative context are presented in the tables below. The first table shows the general conformity assessment environment (see Table 1).

text: By introducing the concept of ‘attestation’ one can narrow the conformity assessment context to a certification context by defining more specifically:

text:  organizations i.e. certification bodies (operating certification schemes) and laboratories (performing some of the conformity assessment activities needed to demonstrate the conformity with specified requirements by a given product or service), and  activities i.e. certification and evaluation.

text: In such arrangements results of all conformity assessments are presented in the form of a statement issued by the respective body showing clearly the scope and relations in a given conformity assessment scheme.

text: This part of the general normative concept is presented in Table 2.

text: It should be emphasized that within the general normative context the attestation adds a mark of certification (certificate or statement of conformity) to show that a given product or service is conformant after performing assessment operations. Further, it is worth to know that the general normative context does not identify which select and demonstrable function is assigned to a particular conformity assessment body. This has to be defined in a particular certification (conformity assessment) scheme.

text: Legal context

text: The legal context of conformity assessment appears from terms whose definitions are taken from the Regulation 765/2008 (see Table 3). The Regulation introduces strict organization of the market surveillance by using accreditation mechanisms for conformity assessment bodies. Only one accreditation body per Member State is allowed. Conceptually, the legal context does not disturb the general normative one.

text: Please refer to chapter 3 for a more detailed overview of the legal context according to Regulation 765/2008.

text: Specific normative context

text: A specific normative context is established by adopting terms and their definitions from the Common Criteria standards . First, the general normative context is narrowed by introducing specific types of objects to the conformity assessment performance i.e. TOE (Target of Evaluation), Protection Profile (Protection Profile), and ST (Security Target). Secondly, ‘conformity assessment’ is expressed as ‘evaluation’. Finally, the standards define a specific framework under which all activities related to evaluation are performed, and impose specific requirements on organizations involved with regard to their activities.

text: A systematic approach to relevant terms and definitions is presented in Table 4.

text: It should be noted that activities related to evaluation are expressed in different terms and meanings than selecting and demonstrating functions to be performed with regard to conformity assessment, as described in ISO 17000.

text: Arranged context

text: This context for conformity assessment of IT products security is given by international arrangements such as the Common Criteria Recognition Arrangement (CCRA) – world-wide, gathering organizations from several countries, and Senior Officer Group Information Security Mutual Recognition Arrangement (SOGIS MRA) – gathering organizations from the EU plus Norway. As these two arrangements both implement the same certification and evaluation framework based on the Common Criteria, the context will be described further using the SOGIS MRA documents, as relevant to European Certification Framework.

text: The arranged context is presented in the following table:

text: Comparison of terms use in different contexts 2.6.1 Equivalents

text: Part of fundamental terms or group of terms represent the same concept and share the same meaning regardless of the context they function in. These terms are presented in Table 1.

text: 2.6.2 Similarities

text: Several terms have similar meanings and could replace each other while mentioning or remembering their specific context. These terms are summarized in Table 7.

text: 2.6.3 Differences

text: Some of the terms have different meanings. Other terms should mean the same but they have significantly different definitions. Terms presented in Table 8 should be used with care, always adding the context or explanation.

text: 3. Legal framework and regulations for evaluation laboratories

text: 1. General requirements as set up by the

text: Regulation (EC) No 765/2008

text: More details on these levels will be described in the following subchapters. The complete overview is summarized in the following figures:

text: Regulation No 765/2008

text: With

text: Regulation (EC) No 765/2008

text: the EU set requirements related to the accreditation of bodies who perform conformity assessments for products. This regulation applies to European national accreditation bodies that in turn perform the accreditation of the evaluation laboratories.

text: It is an essential part of this regulation that each Member State of the European Union shall appoint a single national accreditation body. (Article

text: This national accreditation body shall, when requested by a conformity assessment body, determine whether that conformity assessment body is competent to carry out a specific conformity assessment activity. Where it is found to be competent, the national accreditation body shall issue an accreditation attestation to that effect. This way, a harmonized European structure for the accreditation of laboratories is built.

text: On the European level, one body has been recognized as the European Body for Accreditation. This body is the European co-operation for Accreditation or EA. EA is an association of national accreditation bodies in Europe that are officially recognised by their national governments to assess and verify—against international standards—organisations that carry out conformity assessment services such as certification, verification, inspection, testing and calibration.

text: For the use in this report, EA has the following roles:

text:  EA cooperates with the European Commission in questions of accreditation,  EA shall allow national accreditation bodies within the Members States to become a member of EA, provided that they comply with the rules of EA,  EA shall provide its members with peer evaluation services.

text: In the context of this report, the peer evaluation services of this body are of specific importance.

text: ‘peer evaluation’ refers to a process for the assessment of a national accreditation body by other national

text: accreditation bodies, carried out in accordance with the requirements

text: (EC) No 765/2008

text: , and, where applicable, additional sectoral technical specifications;

text: Each member of EA (i.e. each European Accreditation body) agrees to take part in the process of peer evaluation, both actively and passively in order to ensure that all members of EA follow the corresponding regulations. By the use of the peer evaluation procedures, a consistent quality should be guaranteed throughout the complete system.

text: National level

text: When the scope of our view changes to the national level, [ISO17025] and [ISO17065] are the most relevant standards after which each of the national accreditation bodies will perform the accreditation of the conformity assessment bodies. Further, from the laboratories perspective, [17027] is of the utmost interest.

text: [ISO17025] defines general requirements on competency that will have to be met by the laboratory providing testing (including evaluation), or calibration services. It specifically poses requirements on:

text:  General Management Requirements o Control of Documents o Subcontracting, purchasing and service to the customer (including complaints) o Control of nonconformity testing and/or calibration work o Improvements, Corrective Actions, internal audits and reviews  Technical Requirements o General requirements o Accommodation and environmental conditions o Test and calibration methods o Equipment o Measurement traceability o Handling of test and calibration items o Assuring the quality of test and calibration results o Requirements on reporting the results A new edition of [ISO17025] is being elaborated at the time of writing of this report. Among new features, it will offer a process oriented approach, which constitutes new value and significant changes in accreditation schemes.

text: Even though it is not a strict requirement, evaluation laboratories in the area of IT-security often also maintain a quality management system in accordance with [ISO9000] and an information security management system in accordance with [ISO27000].

text: [ISO17065] defines the requirements that will have to be met by the certification authority. It specifically poses requirements on:

text:  General Requirements o Legal requirements o Impartiality o Liability and financing o Non-discriminatory conditions o Confidentiality o Publicly available information  Structural Requirements o Organization Structure o Mechanisms for safeguarding impartiality  Resource Requirements o Personnel o Ressources for evaluation  Process Requirements o Applications and Application Review o Evaluation and Review o Certification decision and documentation o Directory of certified products o Surveillance o Changes affecting certification o Termination, reduction, suspension or withdrawal of certification o Records o Complaints  Management System Requirements o Options o General management system documentation o Control of documents and records o Management review o Internal audits o Corrective actions o Preventive actions

text: Standard Level

text: Below the level of [ISO17025] and a potentially implemented management system, the view will change to the standards according to which a conformity assessment body (i.e. the actual evaluation laboratory) performs its assessments.

text: Prominent examples include the Common Criteria (aka ISO/IEC 15408 ([ISO15408] and ISO/IEC 18045 9 10 [18045] ) and [ISO19790]

text: Beside their technical criteria for conformity assessment, these standards bring further organizational requirements.

text: As an example, this can be analyzed for the Common Criteria (CC):

text: Requirements from international arrangements

text: The CC distinguishes between the processes of evaluation and certification. The evaluation is the process that can be seen as the assessment of a product (the so-called Target of Evaluation) against defined requirements.

text: The certification process oversees the evaluation and ends with the actual certificate. CC requires that the certification process is performed by a party independent from the evaluation laboratory.

text: 3.4.1 General considerations

text: The international recognition of certifications that are performed according to Common Criteria is one of the major advantages and benefits of using Common Criteria as reference standards. However, from the perspective of an evaluation laboratory, this advantage also goes along with an additional set of requirements that have to be met. The following two subchapters introduce the major areas from which these requirements arise, namely the CCRA and the SOG-IS agreement.

text: 3.4.2 CCRA

text: The arrangement on the Recognition of Common Criteria Certificates in the Field of IT Security ([CCRA]) is the first agreement on the international recognition of certificates issued on conformity assessment against Common Criteria.

text: The purpose of this Arrangement is to ensure that evaluations are performed based on consistent standards and with a high level of assurance, to improve the availability of evaluated products and Protection Profiles, to eliminate the burden of redundant evaluations and to continuously improve the efficiency and cost-effectiveness of evaluations.

text: The primary goal of the arrangement is to ensure that IT products and protection profiles which earn a Common Criteria certificate can be procured or used without the need for further evaluations.

text: A Management Committee, composed of senior representatives from each signatory’s country of the CCRA, has been established to implement the Arrangement and to provide guidance to the respective national schemes conducting evaluation and validation activities.

text: The requirements of the Common Criteria are mainly developed by an international consortium known as Common Criteria Development Board (CCDB) and Common Criteria Maintenance Board (CCMB). Usually, members of these consortia have a governmental background.

text: CCDB manages the technical work program for the maintenance and ongoing development of the CC and CEM and reach agreement on the application of the CC and CEM to evaluations being carried out by the CCRA certificate producing nations to ensure harmonization across qualifying nations. The principal purpose of CCMB is to process requests for inclusion of Change Proposals (CP), based upon national CC and CEM development requirements and taking into account CCRA requirements as specified by the CCDB.

text: For an evaluation laboratory, the work of the CCDB goes along with a set of requirements for the daily work. Beside the criteria that are laid down in CC and a set of supporting documents that are developed and maintained by the CCDB, the certification body of each member of the CCRA is responsible for oversight of the compliance of the evaluation laboratory with the criteria.

text: 3.4.3 SOG-IS

text: Beside the arrangement laid down in the CCRA, an additional recognition agreement exists on the European Level.

text: The SOG-IS (Senior Officers Group) agreement was produced in response to the EU Council Decision of March 31st 1992 (

text: ) in the field of security of information systems, and the subsequent Council recommendation of April 7th (

text: ) on common information technology security evaluation criteria.

text: The participants of the SOG-IS agreement have a slightly different perspective than the participants of the CCRA. SOG-IS mainly focusses on coordinating evaluation activities around Common Criteria among European Certification Bodies (also to gain a strong standing within CCRA) and to coordinate the development of Protection Profiles. It should be noted that the SOGIS MRA scope of interest is limited to two technical domains of IT products i.e. “Hardware devices with security boxes CC” and “Smartcards and similar devices CC”.

text: However, the SOG-IS agreement also comprises a recognition of Common Criteria certificates among the participants of the agreement. In contrast to the CCRA, two things are important to mention about the recognition that are regulated under this agreement:

text: 1) The recognition of certificates covers all evaluation assurance levels up to EAL7 (compared to only EAL2 typically under the CCRA) 2) The recognition above EAL 4 is only covered for defined technical areas when schemes have been approved by the management committee for this level.

text: Similar to CCRA, the arrangement provides for member nations to participate as certificate consuming participants or as certificate producers.

text: The original arrangement signed in 1997 (updated to incorporate the use of Common Criteria in 1999) was updated in 2010 for two reasons; firstly, to provide a robust mechanism allowing new schemes to take part as certificate producers and, secondly, to limit the higher levels of recognition to agreed technical domains where adequate agreement around evaluation methodology, laboratory requirements, attack methods etc. are in place.

text: In order to achieve the higher level of recognition under the SOG-IS agreement, it has been necessary to develop and publish a set of supporting documents that have been developed by different working groups

text: in the context of the SOG-IS agreement. These documents build up the Joint Interpretation Library (aka JIL documents) and comprise mandatory documents that have to be followed during each evaluation of a product that falls into a technical domain covered by the SOG-IS agreement and guidance documents that are optional regarding their use. More details will be given in Chapter 5.

text: The certification bodies which are part of the arrangement ensure that all evaluation bodies will follow those criteria in addition to the criteria that has been published by the CCMD/CCDB under the CCRA.

text: 4. Organisation of laboratories

text: Legal forms of laboratories 4.1.1 Introduction

text: In general, laboratories operate under respective national schemes (see 4.2). Although all are providing services of evaluating the security of ICT products based on an approved and unified methodology, their legal and business context varies, reflecting characteristics of local economies and policies of the Certification Body. By researching all licensed laboratories world-wide this report will try to identify relevant patterns, similarities and differences. The research will focus on the important implication from the European Union’s perspective. Respective European schemes are indicated in pink in every table.

text: 4.1.2 Research methodology

text: Presented information has been gathered by means of Internet based research.

text: For all licensed laboratories the type of legal entity has been determined, using the respective country’s business register. The following types have been identified:  Private company – independent legal entity, usually similar in form to a limited liability corporation,  Traded private company – independent legal entity, traded on a local stock exchange,  Research institute – non-profit entities, set up by industry members or universities, and  Government agency – laboratories which are part of the country’s government.

text: By investigating companies’ websites the importance of the Common Criteria evaluations (and ICT certification more broadly) for their business model has been qualified, estimating them on a range from core to minimal, with labels “IT Security oriented” and “certification oriented” signaling states in between. The assessment is subjective.

text: 4.1.3 Presentation of results 4.1.3.1 Laboratories in numbers

text: All laboratories listed as being currently in operation within the CCRA framework were the subject of analysis. It should be emphasized that laboratories operating in the context of the SOGIS Mutual Recognition Arrangement are a subset of the analyzed set of laboratories.

text: Major findings with regard to licensed laboratories are presented below.

text: This report analyzes 67 laboratories licensed under 16 operating schemes. As some laboratories are providing evaluation services under more than one national scheme (see below) a total of 60 separate entities has been counted in total.

text: The number of licensed laboratories operating under respective scheme range from 1 (India) to 9 (USA).

text: less than 3 6 Australia, India, Malaysia, the Netherlands, Sweden, UK 3-5 5 Canada, Japan, Norway, Spain, Turkey

text: 6 and more 5 France, Germany, Italy, Korea, US

text: 37 laboratories fall under the category of ‘(traded) private company’ which is more than 50% of all licensed laboratories. In the context of this study they are called ‘legally independent’. Legally independent laboratories constitute a majority of licensed laboratories in Norway (100%), Germany (87%) and France (60%).

text: The second group of security evaluation providers consists of laboratories operating as parts of multinational companies with a wide portfolio of ICT products and services. They constitute a majority of licensed labs in Australia (100%), UK (100%) and Canada (75%).

text: The third group includes research institutes (both industry funded and affiliated with universities) and government agencies. There are several cases that highlight their importance. A governmental agency is the only licensed laboratory under the Indian scheme. Research institutes constitute a significant proportion of licensed labs in Japan and Korea.

text: All discussed figures are presented in Table 10.

text: Several schemes allow laboratories based abroad to be licensed. Table 11 provides the list:

text: It should be noted that all licensed laboratories in following schemes: Italy, Japan, Norway and Turkey are providing evaluation services from abroad (they do not have locally based entities operating under the scheme).

text: 4.1.3.2 Common Criteria evaluation as a business

text:  Further analysis comes to business perspective of laboratories. Considering the following classification: o Core: Common Criteria evaluations is the core business of the entity; additionally, the security certification (as in both cases of importance) focus also falls under 'Core' label; o IT security oriented: the entity focuses on ICT security services, however, Common Criteria evaluations play important role in their portfolio; o Certification oriented: the entity provides wide range of evaluation and certification services in many fields; o Minimal: CC evaluations are only a small part of a wide portfolio of products and services, including providing integrated IT or IT security solutions. We have quantified laboratories per business type, with regards to all analyzed schemes (see Table 12).

text: Differences between European laboratories and others with respect to type of business is discussed in next section.

text: 4.1.4 Differences between European laboratories and others

text: While analyzing the legal form with regards to relevant arrangements gathering schemes significant differences can be identified.

text: It appears majority of licensed laboratories operating under the SOGIS scheme are independent private companies focused on evaluation services themselves, and this figure is significantly higher than for laboratories under the CCRA.

text: This finding is summarized in Figure and Figure 4, respectively.

text: Considering the importance of the Common Criteria evaluation from the business perspective one can compare European and non-European schemes. For this purpose, 52 individual business entities ie. local branches counted as singular, multinationals - as one instance have been identified. The results are presented in Figure 5 and Figure 6, respectively.

text: Worldwide, for 46% of the studied laboratories certification under CC is deemed as core business. A much smaller proportion (13%) of the laboratories focuses on IT security, of which certification is only a part. The smallest group (6%) consists of laboratories that provide certification services in many fields. The rest of the laboratories (35%) are parts of large companies, often multinational, for which CC certification is of minimal importance. Results of classification limited to the SOGIS MRA follow similar patterns, with the exception of the relative greater importance of IT security oriented business model.

text: Licensing and supervising 4.2.1 Introduction

text: For the purpose of analysing relevant characteristics, the scheme processes have been divided into two phases: licensing and then supervising. The research has been conducted with respect to all 16

text: certification schemes although finding a complete set of information for all these schemes has not been possible in the given timeframe.

text: 4.2.2 Licensing 4.2.2.1 Application procedure

text: A licencing mechanism has been identified in all analysed schemes. The most distinguished features are:

text:  one- or two-step procedure: the proceeding for granting a license can be divided into two phases. In some schemes it is allowed to obtain a provisional appointment after passing technical examination of the laboratory readiness, and then a permanent license after successfully completing the test evaluation,  technical examination of the applicant capability is sufficient, or additionally, performing a test evaluation is necessary for the applicant to get a license,  getting an accreditation as a prerequisite: being accredited against ISO/IEC 17025 is mandatory in

text: all analysed schemes, however, in some schemes currently it is allowed to start the assessment by

text: a Certification Body without fulfilling the accreditation requirement by the applicant.

text: It should be noted that European schemes are licensing the laboratories under CCRA and - in parallel under SOGIS MRA. The characteristics of the application procedure are summarized below.

text: 4.2.2.2 Licensing fees

text: There is a variety of licensing fee models among schemes. Some do not apply any fee, some others have only an introductory fee (for the Application), and then for renewal of the license, and some implement subscriptions. It should be noted that fees can be applicable to accreditation, but this is out of the schemes control hence not discussed here.

text: All findings concerning the fee models for licensing are summarized in Table 14.

text: 4.2.2.3 Independency of evaluation

text: Preserving independency of evaluation is fundamental for the third-party evaluation. Schemes impose on licensed laboratories various requirements to ensure actual independency of evaluations. The great majority of commercial laboratories provides consultancy services, which pose a significant source of potential conflict of interests. Every scheme addresses the issue, some provide a time period, in which consultancy work prohibits an individual from working as an evaluator, some others set up general rules only for avoiding conflicts of interest. We have found the strongest rule in the Japanese scheme where providing consultancy services in parallel to the evaluation with regards to a given evaluated product is clearly forbidden.

text: A summary of requirements concerning independency is given in Table 15.

text: 4.2.3 Supervision 4.2.3.1 Supervising the evaluations

text: In general, certification bodies are obliged to oversee the evaluation process as they are held responsible for issued certificates. General provisions on the supervision activities can be found in relevant documentation of every scheme. However, schemes differ significantly if it comes to details. In some schemes dedicated certifiers actively participate in the evaluation, in others a lead certifier even validates every part of the evaluation process, while in some others the nominated certifier plays a passive role, and only approves the final Evaluation Technical Report. Various approaches depending on the perceived complexity of evaluation can be applied as well (for example, see the Spanish scheme characteristics).

text: The summary of findings is presented in Table 16.

text: 4.2.3.2 Auditing or monitoring

text: Schemes vary in their understanding of auditing or monitoring with respect to licensed laboratories. Such activities are part of continuous supervision of the laboratories. However, the approach, depth and frequency of such activities show different ways of achieving the objective. Some schemes include a procedure for monitoring the laboratory operations upon defining trigger criteria, others rely on periodic technical assessments, and some others on analysing periodic reports submitted by the laboratory.

text: The findings are presented in Table 17.

text: 5. Standards used in the evaluation process

text: Assurance paradigm in ISO/IEC 15408-3

text: The main use of ISO/IEC 15408 is to assess the security of IT products. However, every IT product may be used in many ways, and in many types of environment, so the notion of security can vary with context. That means that the result of an ISO/IEC 15408 evaluation is never “this IT product is secure”, but is always “this IT product meets this security specification”. In other words, security evaluation is a process of determining and then proving - with sufficient level of assurance – that the IT product is conformant (or not) to defined criteria.

text: As ISO/IEC 15408-3 states, assurance is gained by active investigation performed by an evaluator. This includes the use of various techniques like:

text:  analysis and checking of process(es) and procedure(s);  checking that process(es) and procedure(s) are being applied;  analysis of the correspondence between TOE design representations;  analysis of the TOE design representation against the requirements;  verification of proofs;  analysis of guidance documents;  analysis of functional tests developed and the results provided;  independent functional testing;  analysis for vulnerabilities (including flaw hypothesis);  penetration testing (i.e. finding and exploring vulnerabilities).

text: The ISO/IEC 15408 philosophy asserts that greater assurance results from the application of greater evaluation effort. This increasing level of effort is based upon:

text: 1. scope -- that is, the effort is greater because a larger portion of the IT product is included; 2. depth -- that is, the effort is greater because it is deployed to a finer level of design and implementation detail; 3. rigour -- that is, the effort is greater because it is applied in a more structured, formal manner.

text: To structure all assurance requirements of every IT product subject to evaluation is described in formal language of assurance classes, families and components.

text: It is beyond the scope of this paper to describe all assurance elements presented in ISO/IEC 15408-3 in detail, but one should notice the least grained element i.e. assurance component - relevant to the IT product - includes all information needed to perform an evaluation in the scope of such element:

text: a) Dependencies with other assurance components, either from the same family/class, or different one;

text: b) Developer action elements: i.e. set of actions is further qualified by evidential material referenced in the following set of elements.

text: c) Content and presentation of evidence elements: i.e. the evidence required, what the evidence shall demonstrate, and what information the evidence shall convey.

text: d) Evaluator action elements: i.e. a set of actions that explicitly includes confirmation that the requirements prescribed in the content and presentation of evidence elements have been met. It also includes explicit actions and analysis that shall be performed in addition to that already performed by the developer. Implicit evaluator actions are also to be performed as a result of developer action elements which are not covered by content and presentation of evidence requirements.

text: An example of the assurance component and related actions on the evidence submitted is presented in Table 18.

text: The overall assurance of a given evaluation is expressed then by including building blocks in the form of the relevant assurance components reflecting the scope, depth and rigour of the evaluation effort, as described above. To make comparison among evaluated products easier ISO/IEC 15408-3 comes with

text: EALn+1 is that it includes all assurance components of the EALn plus set of components representing broader scope, deeper or more rigorous approach to the evaluation than relevant components of the EALn.

text: Evaluation process according to ISO/IEC TR 18045

text: There are direct relationships between ISO/IEC 15408-3 assurance structure (i.e. class, family, component and element) and the structure of evaluation process as described in ISO/IEC TR 18045 (see Figure 7).

text: The ISO/IEC TR 18045 provides a description of evaluation process in terms of: a) roles and responsibilities of the parties involved, and b) a general evaluation model, as a direct consequence of relationships presented above. The general model defines the following roles: sponsor, developer, evaluator and evaluation Four distinct roles i.e. sponsor, developer, evaluator and evaluation authority are involved in the evaluation process.

text: The sponsor is responsible for requesting and supporting an evaluation. This means that the sponsor

text: establishes the different agreements for the evaluation (e.g. commissioning the evaluation). The sponsor ensures that the evaluator is provided with the evaluation evidence.

text: The developer produces the TOE and is responsible for providing the evidence required for the evaluation

text: (e.g. training, design information), on behalf of the sponsor. One should observe that these two roles are not necessarily distinguishing each other in every evaluation. In simple instances (e.g. EAL1 evaluations) it could be one person.

text: The evaluator performs the evaluation tasks required in the context of an evaluation: the evaluator

text: receives the evaluation evidence from the developer on behalf of the sponsor or directly from the sponsor,

text: performs the evaluation sub-activities and provides the results of the evaluation assessment to the evaluation authority.

text: The evaluation authority establishes and maintains the scheme, monitors the evaluation conducted by the

text: evaluator, and issues certification/validation reports as well as certificates based on the evaluation results provided by the evaluator.

text: Each evaluation, whether of a PP (Protection Profile) or TOE (Target of Evaluation) with its ST (Security

text: Target), follows the same general model, and has four evaluator tasks in common: the input task, the

text: output task, the evaluation sub-activities, and the demonstration of the technical competence to the evaluation authority task.

text: The input task and the output tasks are related to management of evaluation evidence and to report generation, respectively.

text: Every evaluation sub-activity is performed with respect to an ISO/IEC 15408-3-mapped evaluator action element (explicit or implied) and ends with the evaluator verdict as a result of the corresponding evaluation. An initial state of the verdict is ‘inconclusive’, and can be changed into ‘pass’ or fail’ after the sub-activity is completed. If all conditions for completing the evaluator action element are not fulfilled, the verdict remains as ‘inconclusive’. An overall verdict is ‘pass’ if only if all partial verdicts are ‘pass’ as well.

text: The demonstration of the technical competence to the evaluation authority task may be fulfilled by the evaluation authority analysis of the output tasks results, or may include the demonstration by the evaluators of their understanding of the inputs for the evaluation sub-activities. This task ends with the evaluator authority verdict.

text: This description of evaluation process is summarized in the action flow and presented using BPMN 2.0 notation (see Figure 8 below).

text: Requirements for laboratories operating in international schemes 5.3.1 CCRA

text: An Arrangement on the Recognition of Common Criteria Certificates In the field of Information Technology Security (CCRA) imposes several requirements on the laboratories (called here ‘Evaluation Facilities’). In general, laboratories shall be:

text:  either accredited by relevant accreditation body in accordance with ISO/IEC 17025, or  established under the laws, statutory instruments, or other official administrative procedures valid in the country concerned.

text: In both cases, common requirements apply, as specified in one of the Annexes of CCRA. In particular, the laboratory shall be licensed or otherwise approved by the Certification Body. Furthermore, the Evaluation Facility also has to demonstrate that it is technically competent in the specific field of IT security evaluation and that it is compliant with the rules of the scheme concerned.

text: Finally, the CCRA states each scheme can establish its own requirements in relation to security, personnel training and operating procedures. Differences among schemes with regard to requirements for laboratories are discussed in Chapter 4.

text: Apart from the arrangement itself, CCDB publishes supportive documents, mandatory for use by the laboratories, which include requirements for the evaluator’s activities in specific areas of evaluation such as Stateful Traffic Filter Firewalls or Integrated Circuits. Full list of supportive documents (SDs) is presented in the CCRA official web site .

text: 5.3.2 SOGIS-MRA

text: This arrangement provides the same general requirements for laboratories as the CCRA. However, due to narrower scope of operation than CCRA, and higher requirements with regard to assurance, there are additional obligations imposed on laboratories. These obligations are described in two documents:

text:  Minimum ITSEF Requirements for Security Evaluations of Smart cards and similar devices ,  Minimum ITSEF Requirements for Security Evaluations of Hardware Devices with Security Boxes .

text: The first document contains several requirements concerning:

text: o basic and detailed knowledge and experience of evaluators in several areas related to Integrated circuit (IC) technology, including: the IC design and production processes, security of IC devices and

text: the environment they are operating, hardware physical and software-related attack techniques that could compromise a secure IC, cryptography techniques used in the ICs, o the evaluators’ ability to use dedicated hardware and software tools to perform attacks against the IC components, and further to analyse the data gathered as a result of data-capture and signal processing procedures.

text: Additionally, the document lists types of equipment indispensable for performing specific attacks against the ICs and similar devices, and discuss the way the laboratory can use them. Furthermore, it describes required capabilities of the laboratory to perform composite evaluations of software and hardware of the product, in terms of tools and necessary equipment.

text: Finally, the document contains basic requirements to the organization of the evaluations and use of subcontractors.

text: The second document discusses similar topics but in relation to another technical domain ie. hardware devices with security boxes (sometimes called ‘hardware security modules’ - HSM). It includes specific requirements with regard to:

text:  the evaluators, in terms of their skills and experience i.e: o basic knowledge in the area of electricity and chemistry, o detailed knowledge and experience of design principles of integrated circuits, microcontroller architecture, functionality and packaging, several attack techniques against the hardware and programmable micro-controllers, cryptographic algorithms and random number generators, o ability to use the equipment to perform independent tests and attacks.  the laboratory unrestricted access to dedicated facilities such as o environment control equipment (e.g. to control communication, voltage, clock and temperature) o chemical and mechanical lab equipment (i.e. for sample preparation and analysis) o imaging equipment (e.g. cameras, microscopes) o logical test tools (e.g. for interface testing, vulnerability scanning, operating system testing, randomness analysis, source code analysis, circuit layout analysis, fuzzing tools)  the laboratory organization for evaluations, in term of the use of bespoken equipment, directly or by subcontracting.

text: Additionally, it should be noted there are several detailed supporting documents for evaluations, mandatory to use for laboratories under the SOGIS MRA. These are listed in Table 199.

text: Standards supporting specific areas of evaluation 5.4.1 Standards in support of evaluation methods and techniques

text: There are several types of IT products which require various methods, techniques, tools and procedures for security evaluations. Consequently, specific and detailed knowledge, skills and experiences are expected from evaluators in these areas as well.

text: Several standards which support the evaluation methodology as described in ISO/IEC TR 18045, in particular for vulnerability assessments, are listed in Table 200.

text: It should be noted there is undergoing constant development of standards in new areas of applicability of security evaluations such as white-box cryptography, quantum cryptography, patch management and deployment activities .

text: 5.4.2 European standards supporting security evaluations

text: Significant standardization activities are related to the implementation of Regulation No 910/2014 of the European Parliament and of the Council on electronic identification and trust services for electronic transactions in the internal market (eIDAS).

text: According to the Commission Implementing Decision No 650/2016 laying down standards for the security assessment of qualified signature and seal creation devices (...) , ISO/IEC 15408, part 1-3, and ISO/IEC

text: 18045 are designated as the reference documents for the security assessment of information technology products that apply to the certification of qualified electronic signature creation devices or qualified electronic seal creation devices. Further, the decision refers to multi-part European Standard EN 419211 Protection profiles for secure signature creation device, containing subjects for evaluations and subsequent certifications ie. Protection Profiles for relevant electronic signature/electronic seal devices (see Table 221).

text: There are other European Standards, not referenced in the Decision 650/2016, which are relevant for performing security evaluations of electronic signature related devices such as these included in Table 22.

text: 6. Practices of laboratories

text: Typical processes and timeframes for evaluation

text: The evaluation according to Common Criteria usually happens in a typical ping-pong run. The developer provides evidence to the evaluation laboratory that the product (the Target of Evaluation, to be more precise) meets certain requirements as determined in the Security Target documentation; the laboratory reviews and tests the evidence and provides feedback to the developer. Relevant documentation and the TOE itself are revised and the process starts over.

text: Evaluations are typically organized by examining several assurance classes separately. This means that the following areas of an evaluation are addressed one after the other:

text:  The Security Target specification  The Design Documentation (Class ADV)  The Guidance Documentation (Class AGD)  The Life-Cycle Documentation (Class ALC)  Testing and Vulnerability Analysis (Class ATE and AVA)

text: In the context of the international recognition of the criteria and the overall complexity of the Target of Evaluation, some of the documentation to be provided for an evaluation can get complex. In this context, it is common that the evaluation/testing of a certain piece of information or its revision takes several weeks rather than few days.

text: The overall timeframe that needs to be planned for an evaluation depends on the following factors (in that order):

text:  The chosen Evaluation Assurance Level (EAL). The basis for the estimations is the list of work units from [ISO18045] that relate to the chosen EAL.  The complexity of the Target of Evaluation  The experience of the developer  The experience of the lab and  the maturity of the criteria

text: The question of the overall timeframe of an evaluation is hard to answer but the following lists should provide a first overview:

text: Timeframes after EAL:

text:  EAL 1,2,3: evaluations typically can take less than 6 months  EAL 4: evaluations are typically planned in a timeframe of a year  EAL 5 and above: 1 1/2 years and above

text: In this context, it should be noted that the listed timeframes should just be taken as examples based on the experience of the editors. Timeframes vary significantly from scheme to scheme and are even restricted by some schemes (e.g. in the US).

text: Timeframe considering the complexity of the product:

text:  Simple products: See above

text:  Products of medium complexity: add 10-20%  Complex products: add 50 % or more

text: The experience of the developer plays a very important part regarding the schedule of an evaluation. A well experienced developer can easily cut the timeframes that have been mentioned before even by half, specifically if a predecessor version of the product already has been certified. On the other hand, novices in the field of Common Criteria, are well advised to allow some extra time in their schedule.

text: The experience of the evaluation laboratory and the maturity of the evaluation criteria are two influencing factors that are less often discussed. Even though, no reliable data exists for this question, empirical observations lead to the conclusion that the experience of the laboratory has significant impact on the timeframe of an evaluation. This does not only refer to the experience of the laboratory in general (meaning how many evaluations a laboratory has carried out) but it specifically refers to the experience of the laboratory with respect to similar products. As an example: A laboratory that has already performed evaluations of firewalls according to a certain Protection Profile will have a significant advantage for evaluations in this area (even over other labs that are more experienced in general but have less specific experience).

text: Last, but not least, the maturity of the criteria is an important aspect. This does not relate to the maturity of the Common Criteria itself but relates more to the maturity of dedicated Protection Profiles, interpretations and guidance documents. Specifically, when these criteria are used for the first time, evaluations usually take significantly longer.

text: Operational procedures

text: Operational procedures for working in a laboratory should be seen in different contexts as follows:

text:  Working as an evaluator  Working as a lead/senior evaluator  Managing a laboratory

text: The work of an evaluator comprises the review of the documentation that has been provided by the

text: developer as well as tests and vulnerability assessment of the product itself. In larger laboratories, it is common that evaluators specialize in certain aspects of evaluation procedures. Even though it is usually the case that evaluators are capable to conduct all aspects of an evaluation, they often focus on certain aspects of an evaluation in their daily work.

text: From the evaluator perspective, the daily work is characterized by the four-eyes principle that is inherent to the Common Criteria requirements in many different places. Every step of work that is performed by the evaluator is typically reviewed by at least one competent co-worker. The extensive amount of reviews and checks facilitates the high quality of the evaluation procedures according to Common Criteria.

text: Every evaluation is led by a senior/lead evaluator. The work in such role usually includes strict and active

text: supervision of the evaluator but is augmented by aspects of project management and customer relations. The senior evaluator should be a well experienced professional and therewith is the first point of contact for all evaluators in a project if questions arise. Also, it is typical that sensitive decisions and test results that potentially have an impact on entire evaluation are discussed and double-checked with the senior/lead evaluator.

text: The senior/lead evaluator overlooks the entire project and also consider the effect of a technical decision on the overall project schedule and budget. Also, senior/lead evaluators usually are responsible for

text: contacting customers, establishing modes of co-operation, leading milestone meeting and similar activities.

text: The manager of the laboratory oversees all evaluation processes which are conducted in that laboratory. With respect to the evaluation processes themselves, the lab manager represents another level of escalation. If a technical dispute among evaluators (or between evaluators and the developer or sponsor) comes up, the lab manager can be involved. However, in a large number of laboratories, the position of a lab manager is seen as a management position rather than a technical one. This means that a lab manager does not necessarily have the technical background and knowledge to discuss all technical issues. The primary focus of the lab manager in daily life is:

text:  to ensure that the laboratory follows all regulations,  to ensure that new interpretations and information about the Common Criteria are circulated amongst all evaluators and utilized during evaluations,  to communicate with the certification and accreditation authorities in all substantial issues with regard to the accreditation of licensing of laboratory.

text: Capacity and capabilities

text: Gaining an overview of the capabilities of all evaluation laboratories in Europe that work in the area of Common Criteria is not an easy task. The main reasons for this are:

text: 1) There is no obligation for a certification or evaluation authority to report on finished certification. 2) Evaluations can remain unreported for a variety of reasons as such: a. The evaluation fails, b. The evaluation is performed in the context of confidential project (e.g. a project with military scope or a classified project, in general) 3) The primary, central source for certifications under www.commoncriteriaportal.org often remains obsolete.

text: The analysis in this chapter is based on the information as published on www.commoncriteriaportal.org and augmented by the knowledge of the experts who authored this report.

text: By the time this report has been prepared (November 2017) a total of 1864 certificates have been reported under www.commoncriteriaportal.org by European laboratories.

text: Over the last few years, the numbers are as follows:

text:  2016: 83 certificates  2015: 147 certificates  2014: 163 certificates

text: When looking at these numbers, one should keep in mind that quite a few of these products the certificates are granted to, have actually been re-certified.

text: The following table shows, how the certificates in 2014-2016 are distributed over the European countries that issue certificates:

text: The following figures show how the published certificates are distributed among the various classes of products (the most populated categories shown for clarity of presentation).

text: It becomes immediately obvious that the vast majority of certificates are issued for smart cards and similar devices.

text: If one filters out the smart card related certificates (see Figure 10) and repeats the analysis, the next top 3 are not so obvious as they change over the years. However, it’s probably fair to say that the next most popular categories for certification are databases, products for digital signatures and network devices.

text: Personnel

text: A plethora of requirements that have to be met by the personal are covered by the criteria as introduced previously. In this context, it falls into the responsibility of the certification body to ensure that evaluation labs meet the requirements. This includes competence requirements for evaluators. In this context, some certification authorities also have documented criteria that summarize the requirements.

text: While developing this report, investigation has also been performed with respect to the question, how the various schemes address the requirements on personnel. The outcome of this investigation can be found in Table 22

text: The results in this table show that the actual requirements on personnel for CC certifications differs among the various schemes. In order to facilitate a harmonization of the requirements in this context, ISO/IEC SC27 WG3 has started a project in this area. The three-part standard ISO/IEC 19896-1,2,3 Information technology — Security techniques — Competence requirements for information security testers and evaluators comprises the following parts

text:  Part 1 introduces the general concept and some general requirements  Part 2 specifies Knowledge, skills and effectiveness requirements for ISO/IEC 19790 testers  Part 3 specifies Knowledge, skills and effectiveness requirements for ISO/IEC 15408 (Common Criteria) evaluators

text: The international standardization of these requirements in the context of ISO will lead to a further harmonization of the competence requirements for evaluators in different fields that will - in the end -also help to improve the quality and comparability of evaluations in general.

text: Annex A: List of full members of EA

text: Albania DPA Directorate of Accreditation

text: Austria AA Akkreditierung Austria

text: Belgium BELAC Belgian Accreditation Council

text: Bulgaria BAS Executive Agency "Bulgarian Accreditation Service"

text: Croatia HAA Croatian Accreditation Agency

text: Cyprus CYS-CYSAB Cyprus Organization for the Promotion of Quality

text: Czech CAI Republic Czech Accreditation Institute

text: Denmark DANAK Danish Accreditation

text: EAK EAK Estonian Accreditation Centre

text: Finland FINAS Finnish Accreditation Service

text: France COFRAC Comité français d'accréditation

text: Germany DAkkS Deutsche Akkreditierungsstelle GmbH

text: Greece ESYD Hellenic Accreditation System

text: Hungary NAH National Accreditation Authority

text: Iceland ISAC Icelandic Board for Technical Accreditation

text: Ireland INAB Irish National Accreditation Board

text: Italia ACCREDIA Ente Italiano di Accreditamento

text: Latvia LATAK Latvian National Accreditation Bureau

text: Lithunia Lithuanian National Accreditation Bureau

text: Luxemburg OLAS Office Luxembourgeois d'Accreditation et de Surveillance

text: Malta NAB-Malta National Accreditation Board -*Signatory in testing except ISO 15189

text: Montenegro ATCG Accreditation Body of Montenegro

text: Norway NA Norsk akkreditering

text: Poland PCA Polskie Centrum Akredytacji

text: Portugal IPAC Instituto Português de Acreditação, I.P.

text: Romania RENAR Romanian Accreditation Association

text: Serbia ATS Accreditation Body of Serbia

text: Slovakia SNAS Slovak National Accreditation Service

text: Slovenia SA Slovenska akreditacija - * Signatory in testing except ISO 15189

text: Spain ENAC Entidad Nacional de Acreditación

text: Sweden Swedish Board for Accreditation and Conformity Assessment

text: Switzerland SAS Swiss Accreditation Service

text: The former The Accreditation Institute of the former Yugoslav Republic of Yugoslav Macedonia

text: The RVA Netherlands Raad voor Accreditatie

text: Turkey TURKAK Turkish Accreditation Agency

text: United UKAS Kingdom United Kingdom Accreditation Service

text: Bibliography

text: [ISO15408] ISO/IEC 15408-1:2009 Information technology -- Security techniques -- Evaluation criteria for IT security [ISO19790] ISO/IEC 19790:2012 Information technology -- Security techniques -- Security requirements for cryptographic modules [ISO9000 ISO 9001:2015 Quality management systems -- Requirements [ISO17025] ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratories [ISO17065] ISO/IEC 17065:2012 Conformity assessment -- Requirements for bodies certifying products, processes and services [ISO27000] ISO/IEC 27000:2016 Preview Information technology -- Security techniques -- Information security management systems -- Overview and vocabulary [ISO17065] ISO/IEC 17065:2013 Conformity Assessment – Requirements for bodies certifying products, processes and services [CCRA] Arrangement on the Recognition of Common Criteria Certificates In the field of Information Technology Security https://www.commoncriteriaportal.org/files/CCRA%20- %20July%202,%202014%20- %20Ratified%20September%208%202014.pdf 02.07.2014

text: ENISA

text: European Union Agency for Network and Information Security Science and Technology Park of Crete (ITE) Vassilika Vouton, 700 13, Heraklion, Greece

text: Athens Office

text: 1 Vasilissis Sofias Marousi 151 24, Attiki, Greece Catalogue Number: TP-04-18-008-EN-N

text: PO Box 1309, 710 01 Heraklion, Greece ISBN: 978-92-9204-248-6 DOI: 10.2824/35439 Tel: +30 28 14 40 9710 info@enisa.europa.eu www.enisa.europa.eu