R612.1 Application of Bayes method in evaluation of ROP/NOP trip setpoint – Phase I (ACAN)

1.0	Introduction
The Canadian Nuclear Safety Commission (CNSC) has a requirement to prepare a work plan, and supporting technical basis, for the development and qualification of an independent capability for verification and confirmation of ROP/NOP trip setpoint values (method and computer code). The framework and software, which will be produced through this multi-phase project, are intended to support CNSC staff’s assessment of licensees' proposed ROP/NOP trip setpoints under ageing conditions and risk-informed decision making (RIDM) processes, when necessary.
 The purpose of this advance contract award notice (ACAN) is to signal the CNSC’s intention to award a contract for these services to:
Anthony O’Hagan
26 Scowcroft Drive, Bishops Itchington, Southam, CV47 2YP, UK
Before awarding a contract, however, the CNSC would like to provide other suppliers with the opportunity to demonstrate that they are capable of satisfying the requirements set out in this ACAN, by submitting a statement of capabilities within the posting period for this ACAN, which is 15 calendar days.  
If, during the posting period, other potential suppliers submit a statement of capabilities that meets the requirements set out in this ACAN, the CNSC may proceed to a full tendering process via the Government Electronic Tendering Service or by inviting bids directly from suppliers.
If no other supplier submits, on or before the closing date, a statement of capabilities meeting the requirements set out in the ACAN, a contract will be awarded to the above-noted supplier.
2.0	Background
The intent of the Neutron Overpower Protection (NOP) in-core detectors system is to initiate a reactor shutdown whenever the neutron flux reaches a level that is considered to be too high at any place in the reactor core. Such a condition can occur in any design basis event from a low probability large break Loss of Coolant Accident (LOCA), which is characterized by a rapid void reactivity induced power excursion, to a relatively high probability Loss of Regulation (LOR) events involving a loss of control of the bulk power and/or the spatial power distribution in the reactor.

During an LOR, if the power increase is sufficiently large, then an unstable dry patch on the fuel sheath may develop. This condition is commonly referred to as dryout. Although the onset of fuel sheath dryout does not necessarily lead to fuel or fuel channel failures, elevated fuel temperatures can result in fuel element deformations and, possibly, fuel centre-line melting and eventually pressure tube failure. 

The method employed to determine the NOP set points does not directly simulate the limiting transient (LOR event). The Probability Density Function (PDF) of output parameters is estimated by a stochastic method (convolution of uncertainties or Monte Carlo sampling), rather than by propagation of PDF of key input parameters through simulation of scenario. It is the probability of having a successful outcome of an event that is of interest, rather than knowing the performance parameter (e.g. fuel sheath temperature).

The Heat Transport System (HTS) operating conditions (coolant flows, temperatures, pressures) of a CANDU reactor are affected by the ageing of the HTS components (pressure tube, steam generators and feeders) and this results in less effective fuel cooling which consequently lowers the power at which the fuel will experience dryout. Certain corrective measures, such as mechanical and/or chemical cleaning of the HTS components, and/or other operating measures, could significantly slow down this trend. Eventually, adjustments to the ROP/NOP trip setpoint will need to be implemented.

Regulatory independent evaluation of the required trip setpoint under aging conditions is necessary for verification and confirmation of the adequacy of licensees’ proposed values of the installed trip setpoints, included through a risk-informed decision making process. This project proposes to prepare a first phase work plan for developing a method and computer code, based on Bayes method, for use in regulatory independent verification and confirmation activities.

This project supports the CNSC’s research goal of enhancing capabilities to conduct independent verification and simulations under the Safety Analysis Program Area (E-DOCS-#4391744) in the Deterministic Safety Area Sub-Program Area. 

3.0	Objectives

The objective of this first phase is to prepare a work plan, and supporting technical basis, for the development and qualification of an independent capability for verification and confirmation of ROP/NOP trip setpoint values (method and computer code). The framework and software, which will be produced through this multi-phase project, are intended to support CNSC staff’s assessment of licensees' proposed ROP/NOP trip setpoints under ageing conditions and risk-informed decision making (RIDM) processes, when necessary.

4.0	Scope of Work

The consultant shall prepare the technical basis, starting with the description of the NOP problem, and the work plan for developing and qualifying an independent capability for verification and confirmation of NOP trip setpoints, included through a risk-informed decision making process. Although the use of Bayes method, which is widely used in reliability assessments, is the main option for the software, other statistical methods will also be recommended for implementation through discussion with the chosen contractor, to allow for extension of the range of sensitivity studies. Guidelines for setting performance criteria and parameters, and for manufacturing benchmark cases for testing and qualification shall also be prepared.

5.0	Tasks to be Performed

5.1	Review of the NOP/ROP problem - original general description in Ontario Hydro documentation and SIMBRASS theory manual to be provided by CNSC. This should be the starting point for developing the technical basis for the independent capability. Develop observations concerning the original statistical approach and method, shortcomings and limitations, and potential options for improvement.

5.2	Starting from a CNSC staff paper describing the NOP “effectiveness” and “risk” problems and expected features and needs for the statistical framework, which will be finalized in the form of a “software specification” document based on the discussion at the start up meeting and feedback from the contractor and industry specialists from OPG and Bruce, provide expert opinion on: 

-	separation of uncertainties for NOP analysis and approaches for propagation of "ignorance" and "variability"
-	candidate statistical approaches, other than Bayes method, which should be considered for implementation in the independent capability

5.3	Develop a high level technical description of a statistical framework (statistical method(s), assumptions and requirements for input data) for verification and confirmation of ROP/NOP trip setpoint values which can be used for both “effectiveness” and “risk” problems.

5.4	Develop: 
-	a work plan for implementation of the proposed framework in a computer code, and 
-	generic criteria for manufacturing benchmark cases and testing

5.5	Prepare a draft report and present the results in a seminar with CNSC staff and invited 	guests

5.6	Finalize the report.


6.0	Deliverables


6.1	Start-up Meeting

Date:  	Two weeks after contract award

Location:  	The CNSC Head Office, Ottawa   

Purpose:	To discuss and clarify the proposed approach, work plan and schedule to ensure achievement of the contract objectives. The contractor shall make a presentation with the above purpose in mind.

6.2	Progress Meetings

Dates:	Monthly

Location(s):  	Via Tele/Videoconference

Purpose: 	To assess the degree to which the agreed project objectives are being achieved as planned and thus to facilitate timely adjustments (if necessary) to ensure the project success.

6.3	Draft Final Report (incorporating results of tasks 5.2 to 5.4)

Due Date:	Before end of January 2016

Copies:	One electronic copy via email to the Project Authority

Format and style requirements: 	As specified in the Final Report.

6.4	Presentation 

Due Date:	Before end of February 2016

Location:  	The CNSC Head Office, Ottawa 

Purpose: 	To present the project findings, conclusions and recommendations documented in the Draft Report to the CNSC Commission.

6.5	Final Report (incorporating results from tasks 5.2 to 5.4)

Due Date:	Before March 15, 2016

Copies:	One electronic copy via email to the Project Authority
Format & style requirements:
To be specified by the Project Authority. The font Times New Roman 12 is to be used. Electronic copies must be provided in a format readable by Word 2003 with minor formatting changes. Any electronic files that cannot be read or require major formatting changes when opened are not acceptable and may be returned to the contractor for correction. The CNSC reserves the right, at its own discretion, to have the final report printed under CNSC cover, and to distribute it publicly. Translation of the abstract into French or English, CNSC report covers and the publication number will be provided by the CNSC.

The estimated value of the contract is $70,000 to $80,000 CAD. Applicable taxes are extra.
7.0	Minimum Mandatory Requirements
Any interested supplier must demonstrate by way of a statement of capabilities that it meets the following requirements: 
- Extensive experience (>30 years) in statistics, in particular Bayes theory, with application in scientific and engineering field
- Independence from the Canadian nuclear industry
- Familiarity with the ROP/NOP problem
- Proven expertise in the area of management of uncertainties in complex scientific and engineering models
- Proven expertise in elicitation processes
- Proven expertise in the area of testing and benchmarking of statistical procedures
- Relevant experience in providing advice, opinions, and recommendations for significant risk-informed decisions.
8.0	Justification for the Pre-selected Supplier
Professor Anthony O’Hagan is believed to be the only candidate capable of delivering the required services to complete this contract. He is an internationally recognized leading expert in the use of Bayes method in complex applications.
9.0	Intellectual Property
Canada intends to retain ownership of any foreground intellectual property arising out of the proposed contract on the basis that the main purpose of the contract is to generate knowledge and information for public dissemination. 
10.0	Statement of Capabilities
Suppliers who consider themselves fully qualified and available to meet the specified requirements may submit a statement of capabilities in writing to the contracting authority identified in this notice on or before the closing date of this notice. The statement of capabilities must clearly demonstrate how the supplier meets the advertised requirements.
The closing date and time for accepting statements of capabilities is September 24, 2015, 3pm EDT.
11.0	Contact Information
Inquiries and statements of capabilities are to be directed in writing to:
Dan Simard
Senior Contracting Officer
280 Slater Street
P.O. Box 1046, Station B
Ottawa ON K1P 5S9 
Canada

Telephone: 	613-996-6784
Fax: 		613-995-5086
Email: 		dan.simard@cnsc-ccsn.gc.ca
12.0	Policy Information
Government Contracts Regulations: section 6(d): “only one person is capable of performing the contract.”
Subject to the North American Free Trade Agreement (NAFTA) – (Article 1016.2 b)
Subject to the Agreement on Internal Trade (AIT) – (Article 506.12 b)