Enhancing the Common Attribute Schema for Forest Resource Inventories for use by Natural Resources Canada.

ADVANCE CONTRACT AWARD NOTICE (ACAN)

1. Title

Enhancing the Common Attribute Schema for Forest Resource Inventories for use by Natural Resources Canada.
2. Definition

An Advance Contract Award Notice (ACAN) allows departments and agencies to post a notice, for no less than fifteen (15) calendar days, indicating to the supplier community that it intends to award a good, service or construction contract to a pre-identified contractor. If no other supplier submits, on or before the closing date, a Statement of Capabilities that meets the requirements set out in the ACAN, the competitive requirements of the government's contracting policy have been met. Following notification to suppliers not successful in demonstrating that their Statement of Capabilities meets the requirements set out in the ACAN, the contract may then be awarded using the Treasury Board's electronic bidding authorities.

If other potential suppliers submit Statement of Capabilities during the fifteen calendar day posting period, and meet the requirements set out in the ACAN, the department or agency must proceed to a full tendering process on either the government's electronic tendering service or through traditional means, in order to award the contract.
3. Background

Natural Resources Canada (NRCan) has committed to delivering open science and data to support regional cumulative effect assessments. The most cost-effective way to do this is to leverage and re-purpose datasets that have already been collected.
Forest resource inventory maps (referred to as ‘FRI datasets’ herein) are compiled by provincial and territorial governments and private sector firms for use as key inputs to their forest management planning processes. Digital FRI datasets, in the form of geospatial databases, exist for most forest areas in Canada and contain a wealth of information that could be re-purposed for regional applications, including cumulative effects assessments. However, FRI datasets are difficult to use in regional applications because each jurisdiction employs its own procedures and standards and these have changed over time. This heterogeneity has effectively prevented regional application of FRI datasets. In fact, national FRI dataset collections have only been compiled twice before; once by the Canadian Forest Service for Canada’s Forest Inventory (CanFI), and once by an academic consortium led by Université Laval for Boreal Ecosystems Modelling research.
CanFI compilation was expensive and time consuming. The previous version of CanFI was delivered by the Canadian Forest Service five years late because of the overwhelming complexity of FRI data harmonization and integration (https://cfs.nrcan.gc.ca/publications?id=26795). Université Laval had to reproduce CanFI because it did not retain all of the required information content of the original FRI datasets; CanFI was generalized to serve its function as a national reporting tool whereas Université Laval required all of the detailed geospatial information contained in the original FRI datasets.
Université Laval created the Common Attribute Schema for Forest Resource Inventories (CASFRI) to compile the nationally harmonized FRI dataset needed for their research, and gained access to FRI datasets from provincial governments, territorial governments, Parks Canada, the Department of National Defence and private sector firms.
For NRCan to deliver FRI data as part of its open science and data program, these data must be harmonized and integrated once again. The CanFI approach has proven to be inadequate because too much of the geospatial information contained in the original FRI datasets was lost, but the CASFRI approach very nearly satisfies NRCan’s requirements. Provincial and territorial governments, federal departments and agencies and private sector firms will be invited to contribute their FRI datasets. But first, NRCan needs to have the tool to harmonize these datasets. Enhancing CASFRI for this purpose will be faster, easier and cheaper than developing a new tool from scratch. NRCan requires this tool within three years time to deliver on its commitments.
4. Objectives

The overarching objective of this project is to prepare and deliver the tool needed by NRCan to harmonize and integrate FRI datasets nationally on an ongoing basis (as new FRI datasets become available to NRCan) for regional applications.

The specific objects are:
- enhance CASFRI so that it can be maintained in-house by NRCan staff using open source technologies;
- enhance CASFRI so that it can be used by NRCan to process new and old FRI datasets for the same forest areas and produce a harmonized FRI dataset time series; and
- enhance CASFRI so that it can be used by NRCan to efficiently perform updates as new FRI datasets become available to NRCan.
5. Project Requirements

5.1 Tasks & Deliverables

Project requirements are articulate as a series of tasks. Each task has a series of milestones and deliverables and production and delivery schedule.

Task 1 – Project Meeting and Initial Setup

Meeting between Client and Contractor at Pacific Forestry Centre in Victoria, BC, to agree on strategy and technical solutions to meet NRCan requirements.

Timeline :
-Dec 2018: meeting with Client to discuss implementation options for Tasks 2 through 6 and reach agreement
 -Jan 2019: draft meeting report for review by Client
-Feb 2019: delivery of finalized meeting report and invoice to Client

Task 2 – FRI Dataset Conversion

FRI datasets come in many different formats (shapefiles, coverages, e00 files, geodatabases, etc.). A tool is needed to convert raw FRI datasets into a standardized format for processing.

The task is to implement the conversion step using a well-documented, open source solution that can be maintained in-house by NRCan staff without any dependencies on commercial software, including ArcGIS. In the current version of CASFRI, FRI datasets are converted to a set of CSV files containing all the attribution information and a parallel set of shapefiles for the geometries. The deliverable is a new set of conversion scripts implemented in Python invoking the open source GDAL/OGR library instead of the ArcPy library.

Time schedule:
- Jan/Mar 2019: write conversion scripts for a first suite of FRI datasets (2 or more FRI datasets from different Canadian jurisdictions) and use to test and refine conversion process performance
- March 31, 2019: delivery of conversion scripts for first suite of FRI datasets
- Apr/Dec 2019: write conversion scripts for remaining FRI datasets
- March 31, 2020: delivery of remaining conversion scripts and invoice to Client

Task 3 - FRI Dataset Translation and Validation

Once FRI datasets have been converted into a standardized format for processing, the FRI dataset attributes can be translated from their original standards to the Common Attribute Schema (CAS) standard.

The task is to implement the translation step using well-documented, transparent, easily maintained code that includes validation of input and translated data and is structured so that new translation rules can be easily added as new FRI datasets become available to NRCan.

In the current version of CASFRI, this process is implemented as a set of translation rules for each FRI dataset that are hard-coded in Perl scripts. The output is a set of CSV files formatted for import to the CASFRI database. There is no validation of input or translated data. This limits the reliability of the data in the CASFRI database. Translation rule hard-coding in Perl makes the process difficult to maintain, and makes it difficult to add new translation rules as new FRI datasets become available for integration.

The deliverables are:
(1) a translation engine implemented as a generic ETL (extract, transform, load) process, either written in a suitable programming language or using a well-documented open source solution (such as Pentaho or equivalent);
(2) a set of translation rules loaded into standardized input files or database tables, where they can be accessed by the translation engine and maintained or expanded by NRCan analysts; and
(3) a set of validation scripts to ensure that all translations are implemented correctly and consistently.

Time schedule:
- January 1 to March 31, 2019: translation engine beta version implementation with a first set of translation files for at least 2 FRI datasets from different Canadian jurisdictions, translating all or a subset of key attributes for beta demonstration purposes
- March 31, 2019: beta version of translation engine delivered to NRCan with a first set of translation files
- April 1, 2019, to March 31, 2020: debug, optimize and complete validation and translation engine (API)
- April 1, 2019, to March 31, 2020: write remaining translation files
- March 31, 2020: deliver production version of translation engine and complete set of FRI translation files and invoice to Client

Task 4 – CASFRI Temporalization and Topological Correction

Once FRI datasets have been translated from their original standards to the Common Attribute Schema (CAS) standard, they can then be loaded into the CASFRI database. However, the current CASFRI database does not support multiple FRI datasets for the same forest. FRI datasets have been compiled periodically to replace older FRI datasets as forests change and improved FRI methodologies become available. NRCan requires a tool that can accommodate both new and old FRI datasets. NRCan also requires the capability to geographically and temporally relate other datasets to information in CASFRI. To do this, we must be able to extract the most relevant data from CASFRI for a given location and time. The current version of CASFRI provides the capability to extract the most relevant data for a given location or area of interest, but not a time or time-period of interest. A query run in the current version of CASFRI may return data that pertain to a range of years, with some older data and some newer data, and doesn’t provide information about the vintage of these data. This makes it difficult to know what the forest conditions were at the time of interest.

The task is to implement an SQL script to build a new version of the CASFRI database that can accommodate both new and old FRI datasets, even when these overlap in space, and implement a process for extracting datasets from the new CASFRI database for user-specified areas and dates (or time periods) of interest, write a loading script for loading translated FRI data into the CASFRI database, and develop a system for producing CASFRI outputs. The output process may be implemented as queries that run on demand to produce CASFRI output coverages, or as queries that precompute output coverages for each year and fix overlaps for each of those coverages to provide a set of overlap-free CASFRI coverages for each year. Once NRCan obtains access to all FRI datasets, there will be at least 30 million polygons in its version of CASFRI. The latter approach may be required to meet client needs by storing precomputed datasets instead of waiting for lengthy computations to run. All scripts and processes shall be implemented using open source technologies with well-documented, transparent and easily maintained code.

The deliverables are:
(1) an SQL script that builds a new CASFRI geodatabase implemented in PostgreSQL that can store FRI datasets of different vintages, even when these overlap in space, and retains information on each FRI polygon’s vintage (year that forest conditions were recorded in the FRI attributes, i.e. the year the data pertain to, which in photo-interpreted FRI datasets is typically the year the aerial photography was taken).
(2) a new loading script implemented in Python, invoking the open source GDAL/OGR library
(3) a CASFRI query script that combines the best (most complete attribution) and most relevant (inventory date closest to user-specified date of interest) available FRI data into a topologically clean dataset with no overlaps for the user-specified area of interest.

Note that a CASFRI geodatabase with FRI datasets loaded is not a deliverable. FRI datasets will be delivered to NRCan seperately, by the FRI dataset owners/custodians themselves, for loading by NRCan to ensure that NRCan possesses only those FRI datasets that it has permission to use. The Contractor will need to load all FRI datasets in the current CASFRI dataset into the new CASFRI database for development and testing purposes, but these datasets shall not be delivered to NRCan.

Time schedule:
- April 1, 2019, to March 31, 2020: design new CASFRI geodatabase in PostgreSQL and write SQL script to build this database;
- April 1, 2019, to March 31, 2020: write loading script in Python with GDAL/OGR library to replace the current Perl/PostGIS script for loading into PostgreSQL, using FRI datasets from across Canada to test, validate, identify and fix bugs;
- March 31, 2020: delivery of script to build new CASFRI geodatabase in PostgreSQL, loading script and invoice to Client

Task 5 – CASFRI Update Computation

NRCan requires the capability to import new, updated FRI datasets as these become available. The most efficient way to implement this would be in a manner that requires only the new data to be converted, translated and loaded. Then the temporalization step could be re-run as needed on the full FRI dataset collection.

The task is to implement an efficient update procedure and ensure its robustness using FRI datasets available to the Contractor.

The deliverables are:
(1) an efficient update procedure implemented using open source technologies that is well-documented, transparent and easily maintained, and
(2) an exhaustive list of FRI datasets, new and old, that could potentially become available to NRCan for loading into its version of CASFRI, and a comprehensive set of translation files for all FRI datasets available to Dr. Cumming for BAM research purposes as of April 1, 2020.

Time schedule:
- April 1, 2020, to March 31, 2021: write, test and validate CASFRI update procedure
- April 1, 2020, to March 31, 2021: prepare exhaustive list of FRI datasets that could become available to NRCan for its version of CASFRI, and write translation files for any remaining FRI datasets in the Client’s possession that have not been delivered to the Client yet
- March 31, 2021: delivery of CASFRI update procedure and documentation delivered to Client
Task 6 – CASFRI Parallelization

The CASFRI data processing steps described above will be data intensive because there will be at least 30 million polygons in NRCan’s CASFRI once it has been delivered and NRCan acquires access to FRI datasets from across Canada.

The task is to develop, test, validate and document a parallelization procedure that NRCan can use to run CASFRI. The Client has particular interest in running CASFRI on its own “Boreal Cloud” high-performance computing facility. The parallelization task may be completed in partnership with Compute Canada regional partner organizations such as Calcul Quebec, WestGrid or other, as deemed appropriate by the Client.

The deliverable is a documented parallelization procedure that NRCan can run on advanced research computing systems available to the department, such as the Boreal Cloud.

Time schedule:
- April 1, 2020, to March, 2021: develop, test, validate and document a parallelization procedure for CASFRI
- March 31, 2021: Deliver CASFRI parallelization procedure and documentation to Client
6. Trade Agreements

Applicable Limited Tendering Provision under NAFTA (Article 1016.2)
1016.2(b) - where, for works of art, or for reasons connected with the protection of patents, copyrights or other exclusive rights, or proprietary information or where there is an absence of competition for technical reasons, the goods or services can be supplied only by a particular supplier and no reasonable alternative or substitute exists;

Applicable Limited Tendering Provision under the Comprehensive Economic and Trade Agreement (Article 19.12) - if the goods or services can be supplied only by a particular supplier and no reasonable alternative or substitute goods or services exist for any of the following reasons:
(i) the requirement is for a work of art;
(ii) the protection of patents, copyrights or other exclusive rights; or
(iii) due to an absence of competition for technical reasons;

Applicable Limited Tendering Provision under Canada-Chile (Article Kbis-09)
Kbis-09 (b) - where, for works of art, or for reasons connected with the protection of patents, copyrights or other exclusive rights, or proprietary information or where there is an absence of competition for technical reasons, the goods or services can be supplied only by a particular supplier and no reasonable alternative or substitute exists;

Applicable Limited Tendering Provision under Canada-Peru / Canada-Colombia (Article 1409)
1409 (b) where the goods or services can be supplied only by a particular supplier and no reasonable alternative or substitute goods or services exist for any of the following reasons:
(i) the requirement is for a work of art,
(ii) the protection of patents, copyrights or other exclusive rights, or
(iii) due to an absence of competition for technical reasons

Applicable Limited Tendering Provision under Canada-Honduras (Article 17.11)
17.11 (b) a good or service being procured can be supplied only by a particular supplier and a reasonable alternative or substitute does not exist because:
(i) the good or service is a work of art,
(ii) the good or service is protected by a patent, copyright or other exclusive intellectual property right, or
(iii) there is an absence of competition for technical reasons;

Applicable Limited Tendering Provision under CFTA (Article 513:1)
513:1(b) – if the goods or services can be supplied only by a particular supplier and no reasonable alternative or substitute goods or services exist for any of the following reasons:
(i) the requirement is for a work of art;
(ii) the protection of patents, copyrights, or other exclusive rights;
(iii) due to an absence of competition for technical reasons;
(iv) the supply of goods or services is controlled by a supplier that is a statutory monopoly;
(v) to ensure compatibility with existing goods, or to maintain specialized goods that must be maintained by the manufacturer of those goods or its representative;
(vi) work is to be performed on property by a contractor according to provisions of a warranty or guarantee held in respect of the property or the original work;
(vii) work is to be performed on a leased building or related property, or portions thereof, that may be performed only by the lessor; or
(viii) the procurement is for subscriptions to newspapers, magazines, or other periodicals;

Applicable Limited Tendering Provision under Canada–Panama Free trade agreement (Article 16.10)
16.10 (b) the procurement can be carried out only by a particular supplier and a reasonable alternative or substitute does not exist because:
(i) the requirement is for a work of art,
(ii) a good or service being procured is protected by a patent, copyright or another exclusive right, or
(iii) of the absence of competition for technical reasons

Applicable Limited Tendering Provision under Canada-Peru Free Trade Agreement (Article 1409)
1409 (b) where the goods or services can be supplied only by a particular supplier and no reasonable alternative or substitute goods or services exist for any of the following reasons:
(i) the requirement is for a work of art,
(ii) the protection of patents, copyrights or other exclusive rights, or
(iii) due to an absence of competition for technical reasons;

Applicable Limited Tendering Provision under WTO-GPA (Article XIII)
XIII (b) where the goods or services can be supplied only by a particular supplier and no reasonable alternative or substitute goods or services exist for any of the following reasons:
(i) the requirement is for a work of art;
(ii) the protection of patents, copyrights or other exclusive rights; or
(iii) due to an absence of competition for technical reasons;
7. Title to Intellectual property

4006 (2010-08-16), Contractor to Own Intellectual Property Rights in Foreground Information
8. Contract Period

The contract period shall be from date of award of contract to March 31, 2021.
9. Estimated Cost

The estimated maximum value of the contract is between $330,000.00 and $340,000.00 CAD including all applicable taxes.
10. Exception to the Government Contracts Regulations and applicable trade agreements

Sole Source Justification  Exception of the Government Contract Regulations (GCR):

(d) Only one person or firm is capable of performing the contract

The contractor must:

1) At least ten (10) years of experience in the past fifteen (15) years creating, processing or analyzing FRI datasets for/from at least nine (9) different Canadian jurisdictions (provinces, territories);
2) Access to FRI datasets from across Canada, including at least 9 different Canadian jurisdictions (provinces, territories), for application design and testing purposes;
3) Possesses the knowledge and understanding of FRI attribution standards in at least nine (9) Canadian jurisdictions and FRI dataset formats used by at least nine (9) Canadian jurisdictions;
4) Possess the knowledge and understanding of open source programming and database technologies including Python, GDAL/OGR, PostgreSQL and SQL;
5) Be able to complete all deliverables mentioned in section 5.1 by March 21, 2021.
11. Name and Address of the Proposed Contractor

Université Laval
Département des sciences du bois et de la forêt
Pavillon Abitibi-Price
2405, rue de la Terrasse
Québec (Québec) G1V 0A6
12. Inquiries on Submission of Statement of Capabilities

Suppliers who consider themselves fully qualified and available to provide the services/goods described herein, may submit a Statement of Capabilities in writing, preferably by e-mail, to the contact person identified in this Notice on or before the closing date and time of this Notice. The Statement of Capabilities must clearly demonstrate how the supplier meets the advertised requirements.
13. Closing Date

Closing Date: December 6, 2018
Closing Time: 2pm EST

14. Contract Authority

Len Pizzi
Senior Procurement Officer
Natural Resources Canada
183 Longwood Road South
Hamilton, ON L8P 0A5
Telephone: (905) 645-0676
Fax:  (905) 645-0831
E-mail : len.pizzi@canada.ca