Web Scraping Services

REQUIREMENT

Statistics Canada (StatCan) is seeking one (1) contractor to provide web-scraping services which can provide online pricing data.

The objective of this project is to obtain web-scraped data from a list of Canadian retailers that are in a structured format for Statistics Canada’s estimation of its Consumer Price Index. Statistics Canada is only interested in acquiring data, and not in purchasing software or a license to use an established software as a service (SAAS) solution.

Trade Agreements

The requirement is subject to the provisions of the World Trade Organization Agreement on Government Procurement (WTO-AGP), the North American Free Trade Agreement (NAFTA), and the Canadian Free Trade Agreement (CFTA).

Language

This Statistics Canada office provides procurement services to the public in their official languages.

CONDITIONS OF THE REQUEST FOR PROPOSALS (RFP)

There shall be no direct payment by Statistics Canada for costs incurred for the presentation and submission of proposals in response to this RFP.

ESTIMATED CONTRACT PERIOD

The contract will be awarded for one (1) year plus an additional three (3) one (1) year option period.

OPTION TO EXTEND CONTRACT

The Contractor grants to Canada the irrevocable option to extend the term of the Contract by up to three (3) additional one (1) year periods under the same conditions. The Contractor agrees that, during the extended period of the Contract, it will be paid in accordance with the applicable provisions as set out in the Basis of Payment.

Canada may exercise this option at any time by sending a written notice to the Contractor at any time before the expiry date of the Contract. The option may only be exercised by the Contracting Authority, and will be evidenced for administrative purposes only, through a contract amendment.

SOLICITATION/BID CLOSING DATE AND SUBMISSION OF PROPOSALS

Interested suppliers should express their interest in the procurement to the following;

Name: Francesco Germano

Title: Coordinator

Organization: Statistics Canada
Address: 150 Tunney’s Pasture Driveway, Ottawa, Ontario K1A 0T6

Telephone: 613-882-1902

E-Mail: statcan.macs-bids-smc-soumissions.statcan@canada.ca

The deadline for submissions of bids in response to this RFP is December 4, 2017 @ 2:00pm EST. Any and all bids received after this date and time will be returned unopened to the Bidder. As per Treasury Board Contracting Policy the onus is on the Bidder to ensure that all solicitation documents reach the stipulated destination before the closing date and time of the RFP.

Bids must be submitted only to:

MACS BIDS

statcan.macs-bids-smc-soumissions.statcan@canada.ca

StatCan will not assume responsibility for proposals directed to any other location. Proposals not received in the Bid Receiving Unit by the closing time, will be returned to the Bidder unopened.

ENQUIRIES

All enquiries must be submitted in writing to the Contracting Authority no later than five (5) calendar days before the bid closing date. Enquiries received after that time may not be answered.

Bidders should reference as accurately as possible the numbered item of the bid solicitation to which the enquiry relates. Care should be taken by Bidders to explain each question in sufficient detail in order to enable Canada to provide an accurate answer. Technical enquiries that are of a proprietary nature must be clearly marked "proprietary" at each relevant item. Items identified as “proprietary” will be treated as such except where Canada determines that the enquiry is not of a proprietary nature. Canada may edit the question(s) or may request that the Bidder do so, so that the proprietary nature of the question(s) is eliminated and the enquiry can be answered to all Bidders. Enquiries not submitted in a form that can be distributed to all Bidders may not be answered by Canada.

MANDATORY TECHNICAL CRITERIA

Mandatory Criteria (MC) Pass / Fail

MC1

Corporate Profile

The Bidder must include a description of the bidding organization.

The description must include:

• An overview of the business and the services offered to the private or public sector;
• Number of years of experience providing web-scraped data;
  • Location and contact information of the head office;
• Location and contact information of a Canadian office, if possible.

MC2

Project Officer

The Bidder must include a Project Officer who will be the main point of contact.

At a minimum, the following must be included:

• Name
• Telephone Number
• Email address

MC3

Project Plan

The Bidder must submit a Project Plan describing the approach and schedule for completing all aspects of the project tasks and deliverables.

The Project Plan must include:

• Evidence that the proposed team composite have a collective minimum of five years of experience in providing web-scraped data;
• A description of the software tools that will be used to meet the tasks and deliverables outlined in the Statement of Work and the number of years of experience using the tools;
• A schedule that outlines the stated objectives and requested deliverables;
• A description of how the work will be monitored, including quality control measures. If there were instances of any issues, errors or failures, describe the contingency plan(s) used to resolve them.

MC4

Example of Previous Project

The Bidder must demonstrate experience in at least one previous project of similar size and nature (a minimum of 15 websites, with similar frequency of collection and delivery) that was completed within the last 10 years.

The description must include:

• Overview of the project including the website(s) scraped, the number of unique product prices collected, the duration of the project, and the value of the contract;
• The team composition of the project providing web-scraped data;
• Identify and describe the software tools used to collect the data including the number of years of experience using the tools.

MC5

Sample Structured Data File and Record Layout

The Bidder must provide at least one sample structured data file and record layout with a minimum of 500 unique products from the website of a major Canadian retailer (e.g. Canadian Tire, Hudson’s Bay Company, SportChek).

The structured file must contain the following properties:

• Name and URL of the retailer
• Variables:
  • product price
  • product identifier (e.g. UPC, SKU)
  • product description
  • product characteristics
  • time and date of collection
  • URL associated to each product
• The variables must be properly named and all characteristics defined in the record layout;
• The values reported for each variable must be in the agreed upon format as defined in the record layout;
• Time and date of the scrape;
• In CSV format.

The record layout must contain the following:

• Name and URL of the retailer
• Variable name
• Variable position in data file (identify the column)
• Variable description
• Variable format (e.g. product price reported in numeric characters, dates as DD/MM/YYYY, all other variables reported as text characters)
• Variable length (minimum/maximum characters allowed).

SECURITY REQUIREMENTS

There is no security requirement applicable to this Contract.

Debriefings

Bidders may request a debriefing on the results of the bid solicitation process. Bidders should make the request to the Contracting Authority within 15 working days from receipt of the results of the bid solicitation process. The debriefing may be in writing, by telephone or in person.