Item - 2023.AU2.5

The Audit Committee recommends that:

1. City Council request the Chief Procurement Officer to establish guidelines and / or criteria for client divisions to consider when implementing complex procurement strategies and / or significant changes to the approach for category management and strategic sourcing initiatives, and to test changes on a smaller scale such as through staggered or segmented approaches, where possible, in order to review lessons learned and address potential issues, shortcomings, and risks before implementing wholesale changes.

2. City Council request the Chief Procurement Officer to provide guidance for client divisions on avoiding concurrent or overlapping procurement processes where appropriate, and / or implement appropriate risk mitigation measures to address potential issues that may arise when running concurrent or consecutive procurement processes for the same or related services where there may be the same suppliers bidding for contracts where operational capability is a factor or evaluation criteria.

3. City Council request the Chief Procurement Officer to provide guidance for client divisions to document in the solicitation file the rationale for exercising reserved rights or discretion when conducting a negotiated request for proposal, and to record the rationale for exercising such rights or discretion in the project closeout report.

4. City Council request the Chief Procurement Officer to provide more guidance and / or training for client divisions identifying when it would be appropriate to:

a. Implement a request for supplier qualification (RFSQ) process in advance of a negotiated request for proposal (NRFP) process, to address the risk of an insufficient number of suppliers meeting technical requirements and to increase the likelihood that more qualified suppliers submit pricing proposals.

b. Cancel, review, and reissue an amended RFSQ and / or NRFP where an insufficient number of qualified suppliers meet the City’s requirements or where concentrating risks with few suppliers is not appropriate or acceptable to the City.

5. City Council request the Chief Procurement Officer to review the process and form used for checking references, and establish formal procedures or guidelines for how references are to be used to validate accuracy of solicitation responses and how information from reference checks can be clarified with referees and suppliers.

6. City Council request the Chief Procurement Officer to review the standard process rules (or template) for negotiated requests for proposals and related evaluation criteria for past experience and reference checks, and enhance guidance and/or training for client divisions to ensure solicitation requirements and scoring guidelines for evaluation teams are sufficiently clear.

7. City Council request the Chief Procurement Officer, in consultation with the City Solicitor, to review how affiliate relationships may impact the evaluation of a supplier’s past experience and provide guidance to Purchasing and Materials Management Division staff and client divisions on how to evaluate.

8. City Council request the Chief Procurement Officer to provide guidance for client divisions and ensure that standard process rules (or templates) for negotiated requests for proposals and related scoring guidelines avoid the use of a two-pronged, multi-scenario approach for a given evaluation criteria / sub-criteria wherever possible; and, where the use of a two-pronged, multi-scenario approach is unavoidable, ensure evaluation criteria / sub-criteria is clear and all possible scenarios have been considered and incorporated in the scoring guidelines.

9. City Council request the Chief Procurement Officer to provide further guidance and / or training for client divisions to:

a. Ensure that requirements in negotiated requests for proposals solicit enough information to enable the City to assess whether key risks impacting suppliers’ capability to meet the scope of work and deliverables have been appropriately addressed or mitigated.

b. Clarify the extent of information that can be requested by the City from a supplier or third parties to verify, clarify or supplement the information in a supplier’s proposal response submitted to a negotiated request for proposal, when evaluating technical proposals and / or undertaking negotiations.

10. City Council request the Chief Procurement Officer to provide more guidance to client divisions on “value-added services” to be included in suppliers’ technical proposal content requirements and evaluation criteria contained in negotiated request for proposal solicitation documents, to ensure the City is clear and consistent when communicating to suppliers what it is willing to negotiate and the range of alternatives it is willing to consider through proposals of “value-added services”, (including through addenda containing questions and answers and in examples of value-added services provided within technical proposal forms).

11. City Council request the Chief Procurement Officer to provide guidance and / or training to staff to make negotiated request for proposal process rules as clear as possible, and avoid or clarify process steps that may be interpreted inconsistently or cause supplier confusion.

12. City Council request the Chief Procurement Officer, in consultation with the City Solicitor, to review procurement policies and procedures and integrate additional procedural guidance specific to the implementation of more flexible, non-binding procurement methods (e.g., negotiated requests for proposals).

13. City Council request the City Manager, in consultation with the City Solicitor, to review Toronto Municipal Code Chapter 195, Article 10, as well as the Pre-Award and Post-Award Bid Dispute Procedure, to make recommendations to City Council regarding changes to address or clarify the circumstances or criteria where an alternate senior City Official should be designated to review pre- or post- award bid disputes.

14. City Council request the City Manager to ensure the review of Toronto Municipal Code Chapter 195, Article 10, as well as the Pre-Award and Post-Award Bid Dispute Procedure, to take into consideration any outcomes or recommendations related to the City Council-directed review of the potential for an Inspector General for the City of Toronto.

15. City Council request the Chief Procurement Officer to implement a policy or procedure which establishes the criteria for engaging an independent fairness consultant to monitor a procurement process. Documentation should be retained on file to support why a fairness consultant is or is not engaged before the issuance of such procurements.

16. City Council request the Chief Procurement Officer to provide guidance for client divisions to:

a. Review, document, and retain in the solicitation file the reasons for significant variances between pre-solicitation estimates and actual contract values.

b. Establish a baseline or perform a cost-benefit analysis when changing the sourcing strategy, procurement approach, or contracting model to better assess the actual outcomes achieved on new contracts.

Tara Anderson, Auditor General, gave a presentation on A Review of the Procurement and Award of the Winter Maintenance Performance-Based Contracts.

In December 2021, City Council approved the award of nearly $1.5 billion in contracts to provide winter maintenance services in 11 geographically based contract areas, over a 10-year term (i.e., seven years plus three optional renewal years). The five suppliers awarded contracts were selected through a negotiated request for proposals (NRFP) process.

When it approved the award of the winter maintenance contracts, City Council also added a review of the City’s NRFP process to the Auditor General’s 2022 Work Plan.

This report presents the results of the Auditor General’s review of the procurement and award of the winter maintenance performance-based contracts. The objective of this audit was to assess whether the City’s NRFP procurement process for the provision of winter maintenance services was conducted in a fair, open, and transparent manner, specifically:

·       Negotiated Request for Proposal Doc2970598171 (NRFP1)

·       Negotiated Request for Proposal Doc3136860258 (NRFP2)

·       Non-Competitive Procurement (NCP)

The timeframes established for the issuance of the call document, evaluation of proposals, negotiations, and award, were based on a single NRFP being sufficient. Many of the challenges related to this procurement likely arose from the unexpected need for a second NRFP process (and subsequent non-competitive procurement).

Taking into consideration the atypical circumstances surrounding this procurement, we conducted this review of the two NRFPs and NCP through the lens of identifying lessons learned and opportunities for continuous improvement for NRFPs going forward. For example,

·       Going forward, before adopting large-scale changes to its procurement and contracting approach for services, the City should consider testing out changes on a smaller scale and making adjustments based on the outcomes and lessons learned, where necessary. Overlapping procurements should also be avoided.

·       A key to successful performance-based contracting is conducting an effective evaluation of the suppliers’ ability to deliver required services during the procurement process. Based on our review of evaluation guidelines and records, we identified areas where more guidance and training can be provided to improve the evaluation of suppliers’ past experience and operational capability.

·       The flexible, non-binding NRFP format is an important tool that can allow the City to request and consider relevant information that supports the success of the contracts awarded through the procurement. Going forward, the City should ensure NRFPs solicit enough information to assess key risks impacting suppliers’ capability to meet its requirements.

·       As a result of our audit, we identified several procurement-related policies and procedures that should be clarified or strengthened, particularly taking into consideration the impact of the non-binding/flexible NRFP format.

The 16 recommendations in this report focus on continuous improvement and reinforcing and supporting the City’s ongoing efforts to make NRFP process rules, technical proposal requirements, and evaluation criteria clearer, with the goal of keeping as many suppliers as possible through all stages of evaluation and achieving the best possible outcomes and value for the City.