Audit of CIHR's Management, Resources and Results Structures

Table of Contents


Executive Summary

Introduction

The Internal Audit of CIHR's Management, Resource and Results Structure (MRRS) is part of the 2012–13 Risk-Based Annual Internal Audit Plan, which has been approved by the Canadian Institutes of Health Research's (CIHR) Governing Council (GC).  A horizontal audit of MRRS is currently being completed by the Office of the Comptroller General.  The results of their work are expected to be available in December 2012.

The Canadian Institutes of Health Research

The Canadian Institutes of Health Research is the Government of Canada's agency responsible for funding health research in Canada.  CIHR was created in June 2000 under the authority of the CIHR Act and reports to Parliament through the Minister of Health.  CIHR's mandate is to "excel, according to internationally accepted standards of scientific excellence, in the creation of new knowledge and its translation into improved health for Canadians, more effective health services and products and a strengthened Canadian health–care system."  CIHR comprises 13 "virtual" institutes – each headed by a Scientific Director, who is assisted by an Institute Advisory Board - which bring together all partners in the research process – the people who fund research, those who carry it out, and those who use its results – to share ideas and focus on what Canadians need: good health and the means to prevent and fight disease.  Each Institute supports a broad spectrum of research in its topic areas and, in consultation with its stakeholders, sets priorities for research in those areas.  CIHR funds over 13,000 researchers and trainees in universities, teaching hospitals, and other health organizations and research centres in Canada and abroad.

Management, Resources and Results Structure

The Policy on Management, Resources and Results Structures (MRRS1) supports the development of a common government–wide approach to the identification of a department's programming activities and to the collection, management, and reporting of financial and non–financial information relative to those programs. The policy provides departments with the flexibility and discretion they need to design and manage their programs in a manner that best achieves results for Canadians.

The policy, issued pursuant to section 7 of the Financial Administration Act, reinforces the government's commitment to strengthen public sector management and accountability, consistent with the Management Accountability Framework, by providing a standard basis for reporting to citizens and Parliament on the alignment of resources, programs and results.  The objective of the MRRS policy is to ensure that the government and Parliament receive integrated financial and non–financial program performance information for use to support improved allocation and reallocation decisions in individual departments and across the government.

The establishment of a MRRS in each department is a key element of the Expenditure Management System because it provides a common framework within which financial and non–financial information is linked across government.

The deputy head is accountable to his or her Minister and to the Treasury Board for the establishment of an appropriate MRRS, for the management of the department and for public performance reporting using the MRRS.

The main components of a MRRS include:

Strategic Outcomes: measurable long–term and enduring benefits to Canadians that stem from a department or agency's mandate, vision and core functions;

Program Alignment Architecture: a structured inventory of all programs being delivered by a department or agency that are linked to its Strategic Outcomes, including a supporting Performance Measurement Framework, which sets out the expected results to be achieved and the specific outputs to be produced by a program; and

Governance Structure: the decision–making mechanisms, responsibilities and accountabilities within a department or agency.

CIHR's latest MRRS structure was submitted and approved by Treasury Board in 2009.  Based on this document, the Program Alignment Architecture (PAA) codes were also finalized and are used in both the financial and non–financial reporting carried out by the agency.

Risk Addressed by the Audit

In accordance with the Treasury Board of Canada (TB) Policy on Internal Audit, this audit addressed the risks, controls, and governance processes associated with CIHR's compliance to the MRRS Policy.  The risk addressed is that CIHR is not providing the government and Parliament with accurate and informative program performance information.  This risk is related to the following TBS Management Accountability Framework (MAF) elements:

Objective

The audit objective was to assess CIHR's compliance to the Treasury Board Policy on Management, Resources and Results Structures.  The audit provided assurance that Treasury Board and Parliament are receiving integrated financial and non–financial program performance information to support funding allocation and reallocation decisions from CIHR.

Scope

The audit focused on the Policy on Management, Resources and Results Structures and its implementation and application at CIHR in terms of integrated financial and non–financial program performance information.  It is management's intention to revise CIHR's current MRRS and PAA structure in fiscal year 2013–2014.  At that time management's intention is to align the strategic plan, MRRS, performance measurement, and the reporting framework.  Therefore, the audit relied on the present MRRS and PAA structure with the expectation that it can provide some "lessons learned" to assist with the development and implementation of the new structure.

Overall Audit Opinion

The audit has concluded that there are moderate issues with CIHR's adherence to the MRRS policy and the information contained in the MRRS, however exposure is limited because either the likelihood or the impact of the risk is not high.

Statement of Conformance

In my professional judgement as Chief Audit Executive, sufficient and appropriate audit procedures have been conducted and evidence gathered to support the accuracy of the opinion provided in this report.  The audit of MRRS was conducted in accordance with the Federal Government's Policy on Internal Audit and related professional standards.  It conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of a quality assurance and improvement program.

Summary of Strengths

Through the course of the audit, the following MRRS strengths were observed:

Summary of Improvement Opportunities

The following aspects of CIHR's MRRS require management's attention, should be considered when the MRRS is refreshed, as planned during the 2013–2014 fiscal year:

Internal Audit thanks management and staff for their assistance and cooperation throughout the audit.

Martin Rubenstein
Chief Audit Executive
Canadian Institutes of Health Research

Detailed Report

Methodology and Criteria

The internal audit of CIHR's MRRS was conducted in accordance with the Federal Government Policy on Internal Audit. The principal audit techniques used included:

Controls were assessed as adequate if they were sufficient to minimize the risks that threaten the achievement of objectives. Detailed criteria and conclusions are contained in the Appendix to this report.

The audit was conducted between June 2012 and September 2012.

Observations, Recommendations, and Management Action Plan

The following are audit observations, recommendations, and management action plans to address the weaknesses identified during the audit.

Observation Recommendation Management Action Plan
1. Management needs to adopt a governance process around MRRS that ensures regular and timely updates of its appropriateness.

Section 6.1.2 of the Policy on Management, Resources and Results Structures states that the Deputy Heads are responsible for "approving the departmental MRRS and ensuring regular and timely updates of its appropriateness."

While there is no formal process for senior management to regularly review CIHR's MRRS, there is an informal process embedded in management's review and approval of CHIR's Parliamentary performance reporting (i.e. RPP and DPR) both of which are organized around the MRRS.

2009 MAF Assessment

CIHR's 2009 MAF assessment identified some opportunities for improvement in regards to its MRRS. 

The issues noted during the assessment included:

  • clarification of the term "health research enterprises" contained in the strategic outcome
  • improve the program description for the items contained in the Program Alignment Architecture (PAA)
  • expected results for the performance measures often read as activities or outputs of the program, and CIHR was encouraged to modify the expected results in order to reflect a more outcome oriented statement

These issues have yet to be fully resolved. However, management is planning to refresh the MRRS during 2013-2014 fiscal year, and will be addressing them at that time.

Risk and impact

Notwithstanding policy requirements, a formal review and regular update of MRRS would help to ensure that CIHR's reporting to Parliament continues to be relevant and is as effective as possible.

1.1 When the planned refresh of the MRRS is completed in 2013-2014 fiscal year, it is recommended that CIHR's executive management committee review the document on an annual basis as a standing committee item.

1.2 It is also recommended that any changes identified during these reviews, or as a result of other assessments be addressed in a timely manner.

Responsibility

EVP

Action

1.1 As indicated, CIHR's executive management committee will review and update the MRRS in 2013-2014. This review will be based on observations made during the 2009 MAF Assessment and will be linked to the strategic planning process. It will then be the intent of the committee to review and update, as appropriate, the MRRS annually.

Expected Completion: Pending confirmation of Treasury Board timetable

1.2 As part of the annual review, a process will be developed (including timelines and responsibilities) to ensure any required changes to the MRRS will be addressed in a timely manner.

Expected Completion: July 2013

2. The MRRS document identifies data sources that are either no longer available, or not available in the frequency outlined in the document.

Although data sources for CIHR's Strategic Outcome have been identified, many are no longer available to CIHR, or are not available in the frequency noted in the MRRS.

This is evident in assessing CIHR's strategic outcome, "A world-class health research enterprise that creates, disseminates and applies new knowledge across all areas of health research."

For this key item, the MRRS identified Science-Metrix as the annual data source required to assess some of the aspects of the performance of CIHR's Strategic Outcome.  However, Science-Metrix has not been used to assess CIHR's strategic outcome for a number of years, as the organization no longer has the contract needed to access the required information.

Risk and impact

MRRS is relied on by Treasury Board and Parliament to strengthen their management and accountability of public expenditures by providing a modern expenditure management framework.  If the data sources identified in the approved MRRS are not available, there is a risk that the information and reports prepared will not be accurate or comparable with previous periods.

2.1 It is recommended that CIHR update the data sources available and required to produce the information identified under the MRRS framework.  In the process of assessing the information needs, the organization should also ensure that it has both the capacity and financial resources required to generate and utilize this information.

Responsibility

EVP

Action

As part of the MRRS refresh exercise (see audit observation #1) CIHR will develop and implement a structured Performance Management Reporting Framework. This framework will include key performance measures and indicators for the Institutes, Open Suite of Programs as well as the Strategic and Signature Initiatives. The Framework will also include a data collection and analysis plan (sourcing and ownership of data), the delivery of ongoing and timely corporate-level and program-level performance reports. CIHR's Executive Management Committee will be tracking this priority on a quarterly basis.

Expected Completion: March 2013

3. The performance plans for CIHR's senior executives do not fully reflect their accountability for the outputs and outcomes set out in the MRRS.

CIHR's senior executives abide by the same human resource process for performance management as other employees within the organization.  The annual performance management process for CIHR staff involves:

  1. Preparing an annual performance plan.
  2. Holding mid-year discussion with the employee's manager concerning performance to-date.
  3. Assessing the employee's actual performance versus their plan.

Section 6.1.7 of the MRRS policy states that the Deputy Head must ensure that senior executives are held accountable for the agreed outputs and outcomes set out in the MRRS.

In reviewing the performance plans and annual reviews for all of CIHR's senior executives over the last two fiscal years, it was not clear in the majority of instances where the link was between the performance plans and the outcomes and outputs identified in the MRRS. 

Risk and impact

MRRS identifies the outcomes the organization is trying to achieve for both the government and Parliament.  Therefore, if these outcomes are not accurately reflected in the performance plans of the senior executives there is a risk that they will not be assessed against the similar goals and outcomes as the organization.

3. When CIHR is developing and/or revising the MRRS document, it should consider outputs and outcomes that can be reflected and assessed in the performance plans of the organization's senior executives.

Responsibility

EVP

Action

Once the MRRS refresh exercise is completed, senior level executives' performance plans will be updated to include the outputs and outcomes set out in the refreshed MRRS.

Expected Completion: March 2013

Appendix

Audit Criteria and Conclusions

The audit uses the following definitions to make its assessment of the internal control framework.

Conclusion on Audit Criteria Definition of Opinion
Well controlled Well managed, no material weaknesses noted or only minor improvements are needed. 
Moderate issues Control weaknesses, but exposure is limited because either the likelihood or the impact of the risk is not high.
Significant improvements required Control weaknesses either individually or cumulatively represent the possibility of serious exposure.

The overall conclusion considers the cumulative risk exposure related to the audit observations in the context of the above criteria.

Overall Conclusion

The audit has concluded that there are moderate issues related to application of the MRRS policy, but exposure is limited because either the likelihood or the impact of the risk is not high.

Criteria Reference to Observations Conclusion
1. Strategic Outcomes – With guidance and leadership from deputy heads, departments have established clear and measurable strategic outcomes that provide a basis for horizontal linkages.
1.1 Departmental Management, Resources and Results Structure have clearly defined and measurable strategic outcomes that reflect the department's mandate and vision as well as being linked to the government's priorities.  (Policy on MRRS 6.1.1.1) No exceptions noted Well controlled
1.2 Departmental Strategic Outcomes provide the basis for establishing horizontal linkages between departments.  (Policy on MRRS 6.1.1.1) No exceptions noted Well controlled
1.3 Deputy Heads provide overall leadership in developing, in consultation with key stakeholders in the department, the changes required to the Strategic Outcomes and to the PAA.  (Policy on MRRS 6.1.4) Internal Audit Report Observations #1 Moderate Issues
1.4 Deputy Heads approve the departmental MRRS and ensure regular and timely reviews and updates take place to ensure its appropriateness.  (Policy on MRRS 6.1.2) Internal Audit Report Observations #1 Moderate Issues
2. Program Alignment Architecture (PAA) – Department has explained its PAA in sufficient detail to reflect how it allocates and manages its resources to achieve its intended results.
2.1 Related programs are identified, grouped and linked to the Strategic Outcomes they support.  (Policy on MRRS 6.1.1.2) No exceptions noted Well Controlled
2.2 Planned resource allocations, expected results, and performance measures are linked to programs against which actual results are reported and are used in monitoring performance and for decision-making.  (Policy on MRRS 6.1.1.2) Internal Audit Report Observations #2 Moderate Issues
2.3 The PAA is the basis for resources allocation at all levels of government.  (Policy on MRRS 6.1.1.2) No exceptions noted Well controlled
2.4 The Treasury Board Secretariat (TBS) approves the PAA level that allocates and controls resources.  (Policy on MRRS 8.1.1) No exceptions noted Well controlled
3. Governance Structure – Departments outline responsibilities and accountabilities for decision making mechanisms, monitoring and reporting.
3.1 The Departmental Management, Resources and Results Structure outline the decision making mechanisms, responsibilities and accountabilities of the department.  (Policy on MRRS 6.1.1.3) No exceptions noted Well controlled
3.2 Departmental information systems, performance measurement strategies, and reporting and governance structures support the MRRS.  (Policy on MRRS 6.1.5) Internal Audit Report Observation #2 Moderate Issues
3.3 Senior Executives are held accountable for outputs and outcomes set out in the MRRS.  (Policy on MRRS 6.1.7) Internal Audit Report Observation #3 Moderate Issues
3.4 Departments monitor their compliance with the MRRS Policy and notify the TBS of any changes they intend to make.  (Policy on MRRS 6.2.1) Internal Audit Report Observations #1 Moderate Issues
3.5 The MRRS is the basis for Reports to Parliament. (Policy on MRRS 6.2.2) No exceptions noted Well Controlled

Footnotes

Footnote 1

Policy on Management, Resources and Results Structure, effective April 1, 2012.

1

Date modified: