Benchmarking and Evaluation
Some of the most influential components of programming – benchmarking, consultation and evaluation – can be overlooked in multi-residential recycling program planning or first on the chopping block when resources are tight. Understanding the nuances and impacts of programming however, can substantially improve long-term success and the cost savings.
Benchmarking provides data upon which municipalities can build their programming; it enables goal-setting and the ability to measure the impact of specific initiatives. Read more
To move program performance forward effectively, you must first know where you are starting from. This is benchmarking; taking stock of where you are so that you can set targets and have something to measure against once program changes and initiatives are implemented.
Benchmarking in waste management is a quantitative measurement that can be obtained with an audit (material or financial based). In the multi-residential (MR) sector, material audit is measured in:
kilograms/unit or complex
Depending on the size, residential profile, waste collection model or the resources of your municipality, getting this information for each building is not always cost effective or easy to retrieve. There are, however, other performance indicators – obtained through the site visit – that can supplement benchmarking activities.
Using the Site Visit as Back Up
The initial site visit provides an obvious and appropriate time to assess a MR building or complex’s recycling behaviour.
Though a site visit evaluation provides estimates only, as it doesn’t provide a precise weight, it can yield accurate performance information (to within 10-15% of actual weights) if completed in a concise manner. Have staff use the Guide to Completing the Site Visit Form when completing a Site Visit Form to ensure reliable results.
Gathering this preliminary data for each building in your community will provide a broad understanding of your MR recycling sector as a whole, enabling you to make appropriate cost and program decisions.
A valuable step in recycling planning, consultation can ensure municipalities develop programming that resonates with residents, while preventing costs associated with misunderstood materials or ineffective outreach. Read more
Public consultation is integral to the effective design and delivery of municipal services. By consulting with citizens and other stakeholders, municipal managers can ensure the policy, plans, budgets and programs they develop are meaningful and understood.
In the MR recycling sector, public consultation can be further used to:
- Build general knowledge of municipal recycling expectations and practices
- Build the public profile of MR recycling specifically
- Increase buy-in and support for programming
- Identify and/or work through barriers to MR behaviour change
- Identify unforeseen costs
- Reveal operational issues and needs (of both residents and property managers)
- Trial P&E materials for resonance and comprehension
- Identify program champions, volunteers and partners
- Gauge resident support for proposed programming and policy
Whether consulting with one person or many, anticipate questions and concerns ahead of time and speak to potential issues before they are brought up. This can help avoid bad will, bad press, and reduce fear of change. Where appropriate, ensure the media and local Council is kept up-to-date, and in some cases, invited to your consultation.
MR consultation strategies can include:
- Open houses with displays, presentations and opportunities for feedback
- Staffed lobby displays soliciting feedback
- In-person, phone and online surveys
- Focus groups (or individual informational interviews)
- One on one discussions with key stakeholders
Visit the Depot Public Consultation Approach page for more best practices and tips.
Measuring and monitoring (M+M) performance is critical to program delivery and change. With it you can determine if your activities – and the resources applied to them – need correction mid-delivery or more importantly, have achieved their desired goals. M+M, along with qualitative program evaluation, can provide a comprehensive understanding of a program and its potential, resulting in well-informed, financially-prudent waste management decisions. Read more
Quantitative Data
Measuring and monitoring (M+M) in the waste management sector is the quantitative evaluation of program performance. A critical best practices step, and required task for CIF-funded projects, this activity, enables municipalities to determine if their programs are successful or not, and can highlight areas that need to be addressed.
To determine success, benchmarked data must be compared against interim and final measurements. The resulting numbers can be used to compare diversion, contamination or specific item (e.g. tied grocery bags) rates against program changes or new initiatives.
Performance measurements – secured through waste composition studies or audits – can identify numerical achievements (costs saved or addition tonnes diverted, for example). To complete the picture, they should be considered alongside qualitative information.
Qualitative Data
Qualitative information can provide the insight needed to truly inspire change. Conducted thoughtfully, it can provide you with an understanding of the living context, knowledge, needs and attitudes of the demographic group you wish to engage. And at times, with the aid of those interviewed, can unearth brand new ideas and approaches to behaviour change and barrier removal.
Qualitative data can be retrieved through surveys, focus groups, opinion polls, even a conversation with a multi-residential stakeholder.
Use both qualitative and quantitative data to inform your planning, policy and program development, budgeting, and program tweaking. They work best as a team.
Other Performance Indicators
Other performance indicators to consider when evaluating your program are:
- Revenue changes as a result of the initiative
- Staffing hours (and associated costs) required to provide programming
- Changes to capital or operating assets required (e.g. number of trucks, carts, etc.)
- Costs to sustain programming after initial implementation costs
- Feedback from contractors, residents, property managers/superintendents, and staff
- Greenhouse gas emission changes as a result of programming
Review the Guidance Document for Monitoring and Reporting of CIF-Funded Projects for assistance in identifying your performance criteria and planning your measuring and monitoring strategy.
Information to assist you with conducting a visual or physical waste audit can be found in Curbside Waste Audits: Considerations for Small Communities
Contact Jessica Landry with your questions.
Qualitative analysis during and upon completion of program delivery can highlight effective components, identify unforeseen barriers or challenges, and encourage creative problem solving and program progression. Read more
The value in the practice of program reflection should not be underestimated. Besides being quick and inexpensive, this activity, often referred to as “lessons learned”, can produce very instructive information by leveraging the knowledge, observations and experience of program coordinators and other stakeholders.
Conducted during and upon completion of program delivery, this qualitative program analysis can:
- Breakout effective and ineffective program components and strategies
- Identify unforeseen barriers, challenges, and costs, and
- Encourage creative problem solving
This practice is strengthened when conducted with a project team, with efforts to bring the experince of stakeholders (from conversations with property managers and residents, for example) to the table.
If conducting interviews, surveys or focus groups for the program (which is highly recommended), this information too, can be brought into the discussion.
This important evaluation step is more than a conversation, the ideas and observations generated should be noted formally, in meeting minutes and reports for example, with a plan for follow-up.
Program Reflection Activities
Use one of the following activities to generate a discussion amongst program delivery staff:
1. Three Simple Questions
Ask all stakeholders:
- What worked?
- What didn’t work?
- What should we try next time?
2. SWOT Analysis
Vet your initiative through a SWOT analysis. In this activity, staff use four topic areas to enable a comprehensive examination of their activity. By category, discuss the strengths, weaknesses, opportunities and threats of a program. Follow this analysis up with a brainstorming session to leverage or address the identified outcomes.
