Design

Put together an evaluation team of interested or key staff and managers and begin by developing a ‘program logic’. A program logic (known as a Logic Model) clearly states why a program is being delivered, what issue it is addressing or changing, and what outcomes are expected. The logic model can then be used by your team to agree on an evaluation framework and methodology, depending on what you want to learn or discover.

When you have clearly documented your preferred evaluation approach, you will need to develop some performance indicators which are relevant, meaningful and which you have control over. These performance indicators will help you decide on your methodology. The methodology is the manner in which you are going to collect objective and reliable information related to the performance indicators.

Your evaluation team should decide how extensive the evaluation is going to be. The size of the evaluation depends on your aims, skills, resources, budget and time. If you work in an organisation that does not have the time or budget to engage a consultant, you will need to up-skill your program staff so they can conduct the evaluation themselves. If you work in an organisation needing to collect a large amount of data and utilise interviews or surveys to collect information, you can either choose to conduct the evaluation yourself, if you have the time and skills, or engage an independent consultant to conduct the evaluation for you.

Before you move on to the implementation phase, ensure you have gained formal support and endorsement of the proposed evaluation from your organisation’s board. Without formal support for your efforts, it is likely that the evaluation will be incomplete or selective in its investigations and therefore in its results. The outcomes will not be widely accepted and may lead to inaccurate recommendations.

Implement

Implement your chosen data collection methods and ensure an ongoing commitment to the evaluation. If even one staff member or volunteer does not really understand why they are collecting the data, your results may be inaccurate.

Collect the relevant data over an appropriate period of time. One month may be adequate for short-term projects, but one year may be more appropriate if you are evaluating a larger and ongoing program. Some evaluations may need data collection to occur at the beginning as well as the end of a project, or at regular intervals, and even after the project has finished (to assess long term outcomes).

In most cases, it is best to conduct a ‘pilot’ at an appropriate time after you have started collecting the data. A ‘pilot’ means testing your data collection and results to see if any problems need ironing out before you go any further. After the pilot, continue to monitor the ongoing progress of your data collection throughout the project. If you are collecting a large amount of data, or collecting data over a long period, ensure that it is still relevant and staff remain enthusiastic and clear about the process.

Reflect

Making sense of the information you have collected is the most important task! This is where you really discover if your planning and data collection methods were successful. There are many ways of collecting both quantitative and qualitative data, which means there are many ways of analysing and interpreting this data as well.

You need to assess the information collected and sort it into meaningful evidence. The manner in which it is presented will determine what conclusions are made. For example, if you present the findings that 80% of people completing a vocational course found work experience, it could mean a positive outcome and that your program was a success. However, if this 80% had problems at the work experience and left the placement before completing it, it might demonstrate that your program needs to do more life skill and preparatory work before sending the participants of the program for work experience.

Hopefully, you will find that the information collected confirms you have met your performance indicators, the number of outcomes produced, and what impact the organisation or program is having. The data may also reveal the efficiency and cost effectiveness of the program, including the total cost of delivering the identified outcomes.

Be careful that you do not fall into the trap of only collecting and reporting on positive outcomes and indicators. Evaluations that are objective and willing to state what did not go so well, or identify areas of weakness, are much more helpful and realistic. You will learn a great deal about a program by understanding its challenges and areas that need to be changed or modified.

Review

You should present the findings of your evaluation by organising the information you have collected, analysing it, and then describing the results accurately and objectively. How you do this will depend on the purpose of the evaluation and the needs of your primary audience.

If you decide to include recommendations in your final report, then there should be a clear relationship between the findings and your recommendations. Recommendations should draw directly on what you have learned in your evaluation and what you know about the program.

If possible, involve as many people from your organisation as possible to review the findings of the evaluation. Evaluations should help you to make informed decisions that benefit the organisation, enabling you to make changes to a service or organisation based on the findings. However, be aware that if your data is too ambiguous or unreliable, you could be making decisions based on a lack of evidence or subjective opinions and judgments. This could harm your organisation and cause conflict, rather than improving your services and programs.

After the review, return to the planning process, to evaluate whether your changes have improved your program.

Key points

  • Do not avoid evaluations because they take up too much time. The effort required far outweighs the benefit you will gain from completing an evaluation, however small
  • Start collecting information now. Do not overlook how important collecting information over the life of a program is to an evaluation process
  • Try to include qualitative data in your evaluation methodology. Quotes and personal stories are a powerful way to support findings from questionnaires
  • Avoid reporting just the successful elements of the program. Be objective and report about findings that can help you improve the program
  • Once your evaluation is complete, disseminate it widely. Be generous with your communication and help your funders and community partners learn from the process
The Department of Communities, Child Safety and Disability Services' (DCCDS) Community Recovery section is encouraging individuals, carers and service providers in the DCCSDS North and Far North Queensland area to be prepared in relation to Tropical Cyclone Debbie. The likely impacts of a tropical...
decorative
We have launched a brand new course on the Community Door eTraining website which provides an introduction to the social service sector in Queensland for managers and organisations new to the sector. It is free for anyone to access. It features a profile of the sector in Queensland, insights into...
Back by popular demand The State Councils of Social Service (COSS) are partnering to once again deliver a webinar series to uncover what every not-for-profit organisation needs to know about effectively responding to government procurement processes. The webinars will cover: Getting “tender ready”...
Case studies show social purpose organisations are reaping the benefits of improved process efficiencies. According to Olivia Hilton, Executive Director, SVA Consulting, “A lot of social purpose organisations just do not have the luxury of spending time wondering if there are other ways to achieve...
Are you a QLD-based professional who uses evaluation in their work? Are you someone who works in QLD and is interested in evaluating their programs or work? Are you someone who wants to know more about how to measure outcomes and work out if you are making a difference? Are you interested in...
The Early Childhood Early Intervention (ECEI) Approach is how the NDIS is delivering early childhood intervention for children aged 0-6 years. The overall aim of ECEI is to ensure that parents or primary caregivers are able to provide young children who have developmental delay or disabilities with...
agedcare101 is a new website covering everything you need to know about aged care. It is comprehensive, intuitive, reliable and free. You are almost certainly facing some new and challenging decisions, either about your own life or someone close to you. This is where agedcare101 comes in...
From August, the ACNC will be visiting over 25 regional and metropolitan locations in Australia to talk to charities. The Ask ACNC sessions will provide charities the opportunity to hear directly from one of the ACNC’s Commissioners. The sessions include an update on the work of the ACNC, issues...
The Guide for Community Financial Officers is designed to help Financial Officers of community organisations, of all shapes and sizes, to take on the role with greater knowledge and confidence and to help Board Members to understand what is required of a Financial Officer. This Guide is one of a...
Queensland Housing Profiles make it easy for you to create a profile of your community with maps, information and charts about residents, housing and housing markets. Data can be selected from a range of topics for multiple locations or regions across Queensland. You can view key figures about your...

Pages

Are you looking for support in Queensland, or trying to find a service that meets your needs? Now you can search oneplace , the service directory hosted by the Queensland Family and Child Commission. oneplace is an easily accessible directory of community services to help Queensland families to get...
The Australasian Evaluation Society (AES) is a member based organisation which exists to improve the theory, practice and use of evaluation in Australasia for people involved in evaluation including evaluation practitioners, managers, teachers and students of evaluation, and other interested...
The Queensland Disaster Management website and the Get Ready website have a range of information and useful resources that can assist you to plan and prepare for an emergency. As part of your preparedness and planning for weather events, access regular weather forecast updates from the Bureau of...
The Community Resource Handbooks were launched by Volunteering Queensland on 12 May 2015. They consolidate the knowledge gained through Volunteering Queensland's community leadership work with more than 2,000 community groups over the past fifteen years. The handbooks are aimed at small to medium...
Australian Institute of Company Directors has completed a comprehensive study that examines governance practices and opportunities in Australia's NFP sector. Read more
The Sector Capacity team at QCOSS has worked with the costing methodology developed and kindly shared by the Endeavour Foundation to create an easy to use Unit Costing Tool. This tool supports organisations to understand their cost drivers, and allows them confidently plan and negotiate the level...
The Queensland Council of Social Service (QCOSS) has produced the Planned Support Guide: an approach to case management for people working in a wide range of community services. If you are a front line worker, supervisor or manager this guide is for you. The guide assists in the development of a...
Information technology and online solutions can help your community organisation work better. ACOSS, Infoxchange and the Department of Communications have teamed up to bring you Digital Business Kits which give you the tools and information to do more online. Digital Business Kits are hosted on...
All special fund-raising events are different. Each have their own special set of circumstances and requirements. ourcommunity.com.au's event management team has pooled their many years of experience to create a fund-raising event checklist to help you get started on your planning journey. This...
Many not-for-profit organisations are now developing income-generating activities to reduce the reliance on government and philanthropic funding, and others are establishing social enterprises to respond to client needs in a sustainable way. The development of business skills and systems is very...
This is a very simple and easy to use toolkit that is filled with templates and guides to support your organisation to develop innovative and collaborative ways of working together. From developing a value statement, stakeholder engagement, business planning, SWOT analysis and many more. The...
Dr Mark Griffin gives a basic introduction to survey design for the not-for-profit sector. Mark is Director of the Australian Development Agency for Statistics and Information Systems. Although his talk focuses on developing surveys for international aid, it is useful for anyone starting to develop...

Pages

There are a range of legal structures which may be suitable for Queensland not-for-profit community groups. The four main options are: an incorporated association: Associations Incorporation Act 1981 (QLD) and Associations Incorporation Regulation 1999 (QLD) a company limited by guarantee:...
What do you need to succeed?
Author: 
Shelley Dunlop, QCOSS

Community Door hosts a number of useful tools and resources that can assist you with service delivery. These are all free and available for anyone to download. A few of the most popular tools are highlighted here.

...

Author: 
Tiffany Tento, Queensland Council of Social Service

National Reconciliation Action Week (27 May to 3 June) is celebrated in Australia each year, in commemoration of two significant milestones in the reconciliation journey – the anniversaries of the successful 1967 referendum and the High Court Mabo decision.

This year’s theme is ...

Results
Author: 
Angela Massey, QCOSS

By this time of year many New Year's resolutions may have already fallen by the wayside. Research suggests only about 8 per cent of those who set new year's goals actually stick to them. 

If you had big plans this year for development for you or your organisation, here are our top...

Decorative image
Author: 
Richard Fahy, Lutheran Community Care

Information, knowledge and process management focusses on capturing meaningful and relevant data to assist evidencing multiple areas of an organisation. A few key areas may include:

  1. The organisation is on track and demonstrating it is achieving its strategic goals
  2. The
  3. ...
 Charles Clowes
Author: 
Charles Clowes, Australian Organisation For Quality

Within the organisational development toolkit the capability area of quality systems, innovation and improvement refers to staff and client contribution, innovation, quality culture and management, reporting, adaptability and improvement, legislation and risk management.

The...

Author: 
Dr Ruth Knight, Zark Consultancy

Like many other community organisations, you are probably being increasingly asked to demonstrate the outcomes of your programs and services. Outcomes measurement allows you to identify which of your programs and services are successful, and helps you to test assumptions and communicate...

""
Author: 
Darren Smith, Breaking New Ground

A new chapter in the story of the not-for-profit sector is being drafted. How do small to medium organisations ensure they're written into the script?

The not-for-profit sector has already experienced significant change and uncertainty over the last few years:

  • funding
  • ...
Capability sign
Author: 
Len Airey, DCA Advisory Services

Strategic capability is about capacity, resources and skills that will enhance and improve the long-term sustainability of an organisation. Long-term sustainability, requires people to think differently about their future - not to look at the future through the prism of the past but to get all...

Author: 
Kylie Hogan, National Disability Services

The Community Services Industry is facing a myriad of reforms and challenges now and into the future. 

Challenges such as an ageing population, workforce shortages, sector-wide reforms, technological advances and economic uncertainty are making it more important than ever for...

The road ahead
Author: 
Valmae Rose, Queensland Council of Social Service

In the social service sector we are currently seeing a lot of interest and focus on outcomes. Given the desire to determine ‘value for money’ and other objectives of public investment in social services, there is renewed interest in the outcomes we are achieving, at a client and program level....

Pages

See videos from StudioQ related to this topic

Share or Print