Impact analysis

An impact evaluation can be used to measure change caused either directly or indirectly by the implementation of a National Water Safety Plan. It is designed to capture both intentional and unintentional changes, which can be positive or negative. This type of analysis assesses how well a water safety plan has addressed specific drowning-related issues it was designed to target. The results can be used to advertise the benefits of a water safety plan or conversely, to show a particular intervention implemented as part of the plan to not be suitable for the local context. An impact evaluation can also highlight individual aspects of a water safety plan which were not successful, which can be used to inform the development of a successive plan. 

As changes are being measured over time, the analysis must establish a baseline to compare water safety plan outcomes against. The baseline should be defined carefully and measured accurately before the water safety plan is implemented. It will be easier to recognize changes resulting from plan implementation if the same measurement methods are used to calculate the baseline figures and measure the outcomes of interventions that are part of the water safety plan. 

There are many impacts that a water safety plan may have; a large and complex plan targeting a number of population subgroups and a variety of drowning-related risk factors could cause a number of changes to the health of a population. It is therefore important to identify which impacts are most relevant to demonstrate and select these as a focus of the evaluation. Think outside the box and consider important indirect impacts a water safety plan can have, such as changes to quality of life or improved social convergence.   

Advantages

• Impact evaluation can generate quantitative data to clearly show that a National Water Safety Plan has been successful in reducing the impact of drowning on a population. 
• Positive results can be used to lobby governments and organisations to support and potentially fund the development and implementation of subsequent water safety plans. 
• It can demonstrate that a water safety plan has positive indirect impacts in addition to its intended effects. 

Disadvantages

• Baseline and final measures may be difficult or time consuming to complete. 
• It is easy to overlook some impacts, particularly those not directly linked to the water safety plan. 
• May be complex and costly to conduct, especially for large and complex plans. 

Context

An impact evaluation should be completed as part of the broader evaluation of a National Water Safety Plan, conducted after its implementation has finished. As  quantifying impact requires comparative baseline measures, the impact evaluation will need to be designed during the development of a water safety plan. These baseline measures should then be undertaken directly before the plan is implemented. While impact evaluation is important to measure the effectiveness of any intervention, it is particularly valuable for water safety plans which include innovative interventions or interventions which have not been previously implemented in a certain context.