TAKEAWAYS
The first part of this two-part article highlighted the growing importance of social impact measurement1 (Ang, 2023a; Ang, 2023b), underlining its role in promoting accountability, responsibility, and transparency to strengthen governance and trust within the people sector. It explored common challenges in applying Social Return on Investment (SROI) (Ang, 2023a) – such as managing long-term costs, navigating complex outcomes, and ensuring project coherence – while introducing key concepts of SROI theory and the balanced scorecard approach.
This article builds on the previous discussion on impact measurement for social good, examining initiatives aimed at reducing unemployment among working-age adults and fostering a more inclusive Smart Nation for seniors.
Based on my research, another common issue faced by society is the unemployment rates of vulnerable adults. In this second case study, we examine an organisation (DEF) that supports adults who have been unemployed for over three years. Currently, DEF is implementing a career coaching programme for these vulnerable adults. However, the organisation has received feedback that the programme is not as effective as anticipated, and requires re-evaluation. The career coaching team at DEF is now looking to reassess the programme to enhance its effectiveness.
Figure 5: Balanced Scorecard
Initially, DEF had identified income instability as the primary social issue they aimed to address. They then conducted research to determine the relevant output indicators, thereby identifying the best course of action. Using a Balanced Scorecard, they determined that two key indicators for income instability were unemployment rates, and rates of mental health disorders (Figure 5). Their research revealed that individuals with mental health challenges often struggle to maintain long-term employment. To revise their current intervention, the organisation explored various strategies to help people escape unemployment.
One of the approaches DEF considered was to improve the career coaching process by collaborating with other organisations, such as SkillsFuture, to provide job placements for the beneficiaries. The other approach considered was a community support campaign aimed at raising awareness about mental illness and, more importantly, about the resources available for individuals to seek support, as well as ways to encourage them to reach out to others within their networks.
On balance, to better support the beneficiaries, DEF decided to run a campaign focused on community support, and to create a space for such adults to come together to network and support each other.
Figure 6: SROI analysis: Post-intervention
Kicking off the project, DEF, while considering the need for programme re-evaluation, implemented the SROI framework and began identifying key stakeholders (Figure 6). They included vulnerable unemployed adults, mental health specialists, family members, and the volunteers/counsellors from DEF who worked directly with the beneficiaries.
The next step was to determine the programme’s outcome: to improve the mental health of the beneficiaries during their job search process, and indirectly improve their unemployment rates. Once the desired outcome was defined, the team extrapolated and developed the SROI metrics to be used. These metrics included the beneficiaries’ resilience in their job search, and their satisfaction with work.
To make accurate comparisons, the team collected data to establish a baseline. The information collected included the beneficiaries’ self-evaluation of their resilience and satisfaction with work, using a simple rating scale from 1 to 10, where 1 indicates most unsatisfied and 10 indicates most satisfied. Additionally, the team interviewed selected stakeholders such as family members, counsellors, and mental health specialists. These stakeholders were asked to rate their views of the beneficiaries using the same rating system that the beneficiaries used.
After establishing the baseline, the team at DEF implemented the support group campaign, ensuring weekly catchups and interviews with the beneficiaries. At the end of the campaign period, the team conducted exit interviews with the beneficiaries and evaluated their social impact. This evaluation involved comparing the beneficiaries’ self-evaluation of their resilience and satisfaction with work before and after the campaign, as well as the pre-campaign and post-campaign interviews with key stakeholders. Data analysis would be used to determine the significance of the change.
Following these steps, the team at DEF produced a report on the programme, determining its efficacy and outlining the next steps.
The final case study focuses on Organisation 123 (O123), which supports seniors. With the rise of the Smart Nation initiative and an increasing shift towards digitalisation, O123 has observed a growing trend of scammers targeting elderly individuals who possess limited technological skills. To address this issue, the organisation decided to tackle a gap within the Smart Nation framework. Through research, O123 identified three main areas that were lacking in the digitalisation of seniors: accessibility, participation, and ability.
Accessibility refers to whether seniors have the appropriate technological devices. Participation centres on whether these devices can access the necessary technology. For instance, for seniors to engage in the Smart Nation initiative, their devices must be able to use digital banking apps, connect to 4G networks, and access SingPass. Ability refers to the seniors’ technological knowhow, specifically whether they know how to use the technology to navigate their daily lives.
To address these gaps, O123 decided on the following solutions: providing upgraded smart devices, and improving digital literacy rates (Figure 7).
Figure 7: Balanced Scorecard
At present, scammers are exploiting the seniors’ lack of understanding about technology and its associated risks. By addressing digital literacy – specifically by educating seniors on digital hygiene and how to recognise scams – the number of elderly victims of digital fraud can be significantly reduced, thereby easing the transition towards a Smart Nation. In line with this, it is crucial that seniors have access to smart devices that are compatible with modern technology, allowing them to fully engage in the digital landscape.
Low-cost digital literacy programmes can be introduced to teach seniors basic technological skills, such as how to use PayNow to pay for groceries or how to participate in Zoom calls, enabling them to stay connected with friends in a digital environment. O123 has also proposed a tech exchange drive, where disadvantaged seniors can trade in their outdated 3G phones – soon to be phased out of network systems – for new 4G models at a subsidised rate. Volunteers will be on hand to teach the elderly how to use their new phones.
To address participation issues, a potential project was to collaborate with government agencies to create an app design competition. The goal of the competition would be to revitalise a government app, such as SingPass, with an interface specifically tailored to the needs of seniors. For instance, the app could feature a simpler design to avoid overwhelming users with information, as well as larger icons and text to ensure seniors can easily read and navigate the application.
In order to implement all three, it was determined that additional funding would be required, to provide subsidies for low-income seniors to participate in these efforts.
Through their internal evaluation, O123 decided to proceed with the digital literacy training programme.
Figure 8: SROI analysis: Post-intervention
To develop the SROI of the project, O123 first identified the key stakeholders involved: the seniors, volunteers, families and friends of the seniors, and trainers (Figure 8). The next step was to determine the desired outcomes of the programme to ensure it met the stakeholders’ needs. O123 stated that the main outcome of the programme was to improve the digital literacy rates of their senior beneficiaries.
With reporting in mind, they developed social ROI metrics to measure the digital competency of their beneficiaries. These metrics included the seniors’ ability to identify fraudulent messages from individuals pretending to be banks or the government, and the seniors’ confidence in using technology for daily activities, such as purchasing groceries. These competencies would be assessed using a simple matrix, where 1 indicates not digitally literate and 5 indicates very digitally literate.
Before implementing the digital literacy programme, it was essential to establish a baseline. This involved determining the digital literacy level of the beneficiaries before joining the programme, and tracking changes throughout the programme via biweekly evaluations from the trainers. To gain a comprehensive understanding of the seniors’ digital competency, interviews were conducted with selected stakeholders, such as family members of the senior beneficiaries, long-time volunteers who know the seniors, and some friends of the seniors. These interviews aimed to learn if the beneficiaries faced any issues using technology, such as PayLah app or SingPass, or if they had fallen prey to digital scams before.
After completing these preliminary steps, O123 implemented the digital literacy programme, closely monitoring the beneficiaries’ progress. At the end of the programme, they evaluated the social impact by comparing digital literacy evaluations from before and after the programme, using data analysis to determine any significant changes. Exit interviews with stakeholders were also conducted, and their feedback was considered in the evaluation. This comprehensive assessment helped O123 evaluate its digital literacy programme and make informed recommendations for its future, whether that involved investing more resources into it or altering the training curriculum to be more practical and applied.
To truly advance the effectiveness and accountability of social programmes, it is imperative that organisations adopt comprehensive social impact measurement models. My research has highlighted the critical importance of measurement, which can be summarised into three key points. Firstly, measurement enables an organisation to track and understand the progress it has made. Secondly, it helps organisations identify areas where they can improve. Lastly, measurement facilitates meaningful and accurate reporting, ensuring transparency and accountability.
By doing so, they not only enhance their ability to track and evaluate the outcomes of their interventions, they can foster greater trust with stakeholders through transparency and demonstrated impact.
I encourage leaders in the people sector to explore and implement frameworks like SROI and the Balanced Scorecard, adapting them to suit their unique organisational needs. This not only strengthens programme delivery but also supports sustainable development by attracting continued funding and support. Now is the time to commit to rigorous social impact assessment, and ensure that resources are used effectively to create lasting positive change.
Dr Ang Hak Seng, FCA (Singapore), is Professor of Social Entrepreneurship and Director, Centre of Excellence for Social Good, Singapore University of Social Sciences; and Adjunct Professor, Nanyang Technological University.
1 Ang, H. S. (2023a). “Creating And Qualifying Social Impact Investments For The People Sector”. IS Chartered Accountant Journal, Oct 2023, 26–31
Ang, H. S. (2023b). “Restructuring Charities: Impact Investment For The People Sector”. IS Chartered Accountant Journal, Sep 2023, 12–19
© 2024 Institute of Singapore Chartered Accountants
