TAKEAWAYS
In my previous articles1 (Ang, 2023a; Ang, 2023b) published in IS Chartered Accountant Journal, I had discussed social innovations within the social impact investment sphere. In this piece, I aim to expand on that discussion by providing case studies and elaborating on the social impact measurement models previously suggested.
Before examining the implementation of social impact measurement models, it is essential to first establish why measurement is necessary. Measurement must be purposeful, to ensure that we are evaluating the correct metrics and not measuring for the sake of measuring.
There are three primary reasons to measure social impact: accountability, responsibility, and transparency. These principles are crucial for fostering effective governance in the charity sector, and for building trust between non-profits and their stakeholders.
1) Accountability
Accountability refers to the duty of people sector organisations to own their actions and decisions. As entities governed by the Code of Governance for Charities and Institutions of a Public Character (IPCs), charities are obligated to adhere to stringent standards of governance, fundraising, and reporting. A robust social impact measurement framework allows charities to quantify their impact, demonstrating both financial and legal accountability.
2) Responsibility
Responsibility pertains to the duty of people sector organisations to fulfil the roles expected of them. Charities, as community-focused entities, must remain committed to their mission and vision, ensuring that their resources and actions consistently align with their stated objectives. Regular assessment of social impact allows charities to verify whether their efforts are advancing core goals. This process also enables them to make informed adjustments to activities that are not producing significant outcomes. Additionally, by providing evidence of programme efficacy, charities may attract more funding, particularly as donors increasingly prioritise organisations that demonstrate measurable results.
3) Transparency
Transparency involves the duty of people sector organisations to remain open and clear with their stakeholders, including regulators, volunteers, beneficiaries, and donors. This is especially important for charities that receive public funding and thus act as custodians of taxpayer resources. Being transparent about operations and financial reporting allows stakeholders to see how resources are utilised and how results are achieved, fostering confidence in the charity’s ability to deliver on its promises. This in turn enhances the charity’s reputation and can lead to increased financial support.
For these reasons – accountability, responsibility, and transparency – there is growing pressure on people sector organisations to conduct thorough social impact assessments.
To support the development of this capability, I previously advocated for the use of Social Return on Investment (SROI) as a model for social impact evaluation. However, before delving into the specifics of this model and its recent advancements, it is important to define social impact measurement and address the challenges associated with current methodologies.
The first question to ask is, “What is social impact measurement?” It is a framework and process used by organisations to quantify and attribute social change to their activities. This step is crucial for evaluating the effectiveness of social interventions and for enabling leaders to make informed decisions. As mentioned in previous discussions, I had suggested SROI as a potential method for measuring social impact.
However, given the way SROI is currently practised, its highly transactional and numerical approach may fail to capture intangible benefits. Moreover, SROI presents challenges in terms of implementation. Through my research, where I had focus group discussions with several local social service agencies (SSAs), I have narrowed down these implementation challenges to the following points: the cost of long-term measurement, complexity of social outcomes, and issues of coherence.
1) Cost of long-term measurement
The cost of long-term measurement can be prohibitive. Proper analysis of a programme’s social impact using SROI requires extensive data collection and management, necessitating significant investments of time, money, and human resources. Maintaining continuous stakeholder engagement – vital for reliable data – can be resource-intensive, especially over extended periods. Furthermore, as the project progresses, the volume of data naturally increases, resulting in a need to dedicate more resources to maintain and manage the data. These factors may deter organisations from adopting SROI for long-term social impact measurement.
2) Complexity of social outcomes
The complexity of social outcomes poses another significant challenge. Social outcomes are often multifaceted and influenced by a range of interrelated factors such as socioeconomic variables like job security and social mobility. This complexity makes it difficult to establish causal links between outcomes and specific interventions. Moreover, not all social outcomes are easily quantifiable. For instance, intangible factors, such as a beneficiary’s level of confidence, is difficult to quantify alongside more tangible outcomes, like household income.
3) Coherency of project
Coherence is a concern. As community-based organisations, charities are expected to implement programmes that contribute to societal improvement. However, relying solely on SROI may not adequately illustrate how a given programme aligns with the broader social mission of the organisation. Additionally, SROI alone may not ensure that the programme’s objectives remain consistent throughout the project’s lifecycle. Therefore, it is insufficient to rely exclusively on SROI to maintain coherence between an organisation’s mission and the specific goals of its initiatives.
Social balanced scorecard
To address these challenges, I propose supplementing SROI with the balanced scorecard approach. The balanced scorecard was initially created for businesses. To better cater to the social sector, I have adapted it and created the following template, also known as social balanced scorecard (Figure 1).
Figure 1: Balanced scorecard: Pre-intervention
The social balanced scorecard, which shall be subsequently referred to as Balanced Scorecard, should be employed during the project planning phase, prior to the intervention. Its primary objective is to create a framework that enables organisations to make causal claims about a programme’s social impact while simplifying the creation of relevant metrics. The Balanced Scorecard also allows for the summarisation of a project’s goals into a single visual diagram, facilitating clarity and ease of tracking.
To utilise the Balanced Scorecard effectively, it should be completed sequentially, from top to bottom. The first step is to identify the social issue (or outcome) the organisation seeks to address, such as poverty reduction. Next, a literature review should be conducted to identify the Pareto Principle indicators – those that represent 20% of the causes but generate 80% of the outcomes. These high-priority indicators are crucial for maximising impact. Following this, the appropriate interventions must be identified and mapped to relevant output indicators. The final step involves conducting a literature review to determine the input indicators (resources such as manpower or funding) necessary to achieve the desired outcome. By structuring the project in this way, organisations can draw clear cause-and-effect links between variables, thereby identifying the most impactful interventions.
SROI framework
Once the Balanced Scorecard is completed and the intervention is implemented, it should be followed by an SROI analysis (Figure 2), as outlined in my previous articles (Ang, 2023a).
Figure 2: SROI: Post-intervention
To recap, my SROI analysis strategy involves five key stages: (1) identifying stakeholders; (2) determining desired outcomes; (3) developing SROI metrics; (4) establishing a baseline through research; and (5) evaluating the social impact generated by the intervention (Figure 2).
To conduct a comprehensive evaluation, the Balanced Scorecard should be compared with the SROI analysis to determine whether the planned objectives were achieved. A successful programme implementation would align the Balanced Scorecard’s goals with the results reflected in the SROI analysis.
To better illustrate this, I would provide three case studies involving the following target groups – youths, vulnerable adults, and seniors.
The first case study centred on a youth-focused organisation (ABC) aiming to improve English literacy rates among primary school students. Families of these students have approached organisation ABC for help as ABC’s regular reading programmes were not sufficient for these students to gain a good enough boost in grades. As such, ABC assessed their interventions further, and determined that additional academic support via subsidised tuition may be a possible way forward.
Before identifying input indicators, such as the budget required to fund the programme, the organisation first analysed the programme to determine whether it directly addressed the social issue they sought to tackle. This was achieved through the use of the Balanced Scorecard. An example of a completed scorecard might appear as follows (Figure 3).
Figure 3: Balanced Scorecard: Youth
Once the team at ABC had chosen subsidised tuition as the main intervention method, they developed a work plan based on the SROI framework (Figure 4).
Figure 4: SROI analysis: Post-intervention
ABC first identified the key stakeholders of the digital literacy programme, which, in this case, included the children themselves, their parents, school teachers, and classmates. ABC then determined that the primary mission of the programme was to improve English literacy rates, with a secondary mission of instilling life skills, specifically promoting a growth mindset in the students.
After determining the desired outcomes, the next step for ABC was to develop social ROI metrics for the programme. These metrics included the students’ English test scores, which were obtained from their test results, and the students’ confidence levels in learning, which were measured through pre-programme interviews.
To accurately assess the social impact generated by the programme, ABC needed to establish a baseline by collecting several key pieces of information. Firstly, they recorded the students’ English test scores before, during, and after the programme. Secondly, they interviewed selected stakeholders, such as parents, school teachers, and classmates, to evaluate the students’ attitudes towards studying. These stakeholders were asked to rate their perceptions of the students’ confidence on a scale from 1 to 10, with 1 indicating no confidence and 10 indicating high confidence.
With these preliminary steps completed, ABC proceeded with the tuition programme. At the end of the programme, the team conducted exit interviews with the students and all previously engaged stakeholders, to evaluate the social benefits created by the tuition programme, and compared it with other academic programmes offered by ABC. To achieve this, the team compared the students’ English test scores before and after the tuition programme, as well as the interview scores from both pre- and post-intervention. This comparison was conducted using data analysis to accurately determine if there was a significant improvement in the students’ grades and confidence levels.
As shown, the SROI process guided the team in structuring the overall work plan and identifying the key variables for data collection. Upon completing the SROI analysis, it would serve as a foundation for drafting the final report.
It is essential that the report is communicated to the relevant stakeholders, ensuring they are fully informed about the programme’s impact. Such transparency may also encourage stakeholders to provide additional support, helping to ensure the programme’s long-term sustainability.
In Part 2 of this article, we will apply the same framework to two additional cases: one focusing on vulnerable adults and the issue of unemployment, and the other exploring how an inclusive Smart Nation can better support seniors.
Dr Ang Hak Seng, FCA (Singapore), is Professor of Social Entrepreneurship and Director, Centre of Excellence for Social Good, Singapore University of Social Sciences; and Adjunct Professor, Nanyang Technological University.
1 Ang, H. S. (2023a). “Creating And Qualifying Social Impact Investments For The People Sector”. IS Chartered Accountant Journal, Oct 2023, 26–31
Ang, H. S. (2023b). “Restructuring Charities: Impact Investment For The People Sector”. IS Chartered Accountant Journal, Sep 2023, 12–19
© 2024 Institute of Singapore Chartered Accountants
