Societal engagement and impact are currently a highly topical and widely discussed subjects in HEIs. Strategic plans often emphasize the topic as a key objective for research and education, as well as a long term benchmark, and societal engagement and impact have become one of the key evaluation criterias for higher education accreditation organizations and providers of funds. Depending on the size of the university,
societal engagement and impact is treated on a regional, on a local or on a global level. HEIs see the topic as highly
important, yet the communication of societal engagement practices and impact is rather scarce. Next the societal impact is first discussed through findings from analysis of selected European cases followed by the key insights from qualitaive studies on HEis in Estonia and Finland.
International cases in Europe - Current situation
Societal impact is a key value for universities and higher
education institutions. Most of them address it in one way
or another in their mission statement or elsewhere on their
websites. However, comprehensive and clear communication
to an outside audience seems to still be a challenge for
institutions, and this might be the result of different causes.
Technical universities seem to have a stronger motivation to
present their societal impact than arts universities.
One of the options for presenting societal impact is provided
by other platforms, for example under research program commucation, which might help the HEIs to become
more public about the projects they are implementing. A presentation
to a wider public can result in a more interested
audience, and furthermore can serve as an inspirational example
of good practices for other institutions to implement similar
projects. The management of HEIs might also fund good
projects, which is an input for doing more and increasing
engagement. Providing a platform for communication and
exchange is another potential improvement that can lead to
a more intense debate around the issues involved. The conditions
need to be provided by the highest board of the HEIs,
so as to pave the way for all levels of stakeholders to interact
and develop societal engagement policy.
This desk research initiated by the HEISE project explores
how HEIs define their societal impact; in which documents
they consider their societal impact; what indicators show
they communicate their societal impact, and what are some
of the examples presented.
...
In the early stages of the preparation for this desk research, the challenge of sample selection was encountered. The most prominent examples were from outside Europe, which for this case was not the aim of the research. As the entire HEISE project focuses on the situation in Europe, universities from other continents should not be part of the sample. It should be stated here that HEIs in the United States and South Africa have a strong communication strategy with regard to their societal impact, with publications, courses, programs etc. employing these keywords. HEIs in Europe meanwhile is not that well represented. The aim was to choose examples from different European countries, excluding the countries which are carrying out this project (Estonia, Finland and Spain), to get a broad image of the situation in Europe. The samples for this research were obtained through convenience sampling—the universities were chosen according to their availability online.
In the preparation for the choice of samples it became apparent that within Europe universities in Great Britain and Ireland have better strategy for communicating their societal impact projects and goals. Documentation, including reports, concerning societal impact is easily findable and accessible. At this point it is important to say that this might be influenced by the language diversity of Europe, as many HEIs present the majority of their projects in the native language of their country.
The final sampling was done through purposive sampling, according to search engine rankings.
To assess the universities’ societal impact,
only data provided by them online was
used. Several tools of measurement were
used uniformly in all the examples. Among
others were the examples of good practice
presented on the website of the university,
the university’s own statement about societal
engagement and their role in society, and
the number of results found on their website
connected to certain keywords.
A sample of eight universities from seven different European
countries was chosen:
1. Trinity College Dublin
2. National College of Art and Design Ireland
3. Tilburg University
4. University of Iceland
5. Bilgy University Istanbul
6. Spiru Haret University Bucharest
7. Oxford University
8. Augsburg University of Applied Sciences
A communication issue was encountered in most cases.
Mostly, the path to finding specific examples of societal engagement is complex and takes effort. Furthermore,
in Germany and Ireland, for example, good practices were presented through other platforms, for which
societal Impact was the main focus. Referral links to the universities themselves were mostly missing from
these platforms. A good illustration of this issue involves Augsburg University. On the website of the institution
itself there are only a few examples of good practice, while in other media there is evidence of many projects
which were carried out under the umbrella of Augsburg University, but not presented by the university itself.
This might be due to different reasons in each case, but the importance of the topic does not seem to be crucial
for most of the institutions. Nevertheless, efforts to keep the public updated about recent projects need
investment and can be improved overall.
Societal impact is the main value of the university.
The universities are aware of the importance and the responsibility they as institutions have towards society.
All of the universities address it in one or the other way in their mission statement, and some have separate
sub-chapters, where they dig deeper into the topic. However, while it is a topic touched on by all the institutions,
this in some cases is the only moment when they refer to it.
Project descriptions of good practice mostly lack concrete tools.
The values, aims and outcomes are stated, but they may not be specific enough to serve other parties as concrete
inspiration. It might be useful to give readers the option of going deeper into the topic, to invest more in
the documentation of projects and practices, in order to provide practical advice for people who are interested.
...
This brief review may not capture the whole picture, as the data might be interpreted in a different way. Nevertheless, it can be stated that communication towards an outside audience is still a challenge for institutions, as it needs special resources. An important contribution is made by other platforms, which might be a good starting point for HEIs to make more public the projects they have carried out. More public representation may also bring the university a more engaged following, and furthermore it can serve as inspiration for other institutions to do similar projects.
Introduction
This research was initiated by the HEISE project and was implemented by Laurea
University of Applied Sciences and the Sibelius Academy at Uniarts Helsinki in
Finland in the spring of 2018. Six semi-structured interviews were conducted with
managers and experts in the field of higher education. This particular study provides
illustrative and informative insights into different aspects of societal impact and captures
a general overview of the current situation.
Background on societal impact in Finnish higher education
Since 2005, societal impact has been one of the focal points in the qualitative auditing
of higher education institutions. The Finnish Education Evaluation Center (FINEEC)
evaluates the societal impact of higher education institutions from a procedural
point of view: How is the setting of goals and objectives organized, and how are they
linked with the core tasks of the university (teaching, research and development, regional
development)? Furthermore, evaluation is concerned with how management
and other staff and students interact with society. In the current third round of qualitative
auditing, social impact is a particular focus, testing the boldness of HEIs to
experiment as well as to innovate in all three tasks.
...
The Finnish Education Evaluation Center does not define what the goals and objectives of HEIs should be, but it is interested in how to build and integrate those goals into various activities. The idea is to support higher education institutions in determining societal impacts based on their profile and to translate this into a higher education culture. Also relevant is how the higher education institutes are able to demonstrate the societal impacts of their own activities and thus show evidence of their societal impact.
Introduction
This research was carried out by the HEISE team of the Estonian Business School
in spring 2018. Six semi-structured interviews were conducted with senior managers
and experts in the field of higher education, of which four represented major universities
in Estonia. These semi-structured interviews served three main aims: 1) to
study the prevailing perceptions and variations in understandings of the concept of
societal impact, 2) to map and study current practices in the evaluation/assessment
of societal impact, and 3) to study whether and how societal impact is used as a decision-
making criterion in the managerial decision-making process in HEIs. A summary
of the main findings and insights into different aspects of societal impact are provided
in the following sections.
Serving the public via engagement in the solving of societal issues has become
firmly established as a third major criterion (alongside teaching and research
and development) for evaluation of the performance of HEIs and their academic
staff among the HEIs interviewed in Estonia. While HEIs keep track of
and report their societal engagement, the HEIs seem to have only a relatively
limited understanding of their societal impact, even though societal impact is
reportedly evaluated in one form or another by all the HEIs interviewed. The
concept of societal impact is not understood uniformly by HEIs, but is subject
to various interpretations. Concept-related knowledge and methodological
know-how seem to be rather superficial in most of the HEIs studied. The path
to impact and causality is barely or inadequately addressed. The focus is often
on outputs rather than on impacts. A tendency to report positive intended
impacts and disregard costs to society is quite widespread. The motivation for
HEIs to engage in SI assessment seems to stem largely from external requirements
(accreditation), rather than from internal needs. Financial considerations
seem to dominate consideration of societal impact in decision-making
processes.
Stakeholders’ perceptions and understandings of the concept of societal impact
The stakeholders of the HEIs understood the concept of societal impact as a management
issue strongly linked to the strategy and to the implementation of it in the
institution’s core tasks. This is understandable because of the Finnish higher education
qualitative auditing system described above.
When considering the societal impacts with other stakeholders within the field of
higher education, interesting complementary viewpoints emerged in discussions. In
assessing societal impact you eventually return to the fundamental values of society.
For example tolerance, multiculturalism, inclusion, and preventing radicalization are
all issues we want to promote.
...
According to those interviewed, social impact concerns various levels: individual, functional, project, organization and society - both locally, regionally and globally. The impact of a single person can (also) be generated, for example, as the result of active participation in (social) media and in collaboration networks.
The respondents acknowledge that, aside from intended and positive impacts, unintended as well as negative impacts should also be accounted for when estimating societal impact. Furthermore, they distinguished between impact, outcomes and output and recognized that there is a need to look both at long-term changes and at short-term processes.
Mapping of stakeholders’ current practices in measuring societal impact
All higher education institutions in Finland evaluate societal impact on a regular basis
due to compulsory audits by FINEEC. The basis for assessing the effectiveness of
Finnish universities lies in their three main tasks: teaching, R & D and regional development.
For this reason, it is natural that this triangle also acts as a starting point
for measuring social impact.
Higher education institutions conduct societal impact evaluation both externally and
internally:
1) National evaluation is done every six years by the Finnish Education
Evaluation Center. In this evaluation the focus is on the level of the whole organization.
2) Other evaluations carried out for specific purposes, both internally and
externally. In this field of assessment, not only project-specific evaluations are conducted.
Many higher education institutions also use external evaluators, for example
at three-year intermediate evaluations.
The performance management of the Ministry of Education and Culture determines
certain indicators for higher education institutions. Additionally, each organization
has, in practice, other (including qualitative) indicators/key performance indicators
for internal use. Most of the indicators concern outputs and outcomes rather than
long-term impacts, but not all. ...
According to the interviewees, more qualitative indicators are needed to assess societal impact; currently the focus seems to be too much on performance indicators defined by the ministry. In addition, one of the challenges is that HEIs do not necessarily always take into account and evaluate unpredictable, untargeted and negative impacts. Furthermore, although it is understood that societal impact concerns long-term changes, it is evaluated through short-term quantitative outcomes. The reason for this is the challenge of evaluating long-term impacts, especially the causalities in the long-term. Therefore, it is also understandable that the perceptions within organizations of societal impact can vary. Further discussion on societal impacts is therefore vital.
Research and development is the area where the impact metrics are at the best level. The reality is that the financers of big projects ask that societal impacts be considered in project proposals. Based on the interviews it seems to be the case that indicators of regional development and collaboration with stakeholders are also numerous. What is also remarkable is that the Sustainable Development Goals of the United Nations are also adopted by some higher education institutions.
Students’ experience is one of the areas of qualitative assessment of societal impact that should be further developed, as well as indicators of sustainable education (employment indicators based on career-long feedback, for example). Furthermore, it would be helpful to be able to collect information more broadly to get an overall picture of the question: "What is the impact of Finnish higher education?" Information is now collected at the HEI level. The Finnish Education Evaluation Center already provides metasynthesis, but this work could be more systematic.
Cultural education has been, in many respects, an innovator of operating models in the higher education field. At the same time, cultural education has had to demonstrate its legitimacy using a variety of indicators. The projects carried out by cultural students in local districts have also helped to illustrate that a university´s regional development can also be activities implemented in collaboration with the districts and their inhabitants.
Usually, the measurement of societal impact is not done in monetary terms. All the organizations interviewed for the study are operating in the public and non-profit sectors, which indicates that success in general and success in societal impact cannot be reduced to numbers or money only. However, numbers are clearly important indicators for many organizations, but instead of money, the numbers relate to the number of graduates, publications, memberships in national and international networks, etc.
Even if not all educational organizations are explicitly and systematically measuring the long-term impact of their activities, they do keep track, for instance, of where their graduates end up working and how they value their education after graduation through the national Bachelors, Masters and Doctors Surveys that are annually distributed to all graduates in Finland.
Educational organizations feel that qualitative information about the societal impact of their operations is not valued enough, although that type of information would be crucially important to understanding the variety of impacts each organization is making. They also lack knowledge of different ways – both quantitative and qualitative – of measuring societal impact, which hinders the creation of systematic procedures for measuring societal impact.
Stakeholders’ perceptions and understandings of the concept of societal impact
The results of semi-structured interviews clearly reveal that the term (concept) “societal
impact” is rather differently understood by the representatives of higher education
institutions interviewed. The multiple and often diverging interpretations of the
term “societal impact” seem to stem from the varying degree of (in-depth) knowledge
of the concept, and from the limited attention paid by HEIs to this concept. This is
revealed by the diversity of definitions of “societal impact” offered by interviewees,
as well as by the answers to specific questions which were aimed at clarifying the
understandings (perceptions) of interviewees in a more detailed and systematic way.
Defining “societal impact” turned out to be a difficult task for most of the interviewees.
None of the definitions offered by interviewees contained simultaneously two of the
most important key words associated with the term – “change“ and “social welfare“.
In fact, “change” was never mentioned, while “social welfare” appeared in only one
of the definitions offered. All this seems to suggest that, overall, the concept of SI has
not yet captured sufficient attention in the institutions studied.
...
According to interviews, the multiple understandings of the term SI seem to stem mostly from divergent views on the type of impacts that are relevant for the analysis of SI. The following short summary illustrates the prevailing understandings of the term and the deviations from them.
Intended and unintended impacts. A prevailing, although not unanimous, understanding among the interviewees is that both types of impacts–—intended as well as unintended—should be considered when assessing the SI. This is in line with the suggest ions from conceptual and normative studies of SI. A different viewpoint was expressed by one respondent, who considered only intended impacts to be relevant for the SI assessment. Interestingly, but not surprisingly, such a biased view was held by the representative of an institution in charge of allocating public funds and monitoring their targeted use. This biased viewpoint has probably been induced by pragmatic considerations, as the effectiveness of the use of allocated funds is primarily assessed by these institutions based on the achievement of set (intended) goals by the recipients. But it can also be explained by the type of SI evaluations carried out by this institution, namely ex ante evaluations of SI.
Positive and negative impacts. The majority of interviewees agreed that assessment of SI should focus both on positive and negative impacts. However, according to the opinion of two interviewees, only positive impacts are relevant. Unfortunately, the reasons for this viewpoint were not explained. Hence, it is unclear whether such views are of a conceptual type or induced by the pragmatic considerations of respondents.
Material and non-material impacts. According to interviewees’ prevailing understanding, both material and non-material impacts should be taken into consideration in SI assessment. However, there were also different opinions expressed in this regard. One respondent suggested focusing on material impacts only, while another proposed considering only non-material impacts. All interviewees agreed that the impacts studied should not be limited to quantifiable impacts only.
Monetary and non-monetary impacts. The majority of interviewees share the viewpoint that both monetary and non-monetary impacts should be addressed in the process of SI assessment. However, there was also the opinion that SI assessment should include only non-monetary impact. This opinion was expressed by the same respondent who suggested focusing on non-material impacts only. Although the reasons for their opinion were not clarified, it seems to suggest that some organizations (such as the interviewee’s) distinguish between financial impact and SI, with the latter associated with non-material and non-monetary impacts only.
Direct and indirect impacts. All but one of the interviewees agreed that both direct and indirect impacts should be accounted for in SI assessment. The dissenting opinion—that only direct impacts matter—was expressed by the interviewee who had also adopted the view that SI assessment should focus on intended impacts only.
Short-term and long-term impacts. The results of interviews suggest that SI is predominantly associated with long-term impacts. All the respondents agreed that evaluation of SI should focus on long-term impacts. Only a few also acknowledged the importance of short-term impacts. It was also pointed out that the appropriate time-horizon of impacts could depend on the context and objective of evaluation.
Societal impact vs. social impact. The interviews reveal that the terms “societal impact” and “social impact” are more often viewed as different concepts rather than as synonyms, even though in the literature these two terms seem to be more often used as synonyms. Those who distinguish between these two terms usually view social impacts as a part of societal impacts.
Output vs. outcome vs. impact. All interviewees agreed that the terms “output”, “outcome” and “impact” should be viewed as different concepts. However, clear and consistent distinction between these terms seems in practice to be a difficult task for many. When asked to provide examples of SIs, the respondents often confused impacts and outcome with outputs. The revealed difficulties in distinguishing between impacts and outputs suggests that there are still significant shortcomings in the understanding of the concept of SI in many of the organizations studied.
The conceptions of causality varied among the interviewees. According to most of them, verifying causalities would be interesting in theory, but not (always) possible. However, some stakeholders thought that nowadays it would be possible to an increasing extent.
In addition, societal impact seems to be highly context-specific and depends on the mission of the organization. This means that every organization needs to create their specific way of defining societal impact, which should be carefully aligned with the mission and strategy of the organization.
As this short review of the results of interviews reveals, people and organizations
have diverse understandings of the concept of SI. This is consistent with the existing
literature on SI, which also reveals a multitude of understandings. For many, the concept
of SI is still a vague concept. According to interviewees, some of the measures
that could improve understanding of the concept of SI and increase its importance in
society would be: 1) wider involvement of academics and specialists in the process
of determining the strategic priorities of the state; 2) the introduction of compulsory
SI assessment at the state level; 3) distribution of information on SI, clearer communication,
and promotion of public discussions about SI; 4) improvement of communication
between stakeholders in order to negotiate their goals and assess their
contribution.
Mapping of stakeholders’ current practices in evaluation of societal impact
SI is evaluated in one form or another by all the organizations interviewed. The motivation
for organizations to engage in SI assessment stems from internal needs as
well as from external pressure and stakeholders’ expectations. According to the vast
majority of interviewees, the need to assess SI arises primarily internally. Assessment
of SI is seen as an activity, which: 1) contributes to the elaboration of plans for
the organization’s future development (making it possible to clarify the organization’s
position/role in society, and assess the effectiveness of the organization’s activities
and efficiency of the use of resources); 2) increases the organization’s public presence
(image); and 3) provides justification for the organization’s material needs
and (public) funding. However, the need to undertake SI assessment is often also
determined externally. Externally, the mapping and assessment of SI is imposed by
funders and also by higher education accreditation institutions, which use SI as one
of the evaluation criteria in the evaluation of HEIs and their study programs.
...
Organizations primarily focus on the SI of the organization as a whole. SI of individual projects, programmes or activities is studied rather selectively. Both ex ante and ex post evaluations of SI are practiced. However, ex post evaluations of SI, which are targeted at a wider range of stakeholders, seem to dominate over ex ante evaluations of SI.
The responsibility to assess/report on SI is most often assigned to senior managers, heads of departments and communication managers, but in some organizations also to middle-level managers and project managers. In universities, SI evaluation and reporting is also envisaged at an individual level, but this requirement applies to a limited number of academic job positions only. For the majority of organizations interviewed, public reporting and disclosure of the organization’s SI is not yet a regular practice. According to one frank interviewee, their organization’s SI is reported and disclosed to the public only if there is a positive impact to report.
At the same time, according to interviewees, evaluation of SI (in contrast to its public reporting) is a fairly regular practice in the majority of organizations interviewed. Those organizations which regularly apply mapping and assessment of their SI typically undertake it with annual frequency in connection with the preparation of their organization’s retrospective annual reports and forward-looking (annual) plans. Regular assessment of SI is not always purely voluntary, but often imposed (prescribed) externally. Assessment of an HEI’s (or program’s) SI is part of the process of self-evaluation that HEIs regularly have to undertake in the process of national accreditation envisaged by law. Aside from regular national accreditation processes, self-evaluation of SI is also externally prescribed for HEIs by international accreditation institutions, although these accreditations are optional for HEIs.
Evaluation of SI is mostly carried out by the organizations themselves. Requirements for evaluators or general principles of evaluation have been set only by the organization in charge of allocating public funds. The existence of formal evaluation procedures is fairly typical for HEIs. With the exception of one HEI, none of the organizations follows a single (unified) methodology for measuring SI. The methods used are either adopted from external sources or elaborated by organizations. However, the application of organization-specific (custom-made) methods for assessment of SI is not widespread. Only two HEIs admitted that the methodology applied was elaborated by the organization itself. The essence of methods applied in the organizations interviewed, their appropriateness and their correspondence to internationally recognized standards in SI assessment remain somewhat unclear, as all the details of methodologies applied were not specified during the interviews. However, some generalizations about the nature of methods can be made. For HEIs, it is slightly more typical to account for intended impacts only. However, this approach is applied by only a slight majority of educational organizations. There are also several HEIs which account for both intended and unintended impacts. With the exception of one educational institution, both positive and negative impacts are accounted for. In general, SI assessment takes into account both monetary and non-monetary impacts, although one educational institution in charge of allocating public funds considers monetary impacts alone. Most institutions take into account both quantifiable and non-quantifiable impacts. The exception is again the institution in charge of allocating public funds on a competitive basis, which accounts for quantifiable impacts only. This approach seems to be taken in order to ensure comparability across competing projects. When assessing SI, organizations typically account for both long-term and short-terms impacts. However, there are also other approaches applied in practice. For example, one HEI assesses SI based on long-term impacts only; while one HEI relies on assessment of short-term impacts only. However, as is apparent from the interview in question, the latter approach is chosen for pragmatic reasons and not because of conceptual misunderstandings. As stated by the interviewee: “Short-term goals do not allow for the measurement of long-term impact. The impact may only be achieved in 5-7 years, but the indicators are set for 2-3 years.”
As us apparent from interviews, the lag between output and impact is not always addressed in the organizations studied. As a result, SI assessment in practice often has a short-term rather than a long-term focus, even in organizations which declare that they focus on short as well as long-term impacts. Sometimes, the underlying reason for this seems to be of a conceptual type—namely, an inability to distinguish between outputs and impact, and an inability to recognize the time it takes for impacts to occur. The lag between output and impact seems to be addressed only by a few organizations in their studies of SI. These are universities, which typically monitor the performance of their alumni during a longer time span (typically 5-10 years after graduation), when assessing the organization’s (or a specific program’s) SI.
Another rather typical shortcoming of approaches applied in practice seems to be the ignorance of the need to address causality. Only three organizations interviewed claimed that they address causality when estimating SI. However, none of these organizations explained adequately how causality is established. Hence, doubts can be raised as to whether the reported SIs are always attributable to the particular organization or particular activities undertaken by it.
The main obstacles and issues faced by the organizations in the evaluation of SI
seem to be lack of previous experience, and absence of applicable methodological
guidelines, models or best practices. Changing guidelines, changing priorities in society,
time constraints for data collection and difficulties in establishing the channels
of impact were among other issues outlined during the interviews.
The role of societal impact in managerial decisions
There seems to be know-how in relation to societal impact at lower levels of management
in educational institutions (e.g. among the people who work more closely
with the students), but this knowledge is hard to articulate and translate into a framework
that could reach the top management and policy makers. On the other hand,
funders require the top management of educational organizations to present figures
reflecting their activities, which focus more on outcomes than on impact. In addition,
organizations lack sufficiently varied indicators for them to be used in strategic decision-
making.
...
Management and societal impacts is an area that continuously develops in the field of higher education towards more systematic activities. A couple of years ago, an impressive university website about societal impacts was created that comprises dimensions of effectiveness and practices. This interactive site offers institutions of higher education tools to strengthen their visibility. The site also introduces impressive operating models and good practices. The core tasks of higher education institutions—education, research and development, and innovation—are the basis for effective, goal-oriented and open interaction between universities and their stakeholders. Promoting the development of a coherent and supportive aggregation of such knowledge and knowhow is a key opportunity for better understanding effectiveness and impacts. The societal impacts of universities of applied sciences and of arts universities can be described in the same terms, although the impacts of the sectors are different. On the other hand, societal impact is defined by each individual institution on the basis of its own field, profile and specificity.
Societal impact is also about the fact that stakeholders know what specific organizations and institutions are doing and what their impacts are. This emphasizes the need to provide stakeholders with relevant data both on outcomes and impacts. This should include both quantitative and qualitative indicators. The communication, for example, of results should be re-thought if they are not communicated to stakeholders critically and transparently. In addition, gathering feedback from the field is seen as crucial in managing societal impact.
Important capabilities with regard to societal impacts and management especially concern future orientation and collaboration between higher education institutions. Societal impact is strongly linked to anticipation of the future and to development of scenarios. Furthermore, societal impact should be a joint issue for higher education institutes and mutual appreciation and success should be highlighted as a common achievement. In addition, comprehensive analysis of higher education and the pooling of different materials for assessment of impacts using big data is an issue discussed both in higher education institutions and in the Ministry of Education and Culture.
Conclusions
To conclude, higher education institutes seem to have established procedures for
measuring and managing societal impact. However, they acknowledge the need to
discuss the understanding of the concept more thoroughly within their organizations
and among the stakeholders of HEIs. The concept is regarded as a management
issue strongly linked to mission, strategy, and key performance indicators. The interviewees
identified the need to put more emphasis on developing qualitative indicators,
and to pay more attention to unpredictable and unintended negative and
positive impacts. Although it is understood that societal impact concerns long-term
changes, the challenge remains how to evaluate these long-term impacts, and how
to evaluate any causalities.
The role of information regarding societal impact in managerial decision-making
All organizations interviewed agree that an understanding of an organization’s SI—
and the impact of its activities—is important for the organization. For most organizations
interviewed, SI assessment is considered to be associated with benefits for
the organization, whereas some of the interviewed organizations perceive that there
are also monetary benefits, although, as noted, the monetary benefits are difficult to
measure. The interviews conducted, however, do not allow us to establish whether
the role played by knowledge of an organization’s SI has remained the same, increased
or decreased (and for what reasons) over time, as the vast majority of interviewees
could not determine this. The opinion that the relevance and importance of
SI evaluation in the organization has increased was clearly expressed by only one
HEI, which was also one of the two organizations that perceived that information
disclosed concerning SI is monitored by their stakeholders, matters to them, affects
their choices and, thereby, determines also the future of the organization. The other
educational organization that perceived that their SI-related information matters to
their stakeholders suggested that stakeholders’ interest in their SI-related information
has grown over time.
...
As suggested by the majority of interviewees, expected SI is one of the considerations in the decision-making process in their organizations. However, it is not clear, what weight exactly is generally assigned to SI in the decision-making process in comparison with other criteria. SI seems to be an important and often overriding decision-making criterion above all in NGOs (which rely on publicly raised funds and have to justify the effectiveness of the use of these funds) and institutions which distribute (public) funds (and have to justify the effectiveness of their allocations). Its (SI’s) weight in managerial decision-making seems to be relatively small in HEIs, which also fund their activities from market-based transactions. Interestingly, the results of interviews also tend to suggest that the role of SI in the decision-making process at organizational level is positively correlated with the organization’s perception of monetary benefits from it.
Although it was not explicitly stated as such, there is reason to believe that SI is still not a primary criterion in strategy formation and decision-making in the majority of educational organizations. It is one of the considerations in decision-making, but not the main one. Overall, the decisions of most organizations seem to be still primarily driven by financial considerations as, in general, financial analysis still vastly dominates over SI analysis in terms of extent, depth and the frequency with which analysis is undertaken.
Although organizations recognize the importance of the evaluation of SI, SI has not yet established itself as a key assessment criterion of performance for employees or structural units in most organizations interviewed. SI as a performance indicator at an individual level has only been formally introduced in universities and only with regard to a limited number of academic positions. These are typically research-related academic positions, where SI is relatively easier to measure.
More extensive use of SI evaluations and expansion of the role of SI as a criterion in
the decision-making process seems to be mainly hampered by the limited knowledge
of evaluation methodologies, as well as by the absence of best practices that can easily
be adopted by organizations. On the other hand, wider use of SI assessment by
organizations also requires the raising of public awareness of the concept of SI, and
public recognition of SI as one of the major criteria for evaluating the use of resources
in society. In order to encourage a society-centered way of thinking and social
responsibility, public policies should promote the generation and wider distribution of
conceptual and methodological information on SI.
Societal engagement and arts organizations
For readers interested in arts organization see below a similar study of arts organizations in Estonia, Finland and Spain initiated by the HEISE project 2018-19.
