Introduction

Universities are no longer simply educational institutions, if ever they were. They increasingly have a responsibility to, amongst other things, provide research that is nationally and regionally relevant.  As a result, the engagement of communities in many aspects of a university’s functioning is gaining prominence both in practice and in the literature. And engagement as such is becoming another dimension on which a university is judged, by governments as well as other stakeholders. Seen from a slightly different perspective, it also becomes a set of activities through which the university itself can demonstrate its relevance to the wider society. In line with this, the notion of benchmarks and performance indicators that enable determination of a university’s socio-economic and cultural contributions to its region has gained prominence.  To date, however, little has been achieved in the development of robust measures reflecting the benefits that flow from such partnerships to both universities and the communities with which they engage.

The current pilot project has been designed to investigate the possibilities of drawing up a viable set of indicators for community engagement based upon an inventory of the relevant international literature. A major condition for such an indicator set is that it does not generate yet another demand on universities to generate data, but that it can operate within the framework of the current data collected by universities. To test this, in the pilot project a two-stage approach is used. First, on the basis of a comprehensive international literature review an attempt is made to draw up a set of indicators relevant to the concept of community engagement. Second, using information available in the public domain, i.e. published reports and data contained on university websites, we investigate the extent to which the indicator set can be populated to constitute a reasonable understanding of the community engagement activities with and between universities in Victoria. Since community engagement can mean many things to many people, we first provide a brief overview of this concept.

Executive Summary

Recent discussions about the inclusion of the community engagement agenda as part of the National Protocols for Higher Education Approval Processes and calls for Third Stream funding have highlighted the need for some measures of university-community engagement. Such measures of engagement should offer tangible evidence of the role that universities play in the educational, social and economic wellbeing of local communities and the nation more broadly. In Search of Evidence builds on the self-audit of Victorian universities undertaken for the Department of Education and Training, Victoria in 2005, reported in Beyond Rhetoric: University-Community Engagement in Victoria. One of the recommendations made was for the development of appropriate indicators for engagement in Research and Teaching Quality frameworks.

The pilot project therefore investigates the practicality of using currently collected and published information to report against a set of indictors to determine the type and extent of community engagement activity in Victorian universities. This will involve the examination of national and international literature in order to develop a preliminary set of indicators, which will then be tested against the data available at Victorian universities. A major condition for such an indicator set is that it does not generate yet another demand on universities to generate data, but that it can operate within the framework of the current data collected and reported by universities. The intention therefore of this report is not to suggest a new suite of indicators but to select from those that have been suggested in the literature, a set which could be applied to evaluate the nature of community- university engagement in Victoria. Or, to be more precise, to actually assess whether such an approach might work.

To test this, a two-stage approach is used. First, on the basis of a comprehensive international literature review an attempt is made to draw up a set of indicators relevant to the concept of community engagement. Second, using information available in the public domain, i.e. published reports and data contained on university websites, we investigate the extent to which the indicator set can be populated to constitute a reasonable understanding of the community engagement activities within and between universities in Victoria.

The international literature search resulted in six sets of indicators and protocols that could prove useful in the development of a set of indicators for community engagement. These indicator sets were evaluated for their usefulness to the objective of this project based on the following criteria: Relevance, Reliability, Transparency, Availability, Auditability and Cost. This assessment resulted in the selection of the so- called Russel Group Indicators, which had the highest overall score (according to those criteria) and also provided the most comprehensive coverage of what in the broadest sense may be captured under the heading of community engagement. We opted to continue the project with that set, while adding in a number of other relevant measures. A trial run consisting of an analysis of existing Victorian university reports and other available forms of data was undertaken to see to what extent the suggested set of indicators could indeed be viably used to determine the extent of community engagement. The outcomes of this trial run were disappointing. Our research suggests that at present there is insufficient information reported to even superficially understand the level of community engagement being undertaken by a university. We believe that this is not the result of the indicator set selected, but is the consequence of what universities report – or not report for that matter – in the area of community engagement. Real reporting of achievements against stated intention only takes place to a very limited extent.

Key Findings

• Community engagement can be defined as the mechanism through which universities achieve the goals they have articulated in relation to specific communities in terms of their trinity of basic functions, i.e. teaching, research and service, whether at the strategic university level, or in project specific contexts.
• A number of reporting protocols have been produced to facilitate the assessment of University performance in relation to their function as institutions of teaching, research, and service.
• There seems to be a dominance of indicators to measure commercialisation, technology transfer and spin-offs.
• At the university level, typically the notion of university-community engagement is enshrined in Mission & Vision Statements and/or the Strategic Plan. Yet, these goals and objectives in general do not seem to be well supported by formal reporting.
• Anecdotal evidence suggests that on the whole many university managers are not aware of the breadth of community engagement that currently occurs within their own institutions.
• If community engagement is reported upon, it is done in an inconsistent and qualitative way (which is intended here as a statement of fact rather than implying that quantitative is to be preferred over qualitative).
• Where there is more detailed information, it is generally found at the faculty or, more often, department/school levels. Yet again we need to note that this type of reporting is ad-hoc, primarily qualitative and by no means consistent across an institution.
• It is suggested that in theory it is possible to arrive at a set of indicators that perhaps may not be comprehensive, but that is robust enough in international comparative terms to serve as a workable operationalisation of the concept of engagement.
• Examining the extent of community engagement on the basis of currently reported and readily available information, is as yet an impossibility.

The Project Team

The project team included:

• Leo Goedegebuure
• Jeannet van der Lee

The Final Report:
In Search of Evidence:
Measuring Community
Engagement - A Pilot Study

------------------------------------

Funding Partner:

Victoria State Department of
Education and the Arts