|
Francesco, besides being the co-editor of Ariadne is also a PhD Scholar in “Public Administration Economy and Regulated Sectors Management” at the University of Siena, Department of Business and Social Studies. Below you can read his thoughts on …
|
|
| The Role of Participatory Evaluation in the Assessment of Public Administration Performance |
|
Almost forty years ago, Peter Drucker, the famous ‘guru’ of management, advocated the building up of “an independent agency … independent of pressures from the executive as well as from the legislature” to play the role of ‘performance auditor’ for Public Administrations, that is, to convert people’s expectations into policy goals and to compare with them the results obtained, much in the same way as the Accounting Office, or simply the Courts, protect and enforce accountability of budgets and procedures.
|
|
|
At the other extreme lie Robert Dahl’s conditions for democratic decision making i.e.
|
|
1
|
Effective participation,
|
|
2
|
Equality in voting,
|
|
3
|
Gaining enlightened understanding,
|
|
4
|
Exercising final control over the agenda,
|
|
5
|
Inclusion of all adults,
|
|
|
|
each of them raising sensitive and unsolved issues in the area of political access, equality and legitimisation.
|
|
For instance, it is known from Sherry Arnstein’s seminal work that the development of participation in a selected context can reach different levels of granularity, including ‘loose coupling’ and rhetoric “manipulation”. Moreover, Dahl himself admits that in the real world, it is very unlikely that every citizen can have equal opportunities to influence the policy agenda.
As James Fishkin clearly stated: “The (real) problem of democratic reform is … how to bring people into the process under conditions where they can be engaged to think seriously and fully about public issues”.
However, this can be simply impossible to reach as the number of participants in public debates increases above a certain, very low threshold.
Again it was Robert Dahl to show this paradox with a back-of-the-envelope calculation: “if an association were to make one decision a day, allow ten hours a day for discussion, and permit each member just ten minutes – rather extreme assumptions … - then the association could not have more than sixty members” in order to allow everyone to have their say.
The recent popularity of electronic voting has been partly related with the possibility of enhancing people’s control over policy makers. Yet in the State of California, as reported by Fareed Zakaria, when the people were asked to vote electronically on more than 200 pieces of legislation in one single year, this instead of opening up a new perspective to participatory democracy, simply turned into the creation of “a jumble of laws, often contradictory, without any of the debate, deliberation, and compromise that characterize legislation”.
This may not necessarily imply that ‘the elected’, not to speak of ‘the appointed’, public officials could take better decisions on their own in the interest of the whole population. But at least it shifts the attention on the much more manageable issue of their accountability towards the citizens, instead of following the myth of universal participation in democratic deliberations.
|
|
| What to measure |
|
Born as a statistical control methodology in manufacturing, the concept of quality has evolved during the 1950s in the US and Japan from product to process orientation and from ex post evaluation to ex ante planning, up to continuous monitoring of business activities. Still the notion takes on several different meanings:
|
|
| • |
conforming to specifications (ISO 9000, Service Charter);
|
| • |
meeting corporate and social objectives (Balanced Scorecard I),
|
| • |
aligning inputs, processes, outputs and outcomes (EFQM, Balanced Scorecard II),
|
| • |
meeting or exceeding customer expectations (the ServQual instrument),
|
| • |
producing emotional, passionate commitment on the part of users.
|
|
|
Assessing the quality of public services and the citizen/customer satisfaction have been major concerns for the promoters and supporters of the Government ‘reinvention’ movement in the past 25 years. Together with the goal of continual improvement, these are also known as the three main pillars of a Quality Management System.
During the 1990s, the wide diffusion of service charters has started to provide the citizen/customer with a ‘prior information’ about the kinds and levels of service (s)he can expect from a public authority, as well as a private organisation supplying public services (e.g. garbage collection). This was more related with the original notion of quality as product performance control than with the modern one of process and activity management. However, it has helped integrate the targets of efficiency and effectiveness with a greater attention to end users in the Public Administration ‘process’.
More recently, the need has arisen for a more integrated view of the process and its results, in light of the recognition that not necessarily a good Government leads to good results, or to results that are perceived as such by its constituency; this is partly related to the ‘networked’ nature of modern Governments, being at the centre of complex relationships involving other public institutions, some private/business entities, and the citizens themselves. Shifting from the quality of services concept to the quality of government/governance is like ensuring that “good pianos play good music” together.
|
|
| Why to measure |
Both at conceptual and experimental level, there are now signals that clearly point to the integration of management, transparency and accountability features within modern Public Administrations.
ISO defines a Quality Management System “the organisational structure together with the planning, processes, resources and documentation that are needed to achieve the quality objectives, and to provide continual improvement of the products and services that are being provided”.
However, for a local authority “sustainable success will only result from the implementation of an integral Quality Management System that addresses the needs and expectations of all interested parties”.
This kind of openness is termed ‘transparency’ and it occurs when the “processes, procedures, methods, data sources and assumptions used by the local government” during a Quality Management System building up and maintenance “are made available to all interested parties and citizens and collectively ensure that (they) are made aware of their roles, rights and duties”.
For instance, a well performing street cleaning service also requires that citizens are careful not to throw litter in the streets.
If we take on this notion of ‘accountability towards the community’ that can be read between the lines of the ISO document, the next question is indeed what local government should be accountable for. The answer is quite obvious: performance.
Implementation of a Quality Management System in local government
Starting from a position where some (or none) of the Public Administration’s areas of activity are included in a Quality Management System, it is suggested that the top management should perform a self-assessment test based on 39 ‘reliability’ indicators.
These are typically more related to measuring the success of public interventions in terms of the quality of life changes which they bring about for the constituency, rather than the quality of activities as such. If one or more indicators showed a weak performance level, this would be the sign that the local government is operating ‘below the reliability line‘. Then the first goal for top managers is to reach minimum conditions of reliability for all the key process and services, by implementing an ‘integral’ Quality Management System.
Quality certification according to the ISO 9001:2000 standards must not be regarded as the final objective: once a local government has achieved an acceptable level of reliability, “it should look beyond conformance to performance”, aiming to improve its overall efficiency through the utilization of ISO 9004:2000 and/or other excellence models.
This shifts the focus from managerial excellence to excellent governance, a concept that transcends organizational borders and goes further into the multi-stakeholder environment that was partially sketched by the ‘piano concert‘ analogy in the previous section.
|
|
| • |
conforming to specifications (ISO 9000, Service Charter);
|
| • |
meeting corporate and social objectives (Balanced Scorecard I),
|
| • |
aligning inputs, processes, outputs and outcomes (EFQM, Balanced Scorecard II),
|
| • |
meeting or exceeding customer expectations (the ServQual instrument),
|
| • |
producing emotional, passionate commitment on the part of users.
|
|
|
Assessing the quality of public services and the citizen/customer satisfaction have been major concerns for the promoters and supporters of the Government ‘reinvention’ movement in the past 25 years. Together with the goal of continual improvement, these are also known as the three main pillars of a Quality Management System.
During the 1990s, the wide diffusion of service charters has started to provide the citizen/customer with a ‘prior information’ about the kinds and levels of service (s)he can expect from a public authority, as well as a private organisation supplying public services (e.g. garbage collection). This was more related with the original notion of quality as product performance control than with the modern one of process and activity management. However, it has helped integrate the targets of efficiency and effectiveness with a greater attention to end users in the Public Administration ‘process’.
More recently, the need has arisen for a more integrated view of the process and its results, in light of the recognition that not necessarily a good Government leads to good results, or to results that are perceived as such by its constituency; this is partly related to the ‘networked’ nature of modern Governments, being at the centre of complex relationships involving other public institutions, some private/business entities, and the citizens themselves. Shifting from the quality of services concept to the quality of government/governance is like ensuring that “good pianos play good music” together.
|
|
| How to measure |
It is commonly agreed that an advanced Performance Measurement System, including both financial and non financial indicators, may help to achieve the strategic objectives established within a Quality Management System.
While the latter can ensure the managerial formulation of strategies and their assessment and monitoring, the former will manage the resources to be used to the purpose and control the achievement of the objectives stated. Kaplan and Norton’s Balanced Scorecard has been repeatedly called for in this context. This forms an integral part of a “third wave” of public sector evaluation, characterised by “internal” institutions, practices and tools like e.g. self-evaluative procedures and reports, agency-based performance indicators etc.
While Drucker’s idea of an agency has not taken place so far, modern regulation theories now admit in its place the establishment of rules, procedures and standards, helping to make the Governments more accountable for their performance towards the citizens. Some international comparisons (from OECD Puma to EU Sigma) already show that the ‘power of standards’ can be used to elicit process reengineering, comparability and harmonisation of practices in tge Public Administration. However, an insufficient attention has been given until now to change realised through quality management principles implementation.
More generally, we believe that a unified and systematic approach to public performance evaluation is missing, which should make use of innovative concepts and tools like social and democratic dialogue and an extensive participation of citizens/customers in the governance and accountability process.
The Living Labs concept is a recent innovation approach set forth in Northern Europe, through which all stakeholders of a product, service or application actively participate in its development process. Stakeholders can be public authorities, civic communities, SMEs and large industries, academia, content providers etc. The underlying R&D methodology enables innovation to be created and validated in a collaborative, multi-context, real-life environment, where the person is focused and monitored in all his/her social roles as (e.g.) a citizen, user, consumer or worker. This human-centric, experience-based perspective does not only ensure a user-driven design and development of products, services or applications, but also user acceptance. The idea is to reach a more sustainable innovation by taking benefit of the ideas, experiences and knowledge of the people involved with respect to their daily needs, in their every day lives, encompassing all their societal roles.
Additional methodology reflections are needed to assess whether the Living Labs paradigm can be helpful in reviving and consolidating the theoretical foundations for a citizens-focused, participatory performance measurement system that is more coherent with the increasingly ‘networked’ configuration of modern Public Administration and the need to find a balanced pathway between budget restrictions and quality assessment of ‘government reengineering processes’.
|
|
| Conclusions |
| Methodology reflections conducted besides public sector reform processes management have shown a growing attention to systemic change and the evaluation problems thereof. Both at theoretical and practical level, there are signals that clearly point to the integration of quality management, transparency and accountability features within modern Public Administrations. However, a unified and systematic approach to performance evaluation is missing. This paper argues that ‘small groups participation’ in the assessment of results can be a valid alternative to external audits, avoiding both exclusion and irrelevance, i.e. the major drawbacks of the current representative and deliberative democracy experiences. Work in progress aims to: |
|
|
•
|
refine the results of an ongoing experience within the EU-funded project LexiPation that can serve as a blueprint and provide a supporting tool for future implementations;
|
|
•
|
further explore the connection between quality management, accountability and citizens participation in local governance processes;
|
|
•
|
assess whether the Living Labs paradigm can be helpful in reviving and consolidating the theoretical foundation of a person-centric, participatory evaluation of public activities and outcomes.
|
|
You may contact Francesco at: francesco.molinari@zia.ms.it
For accessing to the bibliography appearing in the article please use the references below – all of them are worth to read
|
|
|
|
