Contenu du sommaire : L'évaluation des politiques publiques : état(s) de l'art et controverses

— The Five Key Challenges for Evaluation — This text presents an overview of the progress made within the field of public policy evaluation in France. This document asserts the need for evaluation as well as its utility. This premise is based on three reasons: transparency, citizens' increasing demand for quality in public services and the increasing complexity of public policies. Examples taken from the evaluations executed by the Cour des Comptes are presented, and by doing so, this article attempts to determine the progress that has been made in this domain, as well as establish what is still left to be accomplished – especially in terms of the deepening cultural change currently under way, within the context of public finance belt tightening. This developing process must overcome five major challenges: the issue of the standards and the definition of the evaluation process itself; the position of evaluators, particularly their condition as independent actors; their varied levels of competency and types of experience; the issue of much needed resources such as means and time; and finally, the link between evaluation, the enactors' appropriation of the policy and political decision making.

— Evaluation : a Recurring Topic in Government Action? — Rationalising budgetary choices, “Renewal of the public service system”, LOLF, RGPP and nowadays, MAP: from the 60s up until today, the evaluation of public policies has addressed a never ending need that is constantly being reinvented: improved governance of public action. Despite this, the concept of evaluation faced a challenging situation in France: one that was characterised by the fragile institutionalisation of this concept, both ever changing and scattered. In order to explain this random trajectory, four main matters are discussed from a socio historical perspective: evaluation as a method for legitimisation and countervailing, as a factor in the improved application of public programmes, as a tool for promoting sectoral policies and as a means for optimising public expenditure.

— Reconstructing a Line of Argument: the Anchoring of Public Policy Evaluation Within the Field of Social Studies in the United States — This article aims to defend the idea that American social sciences – although bashfully approaching the idea of the State, and therefore, of a State run American society – led to the creation of a theory of the State which does not dare mention its name, and that therefore, the analysis and the evaluation of public policy are nothing but mere avatars.

— The Evaluation of Public Policy at the Local Level : Case Study of the Conseil Général des Côtes d'Armor — This article presents the implementation of the evaluation process within a local authority, the Conseil Général des Côtes d'Armor, and the progressive execution of an evaluation strategy that relies both on the dissemination of an internal culture in all its dimensions (operational and decision making contexts) as well as on the execution of a multi annual evaluation plan which was established to address certain issues that French départements face. It describes its infancy as well as the intrinsic characteristics that have allowed for the evaluative dimension to be further implemented within the sphere of local public action.

— Evaluation and Pôle‑emploi: Practices and Relationships Within the Decision‑Making Process — Established by the Board of Directors of Pôle‑emploi, the “evaluation committee” has made a substantial work since the spring of 2009. Focused on evaluating the effects of the service offer of their structure, the working programme of the committee relies on the use of varied disciplines and methodologies of evaluation, be them qualitative or quantitative. It also relies on a relatively formalised procedure for executing evaluations, which encompasses several stages; ranging from the drafting of the annual programme to the presentation of results before the Board of Directors, and from publishing results to publishing reports on the measures taken by Pôle‑emploi as a result of the evaluation process. The article examines the extent and limitation of this process, as well as the resulting harmonisation between the evaluation and decision‑making processes. It sheds light on the importance of generating shared knowledge, which is in turn used as a tool in the decision‑making and collective learning processes. It highlights that the harmonisation between evaluation and decision‑making displays – above all – a trickle‑up effect, in which the knowledge obtained through the evaluation process is disseminated among the decision‑makers, managers and specialists.

— Evaluating Public Policy and Consultancy Services. The Emergence of a New Professional Field and Segmented Roles — With the development of the field of public policy evaluation both within central public institutions and local authorities, many actors – private and public, consultants and civil servants – are now specialised, formalising their expertise or developing niche skills. Within this new wave, two specific sub groups – consultants and civil servants – stand out for their prominent role in the evaluation process; from its conception to its implementation. In order to analyse the initial steps of a professionalisation process, the article relies on several interviews conducted within the framework of a doctoral dissertation.

— Evaluation as an Instrument for Eliminating Political Significance: the Controversy Surrounding the Evaluation of American Schools Voucher — The school voucher was invented by Milton Friedman in the 60s, and tested out in several American states. It also became the subject of a considerable amount of literature on the other side of the Atlantic, as well as of an evaluation campaign during the 1990 2000 decade. This evaluation resulted in numerous controversies: the main ones will be addressed in this article. The most significant effect of this American evaluation fever when it came to the voucher, was that it managed to depoliticise it. This turned the voucher into a purely technical tool which would be more easily exported abroad; finally reaching French local authorities in its theoretically depoliticised form.

— Questioning the Measurement Model: Evaluation from the Perspective of Professional Judgement and Collaboration Within the Field of Educational Science — This article highlights the link between evaluation and measurement within the field of education. The questions that are raised contribute to the ongoing debate on evaluation, also present within the field of administration. Initially, the article recalls the origins of measurement when it comes to education by mentioning the studies on grading practices which demonstrated the importance of possessing the skill set required for “aptly evaluating” student learning. Therefore, in this case measurement serves as a reference model. Next, the article presents some arguments questioning this idea, namely by highlighting the necessary distinction to be made between evaluation and measurement. By doing so, the article briefly outlines some of the major guidelines for broadening the theoretical framework of evaluation, in which measurement becomes merely just another means to an end. So as to illustrate this broadening process, the article concludes by presenting a model which brings evaluation to the forefront, as a social and epistemic practice, based on professional judgement and the collaborative dimensions of the evaluation process. It aims to integrate the characteristics associated with an evaluation that is both of service to the individual and to social intervention, yet encompassing measurement.

— Outline of a Sociological Criticism Towards New Evaluation Standards — Nowadays, all – or almost all – must be subject to evaluation, and this is where France is said to be lagging behind. Yet, the pervasive neo evaluation trend is well ahead of standard evaluation processes. Based on the hypothesis derived from the organic unity and specific challenges attributed to this recent spin off, this article presents different criticisms which have been directed towards the new evaluation standards in the public sphere: at all its levels, from its conception to its implementation. If evaluation is not a good thing in its own right, triggering a zealous mindset, the author suggests that it must be considered as a dated social factor, inherently strange and complex, just one more among other innovative practices leading to “modernisation”. A description of such practices follows the premise. Neo evaluation not only transforms statesmanship but also the working framework of a top down approach, as well as the subjective relationship of “everyone to his own trade”. Next, it addresses the triangular social relations among the evaluator/the evaluated/the users, incurred by the generalisation of this obligation and the proliferation of interventions. Finally, given how the scales tip from the organic values held by the public service towards the competition embodied by market society, the author contemplates the hypothesis of establishing a technocracy that would remain outside of these practices, which are particularly destructive for commerce and result in effects such as overlegislation, social harassment and also, voluntary servitude.

— Indicators, Evaluation and Classification of Goals: Management Control and Social Performance — The evaluation of public policy relies on a model in which the public authorities agree on goals, which the administration seeks to accomplish together with its partners. In one way or another, it attempts to contrast the goals against the results obtained, while explaining disparities. The conclusions derived from such a comparison are then applied to reform bills and policies. This typical and ideal process hardly works when applied to State policies: the LOLF's efforts to “manage performance” were thwarted, the evaluation reports spark criticisms from the moment they are published rendering the implementation of any of their proposals impossible... The article offers an explanation by clarifying the fundamental difference in nature that exists between the definition for “goals” in the domains of politics and management. Therefore, the success of public policies is dependent on the joint construction of the State apparatus, involving all its partners and attempting to reach compromises on the actions to be executed. Evaluation in all its forms is a useful means to achieve this goal, as long as the varied scales of value are taken into account and the stakeholders are engaged.

– Joint Evaluations in the Domain of Higher Education and Research for the Improvement of Quality and Legitimacy: Challenges and Action Levers – An analysis of the challenges and action levers linked to joint evaluation in the domain of higher education and research, with the aim of producing high quality as well as legitimate results. It concerns diverse entities of every size; from higher education programmes and large research organisations, to research units and universities. Joint evaluation itself is varied in form, largely dependent on the different stages of the evaluation process. This method allows to better answer the needs of the different recipients of the evaluation, and constitutes an elemental factor in the quality and legitimacy of the evaluation. On the other hand, the joint evaluation method does not allow for the confrontation of radical criticisms on the evaluation itself, nor to rise to the challenge of demonstrating the impact of the evaluation based on the quality of the results of those entities' activities that were evaluated.

— Taking the Public into Consideration When Evaluating Public Policy — Taking the target audience into consideration when evaluating public policies is essential. Whether it concerns a quantitative or comprehensive approach, awareness of the target audience for public policies – including their characteristics, the manner in which they take ownership of the public offer and the resulting consequences – is a fundamental element of the analysis. The manner in which the public itself is taken into consideration has been split into two groups: on the one hand, the target audience for the policy or evaluative approach in question, and on the other, the non target audiences for a policy's outputs.

— Can Well being Become an Evaluation Criterion for Public Action? — This article examines the conditions which are necessary so that well being may become an evaluation criterion for public action. As a result, it examines the policy, methodology and technical issues that would arise from the adoption of this perspective. These issues are analysed based on two questions that are raised: (1) measuring well being and (2) how to operationalise this strategy, meaning how it is transferred from socio economics into public action. The first part is addressed using quantitative and participatory methods for the construction of well being indicators, which is in turn based on two testing grounds: the Walloon region and the communauté d'agglomération of Grenoble Alpes Métropole. The second part examines how institutions and policies can employ this type of measurement. Is it possible to operationalise both the definition and the measurement of well being? The article does not claim to establish new general evaluation criteria, rather, it attempts to point out current trends that – although still in their experimental stage – are indeed beneficial. Without forgetting of course, the many methodological, political and ideological pitfalls that must be overcome...

— Meta Evaluation of Public Policy and the Quality of Evaluations — Evaluations are executed so as to assess quality standards. However, in the reality of evaluation practices, evaluators adapt in function of the difficulties present when fulfilling their role and certain specifications inherent to their job. Thus, the quality of evaluation ultimately merits an assessment of its own when it comes time to deliver those reports (a counter report). Meta evaluation is more ambitious, seemingly on its way to becoming an activity that is defined throughout international literature as the evaluation of one or more evaluations (second level of evaluation). Can we introduce this concept and refine it? How can meta evaluation be described? Which activities characterise this process? What is its link with quality assurance for evaluation processes? In order to present this little known topic in France, four stances on meta evaluation – and their corresponding examples – are reviewed. The conclusion is the inevitable development of this reflexive evaluation activity to be applied on public policies and related programs in France.

— Internal Auditing in the Public Sector — This article aims to take stock of the implementation of internal audit systems within the domain of administration, in the context of the Cour des Comptes providing assistance to the Parliament in evaluating public policies. The decree from June 28, 2011 championed the professionalisation of public internal audit (made effective via regulations, training, recruitment and external evaluation). Its fulfilment requires a close collaboration among internal audit, external audit and the audit committee, employing specific tools and exchanging documents.

— How to Stimulate Public Services' Productivity? Lessons Learned from the Implementation of the Price per Activity System at a Hospital — The productivity of public services is a recurring topic of debate. One of the solutions suggested – empowering their management capacities by creating a direct link between real activity and financing – needs to be re examined. The activities carried out in public hospitals are now financed by the entry into force of the price per activity system. This in turn has established that there is a contradiction between encouraging productivity at the local level and managing expenditure at the national level. Furthermore, this system incites competition – between establishments and within them – rather than cooperation and amalgamation. Lastly, it has evidenced that the main potential for gains in regards to productivity – payroll – has been exhausted because almost all employees are civil servants.