Sezione Medicina

from Leadership Medica n. 3/2002

Why Evaluate?

ln 1993, The United States Government launched the “Reinventing Government” policy, which has deeply influenced Public Administration practice all over the world. However, contrary to what people may imagine, the “Reinventing Government” theory does not provide for the State to be regarded as a “business”, especially in view of the significant differences existing between the public and private sectors; on the other hand, Public Administration is requested to adopt a more entrepreneurial approach (Osborne and Gaebler). In particular, the “Reinvention” suggests a more adequate distribution of resources, also through benchmarking research among administration bodies, but also by implementing projects aimed at improving administrative activity efficiency, effectiveness and cost control. Nevertheless it is important to remember that the “Reinvention” makes it necessary to focus attention and efforts on results (efficiency and effectiveness controls), rather than on rules (legitimacy controls). In Italy, the phase of legislative history displaying unequivocal elements and indications as to the legislator’s intent to change Public Administration, by introducing evaluative practices of some kind, dates back to the early ‘90s, through a number of measures which ultimately also involved the Public Health sector. The legislator’s intent, which is now expressed through a substantial number of regulations (which by the way are often subject to review), risks being thwarted if the contents of such rules is not be correctly acknowledged, first of all by public administration managers and, all the more so, by the Managers of the National Health Service. A complete understanding of the manager evaluation processes therefore (necessarily) involves a correct interpretation of the role managers are entrusted with as part of the complex “Reinventing Government” process. Only once we have clarified and understood the principles regulating “Reinvention” will it be easier to understand what a manager is expected to do today. In such a context, evaluation can, through a feedback process, change into a major tool for improvement: this is the foundation on which “change” can be effectively built and implemented. The desire to be evaluated is instinctive, although the existence of some sort of conflict between such a desire and the fear of being evaluated is undeniable (Mohrman and Coll.). This motivation (in both its components: desire and fear) becomes even stronger when the performance is related to any kind of gratification, which need not be of an economic nature. Despite the progressive introduction of management evaluation into the Italian legislation covering public administration bodies, the opposition to a concrete and effective implementation stage is chiefly brought about by this inner conflict. The issue is therefore chiefly of a cultural nature, within a context which should mainly involve the political and strategic management teams of public administration bodies, which often appear to be the ones which have greater distrust in the potentials of evaluation in general, but above all when related to the performance of managers. In this regard, Fertonani believes by right that “before asking department heads and team-mates to believe in the importance of evaluation, it is Company Management and Staff Management who should be satisfied of the importance of such methodologies”. This climate of distrust is also fuelled by the lack of “basically objective, homogenous, transparent and shared evaluation processes”, which means that it is essential and absolutely urgent to create a “permanent staff performance evaluation system that is transparent, in other words agreed with trade union representatives, and also well-known and homogenous, and therefore equal for all staff members” (Migliozzi). Also according to Ruffini: “the most critical issue of the evaluation process is represented by the difficulty in making it objective”, which is particularly true in the evaluation of executive staff. The need for “objectivity” in evaluation criteria also results from the distinctive exceeding self-esteem characterising most individuals, and hence the opinion that one’s activity has been carried out in line with (if not beyond) the quality and quantity standards required by the strategic Management. In fact, the Gaussian distribution of staff performance says that only half of the values can be regarded to be above average (Mohrman)!

Valutazione medici SSN tabella 1

Valutazione medici SSN tabella 2

Regulations Governing the Evaluation of Managers
We are here briefly mentioning the chief reference laws regulating the evaluation of National Health Service executive staff, which can be traced back to the “Healthcare reform” decrees (Lgs D .no. 502/1992 and subsequent alteration and additions) and to the 1998 – 2001 edition of the National Collective Labour Agreement (CCNL), which also refers to certain major provisions of the Lgs D. no. 29/1993 and of the Lgs D. no. 286/1999. We can divide such regulations into three categories:
a) Provisions enunciating managers’ duties;
b) Provisions regulating evaluation procedures;
c) Provisions regulating the sanction application system.

a) Provisions Enunciating Managers’ Duties Lgs.

D. no. 502/1992 and Subsequent Alterations and Additions. Art. no. 15, Commas 1 and 2

1.… National Collective Labour Agreements provide for … general criteria for the graduation of managerial functions and for the assignment, evaluation and review of management assignments as well as for the attribution of the related additional pay depending on the functions assigned and on the related responsibilities for results.

2. Healthcare management is regulated by the legislative decree no. 29, of 3 February 1993, and subsequent alterations, save for the provisions of this decree …

[In this regard, the reader is reminded that the Lgs. D. no. 29/1993 – one of the cornerstones of the “Reinventing Government” in Italian legislation – deals with the “Rationalisation of organisation in public administration bodies and with the amendment of the regulations covering state employees”].

Lgs. D. no. 286/1999. Art. no. 5, Commas 1 and 2

1. Also on the basis of management control results, public administration bodies evaluate, in line with the applicable prescriptions of the National Collective Labour Agreements, the performance of their managers, as well as the behaviours relating to the development of professional, human and organisation resources they are entrusted with (organisational skills).

2. The evaluation of managers’ performance and organisational skills takes into special account the results of administrative and management activity …

[ Il Lgs. D. no. 286/1999, quoted under art. 26 of the National Collective Labour Agreement, NHS management area, 1998 – 2001, concerns the “Reorganisation and strengthening of the mechanisms and tools for monitoring and evaluating costs, outcome and results of the activity carried out by public administration bodies”; within the Italian legislation setting, this may be regarded as the most important directive relating to evaluation procedures].

b) Provisions Regulating Evaluation Procedures

Lgs. D. no. 502/1992 and Subsequent Alterations and Additions. Art. no. 15, Comma 5

The manager is subject to a triennial review; a manager with a facility assignment, whether simple or complex, is also subject to review at the end of the assignment. The reviews relate to the professional activities carried out and the results achieved and are performed by a technical committee, appointed by the general manager and headed by the department head. The positive outcome of the review is a prerequisite for the assignment or confirmation of professional or managerial tasks of greater importance.

Lgs. D. no. 502/1992 and Subsequent Alterations and Additions. Art. no. 15, Comma 6

In addition to the functions associated with their specific professional responsibilities, executives with a complex facility management assignment are also entrusted with facility management and organisation functions, to be carried out within the operating and management policy of the department they belong to; this is also accomplished through directives to all the staff employed in the facility and the adoption of the applicable decisions required to ensure proper fulfilment of the service and the adequacy of interventions with preventive, diagnostic, therapeutic and rehabilitative purposes, carried out in the facility they are responsible for. The manager is responsible for the effective and efficient management of the resources he is granted. The results of his management are subject to annual review through the Evaluation Nucleus.

National Collective Labour Agreement 1998 – 2001. Art. no. 31, Commas 1, 2, 3:

1. The organs in charge of the performance appraisal of managers (in the National Collective Labour Agreement healthcare, technical, administrative and professional area, the words “with a healthcare role” are added), pursuant to art. 15, commas 5 and 6 of the Lgs. D. no. 502/1992 are:

a) the Technical Committee;
b) the Evaluation Nucleus;

2. The body mentioned under comma 1, letter a) carries out the review:

a) of the professional activities performed and of the results achieved by all the managers, irrespective of their assignment, on a three-year basis;

b) of the managers who have a complex or simple facility management assignment, upon completion of the assignment itself;

c) of newly appointed managers, with the purpose of confirming their assignment, at the end of the first five-year period of service.

3. The body mentioned under comma 1, letter b) carries out the annual review: a) of the management results of complex facility managers and also of simple facility managers if they are responsible for resources management;

b) of the results achieved by all managers, including those listed under letter a), as they compare to their goals.

National Collective Labour Agreement 1998 – 2001. Art. no. 32, Commas 6, 7

6. The annual evaluation of managers by the Evaluation Nucleus has the objectives indicated under art. 31, comma 3, letters a) and b).

7. To merely provide an indication, the evaluation mentioned under comma 6 for complex or simple facility managers should – if the applicable conditions apply – relate to the management of the budget they are responsible for and to the human and instrumental resources they are actually granted, as well as all the functions delegated pursuant to the corporate deed and the evaluation of the organisation models adopted for the achievement of the goals, whereas for the other managers it concerns the compliance with performance objectives entrusted and the commitment and availability shown in working hour organisation versus goal achievement.

Lgs. D. no. 286/1999. Art. no. 5, Comma 2

… The evaluation is carried out on an annual basis. The evaluation procedure is inspired by the following principles: direct knowledge of the activity of the person subject to evaluation by the first-instance proponent or evaluator body, the approval or review of the evaluation by the second-instance competent or evaluator body and participation in the procedure by the person subject to evaluation.

c) Provisions Regulating the Sanction Application System

Lgs. D. no. 502/1992 and Subsequent Alterations and Additions Art. no. 15 – ter, Comma 3

The assignments covered by commas 1 and 2 are annulled, pursuant to the procedures provided for by the regulations in force and by the national collective labour agreements, in the event of: non compliance with the instructions given by the general or department management; failure to achieve the goals assigned; serious or reiterated liability; in all other cases provided for by the labour contracts. In the most serious cases, the general manager may back out of the employment contract, in line with the provisions of the civil code and of the national collective labour agreements.

Lgs. D. no. 29/1993. Art. no. 21, Comma 2

In the event of serious non-compliance with the instructions given by the competent body or of reiterated negative evaluation … the manager, subject to previous non-compliance notification and discussion, may be excluded from the conferment of further assignments within the same managerial level as the annulled assignment, for a period of not less than two years. In the most serious cases, the administration may back out of the employment contract, pursuant to the provisions of the civil code and of the collective agreements …

National Collective Labour Agreement 1998 – 2001. Art. no. 34

This provides for the following steps:
a) Discussion with the manager, in the event of negative evaluation, with the assistance of a trusted person;
b) Adoption of measures that are proportionate to both the position and the extent of the non-compliance;
c) Typology of measures:
- partial or total loss of the result-related remuneration;
- assignment of a different task on a similar organisational level;
- assignment of a different task on a lower organisational level;
- termination of employment. In view of the complexity of the measures referred to under commas 3 to 7 of art. 34, a summary table has been prepared
(Tab. 3).

Valutazione medici SSN tabella 3

Lgs. D. no. 286/1999. Art. no. 5, Comma 4

The evaluation procedure described under comma 3, represents a premise for the application of the measures covered by article 21, commas 1 and 2, of the decree no. 29, on the subject of managerial responsibilities. In particular, the measures covered by comma 1 of the aforementioned article apply when the negative results of the administrative and management activity or the failure to achieve the objectives emerge from the ordinary annual evaluation procedures. However, when the serious risk of a negative result materialises earlier than the annual performance appraisal deadline, the evaluation procedure may be closed ahead of time. The evaluation procedure is also closed ahead of time in the cases provided for by comma 2 of the aforementioned article 21, of the decree no. 29.

A rather significant aspect is represented by the duplicity of the institutional competence involved in executives’ evaluation: the Evaluation Nucleus and the Technical Committee (keeping in mind that the body responsible for the switch of top executives from the outdated employment relationship to the new term-related appointment contract, headed by the Health Unit Manager, has only acquired temporary legal status, for the time required to enforce the provisions of the Lgs. D. no. 229/1999 and of the Lgs. D. no. 49/2000). In our opinion, the two bodies can only be actually distinguished because of their makeup (which is absolutely free for the Evaluation Nucleus, whereas the Technical Committee can only be chaired by the Department Head) and for the different periodicity of the reviews (which are annual for the Evaluation Nucleus, and triennial or upon assignment completion for the Technical Committee). Indeed, we find that any disquisition as to alleged discordances in the evaluation “contents”, detected by some, are ineffective, since they appear in fact difficult to differentiate, at least in substance. The objectives of the evaluation are therefore contained in the two aforementioned provisions (Lgs. D. no. 502/1992 and subsequent alterations and addition, and the National Collective Labour Agreement 1998 – 2001): a) For the Evaluation Nucleus the provisions refer to the “management results achieved”; the management is to prove “effective and efficient in line with resources and objectives”; art. 32 of the National Collective Labour Agreement also supplies significant specifications; b) For the Technical Committee, on the other hand, provisions refer to “professional activities” and “results achieved”. In our opinion no substantial differences exist, for the following reasons:

1. This definitions ought to be reinterpreted in the light of all the regulations (including, but not limited to, the Lgs. D. no. 502/1992 and the National Collective Labour Agreement) regulating managerial duties;
2. Breaking down a manager’s professional profile in order to allocate to different bodies the evaluation of the resulting “competences” appears to be an illogic and what’s most meaningless procedure, also in view of the fact that it would make no sense for the manager in question to achieve for instance a positive assessment for his organisation skills and a negative one for budget management, since it is in any case necessary to come to an overall final evaluation as to the functional position covered;
3. Aside from the expression “results achieved”, which is used for both typologies, the term “professional activities” (evaluation covered by the Technical Committee) is absolutely generic, with an exegesis which appears to differentiate the dynamic-modality element (activity, therefore actions and management modalities), required to pursue management results (static–teleological element). Therefore, the parallel existence of two performance assessment organs (the Evaluation Nucleus and the Technical Committee) may only be logically justified by the requirement to evaluate the executives’ performance from two different perspectives and with diachronic modalities: one on a strictly annual basis and the other one according to specific time intervals. Nevertheless, in such a context, a close cooperation between the two bodies should anyhow be demanded and encouraged, to make evaluations more effective.

Principles and Subject of Evaluation
The regulations published in Italy over the last decade in connection with the reform of Public Administration have gradually introduced rules and regulations concerning management evaluation, without however going to the heart of the methodology - except for general principles – leaving the solution to the discretion of the Body or Enterprise. This may involve disadvantages, but also advantages. It certainly reveals a lack of homogeneity, especially in the contexts in which managers control absolutely identical outputs (in this regard, the NHS represents the most significant example). The basic principles of the evaluation methodology, quoted in the 1998 – 2001 edition of the National Collective Labour Agreement for the National Health Service management area (art. 32, comma 4), are basically three: the transparency principle, the information and participation principle and the knowledge principle. Zerilli effectively summarises such principles, by announcing that “The evaluation … is a right for the individual himself, who, the larger the organisation he works in, the less is he able to directly perceive how the organisation judges him and what is expected of him. The individual … has first of all the right to be judged objectively and according to a precise procedure and definite criteria; secondly, he is entitled to know this judgement and eventually speak up for himself”. Again in connection with the general principles of evaluation, we ought to recall art. 5 of the Lgs. D. no. 286/1999, where comma 2 prescribes “The evaluation procedure is prompted by the principles of direct knowledge of the activity of the person subject to evaluation by the first-instance proponent or evaluating organ” and “of participation of the evaluated subject to the procedure”. The transparency procedure is a typical feature of the evaluation methodology: all the procedure phases, in terms of modalities, implementation times and criteria employed, must be known beforehand by the strategic Management, by all the managers and by the related trade union representatives. Transparency applies in any case to general methodology, since, on the other hand evaluation results must be exclusively made known to the person concerned (also in compliance with the provisions of the law no. 241/1990) and the Enterprise Management (in this regard, art. 6, comma 2, of the Lgs. D. no. 286/1999 recalls that the bodies in charge of the evaluation “confidentially report to the policy bodies”). The information principle takes shape according to three modalities: a) communication as to the implementation of the procedure (this means the person concerned is to be informed that an evaluation procedure that concerns him has started); b) participation in the procedure by the individual subject to evaluation, possibly also through discussion, especially in the event of negative evaluation; c) final results of the evaluation, for which the Enterprise Management is responsible. The third principle relates to the direct acquaintance with the individual subject to evaluation by the evaluator who “in the first instance prepares the evaluation proposal based on which the assessment body is called to take a stand.”“ (please refer to art. 32 of the National Collective Labour Agreement).

As regards the subject of the evaluation of NHS managers, we hereby refer to the contents of art. 32 of the National Collective Labour Agreement, as well as of art. 5 of the Lgs. D. no. 286/1999 (despite the discretionary power involved in its application), listed in the table 5. As regards the chronology of evaluation, in addition to the annual, triennial and assignment completion deadlines, relating to the specific regulation and contractual indications, it is worthwhile mentioning once again (especially in the absence of a similar reference regulation) art. 5 of the Lgs. D. no. 286/1999, which prescribes under comma 4 that “when the serious risk of a negative result materialises prior to the annual deadline, the evaluation procedure may be closed in advance”, moreover assessing that “the evaluation procedure will be closed in advance … in the cases provided for by comma 2, of the above-mentioned article 21, of the decree no. 29>.

[The reader is reminded, in this regard, that the Lgs. D. no. 29/1993, under art. 21, comma 2, prescribes that “In the event of serious non-compliance with the directives given by the competent body or of a specific responsibility for the negative results of the administrative activity and management, the manager, subject to previous non-compliance notification and discussion, may be excluded from the conferment of further assignments within the same managerial level as the annulled assignment, for a period of not less than two years. In the most serious cases, the administration may back out of the employment contract, pursuant to the provisions of the civil code and of the collective agreements”].

Valutazione medici SSN tabella 4

Valutazione medici SSN tabella 5

Who Evaluates?
The regulations previously recalled clearly refer to the bodies responsible for the evaluation, depending on the case in point; however, both in order to comply with the contractual principle relating to “direct acquaintance” with the manager subject to evaluation, and because evaluators do not always possess all the useful and/or necessary information, the need for other professional figures to take part in the procedure is unquestionable, starting of course form the manager’s immediate superior, but also the latter’s superior, etc. The chief methods suggested in literature are the “one over one” (tab. 6) evaluations (only the direct superior evaluates; however, irrespective of the regulations in force, this hypothesis appears inappropriate), “multiple appraisal” (tab. 7) (the method which is certainly the most advisable) and “multisource feedback” (tab. 8) (not actionable, in Public Health, for obvious reasons). In the Ferrara ASL (Local Health Enterprise) experience, the Evaluation Nucleus employs a “multiple appraisal” method, by entrusting the Direct Superior with the first approach. The contribution by the institutional body only takes place at a further stage, with the acquisition, if necessary, of any further useful information (evaluation by the manager the direct superior reports to, documentation also from other company departments, etc.), especially if critical issues have emerged.

Valutazione medici SSN tabella 6

Valutazione medici SSN tabella 7

Valutazione medici SSN tabella 8

The Evaluation Scales
One last question relates to evaluation scales, which evaluators may use to define the manager’s performance in terms of quality and quantity (for instance: “insufficient”, “sufficient” and “excellent” performance). In this regard, Migliozzi (rightly, in our opinion) advises against the use of scales that either have an excessive number of grades (owing to the difficulty in distinguishing between two subsequent measurements), or have a too limited number of grades (owing to the difficulties that are likely to be experienced in the choice between two subsequent measurements). Nevertheless there is a legitimate doubt as to the use of odd scales, which involve the disadvantage of encouraging evaluators to choose “diplomatic” average values, or of even scales, which on the other hand involve a choice between a positive value and a negative one. According to Fertonani “If there is a high number, it often proves rather difficult to identify the difference between one and the other; if, on the other hand, the number is low , it may become embarrassing for the evaluator to carry out a choice involving judgements that are too “tranchants”. As a rule, one does not drop below three , if there are more than seven grades … If they are odd … there can be a tendency towards average judgements … if, on the other hand, they are even, the evaluator is forced always to choose between a more positive and a more negative grade, and the evaluation can be therefore distorted”.

The Evaluation Protocol: An Example
The Evaluation Nucleus of the Ferrara Health Enterprise – in compliance with the above mentioned principles (transparency, information and knowledge/acquaintance) – has defined evaluation protocols in the management area, in agreement with both the trade union representatives and the Enterprise Managers. The protocol is presented in a concise manner, and includes the parts which are of greater interest for explanation purposes (on the other hand, the original, resolved by the Management, also contains the chief legal and authority references). A necessary condition for the presentation of managers’ evaluation protocols is the warning not to deal with the contents in an excessively strict manner. The evaluation is a process that must flexibly fit in with the context: if the evaluation culture is sufficiently widespread within the company framework, it is possible to introduce subtleties and nuances that characterise individual career briefs. Should this not be the case, at least during the early stages of procedure implementation it is worthwhile to content oneself with the definition of the items characterising homogenous macro-groups and set to tackle the difficult task of convincing employees to “accept” the evaluation, encouraging cooperative behaviours through preliminary meetings, seminars and/or any other activity that proves fit to obtain full support from all managers, also because many of them will be called in turn to personally take part in the evaluation of their co-operators (the department head, for instance, for the complex facility managers, etc.). Furthermore, the evaluation protocols (whose organisation has to be implemented by experts on the subject) must subsequently and unavoidably (also in compliance with the transparency, information and acquaintance principles) be fully shared by the review bodies and the trade union representatives, as well as by the Strategic Management, so as to be made known to all the company managers. Nevertheless, one must always keep into account that the chief aim of an evaluation activity is not that of arranging beforehand a tool aimed at applying sanctions for the Strategic Management, but the improvement of efficiency, of effectiveness and of the quality of service provided, also trough reflections shared by the manager subject to evaluation and the evaluator, especially in the event that the first stage of the evaluation is accomplished with the cooperation of the direct superior.The protocols employed by the Evaluation Nucleus of the Ferrara Health Enterprise (whose basic contents are listed in the chapter) are agreed among the institutional stakeholders before being resolved and becoming operational.

To organise the protocols, it has been thought essential, in order to define the items on which the evaluation forms are based, to first of all identify:

I. The management level

a) enterprise;
b) department;
c) facility;
d) towards clients;
e) towards stakeholders;

II. The reference

That is the management responsibility profiles subject to evaluation, which define the “items”;

III. The methodology

Relating to the contents of the evaluation.
Pursuant to the directives of art. 32 of the National Collective Labour Agreement 1998 – 2001, 10 items have therefore been identified: these must be regarded as absolutely indicative and to be used especially during the implementation of evaluation, and should subsequently be adapted to every context or, what is even better, to every manager. The items involve an evaluation according to a five-point scale (see below). As regards the annual evaluation, the procedure employed in the Ferrara Heath Enterprise is the “multiple appraisal” type: first of all, the evaluation form (see facsimile below) is handed to the direct superior (or direct superiors, if applicable), who fills in the form with the participation of the manger who is the subject of the evaluation; the form is subsequently forwarded to the Evaluation Nucleus, for the necessary integrations and conclusions, also with the use of other offices, bodies or operating units, which, somehow, may contribute to the supply of useful information, but of course also through annual reports concerning the evaluation of the result objectives. In our opinion, the triennial and assignment completion evaluations, peculiar to the Technical Committee, in whatever manner accomplished (in our opinion, owing to the above mentioned reasons, through similar items) must provide for the acquisition of periodic reports of the Evaluation Nucleus and (in view of the significant lapse of time involved by the review and in compliance with the participation principle) also of a curriculum vitae of the manager, which will stimulate self-review opportunities. As mentioned, a five-point evaluation scale has been employed, in line with the strongest suggestions provided in literature and with the prevailing trends of the organisations which have already made use of it. In practice, for each item (see table 9), the evaluator is requested to express his opinion with a score which does not represent a “mark”, but rather a qualitative attribute. The use of a number, rather than of the applicable attribute, also allows a greater speed in filling in the forms.

Valutazione medici SSN tabella 9

The sum of the scores assigned for each item on the form leads to an overall evaluation, which needs to be adequately interpreted (please refer to the hypotheses recalled in table 10, should the items be 10).

Valutazione medici SSN tabella 10

In any case, it is worthwhile pointing out that, irrespective of the result achieved through the performance appraisal form, the procedure adopted by the Ferrara Enterprise Evaluation Nucleus is absolutely “observant of civil rights” for management staff. This is absolutely necessary, especially during the initial (and often long) evaluation system implementation phase, which has to be adequately known, both by the evaluators and by the evaluated subjects, and must in any case always be viewed as a tool for the improvement of the procedures and services supplied by the enterprise, rather than as a tool for applying sanctions. The referenced evaluation procedure is very effectively summarised by a flowchart, which also provides an indication as to final hypotheses.

Valutazione medici SSN tabella 11

Mauro Martini
Direttore del Nucleo di Valutazione dell'Azienda Sanitaria di Ferrara

Link utili e bibliografia

Costa G. (a cura di). Manuale di Gestione del Personale; vol. I, II. III (contributi vv.). UTET, Torino, 1992.

Donati E., Cubello A.L. “”. Le esperienze del National Performance Review e dell’Electronic Government in USA dal 1993 al 1999. FrancoAngeli Ed., 1999.

Fertonani M. Le competenze manageriali. Dalla valutazione delle prestazioni e del potenziale alla valutazione delle competenze manageriali. FrancoAngeli Ed., 2000.

Lewis J.P. The Project Manager’s Desk Reference: A Comprehensive Guide to Project Planning, Scheduling, Evaluation, Control & Systems. Probus Professional Pub, 1993.

Marchetti M. e Quaranta E. La valutazione del personale. Quaderni Pirelli, n.49, 1984.

Martini M. La valutazione dei dirigenti del Servizio Sanitario Nazionale. Essebiemme Ed., Noceto (PR), 2002 (it is printing).

Magginson L.C., Mosley D.C., Pietri P.H. Management. Concetti e applicazioni. FrancoAngeli, 1996.

Migliozzi D. La valutazione del personale degli Enti Locali. Prestazioni, risultati e CCNL. FrancoAngeli Ed., 2000.

Mohrman A.M., Lawler III E.E., Resnick-West S.M. La valutazione dei dipendenti. FrancoAngeli Ed., 1998.

Olivero N. (a cura di). La valutazione dei dirigenti nella Pubblica Amministrazione. FrancoAngeli, Milano, 1998.

Osborne D., Gaebler T. Dirigere e governare. Garzanti, 1995.

Rossetti R., Russo T. Ruolo del Nucleo di Valutazione nelle Aziende Sanitarie e valutazione del personale. Mecosan, 21: 33-47, 1997.

Ruffini R. Evoluzione della Pubblica Amministrazione e sistemi retributivi. FrancoAngeli Ed., 1997.

Spencer S.M., Spencer L.M. Competenza del lavoro. Modelli per una performance superiore. FrancoAngeli, 1999.

Sylvia R.D., Sylvia K.M., Gunn E.M. Program planning and evaluation for the public manager. Waveland Press, 1996.

Toropov B. Manager’s portfolio of model performance evaluations: ready-to-use performance appraisals covering all employee functions. Prentice Hall Trade, 1999.

Zanetti et Al. Il medico e il managment. Guida ragionata alle funzioni gestionali. Accademia Nazionale di Medicina; Forum Service, 1996.

Zerilli A. La valutazione del personale. FrancoAngeli, 1963 e 1997.

We ought to point out that most of the websites available are unlikely to specifically deal with managers’ evaluation; what they normally dwell on (incidentally always in a highly professional way) are the various topics related to evaluation, where you will also find information on staff evaluation.

Italian Evaluation Society

European Evaluation Society

Société française de l’Evaluation

UK Evaluation Society

Germany Evaluation Society

American Evaluation Association

Australasian Evaluation Society – Job Evaluation

Lastly, it is also possible to visit the websites of trade unions operating in the field of medical management and PTSA, a number of which deal with the subject.