ICSOM Conductor Evaluation Program Information 2009

History of the Program

ICSOM’s Conductor Evaluation Program was established at the 1967 conference to help orchestras register their collective subjective opinions and objective appraisals of conductors and to thereby help managers and artistic planners reach decisions in engaging music directors, resident conducting staff, and guest conductors.

For many years results were hand-tabulated by member orchestras and conveyed to ICSOM’s Rapid Communication Center for dissemination. Since 1982, ICSOM evaluations have been tabulated by computer at Wayne State University in Detroit and stored in a data bank at WSU.

Pursuant to resolutions adopted at their 1990 conferences, ICSOM, ROPA, and OCSM may access each other’s conductor evaluation information. Delegates needing information from one of the other conferences should contact the ICSOM administrator. This season, one request for ICSOM information came from OCSM, two from ROPA.

Current statistics

Since 1982, 57 different orchestras have submitted conductor evaluations, 4838 sets of evaluations are in the data bank, and 835 conductors are listed, not including those who are deceased. At the 2004 Conference a resolution was passed encouraging each orchestra to utilize the program at least once. 21 of 51 orchestras (41%) complied with the resolution. Between August 1, 2005, and July 1, 2006 usage was up 40% from the previous year low of 15 orchestras. This year, however, usage slipped back down to 10 orchestras (from 13 last year). 79 sets of evaluations were submitted (down 29%). The average number of responses per orchestra participating was 31 (down from 32 in 2008). There were two special requests during that period from an ICSOM: one ICSOM orchestra engaged in a music director search and one ROPA orchestra.

Expenses for the conductor evaluation program in 2008-09 totaled $640. ICSOM pays for the printing and mailing of forms (50,000 were printed in the 2006 fiscal year), creating and amending computer programs, and generating and mailing of reports. When an orchestra sends in a set of evaluations, an initial report is sent to the orchestra which submitted the evaluations.

Searchable database of conductors, orchestras and dates

All of the conductors evaluated to date are in a searchable Excel file that you can access here.

Securing Blank Conductor Evaluation Forms

Blank conductor evaluation forms are available from Wayne State Univ.. Forms are sent in bundles of 500 copies. Requests for forms, questions, and completed forms with cover sheets, should go to:

For special requests and other questions, contact Michael Moore, ICSOM Conductor Evaluation Program Administrator mooretuba@bellsouth.net or 404-875-8822.

Using Conductor Evaluation Forms

ICSOM conductor evaluation forms are for use by ICSOM orchestras only. Delegates are responsible for distributing forms to orchestra members, collecting completed forms, and sending completed forms with a completed report form to Wayne State University for tabulation and entry into the evaluation information bank. The report form must show the name and address of the orchestra’s ICSOM delegate. Forms without this information will not be processed. A copy of the report form should always be in the delegate manual, available for duplication as necessary.

Instructions for filling out the evaluation forms are printed on the forms themselves. Delegates should urge orchestra members to fill out the forms as instructed. The WSU computer scanner will not read forms improperly marked.

In submitting forms and cover sheets to WSU, delegates are asked to provide complete names of conductors, correctly spelled and clearly printed so that no confusion results in computerized listings. Delegates should also provide specific date information (example: 5,7,8,9 October 2003).

CAUTION: Orchestras wishing to do their own independent conductor evaluations and tabulations may not use ICSOM forms, original or photocopied, for this purpose; doing so will be considered a misuse of the forms, as will using the forms in any way other than as prescribed above.

CAUTION: ICSOM conductor evaluation forms should not be handed out to anyone as samples. Should you be asked to provide a sample form, refer the request to the program administrator.

Conductor Evaluation Reports

Conductor evaluation reports are summary tabulations of evaluations sets. Two types of reports are generated by the computer program:

• An individual report is a tabulation of a single set of evaluations, summarizing the answers submitted by the members of a single orchestra.
• A cumulative history is a tabulation of all evaluations on a conductor, regardless of the number of evaluations that have been submitted, the variety of orchestras that submitted evaluations, the number of musicians who participated in evaluations, and any duplication of orchestras.

Conductor evaluation reports are sent only to delegates, who are responsible for their safekeeping and proper use. Delegates receive conductor evaluation in two ways:

• Delegates automatically receive individual reports on evaluations, which they submit from their own orchestras. Receiving these reports requires no further action from the delegate other than sending in completed evaluation forms with a cover sheet.
• Delegates may receive reports of evaluations of conductors by other orchestras for use by their orchestra’s management or artistic advisory and conductor search committees. Securing this information requires following a special procedure outlined below under the heading “Sharing Conductor Evaluation Information.”

CAUTION: Any evaluation information sent to a delegate is for the use of that delegate’s orchestra only; delegates are not to share information with other orchestras or persons, even if those parties provide a letter of indemnification.

CAUTION: Delegates are not to share with management any conductor evaluation information, including evaluation reports on evaluations by the delegate’s own orchestra, without receiving the proper written request. (See the model letter below under “Sharing Conductor Evaluation Information.”)

Reading Evaluation Results

“Number Of Evaluations” indicates how many musicians participated in the evaluation. The column labeled “Count” indicates how many musicians responded to each question.

Numbers in the column headed “Average” indicate the average opinion of those who responded to each question. The scale runs from 1 to 9 for the first question and from 1 to 5 for the remaining questions, reflecting the number of response categories to that question. As an example, if the mean of a given statement on the 5-point scale were 1.71, the average response to that statement would be between “Strongly Agree” and “Agree”; if the mean were 4.15, the average would be between “Disagree” and “Strongly Disagree”.

Numbers in a column headed “Standard Deviation” indicate the range of response. The larger this number, the greater the difference of opinion among the respondents. As an example, a standard deviation of 0.58 on a given statement would indicate a small range of response; a standard deviation of 2.67 would indicate a wider range of response, although not necessarily strong divergence of opinion within the orchestra. The Standard Deviation in question 1 will generally be larger than in questions 2A-2L.

Numbers in columns headed “1” through “9” for question 1, or “Strongly Agree”, “Agree”, “Neutral”, “Disagree”, “Strongly Disagree” for questions 2A-2L, are percentages of the total number of musicians who responded to that category on the form.

Sharing Conductor Evaluation Information

When a delegate is asked by his or her orchestra manager or artistic committee for access to ICSOM conductor evaluation material to help in reaching decisions about engaging music directors or guests conductors, the delegate must secure a formal signed request to the delegate, written on official stationery. This letter should be based on the following model:

This letter confirms our request for evaluations of [names of conductors]. We request this information for [specific purpose]. We promise to make no other use of this material than that set forth herein, and we understand that it is given to us on this basis only. We agree that we will not show this material to, nor reveal its contents to, any conductor or to anyone outside the management of this organization [or: outside the artistic committee], including the media. We will not copy this data. In the event you or ICSOM suffer any loss or damage as a result of our breach of the above assurances, we agree to indemnify and hold you and ICSOM harmless. Upon conclusion of our review, we will return the material to you promptly.

If management or an artistic committee wants to see evaluations by other orchestras, the delegate must send his or her personal request for this information, written on committee stationery, accompanied by a copy of the management/committee-to-delegate request, to the program administrator.

Delegates should not contact Wayne State University directly asking for release of information. WSU personnel will release information only upon written authorization of the program administrator.

Delegates wishing ROPA or OCSM evaluation information should follow the same procedure as above; they should not call ROPA or OCSM personnel. All ICSOM requests are processed through the ICSOM administrator. Delegates wishing ROPA or OCSM evaluation information should so state in their request.

Unless delegates request orchestra-and-date-specific individual evaluation reports, they will receive one cumulative history for each conductor. Delegates who think that histories may not provide adequate information and who want individual reports from specific orchestras should request specific reports.

The time between sending requests and receiving evaluations is usually at least two weeks and may be longer, depending on mail service and the work load at Wayne State.

Use Of Evaluation Information: Guidelines And Cautions

The question of whether the compilation and distribution of conductor evaluation information might make ICSOM or its orchestras and musicians answerable in any suit for libel arose when the program was adopted. The issues remain the same as outlined by ICSOM legal counsel in 1968: proper and limited use of the evaluations should leave ICSOM and its members free from any liability. Delegates are responsible for this proper use.

In clarifying what was proper and limited, legal counsel has always recommended that statements about conductors be neither recklessly nor maliciously made, that opinions about conductors be distributed only to persons having a common interest those opinions, and that requests by management for evaluation information be made in writing with an assurance of indemnification.

Leaks to the press are a major concern. Publication of evaluation information in any periodical or paper with a circulation outside the limited professional sphere might be protected under the right of the press to make “fair comment” on public figures, as long as such comment has relevance to the person’s public character or performance, but there are many differing views as to how far the press, or anyone, can go in this regard.

Accordingly, ICSOM orchestras are advised to take special care with evaluation results. While this information is appropriately shared with an orchestra’s artistic advisory personnel, wider dissemination of this information is to risk its being carelessly or deliberately abused. Conductor evaluation information is confidential and should not be given out freely or shown to the press, any conductor or music director, the board of directors, or concert managers. However, very general results may now be shared verbally with the orchestra-at-large at orchestra meetings.

If management wants to see any ICSOM conductor evaluation results, the delegate must secure management’s written request for this information. Model wording is provided above under “Sharing Conductor Evaluation Information.” When submitting evaluation information to management, a delegate should include a letter stating that this information is confidential, cannot be given or shown to anyone outside management, should not be copied, and should be returned to the delegate upon completion of management’s review.

If musicians on an orchestra’s artistic advisory or conductor search committee wish to have access to ICSOM conductor evaluation information, the delegate should advise those musicians of the proper use of the material and of the cautions noted in this manual. Delegates are advised to follow the same procedure as above, securing a written request and statement of understanding before showing material to these musicians.

Delegates having questions about these procedures or about any use of evaluation information should contact the program administrator or ICSOM legal counsel.

All ICSOM orchestras depend on the delegate in each orchestra to protect the confidentiality and to observe the professional use of conductor evaluations. Any orchestra whose representatives do not respect this trust may be denied further access to this information and may be barred from further participation in the program.

ICSOM Counsel Leibowitz’s 2003 statement
regarding ICSOM’s confidentiality policy:

Since its inception, the ICSOM conductor evaluation program has been one of ICSOM's more useful innovations. The ability of orchestras to receive the opinions of colleagues throughout the country within hours of requesting them has enabled them to make informed decisions regarding the engagement of music directors and guest conductors without spending hours and hours on the phone or e-mail seeking the same opinions.

However, legal problems have prevented the widest possible dissemination of the results and details of the evaluations, as evidenced by a lawsuit for libel brought by a conductor some years ago. Although we successfully defended against that suit, such litigation is time-consuming and expensive. Thus, the standard letter that has been part of the Delegate Manual for many years was designed to protect the musicians and ICSOM from lawsuits by requiring managements who want the information to take legal responsibility if such a suit is brought.

However, since ICSOM has no desire to impose such potential liability on individual musicians or orchestra committees, it was thought best not to share the evaluation information with the members who actually completed the forms.

In recent years, that prohibition has apparently been, in no small measure, responsible for a diminution in the use of the program. Because the ICSOM Governing Board believes that the program should not wither for disuse, and because some protections can be provided through the Internet, it now proposes liberalizing the rules and establishing a system of online access with password protection. Nevertheless, caution should not be completely thrown to the wind. The risk of "loose lips" remains, and the jeopardy is all-encompassing. At risk is not only the speaker of the alleged defamation, but possibly ICSOM, the local union, the AFM, and the speaker's orchestra association. A substantial measure of protection would be to reveal only the vote tally on the question of whether or not to recommend engaging the conductor evaluated.

If that is sufficient for the musicians, and if an individual musician could say all that is that the vote was, for example, 80-20 to recommend not engaging him or her, then there is no real risk because such a statement, if true, is not actionable. Truth is an absolute defense to an allegation of defamation.

Further discussion about the future of this program will be held at the 2005 Conference and beyond. Please give me a call at 404-875-8822 or e-mail me at mooretuba@bellsouth.net with any questions, requests, or problems.

9A Conductor Evaluation Report & info - 2009