Authors:	Wendy Harris LLB (Hons), LLM (QUT) Lecturer, Queensland University of Technology School of Law
	Melinda Shirley BA, LLB (Hons), Grad Dip Leg Prac, LLM (Bond) Lecturer, Queensland University of Technology School of Law
Subjects:	Law - study and teaching (Other articles) Negotiation Professional Skills for Lawyers
Issue:	Volume 9, Number 3 (September 2002)
Category:	Refereed Articles

Paper presented at the Australasian Law Teachers' Association annual conference hosted by Murdoch University School of Law, Perth, Western Australia September 29 - October 2 2002.

Contents

• Introduction
• Graduate capability integration - Project 1
• An Assessment Framework - Project 2
• The Assessment Task in Trusts - A case study
• Providing an authentic environment
• Example
• Managing student expectations
• Example
• Assessment Criteria
• Example
• Example
• Reflective Role-Play Exercise
• Evaluating the Case Study Assessment Task
• Is it valid?
• Is it reliable?
• Does it help students to develop in the area being assessed?
• Is it manageable for both students and academics?
• Is the assessment as authentic as possible?
• Conclusion
• Notes

Introduction

1. In its major review of the Federal Civil Justice System[1] the Australian Law Reform Commission concluded that legal education should be more concerned with "what lawyers need to be able to do " as distinct from the traditional Australian approach which has been centred around "what lawyers need to know". In response to that recommendation and a number of other recent reports echoing the same theme,[2] the Queensland University of Technology, Faculty of Law undertook an extensive review of the core units within its undergraduate law degree.

2. As a result of a successful university large teaching and learning grant in 2000, the Faculty was able to integrate professional and generic capabilities within the content of all substantive undergraduate law units to ensure incremental development throughout the degree. Through a further university large grant in 2002, the Faculty is now addressing the challenges faced in the re-formulation of assessment and feedback practices to assure their validity and reliability in the new teaching and learning environment.

3. This paper will outline the issues faced in the design of an assessment task and appropriate assessment criteria for negotiation skills, the skill selected for integration into the Trusts Unit. Through a case study developed as an assessment task in Trusts, this paper will then demonstrate the method used to evaluate those criteria to ensure quality and best assessment practice.

   Graduate capability integration - Project 1

4. Essential to the project was the pedagogical aim to embed skills training within the content of learning and to specifically assess competency levels within each of the skills. Ideally this would lead to the development of a "student capability profile". To be effective, such a learning approach required each skill to be developed through a cycle of instruction, practice, feedback and assessment both horizontally and vertically through the LLB degree.

5. The first step required the identification of the generic and discipline-specific capabilities required by, and of, our graduates. These were ascertained from a number of sources including:
   • The 2000 research report of the Evaluation and Investigations Programme (EIP), Higher Education Division of DETYA, undertaken by ACNielson Research Services on Employer Satisfaction with Graduate Skills.[3]
   • The survey produced by the Centre for Legal Education concerning the use of skills by law graduates ("Vignaendra Report").[4]
   • The QUT Manual of Policies and Procedures[5] which sets out desired graduate capabilities;
   • International studies including the list of core skills and values set out in the American Bar Association's MacCrate Report[6] and surveys conducted by the Law Discipline Network of tertiary providers in the United Kingdom.[7]

6. Four broad categories of generic and discipline specific skills were identified to form the basis of the study, being: attitudinal skills; cognitive skills; communication skills; and relational skills. Those four categories were then broken down into specific skills which would be developed across the range of undergraduate core subjects within the LLB degree.

    

   Attitudinal Skills	Cognitive Skills
   Ethical Values	Problem Solving
   Creative Outlook	Legal Analysis
   Reflective Practice	IT Literacy
   Inclusive Perspective	Legal Research
   Social Justice Orientation	Document Management
   Adaptive Behaviour	Discipline & Ethical Knowledge
   Pro-active behaviour

    

   Communication Skills	Relational Skills
   Oral Communication	Work Independently
   Oral Presentations	Teamwork
   Advocacy	Appreciation of race, gender, culture & socio-economic differences specifically and diversity generally
   Legal Interviewing	Time Management
   Mooting
   Negotiation
   Written Communication
   Drafting

    

7. To ensure incremental development, each specific skill was then broken down into three levels to represent gradual attainment. The first level provides a theoretical framework for and application of the component parts of the skill, the second level provides an opportunity to practise the component parts of the skill and the third level offers an understanding of the skill in the context of practice.

8. A table was then devised which set out in relation to each skill, the abilities that a student would be expected to demonstrate at each of the three levels progressed across the LLB course over a 3-4 year period, and at the end of the course (Table of Skills). Each undergraduate core unit undertook to integrate one or more selected skills into their program. Cognisant of the evidence that "the construct (the skill) and the context work best when tightly enmeshed"[8] lecturers were encouraged to assimilate skills development and assessment with existing legal content.

9. In the unit Trusts, it was considered that negotiation skills, a sub-set of communication skills in the table of core skills, could be compatibly enmeshed with the legal content. An understanding of the variety of types of conflict within our society and the advantages of non-litigious dispute resolution was complimentary to a study of trusts and an important addition to the first and second year LLB curriculum. In accordance with the Table of Skills, the specific abilities that the skill of negotiation requires at the various attainment levels are set out below.

10. The first level (which is integrated within a first year core unit) requires students to demonstrate the ability to:
    • identify the advantages and disadvantages of negotiation as a dispute resolution process;
    • identify the principles of effective negotiation;
    • negotiate effectively in a simple one to one negotiation; and
    • reflect upon their own and others' performance.

11. Upon completion of the second level of skills training (which is achieved through the subject Trusts) students should demonstrate the ability to:
    • identify the types of conflict best suited to resolution by negotiation;
    • distinguish between positions and interests;
    • prepare a negotiation plan; and
    • negotiate effectively in a more complex negotiation exercise with the ability to reflect upon their own and others' performance.

12. After completion of this stage, students may go on to achieve a higher level of skills development through a later core or elective unit. At that third level, students are expected to be able to draw upon their previous instruction and translate and use their skills in a variety of different circumstances without guidance. Ideally, students should also be able to adapt and be creative in the way they approach the context and the particular skills that are used, although it is recognised that this may only be possible in the context of a clinical placement program.

13. At this higher level of development in the skill of negotiation, a student should demonstrate the ability to:
    • distinguish between types of conflict and the interventions appropriate for resolution of each;
    • distinguish between positions and interests and use appropriate verbal and non-verbal communication skills; and
    • negotiate effectively in a highly complex negotiation exercise with the ability to reflect upon their own and others' performance.

14. Upon completion of the course, a graduating student would be expected to demonstrate the ability to:
    • identify the factors indicating negotiation as an appropriate dispute resolution process;
    • identify and use the principles of effective negotiation; and
    • prepare for the negotiation process effectively; and
    • conduct and evaluate a negotiation session in a professional and responsive way.

    An Assessment Framework - Project 2

15. As legal educators we were faced with the challenge that existing assessment practices, like those of other legal education institutions, had content as their focus, the result being that only a limited range of skills had been developed and that students' ability to transfer skills from one context to another had not been very well developed.[9]

16. The Law Discipline Network concluded from its surveys that "assessment of skills within law was primarily done indirectly through the traditional methods of assessment: for example problem solving and intellectual skills are demonstrated through exams and/or dissertations; research skills are part of the assessment through the dissertations".[10]

17. There was a need to re-evaluate our assessment methods, keeping in mind the purposes of assessment and best assessment practices. This re-evaluation process is taking place under the second Teaching and Learning Development Grant referred to above.

18. The purposes of assessment are various[11] with its main objectives being to enable certification or classification of students' achievements and to promote and enhance students' learning.[12] These main objectives were also those of the Faculty when determining that skills development was to be specifically assessed in each core unit. Certification or classification was of particular relevance to the desired outcome of a " capability profile" for each student.

19. In relation to best assessment practices, it is generally recognised that good assessment is valid, reliable, and fit for its purpose.[13] A valid assessment method is one which assesses what it purports to assess.[14] The question can be posed "Does the assessed task actually assess what you want it to?"[15]

20. Reliability requires an assessment method which is reproducible. If an assessment task were totally reliable, independent assessors using the same criteria and marking scheme would come to the same result about a given piece of work. The Oxford Centre for Staff and Learning Development conclude that "the connoisseur' approach to assessment (like a wine-taster or tea-blender of many years experience, not able to describe exactly what they are looking for but 'knowing when they find it') is no longer acceptable.[16] Consequently explicitness of assessment criteria is vital for both the students and the markers. Fitness for purpose is a multifaceted concept that would include the task's authenticity and its manageability for class size. Authenticity relates to the extent to which the assessment uses tasks as close as possible to those which are the goal of the program.[17]

21. The project team has set itself the task of developing an assessment framework that will provide the theoretical basis for models of best assessment practice for certain of the graduate capabilities including oral communication. The hypothesis is that it should be possible to assure the quality of assessment methods for capability development by evaluating the efficacy of those tasks as against the framework developed. At this stage of the project, the team has developed a draft framework which incorporates, amongst others, the above criteria for best assessment practice. The project team intends to filter a range of assessment tasks through the framework in order to identify the best practice for the targeted skill areas. As a result of these trials and evaluations the framework may itself be modified for future use. It may also be that individual assessment tasks may require modification.

22. The assessment framework in its draft form consists of structured questions that will assist the trial and evaluation process:
    1. Is it valid?
    2. Is it reliable?
    3. Does it help students to develop in the area being assessed?
    4. Is it manageable, in the sense that it is efficient and effective for both students and academics?
    5. Is the assessment as authentic as possible?

23. The draft framework provides further details and questions under each of these criteria which will be referred to when applying the draft framework to the Trusts assessment task.

    The Assessment Task in Trusts - A case study

24. As Trusts is a second year subject in the LLB course, the skill of negotiation was required to be developed and assessed to the second level in accordance with the Table of Skills. The specific abilities for level two have previously been set out. Importantly, at this level students build upon the theoretical and practical foundations laid in level one, and may be provided with some additional guidance of an advanced level of skills.

    Providing an authentic environment

25. In re-designing the unit to accommodate the new initiative from the first project, it was essential to create an authentic assessment task. An authentic environment is sought at this stage to practice the skill, so that the student experience is linked to a real world legal scenario. Conscious of the criticisms that the traditional law school curriculum is tilted towards preparing students for 'legal combat' rather than the gentler arts of reconciliation and accommodation[18] the teaching team decided to use a problem-solving mode for the exercise.

26. Role-plays enable the exploration of communication skills and promote active learning in skills education. However, the economics of legal education generally in Australia make individual role-play exercises a problematic and unaffordable method of summative assessment. It was therefore decided to teach the interest-based approach to negotiation advocated in Getting to Yes[19] and set a case study in the context of a family estate dispute. The aim was to require students to step outside of the appellate judge role and appreciate the complexity of real life conflicts. Asking students to look for creative and integrative solutions on behalf of a client would hopefully raise awareness of dispute resolution choices and how to assist others to develop real life solutions as distinct from legal solutions to their problems.

27. The case study assessment task was ultimately titled "Life Problem" and required students to prepare a written analysis of a given problem together with a negotiation plan. Following submission of the plan, students were given an opportunity in tutorials to role-play the exercise and compare their performance with their plan. The following examples are from information and instructions provided to students through a number of sources including a study guide made available at the beginning of the semester and an on-line site also available from the beginning of semester and updated from time to time.

    Example

    Negotiation “Life problem” Exercise       20%   You will be required to prepare a negotiation plan (which may be done in teams of up to four (4) students) and conduct a negotiation based on a factual scenario to be provided.  Your written assignment will be worth 20% and will be due in week 5.  In week 7 of the tutorial program you will participate in a negotiated role-play based on the same factual scenario and attendance at that tutorial is compulsory for all internal students.  Parts of this exercise will take place during tutorials in weeks 2 and 7.  You will be also required to submit a report reflecting on your negotiation plan to your tutor in your tutorial.   For external students the negotiation role-play will take place during the attendance school and therefore attendance at the attendance school is compulsory.  You will be required to submit a report, reflecting on your negotiation plan to your tutor at the end of your tutorial at the attendance school.    Unit objectives (2) and (4) will be assessed by the Life Problem Exercise.

     

    Managing student expectations

28. Another important aspect of unit re-design was to ensure that student expectations were managed and that students were explicitly informed in relation to the changing objectives and teaching and learning practices in the unit.

29. The objectives of the unit were reformulated to expressly include the ability to negotiate effectively. The teaching and learning approaches in the study guide were amended to specifically alert students to the fact that skills theory and practice would be incorporated at various stages in the lecture program. This necessitated compulsory attendance for some classes, which had not existed under the previous regime, and given the number of externally enrolled students in the cohort, it was also necessary to specifically provide for alternative assessment should attendance at skills sessions prove impossible.

30. As the new teaching and learning approach required each skill to be developed through a cycle of instruction, practice, feedback and assessment it was also necessary to explicitly define for students the opportunities for formative assessment. Good teaching practice also seemed to require that students should be given a rational explanation for why the particular skill had been targeted in the unit in addition to an explanation of what was expected in the particular task.

    Example

     

    What is the relevance of this assignment?   The skills module in this course builds upon the principled negotiation theory you studied in first year.   In the first lecture of this course you will look at how to identify different types of conflict, how to determine which disputes are best suited to resolution by negotiation and how to prepare a basic negotiation plan.   The Life Problem gives you an opportunity to apply that theory in the context of a trust dispute. After handing in your negotiation plan in week 5 you will participate in a role-play in week 7 tutorials or the attendance school, to explore how well your plan prepared you and to further develop your negotiation skills.   The Life Problem is NOT a research paper on negotiation theory or the substantive law.   To do well in this assignment you will need to apply the theory covered to the problem posed. If you have not completed your first year law studies at this university, you will need to make yourself familiar with the theory of principled negotiation, which is well explained in the text: Fisher,R & Ury,W -  Getting to Yes, Business Books Ltd, UK, 1991, which you will find in the course reserve section of the law library.   Some past students’ comments on the Life Problem:   “It was interesting to deal with a real life situation and to see how many factors, however small, can affect the situation drastically”   “It allowed me to look at and deal with a problem which required application of skills that would not otherwise have been covered in a straight law degree, especially given that negotiation is such an important part of being a solicitor”   “The skills tute at the external school was excellent. Part of my initial defensiveness against this exercise was that it was hard to see the benefit of the skills exercise when just listening to the tapes.”

     

    Assessment Criteria

31. The Life Problem instructed students to prepare an advice for one of the parties to the dispute (maximum 1500 words) including the following:
    • Identification of the conflict/s
    • Consideration of negotiation as an appropriate resolution process
    • A negotiation plan

32. Students were provided with readings on conflict identification and option generation and were lectured specifically on conflict identification and the matching of types of conflicts with dispute resolution processes. A lecture was also dedicated to revising the principled negotiation theory taught at level one and demonstrating how to use that theory in the preparation of a negotiation plan.

33. The assessment criteria were devised for each of the prescribed tasks with the majority of marks being allocated to the negotiation plan itself. Marks were allocated for the application of the theory to the problem on the basis that a merely descriptive paper could not satisfy the demonstrated abilities required at this level.

    Example

     

    1. Identification of types of conflict (6 marks)   Explanation of type/s of conflict identified: - named and applied appropriately (up to 6 marks)   2. Discussion of negotiation as an appropriate process (6 marks)   - Identification of conflicts best suited to principled negotiation (up to 2) - Identification of conflicts which might require 3rd party process (up to 2) - A conclusion as to preferred option with reference to appropriate considerations eg: ongoing relationship for family members (up to 2)   3. Negotiation Plan -  (8 marks)   Discussion of people problems that may arise (up to 1.5) Discussion of likely interests underlying positions (up to 1.5) Identification of possible options for mutual gain (up to 1.5) Identification of ways to use objective criteria (up to 1.5) BATNAs & WATNAs[20] (up to 2)   Detailed explanation of these criteria as they applied to the problem was also provided.

     

34. Upon return of their paper each student received an individual feedback sheet which specified the areas of discussion and application which did not receive full marks along with comments from the examiner and a mark out of 20 for the exercise.

    Example

     

    Life Problem Feedback Sheet Trusts     Name/s: You could have scored a higher mark for section 1 if you: q       Used Moore’s sphere to classify the types of conflict q       Identified more examples of types of conflict q       Applied the theory to the problem more and explained how your classification of the types of conflict were justified q       Addressed conflicts also involving other parties, which may impact upon the negotiated outcome     You could have scored a higher mark for section 2 if you: q       Used Moore’s sphere to identify the most appropriate process for each different type of conflict q       Recognised that the interest and data conflicts were best suited to negotiation q       Recognised the possibility that a third party mediator may ultimately be required to assist with the value/relationship/structural conflicts q       Discussed the parties’ probable need for an ongoing relationship     You could have scored a higher mark for section 3 if you: q       Followed the process set out in the lecture on this topic q       Discussed the people problems that may arise q       Analysed the parties underlying interests q       Identified possible options for mutual gain q       Identified ways to use objective criteria q       Identified both sides BATNA & WATNA[21]   Comments:

     

    Reflective Role-Play Exercise

35. The tutorial role-play exercise gave students an opportunity to practice their principled negotiation skills and trial their negotiation plans in a "real life" context. They were each assigned a role randomly, and whilst they may have written their negotiation plan from the perspective of the other negotiator, their preparation should have encompassed viewing the dispute from both sides making the exercise a valuable way to explore the effectiveness of their plans.

36. Students were given 25 minutes to negotiate and 10 minutes to record any agreement reached. They were then debriefed as a large group in order to give formative feedback on their oral communication skills and to enable further exploration of the success of their negotiation plans. All students were asked to consider to what extent their negotiation plans had predicted the positions and interests of the parties, they were then asked to complete a reflection sheet to record the outcomes of that reflection process.

    Evaluating the Case Study Assessment Task

37. The assessment framework used to evaluate the Trusts "Life Problem" assessment task is still in its draft form. This evaluation is part of the project team's efforts to identify the best practice tasks for the targeted skill areas. Both the assessment framework and/or the individual assessment task may require modification upon completion of this process. As previously discussed, the draft framework is sourced from educational research world-wide and the key questions posed reflect the most fundamental issues on assessment best practice.

    Is it valid?

38. Does the Life Problem actually assess what it purports to assess? Can it be used to discover whether students have achieved the learning outcomes identified for the unit studied and does it allow students to demonstrate those achievements?

39. The Life Problem purports to assess a student's ability to:
    * identify the types of conflicts which may arise in a given fact situation,
    * assess the appropriateness of negotiation as a resolution process for the parties involved in those conflicts; and
    * prepare a negotiation plan in accordance with the Fisher & Ury principles.

    All of the available marks are allocated to the achievement of those three tasks.

40. The identified learning outcomes for the negotiation module of this unit (from the Table of Skills) are that students should demonstrate the ability to:
    • identify the types of conflict best suited to resolution by negotiation;
    • distinguish between positions and interests;
    • prepare a negotiation plan; and
    • negotiate effectively in a more complex negotiation exercise with the ability to reflect upon their own and others' performance.

41. The Life Problem requires students to demonstrate the first three abilities and provides for direct feedback on their strengths and weaknesses in relation to each task. The linked tutorial exercise then provides the opportunity for exploration and demonstration of negotiation skills (although the assessment and feedback are purely formative) and reflection on the success or otherwise of their performance.

42. This exercise scores very highly in relation to this first criterion.

    Is it reliable?

43. Would the marking of the task give the same result no matter who did it? The marking criteria applied to the written component of this exercise appear objective, although it is acknowledged that this may not reduce the subjectivity associated with grading.[22] To remedy this Rowntree[23] recommends carefully constructed marking schemes, repeat marking of some assessment scripts so as to make sure an individual marker maintains the same standards over time, and, where several markers are involved cross-marking to ensure consistency.

44. As resources demand that more than one assessor is involved in marking this exercise, it is proposed that to avoid inconsistency an experienced member of the teaching team will review a selection of students' work for the purpose of grading. Examples of each grade will then be provided to each marker as a guide to the standard required to achieve each grade.

45. The formative feedback to be given by tutors following the role-play is also problematic. A guide to appropriate feedback and what to look for in giving feedback is provided to each tutor, however it is still possible that there is variation in the way the principles are modelled by each. Whilst that assessment is purely formative in this exercise, it is an area of weakness that should be targeted in the formulation of a best practice model.

    Does it help students to develop in the area being assessed?

46. The Life Problem exercise is constructive in the sense that it builds upon the theoretical knowledge acquired in first year and offers students an opportunity to apply that knowledge. The written exercise also incorporates the higher levels of Bloom's hierarchy[24] by requiring students to classify conflict and then combine separate elements into a whole by developing a negotiation plan.

47. The oral communication and reflection components of the exercise also encourage students to judge the effectiveness of their performance to improve their learning opportunities. In those ways this exercise should promote student learning as it is delivered through an appropriate mix of written analysis and active learning.

48. Does the task build in opportunities for practice of the skill and for the provision of effective feedback and guidance to learners? In its current form the written component of this exercise does not provide an opportunity for formative feedback in the sense that students do not have an opportunity to practice identifying types of conflict from a factual scenario or to prepare a negotiation plan. As a result they are also denied feedback on such an exercise before being assessed upon it.

49. One way of addressing this deficiency may be to include the previous year's problem in the study guide and give students an opportunity to identify the types of conflict and prepare a negotiation plan in groups in a tutorial or lecture. The results of that exercise could then be peer assessed in light of an example of a good answer from the previous year's cohort. Such a modification would not only provide opportunities for formative feedback and practice of the skill, it would also ensure that students were aware of and familiar with the marking criteria to be applied so that there is a shared understanding between staff and students.[25]

    Is it manageable for both students and academics?

50. In determining the efficiency and economy of an assessment task Bone[26] focuses on the problems caused by excessive questioning, reading, commenting and grading of students' work. Alternatively the AUTC Project 2001 [27] refers to Gibbs's suggested strategies for making assessment more manageable which includes the recommendation that marking criteria be provided to students at the time an exercise is distributed along with opportunities for practice and formative feedback in order to understand the criteria in use.

51. On the Bone criteria, the summative aspects of this exercise appear to be manageable from the academic perspective. Although it is a compulsory assignment for a cohort which averages about 450, students are encouraged to work in groups (a skill which is developed in an earlier subject) to take advantage of the different perspectives that group work can bring to the analysis of a real-life problem. Gibbs also highlights the benefits of group work. [28]

52. On the AUTC criteria however this exercise again scores poorly for the reasons discussed above in the context of effective formative feedback. It is submitted that the same modifications discussed above should also be useful in making the exercise more manageable from the student perspective. By incorporating group work and peer assessment into this formative assessment, the task also becomes more manageable for staff when dealing with large classes.

53. The summative feedback sheet in so far as it contains generic responses to be marked where appropriate by the assessor, further reduces time spent on identifying common errors or deficiencies. The sheet does provide for individual comments where necessary.

    Is the assessment as authentic as possible?

54. Marcel and Wiseman[29] comment that many client problems might be solved with more satisfying results at less expense and in less time if lawyers understood what their clients really wanted and were able to respond to their clients' real needs. The Life Problem exercise is designed to teach and assess negotiation skills within that real life context.

55. The problem itself is adapted from an actual reported trusts case, in an attempt to ensure authenticity. Further, the tasks set are those recommended as best teaching practice for the development of negotiation skills[30] in a practical legal training program. Whilst it is acknowledged that the use of videos, clinical experience and observations of negotiation sessions may be even more valuable vehicles for the delivery of skills modules, the ever-shrinking resources of most law schools renders such methods "unmanageable" under the criteria already discussed.

    Conclusion

56. Graduate capability integration in the unit Trusts has introduced a new and challenging dimension to teaching and learning in the unit. The qualitative evaluation process being undertaken through the second large grant is adding significant value to that initial project.

57. The assessment framework has proved to be a practical and useful tool in approaching the difficult task of assuring quality in one's own assessment practice. As indicated, it has already led to proposals for change in the assessment methods of the case study assessment task. Through this evaluation, aspects of best assessment practice for negotiation skills can now be incorporated into the development of that skill at other levels throughout the course. To the extent that some of these practices are generic, they will also be transferable to assessment methods in other units.

58. The range of questions encourages the assessor to consider the student perspective as well as pedagogy and the increasingly significant resources issue. The framework and the best practice assessment methods it identifies will provide valuable assistance to academics designing new assessment tasks and methods, for academics engaged in reflective practice and for students seeking high achievement.