Author:	Kelley Burton B Bus, LLB (Hons) Associate Lecturer, Queensland University of Technology Faculty of Law
Subjects:	Law - study and teaching (Other articles) Lecture Method in Teaching Teaching -- Aids and Devices
Issue:	Volume 11, Number 4 (December 2004)
Category:	Refereed Articles

Contents

• Introduction
• The Audience Response System
• Audience Response System Trials in Other Disciplines
• Audience Response System Trialed in a Law Lecture at QUT
• Lawyering Skills and Generic Skills Developed by the Audience Response System
• Feedback from Students about the Audience Response System
• Feedback from the Lecturer about the Audience Response System
• Words of Advice and Caution for Law Lecturers
• Conclusion
• Notes

Introduction

1. The Queensland University of Technology (QUT) law school is a leader in legal education because it uses innovative teaching and learning tools. Recently, the QUT law school introduced an audience response system, which gives students the power to respond to questions or statements in lectures posed on Powerpoint slides. The students respond by pressing a button on a keypad, which is similar to a television remote. The whole process of the audience responding to a multiple choice question has been demonstrated on television show, Who Wants to be a Millionaire.

2. This article considers the additional resources required by a law school before it could use an audience response system in lectures. It considers how other disciplines have taken advantage of an audience response system and how it was trialled in a second year undergraduate law subject at the QUT. The trial assisted the law lecturer’s ability to deliver the content and developed the lawyering skills and generic skills of the second year undergraduate law students. The feedback from the second year undergraduate law students and the law lecturer at the QUT are extracted and the transparency of the audience response system to other law lectures and tutorials is examined. Other law lecturers should embrace such innovative and interactive technology, but should take into consideration the words of advice and caution provided at the end of the article.

   The Audience Response System

3. A law school needs extra resources before they can use an audience response system in their lectures. In particular, they need to install Turning point software and purchase a sufficient number of keypads so that each law student can be issued with one in the lecture. Staff must be trained on how to use the audience response system.

4. The audience response system used by the QUT law school allows students who attend the lectures to respond to questions or statements posed on a Powerpoint slide by pressing a button on a keypad. The questions may consist of multiple choice questions, true or false questions and yes or no questions. Law lecturers may pose statements and the law students may respond by selecting, for example, strongly agree, agree, disagree or strongly disagree. The number of buttons on the keypad limits the number of responses. The law lecturer can set a response reminder when they create their Powerpoint slides as this requires the law students to respond within a certain time limit.

5. A grid at the bottom of the Powerpoint slide has a cell for each keypad. A cell is highlighted once the student holding that keypad has responded. This enables the student to confirm that their response has been recorded by the audience response system. It also informs the lecturer how many students have responded. The audience response system can be used so that the lecturer and other law students can or cannot determine how a particular student has responded to a question or a statement. Maintaining anonymous responses encourages students to respond because they feel protected. This is important where a large cohort of students uses an audience response system. Workloads of law lecturers or administrative staff increase if responses are not kept anonymous because student name and the number of their keypad must be entered into a spreadsheet prior to the lecture. This increased workload may not be feasible, particularly if the audience response system is only used for formative assessment.

6. Once the audience response system has received the responses, it can generate a pie or bar graph to display the results. Each possible response shows a percentage, which represents the proportion of students that selected that response. The correct response can be indicated in a different colour. The audience response system is able to generate results in a report, which is presented in the form of an excel spreadsheet.

   Audience Response System Trials in Other Disciplines

7. Academics across the globe have used audience response systems in disciplines other than law, for example, medicine, engineering, science, maths, economics and psychology.[1] However, audience response systems are a novel teaching and learning tool and the literature discussing the pedagogical effectiveness of them is limited. Two notable contributions have been made by Caroline Elliott from the Lancaster University in the United Kingdom and Jeremy Williams from the Queensland University of Technology in Australia.

8. In the United Kingdom, Elliott trialed an audience response system in a second year undergraduate economics subject at the Lancaster University.[2] Only 47 students were involved, which is a small cohort of students. Elliott used the audience response system to anonymously test the students' understanding of the lecture material and tailored the content of subsequent tutorials and lectures to remove any confusion.[3] She also claimed the audience response system enabled her to vary the lecturing learning experience of her economics students and thus was able to encourage active learning, increase their interest and concentration levels.[4] Elliott also recognised that the audience response system provided students with “an easy method of gauging their own understanding, and comparing their performance against that of their peers”.[5] However, she did not provide any statistical evidence to support this statement.

9. Elliott did caution that lecturers cover less lecture materials if they use an audience response system.[6] However, she considered that this was overcome by the fact that she had a better understanding about the “amount of material that students understand”[7] and the “pace of student learning”.[8]

10. In July 2003 in Australia, Williams from the QUT Brisbane Graduate School of Business trialed an audience response system in a postgraduate program, the Masters of Business Administration (MBA).[9] One of the reasons for introducing such innovative technology was because he received negative feedback about the use of Powerpoint slides. He claimed the popularity with Powerpoint slides “seemed to have waned to the point where, faced with a torrent of multi-coloured, animated slides, students began to describe the experience as being akin to “death by Powerpoint”.[10]

11. Williams' trial only involved a small cohort of students, that is, 43 students.[11] At the end of the trial, he recommended that an audience response system be utilised as opposed to a normal Powerpoint presentation because “the students interact with the content on screen instead of staring at it passively”.[12] The MBA students overwhelmingly supported the audience response system, indicated that they were more likely to respond because of the anonymity and thought the audience response system could be used more in the MBA program.[13] Perhaps the positive feedback from the MBA students was influenced by the novelty of the new technology and different feedback would be received in five years time, when a new learning tool is likely to be captivating academics and students. This would mean that the audience response system would have the same fate as Powerpoint.

12. Typically, as students are exposed to new learning tools, their expectations of tertiary education lifts. Perhaps as the audience response system gains wider acceptance amongst academics, the students will expect the audience response system to be utilised in lectures and the audience response system will “constitute normal service”.[14] If students studying combined degrees witness other disciplines taking advantage of new learning tools, they will expect law lecturers to do the same.

13. This article builds upon the research and trials conducted by the academics in other disciplines because it demonstrates that law schools can take advantage of an audience response system in their undergraduate law program even if they have a much larger cohort of students.

    Audience Response System Trialed in a Law Lecture at QUT

14. In semester 1, 2004, the audience response system was trialled in a second year law subject, Real Property A. The size of the internal Real Property A cohort was approximately 392 students. All internal full-time and part-time students were encouraged to use the audience response system during their week five lecture. In 2004, attendance at lectures by internal students decreased because of the availability of audio streaming, that is, students can listen to the lectures through the online teaching websites. Approximately 120 full-time and 40 part-time students participated in the trial of the audience response system. Unfortunately, the external students were unable to take advantage of the audience response system because they do not physically attend lectures on campus. They listen to the lectures on tape or via the online teaching website.

15. During the week 5 lecture, Real Property A students learned about adverse possession. The students were lectured on cases and legislation pertaining to adverse possession. In particular, they learned when a right of action to recover land from an adverse possessor accrued and when it expired. Once students have been lectured on a legal topic, they are encouraged to apply the legal authorities to a set of facts. In an effort to assist the students to apply the legal authorities they had been lectured, the students were given four factual scenarios relating to adverse possession time limits. A multiple choice question was devised for each of the four factual scenarios. Each multiple choice question had four responses. One of the responses was correct. Each student was issued with a keypad and they were given approximately five minutes to read and respond to each question.

16. As alluded to in part 1 of this article, law lecturers could use an audience response system irrespective of what area of law they are lecturing. If the area of law is black and white, the law lecturer could create Powerpoint slides that have multiple choice questions, true or false questions or yes or no questions. If the area of law is grey, perhaps the lecturer could create a statement and ask the law students to agree or disagree. The responses to the statement could lead to a discussion within the lecture group. As a result, it is difficult to envisage an area of law where the audience response system could not be used in law lectures.

17. In addition to using the audience response system in lectures, it could be used in tutorials. It would be more manageable to issue keypads and to collect keypads from students in tutorials rather than lectures because fewer students attend. It would also improve the short-term retention of law students that attend tutorials.[15] However, retention rates are likely to be a larger issue for lectures, which are longer in duration.

18. Most of the tutors in the law school at QUT do not provide Powerpoint presentations and their workload would increase dramatically if they were required to create interactive Powerpoints for the audience response system. The QUT tutors would also need to hire the audience response system technology from audio-visual services and organise for it to be set up in the tutorial room prior to the tutorial. Given this extra workload, it may not be worthwhile using the audience response system in tutorials.[16] Students already contribute to discussion and apply the legal authorities to the facts in tutorials, without using an audience response system. Another argument against using the audience response system in tutorials is that the content of tutorials should be consistent, irrespective of the tutor. Therefore, if one tutor uses the audience response system in a subject in a certain week, all tutors who teach that subject in that week should use the audience response system. Preferably the same questions and statements would be posed in each tutorial group. This would present a tremendous strain on resources where there are many repeated tutorials in a week and where there are many tutors.

19. Irrespective of whether the audience response system is used in lectures or tutorials, it does have a place in legal education and not just on television programs or in lectures in other disciplines. In an effort to consider how valuable the audience response system is to law students, the next part of this article discusses the skills developed by the Real Property A students when they used the audience response system.

    Lawyering Skills and Generic Skills Developed by the Audience Response System

20. The Australian Law Reform Commission recommended that legal education should focus on skills as well as the content of the law.[17] This has prompted law schools across Australia to embed lawyering skills[18] and transferable generic skills into their curriculum so that they can provide their law students with a competitive edge when they enter the “ever-changing global workplace”.[19]

21. Christensen and Kift have categorised skills as attitudinal, cognitive, communication and relational.[20] The audience response system developed attitudinal skills because it required the law students to be proactive in lectures and engage in the lecture materials. It developed their ability to solve problems because the students were given an authentic factual problem with four possible responses, one being correct. The students developed their information technology skills because they were exposed to how the system collected and processed the responses. Therefore, the students were able to develop cognitive skills. It also developed their communication and relational skills because the students were able to discuss the problems and the responses with their fellow students or work independently. They were able to exercise their time management skills because they were given a time limit within which to answer questions.

22. The development of skills in law schools has received widespread attention across the globe, for example, the American Bar Association has devised a list of lawyering skills.[21] The list includes problem solving, legal reasoning, communication, organisation and management of legal work. As discussed above, using an audience response system in lectures enabled students to develop all of these skills.

23. As the audience response system is a novel learning tool, there have not been any long-term pedagogical studies done on its effectiveness.[22] However, it does assist in the delivery of content and the development of skills.

    Feedback from Students about the Audience Response System

24. Feedback was obtained from the Real Property A students about the audience response system because they had not been previously exposed to this novel technology. During the week 5 lecture, the students were asked to respond to the following statements using the audience response system. Only three statements were put to the students because otherwise too much of the lecture time would be used evaluating the audience response system. The possible responses were strongly agree, agree, neutral, disagree and strongly disagree. Approximately 120 full-time and 40 part-time students provided feedback on the audience response system. The proportion of responses for each possible response is represented as a percentage. Table 1 indicates the responses of the full-time students, whereas Table 2 indicates the results of the part-time students.

     

    Table 1

    Full time students

     

    Statement posed to students on the audience response system	Strongly agree	Agree	Neutral	Disagree	Strongly disagree
    Audience response system helped me to learn the lecture materials	45%	39%	7%	3%	5%
    It was useful to compare my responses with other students	44%	30%	18%	2%	6%
    Audience response system should be used in more lectures	50%	27%	7%	6%	10%

     

    Table 2

    Part time students

     

    Statement posed to students on the audience response system	Strongly agree	Agree	Neutral	Disagree	Strongly disagree
    Audience response system helped me to learn the lecture materials	53%	26%	11%	8%	3%
    It was useful to compare my responses with other students	32%	30%	19%	8%	11%
    Audience response system should be used in more lectures	53%	21%	18%	3%	5%

    There was no major disparity in the feedback from the full-time and part-time internal cohorts of Real Property A students as both cohorts were overwhelmingly positive about using an audience response system in lectures.

25. In addition to these three statements, the Real Property A students were invited to provide feedback by emailing the lecturer. The following comments were received by the lecturer.

    I thought it was great, because we were able to get immediate feed back as to whether we were on track or not with your questions.

    Someone should give you a hand with the distribution and collection so less lecturing time is used.

    You obviously put a lot of time and effort into the preparation of your powerpoints. It was also very light-hearted so that makes a nice change when assignments are looming.

    Congratulations on using technology to create a genuinely interactive lecture experience that is sure to improve the usually poor knowledge retention rates of this format.

    The system's anonymity is non-threatening and your discussion of the answer after receiving results gives students the opportunity to pay special attention to areas in need of further work.

    It would also be useful for the teacher to determine how well their points are coming across.

    I hope this system will be used for more law lectures. There's a real problem with some lecturers relying very heavily on reading out their prepared notes already delivered on the overhead slides. Requiring these interactive questions to be incorporated into these lectures would force them to extemporize and prevent the audience from sleeping. It would be a quantum improvement in the learning process in these lectures.

    Feedback from the Lecturer about the Audience Response System

26. One of the main weaknesses with the trial of the audience response system in Real Property A is that only one lecturer, the author of this article, was involved. In an effort to counteract the skew of the lecturer’s feedback, the lecturer’s feedback has been compared with the feedback from, Caroline Elliott, a lecturer in the United Kingdom and Jeremy Williams, a lecturer in Australia, who have trialled an audience response system in the microeconomics and business disciplines.

27. The Real Property A lecturer agreed that the audience response system helped them to teach the lecture materials. In 2003, the lecturer gave the Real Property A students the same factual scenarios, but did not give them the multiple choice responses. The lecturer asked the Real Property A students to discuss the answer with the person next to them. After giving the students a few minutes to work out the answer, the lecturer asked for volunteers to report back their answer. Usually, only a couple of students would be involved in reporting back their answers. In 2004, the lecturer gave the Real Property A students the same factual scenarios, but also gave the students multiple choice responses. The lecturer agreed that the audience response system helped them to teach the lecture materials because it gave more Real Property A students an opportunity to participate in lectures and demonstrate their knowledge of the legal authorities and their ability to apply the legal authorities to the facts.

28. This is consistent with Elliott’s feedback from her trial in microeconomics in the United Kingdom.[23] Elliott stated that the audience response system “offers a method of active engagement: ‘if students are to learn to think, they must be placed in situations where they have to do so. The situations in which they are obliged to think are those in which they have to answer questions because questions demand an active mental response”.[24] Similarly, Williams stated that he trialled an audience response system to “inject some life into didactic sessions”[25] and to get the students to engage in the content on the powerpoint slides.[26]

29. The Real Property A lecturer strongly agreed that it was useful to see the different responses of the students. In particular, in 2003, the lecturer only heard the responses of a couple of students. However, in 2004, the lecturer saw the responses of many more students and was alerted to misunderstandings held by the students. This allowed the lecturer to quickly revise the legal authorities, resolve confusion and immediately provide feedback. Correspondingly, Williams debriefed his students after each question.[27] Elliott from the Lancaster University in the United Kingdom, went a step further and tailored the content of her subsequent tutorials based on the responses.[28]

30. The Real Property A lecturer does not agree that the audience response system should be used in every law lecture because of the additional workload involved in creating the interactive Powerpoints and setting up the audience response system.

    Words of Advice and Caution for Law Lecturers

31. Before using an audience response system, law lecturers should consider the following words of advice and caution.
    • The cost of purchasing the additional resources to operate the audience response system. In particular, determine how many students will use the audience response system in a lecture and purchase a corresponding number of keypads. Lecturers will need to ensure that they have access to the Turning Point software, which will enable them to create interactive Powerpoint slides.
    • Law lecturers will need to be trained on how to create interactive Powerpoints before using the audience response system in lectures. Law lecturers will also need to be trained on how to set up the audience response system in the lecture theatres and how to pack the equipment away, if the audience response system is not affixed in the lecture theatres. Alternatively, audio visual support staff may be trained to cope with setting up and packing away of the equipment.
    • If the audience response system is not permanently affixed in the lecture theatre, arrive at the lecture theatre before the lecture in order to set up the audience response system and ensure that it is working. Perhaps law lecturers could book the lecture theatre for half an hour before the lecture starts to set up the audience response system. If the audience response system is not permanently affixed in the lecture theatre and the lecture immediately follows another lecture, it might be unwise to use an audience response system because there is not time to set it up. Also make sure you have enough time at the end of the lecture to pack the audience response system away and collect the keypads.
    • Use the audience response system for formative assessment rather than summative assessment. Before the audience response system could be used for summative assessment, you would need the system to identify which keypad each student holds and you would need to ensure that students cannot see the responses chosen by other students. In addition, you would need to monitor the students and ensure that they do not have the opportunity to talk to each other or pass notes about the responses.
    • Remember that you could create multiple choice questions, true or false questions or yes or no questions. These questions could be used to quiz students on case law or on legislative provisions. You could also use these questions to assist students to apply legal authorities to factual scenarios. You could also create statements for the students to agree or disagree. Statements are useful for grey areas of the law because they will generate discussion amongst the students.
    • When creating the interactive Powerpoint slides, limit the amount of time that students have to answer the question in by setting a response reminder. Also remember that you will cover less materials in the lecture if you use an audience response system because of the amount of time students need to read and respond to the questions and/or statements.
    • The more responses given to students the longer it will take students to read and respond.[29]
    • Determine if you are going to identify a certain student response by the number on the keypad they are holding. Remember that students are more likely to respond if their responses are anonymous. If you decide to receive anonymous responses, hand out the keypads to students in the order they are sitting and you will be able to immediately identify if any keypads are missing. This will make the collection of the keypads at the end of the lecture easier.
    • Inserting a grid at the bottom of the interactive Powerpoint slide will help the students to identify if their response has been recorded by the audience response system. If there are more than 50 students in the lecture, a rotating grid must be used. Fundamentally, a rotating grid shows some of the keypad numbers at one time and rotates so that eventually all keypad numbers have been displayed.
    • Be selective in your colour scheme. Each response could be displayed in the same or a different colour. When the results appear, the correct response could be displayed in a different colour to the incorrect responses.
    • Law lecturers should advise students that the keypads are useless outside of the lecture theatre, for example, they look like a television remote but they cannot operate a television. This information may deter students from stealing the keypads. Ensure that the batteries are screwed into the keypads in an effort to deter students from stealing the batteries.
    • If you are using the audience response system for your full-time internal students, make sure you make the system available to your part-time internal students as students should have an equal opportunity to access such technology. Unfortunately, external students are unable to access such technology because they listen to the lectures on tape or via the on-line teaching site. They do not physically attend lectures. However, if external students are required to attend an external student attendance school, perhaps the audience response system could be made available at that time.

    Conclusion

32. Law lecturers should catch up to lecturers in other disciplines and embed an audience response system in their lectures because they create an interactive and innovative learning environment. The feedback obtained from the law students and the lecturer after the trial at the QUT law school was positive and other law lecturers could successfully implement an audience response system if they consider the above advice and caution. Law students benefit from using an audience response system because they develop lawyering skills and generic skills essential to the legal profession. Consequently, there is a point to giving power to law students in lectures.