USER GUIDE CONDUCTING CLASSROOM OBSERVATIONS: Stallings “Classroom Snapshot” Observation System for an Electronic Tablet World Bank Education Global Practice July 2017 This manual was prepared by Barbara Bruns, Center for Global Development and World Bank, Audrey Schuh-Moore, Mathematica and FHI 360, and Elizabeth Adelman, Harvard University and AED. It was edited by Aliza Marcus. The World Bank’s Strategic Impact Evaluation Fund supported this work. Table of Contents 1. Why Observe Classrooms Using The Stallings Method?. . . . . . . . . . . . . . . . . 5 2. Observer Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3. Using The Tablet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 4. Using the Stallings Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 5. Classroom Materials and Group Size. . . . . . . . . . . . . . . . . . . . . . . . . 17 6. Classroom Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 7. Analyzing Stallings Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 8. Coding Practice Examples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Appendix 1: Sample School Director Questionnaire. . . . . . . . . . . . . . . . . . . 45 Appendix 2: Classroom Background Data. . . . . . . . . . . . . . . . . . . . . . . . 49 Appendix 3: Original Stallings Coding Grid. . . . . . . . . . . . . . . . . . . . . . . 50 Appendix 4: Coding Worksheet. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 TABLE OF CONTENTS 3 Why Observe Classrooms Using the Stallings Method? The “Stallings Classroom Snapshot” instrument, technically called the “Stanford Research Insti- tute Classroom Observation System”, was developed by Professor Jane Stallings for research on the efficiency and quality of basic education teachers in the United States in the 1970s. (Stallings, 1977; Stallings and Mohlman, 1988). The Stallings instrument generates robust quantitative data on the interaction of teachers and students in the classroom, with a high degree of inter-rater re- liability (0.8 or higher) among observers with relatively limited training, which makes it suitable for large-scale samples in developing country settings. (Jukes, 2006; Abadzi, 2007; DeStefano et al, 2010; Schuh-Moore et al, 2010). The instrument is language and curriculum-neutral, so results are directly comparable across different types of schools and country contexts, and a growing body of comparative country data from the US and developing countries is available. The Stallings instrument generates quantitative measures -- at the classroom, school, and school system level -- of four main variables: • Teachers’ use of instructional time • Teachers’ use of materials, including Information Communication Technology • Core pedagogical practices • Teachers’ ability to keep students engaged Key features of the Stallings instrument make it well-suited to large-scale use in developing coun- try contexts. However, several factors need to be borne in mind when interpreting its results. First, there is a clear potential for Hawthorne effects, as teachers are aware of the observer (and sometimes pair of observers) physically present in the classroom – unlike observation methods that place a video camera in the classroom for extended periods so as to minimize these effects. One operating assumption, therefore, is that Stallings observations capture teachers’ performing at their very best – or production possibility frontier - which is in fact useful to measure. A second issue is the potential noisiness of the variables being measured; if the same teacher were observed on different days or with different student sections on the same day or with a different WHY OBSERVE CLASSROOMS USING THE STALLINGS METHOD? 5 cohort of students the following year, how consistent would the measured performance be? Ini- tial studies in the US called for visits to each classroom on two different days. To lower costs, the protocol followed by the World Bank prioritizes observing large cross-sections of classrooms – on average five different classrooms in each school over the course of a single school day - rather than repeat visits to the same schools and classrooms. Results are not assumed to be valid for individual teachers and the World Bank research protocol preserves the anonymity of individual teachers. But the method generates robust, representative results at the school, region and school system level. A third issue is the non-random assignment of teachers to classes in most of the school systems observed. Even when students are not explicitly ability-tracked, classroom assignment rules may de facto result in some teachers facing much more gifted or docile students than others. Thus, when we evaluate the correlations between teacher practices and student learning, we cannot be sure of the direction of causality. Are students learning more because their teachers are managing the classroom better? Or are teachers able to manage the classroom better because their students are more motivated? Finally, what makes the Stallings instrument versatile and robust across different grades, sub- jects, languages and countries is that it does not try to measure the content of what is being taught – either the depth or sophistication of the curriculum content itself or the teacher’s mastery of that content. Two sixth grade classrooms in different countries could appear identical in terms of the classroom dynamics measured by the Stallings instrument, even though one is teaching a much higher level of science content than the other. Similarly, a teacher’s practice may look high- ly interactive, while he or she is making factual errors that are not captured in the Stallings data. Therefore, it is clear that the dimensions of classroom practice captured by the Stallings instru- ment are not a complete measure of the quality of teacher-student interaction and cannot be ex- pected to explain all of the variance in teacher effectiveness (whether measured as value-added learning gains or average student learning performance) across different classrooms. However, studies using the Stallings instrument to date have established that the four main dimensions of teacher practice that are captured show relatively consistent correlations with student learning results, both in cross-sectional and value added learning analysis. (Bruns and Luque, 2014) The simplicity and robustness of the data generated by Stallings classroom observation studies have contributed to its influence with policymakers. Several Latin American countries have in- stitutionalized annual Stallings observations in a representative sample of schools in order to track progress in raising teachers’ classroom effectiveness; it has had an impact on the design of in-service training programs for teachers and school directors in virtually every country that has carried them out; it has stimulated changes in the way teachers are screened pre-employment in some systems; and it is increasingly being used in impact evaluations of programs and policies aimed at raising teacher effectiveness. Use of the Stallings classroom snapshot in more than seven countries in the Latin America and the Caribbean region in recent years has generated a global data base of more than 20,000 dif- ferent classroom observations in more than 3,600 schools. A public use online database is being created on the World Bank/SIEF (Strategic Impact Evaluation Fund) website. These data pro- vide valuable reference benchmarks for any country or education system that uses the Stallings instrument following the protocol outlined in this guide. 6 User Guide: Conducting Classroom Observations Observer Protocol 1. Introduction to the School Director. On arrival at the school, you should contact the school principal, introduce yourself, and give a brief explanation of the study. You should empha- size that the observations are for research only. There are no consequences for any teacher or school that is observed. Teachers remain anonymous; neither their names nor any other identifying characteristics are noted. The ultimate objective is to identify effective teaching practices that can be shared across classrooms and schools. 2. Director’s Questionnaire. It is useful to collect complementary information about the school from the school director. A paper questionnaire may be left with the director to fill out while you are observing classes. But it is usually preferable to fill out the questionnaire (wheth- er paper or tablet-based) yourself in a separate interview. Specific questions for directors should be identified by the research team before the school visits, but an example of a Direc- tors’ survey is included in Appendix I. 3. Schedule of Classes and Substitute Classes. Observers or observer teams typically spend at least one full day at each school, to observe a maximum number of different teachers. An hourly schedule of classes to be observed should be provided to each observer in advance of the visit. Classes should be identified by subject, grade, section, class code (if available) and room number. For each class hour, there should be 1 to 2 acceptable substitute classrooms identified, in case the scheduled class is not operating normally (for example, a test is being administered) or the regular teacher is absent. It is preferable to concentrate observations on the school’s regular teachers. 4. Arrival in the classroom. You should arrive in the classroom you are scheduled to observe before the scheduled class time. Introduce yourself quickly to the teacher and inform her/ him that this observation is part of a research study on classroom dynamics in a randomly selected sample of schools and not an evaluation / monitoring of the teacher’s performance. Assure teachers that the classrooms to be observed have all been selected randomly and indi- vidual teachers will remain anonymous. Finish the conversation with the teacher before the official class starting time. OBSERVER PROTOCOL 7 5. Preparatory work: Record the exact observation times and school background informa- tion. Come prepared with your tablet fully charged and one blank worksheet for each classroom you will observe. Find a seat at the back of the classroom where you can have a good view of the entire classroom without disturbing the students. There are two things that are helpful to pre- pare before you begin your observations, and in advance of arriving in the classroom if possible. First, in the boxes in the upper right corner of your worksheet, fill in the exact times at which you will make your observations in that class, based on the official class start time. How to do this is explained in more detail in Section IV of this manual. Second, fill in basic information about the school and class that is requested on the first five screens. How to do this is ex- plained in more detail in Section III of this manual. You will find it is helpful to fill in as much of this information as possible before arriving at your first classroom! 6. First observation. Allow three minutes to pass after the bell rings (or the official start time of the class) before you make your first observation. Then, scan the classroom for 15 seconds, starting with the teacher. How to do this is explained in more detail in Section IV. 7. Write notes on your paper worksheet first. This will be a back-up in case you have any trou- ble with the tablet. It will also help you in answering the questions on the tablet. 8. Coding your observation. When you start coding your observation on the tablet, the screens will guide you through six or seven different questions about what you saw the teacher and students doing during the 15 seconds of your observation. Answer all the questions on the tablet. You must fill in all blanks or you cannot advance to the next screen. 9. After you have answered all the questions (which means that the observation has been ful- ly coded) you reach a screen that asks you to write on the tablet (in words) what you saw at the moment of observation. This keeps an electronic record of what you wrote on your worksheet – which makes it more efficient to store this information. Use the keyboard on the tablet to write what you saw. Normally, you will have enough time to complete this before the next observation. If you cannot finish, you can leave this section incomplete on the tablet and come back to it later. (Note, you must enter at least some text in this section, such as “xxx”, or the tablet will not let you advance to the next screen.) 10. Your highest priority is to make each observation at the exact, correct time. Do not wor- ry if you have not finished entering a word description of what you saw on the tablet if it is time for the next observation. Continue onto the next observation. If you have filled out your worksheet correctly, you will always be able to go back later and enter the short description (usually a maximum of 1-2 sentences) of what you saw. 11. Description of the Classroom. While you are getting settled in the classroom and making observations, pay attention to the facilities and classroom materials. What kinds of equip- ment does the classroom have? (Blackboard, whiteboard, digital projectors, laptops, etc.) What is on the walls? (Posters, art work, maps) What are the students wearing? (Do they all have uniforms? Shoes? Backpacks?) What learning materials do the students have? (Do they all have textbooks? Workbooks? Pens and pencils? Tablets? Calculators?) This kind of descriptive information can provide useful correlates of the classroom dynamics data. 12. Since you will not have much time before your first observation, you will be asked to answer these classroom description questions at the END of the class – after you have completed 8 User Guide: Conducting Classroom Observations your ten observations -- when you will have more time. You will also be asked to make a final count of the number of students in the classroom, to make sure that students who arrived late are counted. See Appendix II for the specific questions asked. 13. Thank the teacher. After you have finished your coding and the class period has ended, quickly thank the teacher and proceed quickly to the next classroom you are to observe. OBSERVER PROTOCOL 9 Using the Tablet 1. Turn on the tablet. 2. Touch the ODK Collect icon on the home page. 3. Choose the first option Fill Blank Form 4. Choose Stallings Observation System 5. The first page you will see is the Observer Protocol, as a reminder. 6. The next page is Background Data. It is helpful if you can have this information filled in BEFORE you arrive at your first classroom. This information includes: a. NAME OF OBSERVER b. OBSERVER CODE c. DATE OF OBSERVATION d. SCHOOL NAME e. SCHOOL CODE f. CLASSROOM CODE g. SHIFT OBSERVED: Morning or Afternoon h. WAS THE REGULAR TEACHER PRESENT? Yes/No i. IF THE REGULAR TEACHER WAS ABSENT, WAS THERE A SUBSTITUTE TEACHER? Yes/No i. GRADE (1-8) j. SUBJECT (Language, Math, Science, … Other) k. NUMBER OF STUDENTS: it will ask you to count the boys and girls separately 7. The next screen is Schedule. You are asked to note the official start time and end time of the class. Please use military notation for this (as in 0000 to 2400 for showing time). 8. The next screen is Level: Pre-school/Primary/Secondary 9. The next screen is Grade: 1-8 10 User Guide: Conducting Classroom Observations 10. The next screen is Subject Matter: Language/Math/Biology/Physics/Chemistry/History/ Geography/Other 11. How many students are present in the classroom? Count the Boys and Girls separately 12. The next screen is: Observation 1 a. Class time: use military notation to note the exact time of your first observation (this time should already be written in the top right corner of your worksheet) b. Starting with the teacher, please observe the classroom for 15 seconds, then record your observation. 13. Follow the same process for all ten observations. 14. Description of the Classroom. After you complete the ten observations, fill in the questions about a description of the classroom. These are: a. Is there student work on the walls? Yes/No/NA b. How many students have a textbook or workbook? Entire class/More than half/Less than half/No one/NA c. How many students have a notebook, pencil, pen, or other writing material? Entire class/More than half/Less than half/No one/NA d. How many students are wearing uniforms? Entire class/More than half/Less than half/ No one/NA 15. Materials a. Does the teacher have the following materials? (You may select more than one option.) Blackboard/whiteboard; Chalk/markers: Textbook; Teaching guide; Laptop; Digital whiteboard; LCD projector; Television; Audio equipment b. Are the following materials available for children in the classroom? (You may select more than one option.) Science equipment; Maps; Blocks; Toys; Games; None of the above USING THE TABLET 11 Using the Stallings Method 1. What does the classroom snapshot method do? The classroom snapshot records the activities of the teacher and students, and the materials being used in the classroom, at ten separate instances throughout a class period. 2. Why is it called the classroom snapshot? The method makes a record of the people and activities in a classroom at a single moment in time as if they were being photographed, hence the word “snapshot." Each “snapshot” obser- vation lasts for 15 seconds. 3. What is recording or coding an observation? Recording (or coding) an observation is the process of observing the classroom and answer- ing a series of detailed questions on the tablet about what you saw. You respond to each ques- tion by selecting one of the multiple-choice options and filling in the circle in front of that option. When you have answered all the questions about what you saw during that observa- tion, you will have recorded or coded the observation. It will generally take you less than one minute to complete all the questions for each “snapshot,” or to complete the coding for that observation. Timing The Observations DURATION OF THE CLASS 1 2 3 4 5 6 7 8 9 10 (MINUTES) 30 3 4 7 10 13 16 19 22 25 28 40 3 7 11 15 19 23 27 31 35 39 50 3 8 13 18 23 28 33 38 43 48 60 3 9 15 21 27 33 39 45 51 57 70 3 10 17 24 31 38 45 52 59 66 80 3 11 19 27 35 43 51 59 67 75 90 3 12 21 30 39 48 57 66 75 84 120 3 15 27 39 51 64 75 87 99 111 12 User Guide: Conducting Classroom Observations 4. The strength of the Stallings method is that it is a way of converting the qualitative ac- tivities and interactions between a teacher and students that occur during a class into robust quantitative data on teachers’ instructional practice and students’ engagement. Since all Stallings variables are expressed as a share of class time, it is very important to keep close track of time and to complete your observations at exact intervals. To determine the exact time for each observation, divide the total class time by 10. For example, in a 50-min- ute class, you will take a snapshot every 5 minutes for a total of 10 snapshots. In a 90-minute class, your observations will be 9 minutes apart, etc. Step 1: To help you in coding on the tablet, you will have a one page paper worksheet where you can make brief written notes about what you observe at the exact moment of each observation. Each worksheet is divided into 10 sections, to provide a few lines of space for notes about each of your 10 observations. In each of these 10 sections, there is a box in the upper right corner for you to write the time of each observation. Step 2: Use the table above to determine the exact time at which you will make each observation. For example, for a 50-minute class that starts at 9:00 AM, your observations will be at 9:03, 9:08, 9:13, 9:18, 9:23, etc. Write these times in the boxes in the upper right corner of each of the 10 sec- tions of your worksheet. This will help you keep track of the time. Step 3: You make the first observation exactly three minutes after the official start time of the class. Note that this is the procedure no matter how long the class is. Note also that you must make an observation at this time, even if the teacher and students are not yet in the room or the teacher has not yet started to teach. Step 4: Write on your worksheet first. Immediately after you finish your observation, write a few lines of notes on your worksheet about what you saw. Step 5: Code the observation on your tablet. When you have completed your notes, begin an- swering the questions on the tablet about that observation. There are six to eight different ques- tions about each observation (depending on what the teacher and students are doing). You will have plenty of time to answer all of them. After you have answered the questions, the tablet asks you to copy the notes you wrote on your worksheet in electronic form on the tablet. This makes it possible to store this information long term. It is sometimes frustrating to type on the tablet keyboard. Do not worry if you cannot finish transcribing your notes before it is time for your next observation – you will have time to com- plete this later. The priority must be to go on to your next observation at the correct time. Step 6: Continue to make observations at the appropriate times until you complete and code all 10 observations. (Note the interval between your final observation and the end of the class will be shorter than the other intervals.) Step 7: After you have completed all 10 observations, complete Description of the Classroom questions. The tablet also asks you for a final count of the number of girls and boys in the class- room. Answer these questions as quickly as you can, so that you are ready to move to the next classroom. USING THE STALLINGS METHOD 13 How to Record an Observation When it is time to start your observation, first locate the teacher, and then scan the classroom going clockwise around the room. Remember always to begin by locating the teacher. Then continue scanning the room and observing the activities and materials of all students until you have observed the entire classroom. In many cases, some groups of students will be engaged in activities that are different from the activity the teacher is leading. Be careful to note all these on your worksheet. The classroom “snapshot” Observing a classroom: • Divide the total class time by 10 to determine the interval between observations. For a 50- minute class 50/10 = 5, meaning an observa- tion every 5 minutes • Wait three minutes after the official start time of the class for the first observation • Starting with the teacher, scan the classroom in a circular direction to note what activity the teacher is leading and what the students are doing • After 15 seconds, write what you observed on your worksheet and then code it on the tablet • Repeat this 10 times during the class at regu- lar intervals After 15 seconds, make your notes on the worksheet about what you saw. Immediately after writ- ing your notes, begin answering the questions on the tablet. The first screen asks: What is the teacher doing? Select ONE of the activities on the list. You must select one. At any given moment, the teacher can only be doing one activity. The second screen asks: What material is the teacher using for the activity? Select ONE of the materials on the list provided. The second screen also asks: How many students are engaged in the activity with the teacher? Select ONE of the options: One; a Small group (5 or fewer students); a large group (6 or more students); All The third screen asks: For students NOT engaged with the teacher, what are they doing? (Choose all that apply) You may select more than one answer here, as there can be multiple small or large groups of students involved in distinct activities, such as texting, doing homework, reading, chatting with other students, looking out the window, sleeping. 14 User Guide: Conducting Classroom Observations For each activity that you observed students engaged in, there will be subsequent screens asking: What materials are students using in the activity? (Choose all that apply) and How many stu- dents are using material (xx) in activity (xx)? Here again the options are: One; a Small group; a Large group; and All Make sure that you code only what happened at the exact moment of the observation. For example, if after six seconds, the teacher changes the activity, you will not change what you coded before. Never forget that the first step is to locate the teacher and the first seconds of the obser- vation focus on what the teacher is doing at that instant. The following sections of this manual discuss the different coding options in more detail, and provide practice in making important distinctions about the activities going on in the classroom and the materials being used. USING THE STALLINGS METHODS 15 Classroom Materials and Group Size Descriptions of the materials used in the classroom snapshot 1. No Material: No material of any kind is being used in the classroom at that moment. 2. Textbook: This category refers to any printed materials that students do not write in directly. This category includes textbooks, reading books, and magazines. This can also include pho- tocopies or newspapers. 3. Notebook and writing implements: This category refers to any materials that students work with and write in. For example: notebooks, workbooks, worksheets, journals, slates, or blank sheets of paper on which students work on problems, write answers, or write essays and sto- ries. This category also includes writing materials such as pencils, crayons, or erasers. 4. Blackboard/whiteboard: Blackboard, chalkboard or whiteboard. 5. Learning aids: This category includes all kinds of visual aids and manipulatives that teach- ers use to help students understand. Learning aids include maps, charts, photos, posters and flipcharts. Manipulatives include materials used in science experiments, rulers, compasses, currency, calculators, blocks, flash cards, sticks or human bodies. Note that electronic calcu- lators are coded as learning aids and not ICT. 6. ICT: This category includes electronic learning aids such as digital whiteboards, projectors, radios, televisions, videos, computers, laptops, tablets and smart phones being used for in- structional purposes. Note that digital whiteboards are coded as ICT, even if teachers use them exactly the way they would use a traditional blackboard. Note that smart phones are only considered ICT if the TEACHER is using them for instruction – not if the students are texting or playing games. 7. Cooperative activity and shared materials: The Stallings method gives priority to measur- ing teachers’ use of cooperative, or group, learning activities. A cooperative activity is one where a pair of students or a group of students is working together, and there is only one product of their work – a group product: for example, a map they all help fill in, or a single report that they write and present together. In this case, you should code “cooperative activ- ity and shared materials” as the material used. If “cooperative” is recorded, no other material should be noted. Note: if students are sitting in groups, but they are each working on individ- ual worksheets, it is not considered a cooperative activity. Definitions of group size 1. Small group: Five or fewer students 2. Large group: Six or more students CLASSROOM MATERIALS AND GROUP SIZE 17 Classroom Activities Activities 1-6 represent instructional activities. ACTIVITY 1: READING OUT LOUD The teacher or one or more students are reading aloud. They may be reading from a textbook, the blackboard, their own writing, or a handout. The teacher or student may be reading out loud while the rest of the class follows along in their own books. Question: What is the teacher doing? Choose Reading out loud if: • The teacher is reading out loud • A student is reading out loud • A group of students is reading in unison • ALL the students in the class are reading in unison Example The teacher is reading the class a story from a picture book. She is pointing to the pictures as she reads. All of the students are listening intently. What is the teacher doing? Reading out loud 18 User Guide: Conducting Classroom Observations What material is the teacher using for the activity? Textbooks How many students are engaged in the activity with the teacher? All ACTIVITY 2: DEMONSTRATION/LECTURE The teacher, radio, television or some form of media is informing, explaining or demonstrating academic content to the students. Generally, this category is used when the teacher is presenting new academic content to the students (i.e. the activity is teacher-led) – although a student or group of students making a presentation to the class would also fall into this category. Question: What is the teacher doing? Choose Demonstration/Lecture if: • The teacher is lecturing to the class • The teacher is demonstrating something – a science experiment, a math problem, how to pronounce a word • The teacher is singing a new song • The teacher is reviewing material that was taught in a previous class • A student or group of students is making a presentation and the teacher is listening (in this case, the teacher is considered to be leading the activity and the students are coded as engaged with the teacher) • A student or group of students is making a presentation and taking the role of the teacher, but the teacher is not present (in this case, select “Teacher out of the classroom” and code what the students are doing in the next series of questions) Example The teacher is explaining a history lesson to a class of 40 students. She is using a map to point to regions as she describes them. The entire class is listening to her explanation. CLASSROOM ACTIVITIES 19 What is the teacher doing? Locate the teacher. She is presenting a history lesson to the students and using the map as an aid. Choose: Demonstration/Lecture What material is the teacher using? She is using the map. Choose: Learning aids How many students are engaged in the activity with the teacher? All students are listening to the teacher. Choose: All ACTIVITY 3: QUESTION AND ANSWER/DISCUSSION The teacher is asking or answering questions or exchanging ideas with the students about an academic topic. Question: What is the teacher doing? Choose Question and Answer/Discussion if: • The teacher is asking the students questions • A student is asking the teacher a question • A student is asking another student a question (or responding to another students’ ques- tion) • The students are debating a topic suggested by the teacher • The teacher is exchanging ideas with the students • DO NOT code this if: the students are asking questions about a non-academic subject (such as the date that homework papers are due, or what will be served for lunch.) Ques- tions that do not relate to the academic topic being studied are coded under “Classroom Management” or “Social interaction” as discussed later Example The class is divided into six groups of five students each. The groups are working on a group assignment, to create a map of the countries in Latin America. Students are exchanging ideas on what to include, but the product will be one map per group. The teacher is in one corner of the room discussing a math problem with one student at his desk who is not involved in any of the groups. They are looking at the student’s notebook. What is the teacher doing? Locate the teacher. She is in the corner, working with a single stu- dent, asking the student questions about a math problem. Choose: Question and Answer/ Discussion CLASSROOM ACTIVITIES 21 What material is the teacher using? She is using the student’s notebook. Choose: Notebook/ writing implement How many students are engaged in the activity the teacher is leading? Only one student is working directly with the teacher at that moment. Choose: One Note that your scan of the classroom has also observed the other students in the room engaged in a different academic activity. This must also be recorded, as it is part of the overall class learning activities. For students NOT engaged with the teacher, what are they doing? (Choose all that apply.) The other students, working in groups, are also engaged in question and answer/discussion. Choose: Question and Answer/Discussion 22 User Guide: Conducting Classroom Observations What materials are these students using? The students are working in small groups on an ac- tivity with a joint (group) product. This is a cooperative learning activity. Choose: Cooperative activity and shared materials. How many students are cooperative in the Question and Answer/Discussion activity? A large group (6 or more students) Choose: Large group ACTIVITY 4: PRACTICE & DRILLING Activities that are undertaken with the objective of memorizing material such as multiplication tables, vocabulary, spelling words, poems or songs. The activities involve repetition with the main goal being to simply memorize the information. Question: What is the teacher doing? Choose Practice and Drilling if: • If the teacher is leading practice – calling out multiplication problems or vocabulary and having the students repeat it • If two or more students are practicing addition facts, spellings words, the alphabet, or any rote learning activity • If students are singing a song they already know CLASSROOM ACTIVITIES 23 • Pay close attention to whether the students are repeating or reading. If the students are repeating after the teacher, code this as practice/drill; if they are reading along with the teacher, code this as reading aloud • Keep in mind that this activity is about memorizing information – it is a rote activity. If a student is actually solving a multiplication problem, it should be coded as Question and Answer. If students are simply memorizing and repeating multiplication tables, code that as Practice and Drilling. • If students stand at their desks to do a routine physical activity, such as stretching their arms or jumping up and down, code this as Practice and Drilling Example The teacher is in the front of the class and she is pointing to the blackboard. She is reading out loud the multiplication table for the number 2 and asking the class to repeat the table multiple times. The entire class repeats after her. What is the teacher doing? She is in front of the class and leading the repetition of the times tables for two that are written on the blackboard. Choose: Practice and Drilling What material is the teacher using? She is using the blackboard. Choose: Blackboard/ whiteboard How many students are engaged in the activity with the teacher? The whole class is chanting the drill in unison. Choose: All 24 User Guide: Conducting Classroom Observations ACTIVITY 5: ASSIGNMENT/CLASS WORK One or more students are writing papers, solving problems on the black board, reading silently at their desks, or working on a test or a quiz. Question: What is the teacher doing? Code Assignment/Class Work if: • The teacher has assigned the students to work on an exercise in their workbooks • The teacher has asked the students to solve several math problems written on the black- board • The teacher is moving around the room monitoring the students as they write an essay • The students are taking a written test or quiz. Please note in your written comments if a test or quiz was administered. DO NOT code the following activities as Assignment/Class Work: • A student is reading a comic book under his desk while other students work on problems in their workbook • A student is texting while other students are working on a work sheet Example The teacher is watching a student solve a math problem on the blackboard. The other students are all observing the student at the blackboard. CLASSROOM ACTIVITIES 25 What is the teacher doing? The teacher is monitoring the student solving the problem on the board. Choose: Assignment/Class Work What material is the teacher using for this activity? Choose: Blackboard/whiteboard How many students is the teacher engaging in this activity? Even though only one student is actively solving the problem at the blackboard, all the other students are watching this student and therefore engaged in the same learning activity. Choose: All 26 User Guide: Conducting Classroom Observations ACTIVITY 6: COPYING Students are copying from the blackboard. The primary purpose of the activity is to transfer the text on the board verbatim to the students’ notebooks or paper. Question: What is the teacher doing? Code Copying if: • The teacher is copying something onto the blackboard (from a teachers’ manual, text- book, or other material) that the students are expected to copy • Students are copying something that the teacher has written on the blackboard or copy- ing something from a book or handout (Note that even though the students are doing the copying, the teacher is leading the activity the students are engaged in) • The teacher is dictating a passage and the students are copying down what they hear • The teacher is monitoring students while they are copying from the blackboard • If some students have finished copying math problems or questions from the blackboard and are beginning to solve the problems or answer the questions, code these students as engaged in Activity 5 Assignment/Class Work. If other students are continuing to copy the material from the blackboard, code these students as Copying. In this situation, you need to make a judgment about which activity the teacher is leading: at the exact moment you began the observation, was the teacher more engaged in monitoring the students who are beginning to solve the problems or was the teacher continuing to monitor those stu- dents who are still copying? That will determine how you code the teacher. • Always code the material used by the teacher to teach the lesson. For example, if the teacher is writing a text on the black board while the students are copying in their note- books, code the teacher as copying and the black board as the material being used (since it is what the teacher is using in the process of teaching the students). If the teacher is copy- ing material from her own notes or manual onto the blackboard, still select the blackboard as the material being used, because it is what the teacher is using to teach the students DO NOT code Copying if: • The instructions or information the teacher is writing on the blackboard are not related to academic work (for example, the date and time of the school play). In this case, the teach- er is engaged in Classroom management (described in a later section). Use Copying for the teacher only if the teacher is writing lesson-related material on the board, or monitor- ing the students while they copy lesson-related material. Example The teacher monitors the students as they copy a text from the blackboard into their notebooks. What is the teacher doing? She is monitoring the class, in the activity of copying. Choose: Copying CLASSROOM ACTIVITIES 27 What material is the teacher using? Choose: Blackboard/whiteboard How many students is the teacher engaging in this activity? Choose: All 28 User Guide: Conducting Classroom Observations Activites 7-10 are not considered instructional activities. They are classroom management. ACTIVITY 7: VERBAL INSTRUCTION You can ignore this category. It is no longer used. ACTIVITY 8: DISCIPLINE One or more students are being reprimanded for their behavior, being sent out of the room for disciplinary reasons, or otherwise being punished for their behavior. Question: What is the teacher doing? Code Discipline if: • The teacher is reprimanding one or more students: “Marina, stop talking to your partner” or “Roberto, you are late for class again” • The teacher is giving behavioral commands or reprimands to the whole class such as: “Everyone sit down and be quiet now!” or “I don’t want to see anyone with their phone on their desk” or “One at a time, as you line up for lunch” • The teacher is sending a student out of the room or to sit in a corner, as punishment for bad behavior • The teacher hits or slaps a student as discipline (corporal punishment). If this occurs, please describe it in your written notes. NOTE: This code does not require identifying any material used by the teacher. Example The teacher is reprimanding the entire class for not learning their French homework. The stu- dents are all sitting sullenly with their heads bowed in shame. What is the teacher doing? He is reprimanding the class. Choose Discipline CLASSROOM ACTIVITIES 29 How many students are engaged in the activity with the teacher? Choose: All ACTIVITY 9: CLASSROOM MANAGEMENT WITH STUDENTS The teacher and students are engaged in activities of an organizational or management nature: taking attendance, passing out papers, preparing to leave. Question: What is the teacher doing? Choose Classroom management with students if: • The teacher is engaged in an administrative activity and one or more students are also involved • The teacher is taking attendance and the students are responding as their names are called • The teacher is handing out papers that the students are helping to distribute • The teacher is giving instructions on HOW a homework assignment should be completed or the due date (note: the teacher is not answering questions about the substance of the assignment) • The teacher asks two students to put away the textbooks • The teacher asks a student to clean off the blackboard • The teacher is out of the classroom, but the students are passing out papers or collecting books. In this case, the teacher will be coded as out of the classroom, and the next prompt will be “For students not engaged with the teacher, what are they doing?” The response is: Classroom management with students. This is the activity the teacher launched with the students, even though he/she is not there. NOTE: This activity does not require identifying which material the teacher is using. Example The teacher is passing out papers and the students are helping. 30 User Guide: Conducting Classroom Observations What is the teacher doing? She is distributing papers with the students. Choose: Classroom management with students How many students are engaged in the activity with the teacher? Choose: All CLASSROOM ACTIVITIES 31 ACTIVITY 10: CLASSROOM MANAGEMENT ALONE The teacher alone is involved in classroom management: passing out papers, changing activities, putting away materials, preparing to leave. Question: What is the teacher doing? Choose Classroom Management Alone if: • The teacher is engaged in an administrative activity and the students are not involved • The teacher is writing the date and lesson title on the board (but not academic material) while the students sit and wait • The teacher is cleaning the blackboard • The teacher is sitting at a desk grading papers, while the students work on an assignment • The teacher is setting up an LCD projector, turning on a computer, or turning off and putting away equipment Example The teacher is putting away science materials in a cabinet in the corner of the room. The students are all chatting amongst themselves. One girl is looking out the window. What is the teacher doing? She is putting away the science materials by herself. Choose: Classroom Management alone 32 User Guide: Conducting Classroom Observations Note: Since the students are not engaged in the activity the teacher is leading, the next question will be: For students NOT engaged with the teacher, what are they doing? (choose all that apply) Most of the students are talking amongst themselves, but one girl is looking out the window. Choose: Social Interaction and Student Not Engaged ➢ How many students are engaged in social interaction? Choose: Large group How many students are not engaged? Choose: One CLASSROOM ACTIVITIES 33 Activities 11, 12 and 13 are neither instructional activities nor classroom management. They are considered “Off-task” activities for teachers. ACTIVITY 11: TEACHER IN SOCIAL INTERACTION WITH STUDENTS Two or more students are talking or laughing about non-academic activities. Both verbal and non-verbal (students passing notes to each other, or looking at a magazine together) interactions are included in this category. This category also covers disruptive activities such as students mov- ing around the classroom, shouting, shoving, etc. What is essential is that there is a non-academic interaction between at least two students. If the students are discussing a class assignment, do not code this as Social Interaction. Code this as Question and Answer/Discussion. If the teacher is interacting socially with the students, use this code. If this happens during class time, it is considered “off-task” behavior for the teacher. Question: What is the teacher doing? Choose Social Interaction between Teacher and Students if: • The teacher is chatting with the students about non-academic topics • The teacher is listening to the students chat about non-academic topics Example It is near the end of class and the teacher has finished describing the homework assignment for the next day. The students, who know that she is going to be married soon, begin asking questions about the wedding plans. The teacher spends a few minutes describing the plans. The students are listening and chatting. What is the teacher doing? She is chatting with the students about a non-academic matter. Choose Social Interaction with Students. 34 User Guide: Conducting Classroom Observations How many students are engaged with the teacher in this activity? Choose All ACTIVITY 12: TEACHER IN SOCIAL INTERACTION WITH ANOTHER ADULT OR TEACHER NOT INVOLVED The teacher and another person (parents/visitor/community members/another teacher) are in- teracting. The teacher is in the classroom but not involved in any academic activity and is not engaged with the students. Question: What is the teacher doing? Choose Social Interaction with another adult or Teacher Not Involved if: • The teacher is not working with or talking to any of the students • The teacher is not engaged in any visible classroom management activities (such as grad- ing papers, preparing a lesson) • The teacher is looking out the window, using a cell phone or reading from a book CLASSROOM ACTIVITIES 35 • The teacher is talking to a visitor at the door of the classroom (Note: even if the teacher is talking to a parent about the academic performance of a student in the class or talking to another teacher about an academic topic, such as the correct answer to a test question, code the teacher as Teacher in Social Interaction with another adult or Teacher Not Involved because the teacher is not engaged with any of his students at that moment). Example The teacher is at the door of the classroom talking with the school secretary about a meeting with the newly appointed school director. The students are copying the day’s lesson from the black- board in their notebooks. What is the teacher doing? She is talking with another adult about a subject not related to the class. Choose Social Interaction with another adult or Teacher not involved Because the teacher is not involved with any students, the next question will be: For students not engaged with the teacher, what are they doing? Choose Copying 36 User Guide: Conducting Classroom Observations What materials are students using in the copying activity? Choose Notebooks How many students are engaged in copying with their notebooks? Choose All CLASSROOM ACTIVITIES 37 ACTIVITY 13: TEACHER OUT OF THE CLASSROOM The teacher is not present in the room at the moment of your observation. Question: What is the teacher doing? Choose Teacher out of the classroom if: • The teacher has not yet arrived, but the official class hour has begun and there are stu- dents in the classroom • The teacher has left the room to go get materials. • The teacher has dismissed the class early and left the room, although the official class time is not over • The teacher has stepped into the hall to talk to the principal or a parent (and is out of the classroom and out of view) Example The teacher left the room to go get some chalk. The students are all chatting amongst themselves. What is the teacher doing? He left the classroom to get materials. Choose Teacher Out of the Classroom 38 User Guide: Conducting Classroom Observations For students NOT engaged with the teacher, what are they doing? Choose Social Interaction How many students are engaged in Social Interaction? Choose: Large group ACTIVITY 14: STUDENT(S) NOT ENGAGED This category is used when one or more students are visibly not engaged in activity with the teacher or with other students. For example, if a student is staring out the window, resting his/ her head on the desk, sleeping, or doodling on a piece of paper, he or she is uninvolved. This category may also be used when students are sitting quietly and waiting for the teacher to begin the lesson, or waiting for instructions about what to do next. Students who are walking in or out of the classroom for reasons that are unclear should be coded as Student Not Engaged. Students who are sent out of the room by the teacher for disciplinary reasons are coded under Discipline. NOTE: This category is used for students only. By definition, the teacher will be engaged in another activity. Example The teacher is leading the class in a foreign language drill conjugating verbs. She states the verb forms and then asks the class to repeat. The students listen to the teacher’s words and repeat after her. Three students walk into the classroom late during the activity. CLASSROOM ACTIVITIES 39 What is the teacher doing? She is leading a drill. The class is repeating after the teacher. Choose Practice & Drilling What material is the teacher using? Choose: No Materials How many students are engaged in the activity with the teacher? Choose: Large group 40 User Guide: Conducting Classroom Observations For students NOT engaged with the teacher, what are they doing? (Choose all that apply) Three students entered the classroom late. Choose: Student Not Engaged How many students are not engaged? Three students. Choose: Small group CLASSROOM ACTIVITIES 41 Analyzing Stallings Data In analyzing Stallings data, activities are grouped into four categories: a) Learning Activities b) Classroom Management c) Teacher Off-Task and d) Student(s) Off-Task. a) Learning Activities: 1. Reading Aloud 2. Demonstration/Lecture 3. Discussion/Debate, Question and Answer 4. Practice & Drill 5. Assignment/Class Work 6. Copying b) Classroom Management: 7. Verbal Instruction (not used) 8. Discipline 9. Classroom Management with Students 10. Classroom Management Alone c) Teacher Off-Task: 11. Social Interaction between Teacher and Students 12. Social Interaction with another adult/Teacher not Involved 13. Teacher out of the classroom d) Students Off-Task: 8. Discipline 11. Social Interaction 14. Student Not Engaged 42 User Guide: Conducting Classroom Observations Coding Practice Examples 1. The teacher gives directions to a group of five pupils. Twenty pupils are idle, and three play with their chairs. 2. A group of four students is solving a math problem on the blackboard. The teacher is watch- ing them. The rest of the class is solving the same problem on their slates. 3. All students are singing an "alphabet song". The teacher is singing with them. 4. In a science class, the teacher is using a large picture chart to talk about snakes. The entire class is listening to the teacher except two students who appear to be looking out the window. 5. The teacher is at her desk and appears to be grading papers. Students are sitting in groups of four to five. There are five groups in all. Two groups are working on an assignment at their desks; the other three groups are talking and laughing. 6. In a class of 20 students, almost everyone is repeating verb conjugations from the blackboard as the teacher points to the writing. One student is staring at you, the observer. 7. The teacher is in front of the class and is giving instructions to students on what to do next. All the students are listening to her. 8. Class has begun (according to the time in the class schedule) but the teacher is nowhere in sight. 9. The teacher is writing on the blackboard. 10. The teacher is listening to one student, who is standing, read from the blackboard. The other students are listening to the student as well. CODING EXAMPLES 43 11. The teacher is writing on the blackboard. Fifteen pupils copy in the exercise book and five students are not copying because they do not seem to have a pencil. 12. The teacher is explaining to the students the political system of Brazil. He is using a big post- er which shows the pictures of different government officials. The majority of the students are listening to him. Three students are looking at comic books and four others are talking amongst themselves. 13. The teacher is writing a math lesson on the blackboard. Eight students are playing cards in one corner. Thirty students are waiting for her to finish writing the lesson. 14. The students are writing an essay in their notebooks. The teacher is reprimanding two stu- dents who are not doing their work. One student enters the class late. 15. The teacher is asking a question and pointing to a student to give an answer. The student is standing and responding to the question. The rest of the class is listening. 16. In the back of the class, the teacher is speaking with a school administrator about a party. Twenty-six students are reading silently. Two students are looking out the window and 11 students are talking or laughing amongst themselves. 17. The teacher and the school director are talking about the exam. Most of the students are reading in their seats. Three students have finished their work and are erasing the black- board. 18. The teacher is helping several students do their assignments on worksheets. The rest of the students are reading at their seats. 19. The teacher is explaining a math problem. She is using little stones to demonstrate how to do addition. All the students are watching her. 20. The teacher is using bottles and water to show how a water filter works. All the students are listening except two who are talking to each other. 44 User Guide: Conducting Classroom Observations Appendix 1: Sample School Director Questionnaire PRINCIPAL'S QUESTIONNAIRE - JAMAICA Stallings Classroom Observations Dear Principal, We would like to take the opportunity of this school visit to solicit feedback from you about sector policies that affect your school. The questionnaire also includes several items that are part of an international research effort to collect data on trust and teamwork at the school level in different countries. Please kindly complete this questionnaire and return it to the visiting observer(s) before the end of the school day. We emphasize that this information, as well as the classroom observation data, are completely confidential. Thank you for your collaboration. School Name: School Code: Date of Visit: 1. How many years have you been the principal of this school? Totally Disagree Agree Totally Please mark only one answer. disagree more than more than agree Agree Disagree 2. My school’s results have improved in the last two years. a. b. c. d. 3. The tests in grades 4, 6 and 9 are well designed to reflect student learning. a. b. c. d. 4. Bonus pay for teachers based on annual student learning would stimulate stronger effort from teachers and better school results. a. b. c. d. 5. Bonus pay for teachers based on student learning would not be fair because some schools have better students than others. a. b. c. d. 6.The new teacher licensing standards will improve the quality of a. b. c. d. education. 7. The new teacher licensing standards will not achieve much unless salary levels also improve. a. b. c. d. 8. The proposed decentralized structure will give you more support in managing your school and will improve school results. a. b. c. d. 9. I have the resources and tools I need to deal with teachers whose performance is weak. a. b. c. d. 10. If I could reward my best teachers with higher pay, my school’s results would improve. a. b. c. d. ANNEXES 45 11. If I could remove my worst teachers, my school’s results would improve. a. b. c. d. 12. The emphasis on accountability and target setting by the Ministry of Education will result in improved school performance. a. b. c. d. 13. Inspection of schools is the missing critical element to support improved school performance. a. b. c. d. 14. The instructional approach used by most teachers motivates learning. a. b. c. d. 15. The teachers in this school are equally effective. a. b. c. d. 16. Approach to student discipline is influenced by the poor behavior of a few. a. b. c. d. 17. Student behaviour reduces the impact of teaching & learning in this school. a. b. c. d. 18. Students are not driven to perform. a. b. c. d. 19. Parents in this community are very involved in the school. a. b. c. d. 20. Teacher tardiness and absence are serious issues at this school. a. b. c. d. 21. Lack of learning materials is a serious issue at this school. a. b. c. d. 22. Crime in our community is a serious issue for our school. a. b. c. d. Totally Disagree Agree Totally The most serious constraints to improved results for my school are: disagree more than more than agree Agree Disagree 23. Lack of financial resources. a. b. c. d. 24. Lack of time for planning. a. b. c. d. 46 ANNEXES User Guide: Conducting Classroom Observations 46 25. Lack of time for teachers to work together to share lesson plans, ideas, etc. a. b. c. d. 26. Lack of time for me to observe teachers’ practice. a. b. c. d. 27. Lack of support from the community. a. b. c. d. 28. Lack of performance information and feedback. a. b. c. d. 29. Lack of effective training opportunities for me and my staff. a. b. c. d. 30. Other issues not mentioned here (please note these in the final comments section, if you’d like). a. b. c. d. Totally Disagree Agree Totally The majority of my time is spent on: disagree more than more than agree Agree Disagree 31. Administrative tasks and reporting. a. b. c. d. 32. Observing teachers in the classroom. a. b. c. d. 33. Dealing with parents and the community. a. b. c. d. 34. Trying to raise additional resources for the school. a. b. c. d. 35. Strategic planning. a. b. c. d. 36. Dealing with other issues not mentioned here (please note these in the final section if you’d like). a. b. c. d. Totally Disagree Agree Totally Social Capital Questions - Please mark only one answer. disagree more than more than agree Agree Disagree 37. Staff at this school are willing to help their colleagues. a. b. c. d. 38. Staff at this school have the same opinions about what is right a. b. c. d. and wrong. 47 ANNEXES User Guide: Conducting Classroom Observations 47 39. Staff at this school are people whom I can trust. a. b. c. d. 40. If I needed to borrow $ 1,000 in an emergency, I might ask a a. b. c. d. coworker at this school. 41. I am a person whom others can trust. a. b. c. d. 42. If an employee of this school needed to borrow $ 1,000 in an a. b. c. d. emergency, he(she) could ask me. 43. I trust most people in my community. a. b. c. d. 44. I trust most of the employees in my school. a. b. c. d. 45. Most of the people in my school would try to take advantage of me if they had the chance. a. b. c. d. 46. Most of the time people are preoccupied with themselves. a. b. c. d. 47. I can achieve success by myself, without help from those around a. b. c. d. me. 48. Money is important for happiness. a. b. c. d. 49. People who work hard usually end up in a better situation than a. b. c. d. those who do not. 50. Additional Comments: 48 User Guide: Conducting Classroom Observations Appendix 2: Classroom Background Data Classroom Observation Manual APPENDIX II: CLASSROOM BACKGROUND DATA CLASSROOM BACKGROUND DATA day month year 1. Date of Observation 2. School Name___________________________________________________________________________ ___________________________________________________3. School Code 4. Location (Name of City/Town/Village)___________________5. Location Code 6. Observer Name ____________________________________7. Observer Code 8. Grade Observed (circle grade) Grade: 1 2 3 4 5 6 7 8 9 10 9.Subject(s) Observed _____________________________________________________________________ hour minute 14. Official start time of class : 15. Time class started : 12. Official end time of class : 13. Time class ended : 14. How many students are present in the classroom After the first observation - Total GIRLS BOYS Before the last observation- Total GIRLS BOYS 15. Blackboard/whiteboard in the classroom? no yes not applicable 16. Display of charts, pictures, maps on the wall? no yes not applicable 17. Textbooks/Other book available? entire class more than half less than half very few nobody not applicable 18. Notebooks/Paper and writing utensils? entire class more than half less than half very few nobody not applicable 19. School Uniforms? entire class more than half less than half very few nobody not applicable 23. Comments 44 ANNEXES 49 Appendix 3: Stallings Classroom Snapshot Coding Grid Classroom Observation Manual APPENDIX III: STALLINGS CLASSROOM SNAPSHOT CODING GRID NO.: EXACT TIME OF OBSERVATION: CLASSROOM OBSERVATION SNAPSHOT MATERIAL NO BLACK LEARNING TEXTBOOK NOTEBOOK ICT COOPERATIVE MATERIAL BOARD AIDES ACTIVITY T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E S L E 1. READING ALOUD I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L CHECK IF CHORAL READING 2.DEMONSTRATION/ T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E S L E LECTURE I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L 3.DISCUSSION/ DEBATE/QUESTIONS T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E S L E AND ANSWERS I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L 4. PRACTICE T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E S L E & DRILL I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L 5. ASSIGNMENT/ T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E S L E CLASS WORK I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L 6. COPYING T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E S L E I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L 7. VERBAL T 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E 1 S L E S L E INSTRUCTION I 1 S L 1 S L 1 S L 1 S L 1 S L 1 S L S L 8. SOCIAL INTERACTION T 1 S L E Grade/subject being observed: Notes: I 1 S L 9. STUDENT(S) I 1 S L UNINVOLVED 10. DISCIPLINE T 1 S L E 11. CLASSROOM T 1 S L E MANAGEMENT I 1 S L 12. CLASSROOM MANAGEMENT ALONE T 13. TEACHER SOCIAL INTERACTION OR TEACHER UNINVOLVED T 14. TEACHER OUT OF THE ROOM T 50 46 User Guide: Conducting Classroom Observations Appendix 4: Coding Worksheet School Name Observer Name Grade Subject Date 1 2 3 4 5 6 7 8 9 10 ANNEXES 51