Educational Quality: Defining What's Important Page 1 of 6 THE WORLD BANK GtOUP 2 2 7 9 6 A | f PIvnrrr Fi idings l Africa _ _ rID Findings reports on ongoing operational, economic and sector work carried out by the World Bank and its member governments in the Africa Region. It is published periodically by the Africa Technical Department on behalf of the Region. Educational Quality: Defining What's Important Most primary schools in Sub-Saharan Africa suffer from very poor conditions for learning: dilapidated or half-completed buildings, insufficient desks, overcrowded classrooms, few or no learning materials, poorly educated and motivated teachers, and recitation as the dominant vehicle for learning. Recent attempts at improving the quality of African primary education have tended to rely on national educational policy reforms that build on the rich research findings of the last fifteen years linking educational inputs to student achievement. Most of the national reform efforts seem to assume that a national policy and the delivery of inputs to schools will be sufficient to change the quality of the learning process. However, the impact on students of policy-dependent changes that ignore the internal life of schools is usually limited. The guidelines presented in the study, Planning and Monitoring the Quality of Primary Education in Sub-Saharan Africa, offer an approach to improving educational reform policies and investments by revising the assumptions underlying current methods of planning and evaluation. In sum, it is the central role of the individual school, in all its complexity, that needs more attention in the planning and evaluation of educational quality in Sub-Saharan Africa. These guidelines for planning and monitoring the quality of education are not prescriptive and individuals involved in primary education in Sub-Saharan Africa will be disappointed if they look here for a direct solution to make primary education more effective. Rather, what is presented are a list of ingredients (the definitions and indicators drawn from research around the world) and an identification of the kinds of uses to which they may be put (assessment, planning, evaluation). The reader must be his or her own cook, choosing the ingredients (and changing them as desired) and planning their use according to the needs and sophistication of people in the concerned education http://www.worldbank.org/afr/findings/english/findI 6.htm 9/13/01 Educational Quality: Defining What's Important Page 2 of 6 system. The Conceptual Framework The framework consists of an interrelated network of sixteen school-related factors, organized into four groups, that influence student outcomes. The factors and their inter-relationships are presented in the figure below. The school itself, and the inputs to it, are themselves influenced by the institutional, cultural, political, and economic context surrounding it. Within this context, the supporting inputs flow into each school where the enabling conditions, school climate, and teaching/learning process combine to produce student outcomes. The student outcomes are characterized in four ways - participation, academic achievement, social skills, economic success - that go well beyond the narrow reliance on student testing that prevails in most African countries. On the basis of a literature survey of school effectiveness research, definitions have been identified for each of the sixteen factors, and generic indicators have been identified for each definition. For example, one of the enabling conditions is a capable teachingforce. A capable teaching force is defined by (with the indicators in parentheses): * the teachers' knowledge (demonstrated subject mastery); * their experience (more than a year teaching) and stability (years in the school); - their time input (hours per day in the school). As another example of a less quantifiable factor, one of the elements of the school climate is positive teacher attitudes. Positive teacher attitudes exist when: * teachers have confidence in their ability to teach (they are at ease with learning materials and teaching ideas, and exhibit and report their own sense of confidence); * they are committed to teaching and care about their students (teachers set high standards of work and behavior, all actors report that the school is a caring place, low teacher absenteeism and tardiness). * they cooperate in efforts to improve the school and to help each other with instructional problems (they plan school activities and their teaching collaboratively, they share ideas with each other, and they and administrators work together on whole-school issues). The conceptual framework of factors, definitions, and indicators can be used to plan improvements in the quality of primary education, to conduct situation analyses and sector work on school quality, and to monitor and evaluate educational reforms. Conceptual Framework: Factors that Determine School Effectiveness http://www.worldbank.org/afr/findings/english/findl6.htm 9/13/01 Educational Quality: Defining What's Important Page 3 of 6 Ths Scftol: Factors RelItedto Effeinemss 3D SCHOOL CLIMATE 3.1 High Exlacktb3 of Stwim 3.2 rbiite IMeabl Attitls 33 0hsan cDidipl1 .34 hnkeitdCxdcuun 1D SUPPORTINfGIJFUTS 3.5 Favde and Ito1t 1.1 SAonLPALntan- Connity Suppt 1 2 Efcthe Suppoa t fio _ the Edusatinn gptt IDF 13 AdeqlteMattul Suppit 203 ENABLINGComDITONS sD STULENT OUTCOMES 1.3.1 FequzramAppzopiata leadDs ebpmnt _ 2.1 E=TsLaiiszship 5.1 zddpiLion AciAtie 22 ACapabla IeacilFozce 5.2 Acalemc AdAinm 1.3 2 Sufan IextbDDko and 23 FPwabliyanldAimnny 53 Sodilr Sil 0MIl ijtmek 2.4 High inB-ifSdrol 54 Eno-c Siow 1.3 3 Adffxte Fadhtie 4 41D TEACFHfNGLEARIUNG PROCESS 4.1 HighLauing lime CONTETSTUAL FACTORS 42 Vualin IehingSutIagi- tioni CHILDREtS CHARACTERISlCTI 43 FIeqjinHcWmvxk CutLura 44 FzqntSmi Stud= Ast Tdcal an Feelted Planning Educational Reform The planning of educational reform using the conceptual framework involves two steps. First, the definitions and indicators need to be translated into a concrete and realistic statement of the conditions that are sought for the system's primary schools. Then, the implementation processes that will bring about these conditions in schools have to be planned, including the specification of needed resources, the needed changes in organizational structure, and the time required. The framework has been applied to planning in a few countries. For example, in early 1992, about thirty senior Zimbabwean educators spent four days identifying the effectiveness conditions they wanted to establish in Zimbabwean primary and secondary schools. The level of specificity and the practicality of these conditions are striking. For example, school buildings should include "a store room or cupboard for each classroom" and "clean water supply within 500 meters of the school," realistic expectations given the current condition and location of rural primary schools. Additionally, in the area of learning, each student should be tested every two weeks, and should read for at least two hours per week. In arriving at this detailed list of desired conditions, participants continually referred to resource constraints and current operating habits as they debated the inclusion or non- inclusion of certain conditions and the amount (one chair, a textbook for two children) or frequency (daily, once a fortnight) assigned to those conditions they did include. Similar exercises, though with varying purposes, have been conducted successfully with Ministry of Education officials in Uganda, with a research team for the Ministry of National Education in Madagascar, and with a group of http://www.worldbank.org/afr/findings/english/findl6.htm 9/13/01 Educational Quality: Defining What's Important Page 4 of 6 Kenyan primary school teachers. Once an education system has determined those conditions it wants to improve in its schools and has set targets for amounts and frequencies, the next stage is to establish what change processes will be most effective in bringing about the desired changes at the school level. In Zimbabwe, this next step began with discussion among senior officials in the Ministry of Education. With a consultant's help, they outlined a program that would use existing Inspectors as facilitators of whole-school-staff change teams and annually programmed regional budgets for school improvement activities. However, to date, conditions in Zimbabwe have prevented this plan from being implemented. In sum, the planning of the concrete conditions that a system wants to create in its schools must start with the system's definition of these conditions at the school level. If these school conditions - inputs, climate, and teaching/learning processes - are operationally defined and agreed upon beforehand, planning will focus on their creation. The resulting projects and programs will tend to be more effective because the design of implementation processes will be forced to respond directly to these school-level objectives. Situation Analyses and Sector Work The conceptual framework can also be applied to situation analyses and sector work on the quality of primary education. Most assessments of education quality in Sub-Saharan Africa have relied mainly on quantitative analyses of a system's current performance. System-wide statistics are used to create indicators such as student/teacher and student/textbook ratios to show how well the system is doing. In reality, these indicators only indicate how well-endowed the system is. There is very little, if any, effort to explore the qualitative dynamics of the interaction among the conditions within schools. Consequently, researchers often make general recommendations about program design based on a sterile assessment of what the numbers say. For educational reform and program design to be successful, these quantitative analyses need to be enriched by systematic qualitative information on the dynamics within the schools. A "structured case study" methodology has therefore been adopted to examine school processes in detail using the conceptual framework. The methodology comprises a series of visits to individual schools in order to build an information base about the enabling conditions, school climate, and teaching/learning processes that characterize the system. The sample size may be small, given the intensity of the methods used, and generalizations need to be tempered by how many and what kind of schools are studied. A pilot application of the method in Madagascar has produced written case studies on 36 primary and secondary schools and overall findings, with "best practices" examples, on the twelve primary schools in the study. Monitoring and Evaluation Finally, the conceptual framework provides a starting point for monitoring and evaluating education systems over time. Using the outlined methodologies, education managers, planners, and researchers can select the factors they consider important and keep track of the dynamics in schools. Information on individual schools can be collected and used by supervisors and inspectors, school by school, to help provide feedback on their operation, or the data can be aggregated by system evaluators to obtain a snapshot of how schools throughout the system function. When combined with other tools for assessing school performance, such as national assessments, production function research, and more -rn .1 -n,rrn hon,tnn rnarnh tb0 -nt+n-;1 Ua.- n-r 0AA rj r;- ,,,, n-a1 4; http://www.worldbank.org/afr/findings/english/findl6.htm 9/13/01 Educational Quality: Defining What's Important Page 5 of 6 lullliai %ia;aai%1LU vuOI. vaL1UII m%, a11 t 11iaLiAa 1m.,i. wall auu a ai6,niii.auLi MualLaLav%., Ullll..1ivliJ11 for understanding the locally-defined "cultural meaning" of different mixes of school factors. The framework has been used by the Federal Department of Inspectorate Services in Nigeria to assess the quality of 200 "good" Nigerian secondary schools. A 52-item questionnaire was developed based on 16 variables identified and defined using the conceptual framework. Inspectors were then trained to administer the questionnaires, and data was collected from a sample of 200 schools. By defining quality in operational terms before conducting these inspections, the study found that most of the schools did not measure up to the criteria. Conclusions These guidelines stress their local adaptation precisely because of the complexity of applying them to an education system. The usefulness of the definitions and indicators depends on how the people in a given education system relate them to each other and on how they use them. Planners and policy makers must involve teachers, school heads, and supervisors in the application of the definitions and indicators to any education system. Also, there must be a continuing critical dialogue about their validity and applicability to the system's primary schools. Using the experience and knowledge of the people who work in and seek to improve the quality of their primary schools, the framework presented here offers an opportunity to improve school effectiveness in Sub-Saharan Africa. Ward Heneveld. 1994. Planning and Monitoring the Quality of Primary Education in Sub-Saharan Africa. AFTHR Technical Note No. 14, Human Resources and Poverty Division, Technical Department, Africa Region. Washington, D.C.: World Bank. SEARCH ?EUDACK SITE MAP SHOWCASE http://www.worldbank.org/afr/findings/english/findl6.htm 9/13/01 Educational Quality: Defining What's Important Page 6 of 6 http://www.worldbank.org/afr/findings/english/findl 6.htm 9/13/0 1