Policy Research Working Paper 11026 Improving the Quality of Survey Estimates from Longitudinal Studies An Application to LSMS Panel Surveys Piero Falorsi Paolo Righi Giulia Ponzini Development Economics A verified reproducibility package for this paper is Development Data Group available at http://reproducibility.worldbank.org, January 2025 click here for direct access. Policy Research Working Paper 11026 Abstract Longitudinal surveys are an invaluable source for analyzing dynamics on sample representativeness. Empirically, the the current state of and changes in human populations paper focuses on the estimation procedures. Using data over time. However, maintaining the accuracy of estimates from the Uganda National Panel Survey, the longest Living from a panel sample becomes more difficult as the length Standards Measurement Study panel survey, it experimen- of the panel survey increases. Key concerns are the lack tally evaluates the suggested technique. In summary, the of sample representativeness, due to the sample erosion findings show that the calibrated generalized weight share caused by deaths and movers and the impact of new births, method base estimator yields individual-level statistics that and migration flows. Moreover, sample fatigue introduces appear to be more accurate than those produced by the an increasing measurement error. Correct design, imple- current Uganda National Panel Survey estimator. Addi- mentation, and use of a panel survey considers a set of tionally, the calibrated generalized weight share method methods to deal with these problems at different stages of base cross-sectional estimates on the transition matrix show the statistical process: the sampling design, the data collec- a generally higher degree of stability when the sample is tion, and the estimation. This paper focuses on the case of changed compared to the current Uganda National Panel panels with a rotating sample design. This case represents Survey estimates. a powerful hybrid solution for facing the impact of panel This paper is a product of the Development Data Group, Development Economics. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at gponzini@worldbank.org. A verified reproducibility package for this paper is available at http://reproducibility. worldbank.org, click here for direct access. RESEA CY LI R CH PO TRANSPARENT ANALYSIS S W R R E O KI P NG PA The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Improving the Quality of Survey Estimates from Longitudinal Studies: An Application to LSMS Panel Surveys Piero Falorsi 1, Paolo Righi 2, Giulia Ponzini 3 JEL codes: C81, C83, C87, Y80 Keywords: Panel surveys, attrition, tracking, Generalized Weight Share Method, survey methods. 1 International Consultant, Development Economics Data Group, World Bank, 1,2 Italian Insititute of Statistics (ISTAT), 3 Development Economics Data Group, World Bank; This paper was produced with financial support from the 50x2030 Initiative to Close the Agricultural Data Gap, a multi- partner program that seeks to bridge the global agricultural data gap by transforming data systems in 50 countries in Africa, Asia, the Middle East and Latin America by 2030. For more information on the Initiative, visit 50x2030.org. 1. INTRODUCTION Longitudinal surveys, based on repeated observations of the same statistical units over time, are an invaluable source for analyzing the current state and changes in human populations over time. The Living Standards Measurement Study – Integrated Surveys on Agriculture (LSMS- ISA) initiative demonstrates the value of longitudinal surveys by disseminating household panel data on issues such as health, agriculture, access to basic services, nutrition, poverty status, and much more, for countries across Sub-Saharan Africa. The European Union’s Statistics on Income and Living Conditions (EU SILC) and the US Bureau of Statistics’ Survey of Income and Program Participation (SIPP) are other key examples of the value of longitudinal studies. In panel surveys, accuracy depends on many of the same factors as in cross-sectional surveys. However, maintaining the accuracy of estimates from a panel sample becomes more difficult as the length of the panel survey increases. One key concern is the panel attrition of respondents who drop out of the survey, which can become increasingly troublesome for long-running panels. Additionally, movers – that is, people who have moved away from where they were interviewed in the first observation of the longitudinal study – represent a dynamic sub- population and are complex and costly to interview. Finally, new entrants to the population, such as immigrants and newborns, are particularly difficult to capture in traditional panel surveys. The correct design, implementation, and use of a panel survey considers several methods to address these problems at different stages of the statistical process, namely the sample design, data collection, and estimation stages. To the extent possible, sample design should minimize the loss of representativeness of panel data due to attrition, leavers, and new entrants. Practical approaches to this include the use of refreshed samples (i.e., rotating panels or split panels) and protocols for incorporating individuals not previously involved in the study to capture population dynamics. Based on tracking rules to follow the movers, the data collection stage should be designed to retrieve essential information on individuals or households that have dropped out of the sample. The use of flexible modes of data collection (e.g., telephone interviews) makes it possible to reach those who may not respond to traditional techniques. Additionally, collecting a minimal set of variables on dropouts (e.g., by proxy or doorstep interview) can improve the quality of the survey estimates. The estimation stage considers dropouts, movers, new entrants, and the dynamics of the target population over time. This is achieved by defining a weighting procedure that updates the direct sampling weights to account for new individuals in panel households and by using calibration estimators with up- to-date known population totals. This work proposes to improve the quality of panel surveys by considering the three aspects of survey design outlined above. We consider the estimation at the current time of cross-sectional and longitudinal parameters of a target population. Estimates are computed from sample data collected at the current time on individuals who entered the sample previously and were subsequently followed according to the rules of longitudinal observation considered in the survey. This approach clarifies many aspects of the representativeness of data collected in panel studies. It applies to the many cases that characterize current large-scale household panel surveys (e.g., EU SILC, LSMS, etc.) and can be easily extended to more complex issues in actual survey situations. The paper is structured as follows: Section 2 gives a formal definition of longitudinal and cross- sectional populations and the target parameters to be estimated in longitudinal studies. Section 3 presents the sampling framework and the estimator based on multisource (Singh and Mecatti 2011) and indirect sampling (Lavallée 2007; Falorsi, Righi and Lavallée 2019), which addresses 2 the different representativeness of the various sampling subpopulations and the changes in household composition over time. Section 4 presents improvements in field operations and data collection to facilitate the implementation of the proposed methodology. Section 5 briefly introduces the Living Standards Measurement Study (LSMS), defining its objectives and key aspects of the quality of the estimates. Section 6 illustrates the empirical application of the proposed methodology to the LSMS-ISA data, using the 2009, 2013 and 2015 waves of the Uganda National Panel Survey as a case study. Section 7 summarizes the main findings and concludes. 2. FORMAL DEFINITION OF CROSS-SECTIONAL AND LONGITUDINAL POPULATIONS 2.1. Populations observed at different points in time Let be the population of individuals of a given country at the current time t. is partitioned into subpopulations of households denoted as ,1 , … , , , … , , . We represent the set of households as = {1, … , , … , t }. The household , has , individuals and = ∑ =1 , . We can refer to and as cross-sectional populations of individuals and households since they refer to a specific point in time. To analyze the dynamics of cross-sectional populations in a longitudinal setting from an initial point in time ∗ , to the current time , we introduce the concept of longitudinal-populations. For the sake of simplification, we first introduce a population not partitioned in households. Thus, let ← ∗ denote the longitudinal population of ← ∗ individuals. The longitudinal-population identification is complicated by both the scope of study and the longitudinal observation's operability over time (Helliot et al. 2009). The most common definitions are the intersection- population, ← ∗ = ∗ ∩ , and the union-population ← ∗ = ∗ ∪ . The intersection-population is a proper subset of both ∗ and , and includes the individuals of ∗ who are still living and resident in the country at the time t. The longitudinal survey observes the same individuals over time, excluding both births and deaths during the period of study. However, the longitudinal population definition based on the intersection-population has some drawbacks related to the representativeness at time t and to the limitations in the study of the dynamics in the households’ composition. While the union-population best describes the population dynamics, it requires a dynamic sample design which introduces births and entrants to avoid bias. For this study, we propose a definition that takes the advantages from both approaches. Based on this, the ← ∗ individuals of ← ∗ , include: - all the people of the intersection-population of households ∗ ∩ ; - all new members of the household of the individuals in the intersection-population, even if they were not part of ∗ . With this definition, the newborns and those immigrants who become be part of the households of the individuals in the intersection-population are new entries into the population from time ∗ . If immigration to the country and newborns are numerically insignificant phenomena, then the longitudinal population ← ∗ approximates the cross- sectional population at time . Moreover, the longitudinal observation of ← ∗ is conceptually simple. We select a sample at the initial time, ∗ , and then in the following survey we observe all household members for the initial sample of individuals. 3 Given the definition of the longitudinal population of the individuals, we introduce a definition for the longitudinal population of households, denoted above as ← ∗ , with ← ∗ units. The households over time may experience three types of change: disappearance, fusion, or division. These changes directly affect the cross-sectional and longitudinal analysis and may significantly affect the sample representativeness (FAO, 2015). As an example, consider three households A, B, and C at time 1 as illustrated in Figure 2.1. At time 2, household A disintegrated and its members moved to three different households: A2 joined household c, A3 formed a new household, d, and A1 joined household a with new member E. Furthermore, member B3 of household b has disappeared at time 2. Figure 2.1. Longitudinal households over time (yellow refers to ∗ , blue refers to ) The picture above illustrates the problem of defining the longitudinal household over time. We may adopt two broad approaches. The traditional way of defining longitudinal households is to map them one-to-one over time: one household from time ∗ generates only one household at time . The main drawback of this approach is that we run the risk of excluding from the longitudinal analysis some of the households in which individuals of ← ∗ live. At this point, it is helpful to introduce the concept of continuity rules. To understand the concept of continuity rules, consider a household that observed at time t * consists of a father, mother and two children. Following the same people two years later, the surveyor faces two households, as the father has separated from the mother and is living with one of the two children elsewhere. The continuity rules precisely define which of the two households observed two years apart constitutes the continuation of the initially observed household and which is 4 the new household that has been created. Thus, the continuity is used to define operational rules that make it possible to identify which household at the current time is the continuation of a household observed at a previous time. The continuity rules for identifying the longitudinal household at the time may be different (Falorsi et al., 2009). In figure 2.1, we see that if the continuity rule identifies as the longitudinal household the one with the same head of time ∗ , we exclude household d from the analysis. To overcome this problem, we propose a many-to-many approach: one household from time ∗ may generate many households at time ; conversely, one household at time may derive from several households from time ∗ . The households of population ← ∗ are those comprising people of ← ∗ . In figure 2.1, we see that household A at time 1 continues at time 2 with households a, b, and d. Formalizing the many-to-many criterion and without loss of generality, household of is one of the longitudinal households that derive from the household ℓ at time ∗ , if ∗,ℓ , ∗ ∗ ← ← (2.1) � � ,ℓ = ,ℓ > 0, =1 =1 ← ∗ where ,ℓ is the link {0,1} variable which assumes value 1 if individual of the household ℓ at time ∗ is the same individual of household at time . With this approach, there is a perfect correspondence between the longitudinal population of people and that of households. For each person of ← ∗ , we may define their household at the initial time, ∗ , and at the current time . Definition 2.1 includes as a particular case the one-to-one continuity rule. In this ∗ case, ← ,ℓ is a {0,1} dichotomous variable, which equals to one if household is the only one household of which is the longitudinal continuation of the household ℓ of ∗ . 2.2. Parameters of interest 2.2.1. Cross-sectional parameters Let and be respectively two quantitative variables observed on individuals and households. The parameter of interest for inference is the total referred to the population where: , (2.2) = � � , = � , =1 =1 =1 in which , is the value of of individual of household , and , (2.3) , = � , =1 is the total of for household . Now we assume that the variable is a categorical variable with categories, where , = if person of household belongs to the p-th category (being = 1, … , ) . For instance, if the variable describes the employment status with three categories (employed, unemployed, not in the labour force). Let ( = 1, … , ) denote a dichotomous {0,1} variable which, for individual of household , is equal to 1 if , = , = � . 0, otherwise The parameters of interest are the frequency totals of the different P categories of the variable : 5 , (2.4) = � � , = � , ( = 1, … , ), =1 =1 =1 where , indicates the total of the variable , that is, the number of individuals in the household k characterized by category of variate . Finally, we may be interested in variables typically defined at the household level describing a specific household condition (e.g., poverty or non-poverty, the status of malnutrition, etc.). We indicate with the categorical variable with values , = ( = 1, … , ) if household ∈ belongs to the p-th category and we denote as ( = 1, … , ) the dichotomous (0,1) variables with values 1 if , = , = � . 0, otherwise In this case, the parameter 2.4 counts the number of households in the p-th condition. 2.2.2. Longitudinal parameters of individuals The net change parameter is given by ∆ (2.5) − ← ∗ = ∗ . Parameter 2.5 expresses the difference between the cross-sectional total at the time t with the corresponding total at time ∗ . The net change reflects changes in both the characteristics and composition of the population. Analysis of net change using cross-sectional aggregate estimates may conceal significant status changes at the individual level, from now on referred to as gross changes (Steel et al. 2009). Gross change implies the measurement of the phenomenon in the same units over time. Gross changes may be revealed from longitudinal surveys, following the same people through repeated interviews. In this way we can estimate yearly trends as well as persistence (Lohor 2009). The gross change of individuals who at time ∗ were in category of variable , and are in category p of the same variable (with , = 1, … , ) at the current time t can be expressed as: ′ ∗,ℓ , , ⃖� ,∗ = � ← ∗ , (2.6) ← � � � ,ℓ , ∗,ℓ =� � ⃖← ∗, , ℓ=1 =1 =1 =1 =1 =1 where ′ ∗,ℓ ∗ , ← (2.7) ⃖← ∗, = � � ,ℓ , ∗ ,ℓ ℓ=1 =1 is the flow variable which equals 1 if the individual changed from to in time interval ( ∗ − ), and 0 otherwise. Thus, the gross change may be expressed as a cross-sectional total (at time , t) of the flow-variable ⃖← ∗, . This variable can be measured directly only in the units of the intersection population, ∩ ∗ . 2.2.3. Changes of the households over time The net change of households may be expressed as in 2.5 except for the fact that the totals and ∗ refer to the variable introduced above. 6 The gross change measures the amount of change conditions, that is, how many households shift from negative to positive condition (e.g., poverty to non-poverty, malnutrition to non- malnutrition), from positive to negative condition (e.g., non-poverty to poverty, non- malnutrition to malnutrition), or remain in the same positive or negative condition. As for the individuals, the gross change is observed on the categorical variables. The gross change of households that changed from category to category p of variate (with , = 1, … , ) can be expressed as ∗ ∗ ← ,ℓ (2.8) ⃖� ,∗ =� � ← ← ∗ ∗ ,ℓ , =1 ℓ=1 in which ∗ ∗,ℓ , ∗ ∗ ← (2.9) ← =� � � ,ℓ ℓ=1 =1 =1 indicate the number of potential links of household . This quantity denotes the number of individuals of household k which belong to ∗ . It plays a central role for both the definition of the longitudinal parameter of the gross change and for producing unbiased estimates of the parameters of interest. To clarify how to calculate this number in the following Schema 2.1, we ∗ provide the value of ← for each of the four households (a, b, c, and d) at time 2 given in Figure 2.1. ∗ Schema 2.1. Values of ← for the households at the time 2 in Picture 2.1 ← ∗ Household One-to-one One-to-many A 1 1 B 1 3 C 1 3 d 0 1 ∗ ∗ If ← = ← ,ℓ , then household k is the only household at time t which derives from original household ℓ of ∗ . Reconsidering Formula 2.8, we note that it can be expressed as ⃖� ,∗ = � (2.10) ⃖� ,∗ , ← ← , =1 where ∗ ∗ ← ,ℓ (2.11) ⃖� ,∗ =� , ← , ← ∗ ∗ ,ℓ , ℓ=1 is the flow variable indicating the change in the time interval ( ∗ − ) of household k (as identified at time ) from category to category of variable . ⃖� ,∗ is a real value variable defined in the [0,1] In the case of the one-to-many continuity rule, ← , interval. ⃖� ∗ is a {0; 1} dichotomous variable. , In the case of the one-to-one continuity rule, ← , Schema 2.2. below shows the computation of the household gross change according to the continuity rules and the household relationships depicted in Figure 2.1. 7 Schema 2.2. Gross change parameter computation from household continuity rules for Figure 2.1 One-to-one continuity rules* One-to-many continuity rule Household ∗ a b c D Household ∗ a b c d t Poverty status at given time^ 0 0 1 0 t Poverty status at given time^ 0 0 1 0 A 0 1 0 0 0 A 0 1 0 1/3 1 B 1 0 1 0 0 B 1 0 2/3 1/3 0 C 1 0 0 1 0 C 1 0 1/3 1/3 0 Household Household Computation Parameter % Computation Parameter % 2←1 2←1 Positive change event 1 (Bb) 1 33.3 Positive change event 2/3 (Bb)+1/3(Cb) 1 25.0 Negative change event 0 0 0.0 Negative change event 1/3 (Ac) 1/3 8.3 Stationary event 1 (Aa)+ 1(Cc) 2 66.7 Stationary event 1 (Aa)+1(Ad)+1/3 (Bc)+ 8/3 66.7 1/3(Cc) Total households 3 100 Total households 4 100.0 *Continuity rule follows the head of household ^Cross-sectional poverty status=1, non-poverty status=0 ^Cross-sectional poverty status=1, non-poverty status=0 3. SAMPLING SETTING AND ESTIMATOR As anticipated in the introduction, we consider the case where longitudinal and cross-sectional estimates referring to the current time (e.g., the year 2024) denoted as time are to be computed using data from a longitudinal survey in which individuals were first observed at time ∗ (e.g., the year 2015) and subsequently observed again in year ∗∗ intermediate between and ∗ (e.g., the year 2020) in a refresh of the initial survey (based on well-specified tracking and continuity rules), in which a new sample is also independently selected. This observational setting is very attractive as it summarizes most of the estimation and representativeness problems that can arise in actual large-scale longitudinal studies. The following subsections formalize, using specific notation, the aspects related to sampling design, movers, and direct estimation. 3.1. Sampling design We introduce below some symbols to better illustrate the operations just described. Initial time ∗ . We select a fixed sample size of households, ∗ , from the cross-sectional population ∗ , where household ℓ ∈ ∗ enters the sample with inclusion probability ∗,ℓ . ∗ includes ∗ households, being ∗ = ∑ℓ∈∗ ∗,ℓ . We observe all ∗ ,ℓ individuals of sample household ℓ. Thus, the -th ( = 1, … , ∗,ℓ ) member of household ℓ is included in the sample ∗ with the same probability of inclusion in their household. Let ∗ = ∑ℓ=1 ∗,ℓ denote the realized sample size of ∗ in terms of individuals. Intermediate time ∗∗ . We select a new sample, ∗∗ , of households from the cross-sectional population ∗∗ . ∗∗ is selected independently from ∗ and is a fixed-size sample of ∗∗ households. Household ∈ ∗∗ enters in the sample with the inclusion probability ∗∗ , . We observe in the survey all ∗∗, individuals of sampled household . In this way, we observe a sample of ∗∗ of individuals. Moreover, starting from sample ∗ interviewed in the previous ∗ time, the enumerators track and collect the information of interest on all ∗ people who they are able to track, as guided by specific tracking rules. Tracking rules are fundamental in deciding whether to follow people who moved from where they were interviewed at time ∗ . For example, as mentioned in the 8 specific LSMS-ISA Uganda National Panel Survey (UNPS) survey documentation, all individuals older than 15 years that are biologically related to the head of household who leave a sampled household are found and interviewed, as long as they do not move out of the parish where they were initially located. The latter rule implies that people who left their parish after time ∗ are not tracked and their characteristics of interest are not recorded. However, referring again to the specific example, it is useful to note that regardless of the tracking rules, UNPS survey enumerators often track all those who remained within the Uganda border, even if they moved out of the original EA or parish. Once an individual selected in the original sample ∗ is re-contacted for the survey, the enumerator identifies all of their household members and collects the survey data. This ensures that the study captures changes in the household composition (due to births, deaths, marriages, etc.) of all the individuals tracked in survey ∗ . Thus, starting from the original sample ∗ , the longitudinal sample ∗∗← ∗ of ∗∗← ∗ households with ∗∗← ∗ individuals is formed. ∗∗← ∗ contains all persons of ∗ that were tracked, along with all members of their households at time ∗∗ . Current time . No new sample is selected. The ∗∗ and ∗∗← ∗ samples formed at the time ∗∗ are tracked and surveyed. The rules of tracking and data collection are the same as those already described for the formation of the ∗∗← ∗ longitudinal sample from the ∗ sample. Thus, the enumerators track all ∗∗ individuals of sample ∗∗ according to the specific survey’s tracking rules. This way, the enumerators form longitudinal sample ← ∗∗ , with ← ∗∗ households and ← ∗∗ people. Furthermore, the enumerators track all the people of ∗ ∩ ∗∗← ∗ who are the people selected in original sample ∗ and observed in longitudinal sample ∗∗← ∗ . So, the enumerators form longitudinal sample ← ∗ , with ← ∗ households and ← ∗ people. The samples for the estimation at time are ← ∗∗ and ← ∗ . 3.2. Movers and representativeness of the samples Movers create two types of problems in longitudinal surveys. First, there are additional costs. Surveys that employ face-to-face interviews use cluster and multi-stage sampling designs to control costs. Following a mover to a new address may incur considerable extra expenses if the new address is not in one of the original sample areas. Also, there is a risk that no interviewer may be available to visit a mover if the move is discovered during the field operations. Thomas et al. (2001) discuss the cases of certain surveys in low-income countries that suffered significant attrition and bias because they did not track movers. Therefore, new contact information should be obtained to facilitate collecting data on the mover through phone or online interviewing. Another solution is to collect the data of interest on movers through proxy interviews; however, using proxies requires implementing adjustments in the estimation phase to reduce bias. Based on the above, tracking protocols should be defined to determine a reasonable approach for balancing increased costs on one hand and distortion or bias on the other. The target current population of households can be defined as the union of the three disjoint subpopulations: (3.1) = ← ∗ ∪ ℰ ∗∗← ∗ ∪ ℰ← ∗∗ where 9  ← ∗ is the household subpopulation that remains fixed from time ∗ or is eligible for tracking from time ∗ to time according to the tracking rules adopted for the survey. For example, for UNPS, the households eligible for tracking are those in which at least one individual was already present at time ∗ and lives in the original parish until time . ← ∗ does not include the households of all immigrants, which contain only new people entering the country after time ∗ , and the households consisting only of people who moved after time ∗ out of their parish.  ℰ ∗∗← ∗ comprises the households whose members consist only of immigrants or movers in the time interval [ ∗ , ∗∗ ].  ℰ← ∗∗ includes the households whose members consist only of immigrants or movers in the time interval [ ∗∗ , ]. Partition 3.1 and Figure 3.1. define the representativeness of the two longitudinal samples (← ∗∗ and ← ∗ ) regarding the sub-populations of . Herein, we refer to representativeness as the possibility of computing direct unbiased estimates of a given population from the sample data. For the Sample ← ∗ , the tracking rules limit the representativeness of sample ← ∗ only to the sub-population ← ∗ of households of time who have been eligible for tracking from time ∗ to time t. For Sample ← ∗∗ , the tracking rules restrain the representativeness of the sample data only to the sub-population ← ∗∗ of households of time who have been eligible for tracking from time ∗∗ to time . ← ∗∗ can be partitioned into the two subpopulations ← ∗ and ℰ ∗∗← ∗ , being ← ∗∗ = ← ∗ ∪ ℰ ∗∗← ∗ . Finally, we see that in sample ← ∗∗ , we have no people belonging to the subpopulation ℰ ∗∗← ∗ ,, who are the people entered in the population after the last panel refresh. As we have no sample data on this subpopulation, we cannot compute direct estimates. 10 Figure 3.1. Timeline and representativeness of the longitudinal samples at the time For each household of , we can define the two dichotomous variables 1 if ∈ ← ∗ 1 if ∈ ℰ ∗∗← ∗ (3.2) ,← ∗ = � , ℰ, ∗∗← ∗ = � . 0, otherwise 0, otherwise given in (3.2) may be expressed as the sum of three addenda, each of which is the total of related to one of the three sub-populations ← ∗ , ← ∗∗ , and ℰ← ∗∗ (3.3) = ←∗ + ℰ∗∗←∗ + ℰ←∗∗ being , (3.4) ←∗ = � � , ,← ∗ = � , ,← ∗ , =1 =1 =1 , (3.4) ℰ∗∗←∗ = � � , ℰ, ∗← ∗ = � , ℰ, ∗← ∗ , =1 =1 =1 , (3.4) ℰ←∗∗ = � � , ��1 − ,← ∗ ��1 − ℰ, ∗∗← ∗ �� =1 =1 =� , ��1 − ,← ∗ ��1 − ℰ, ∗∗← ∗ ��. =1 The representativeness of the samples allows to compute a direct unbiased estimate only for the aggregate: (3.5) ←∗∗ = ←∗ + ℰ∗∗←∗ . ←∗∗ represents an accurate (proxy) measure of the total only if the aggregate ℰ←∗∗ is negligible. This condition holds if time point ∗∗ is close to current time . 11 3.3. Estimation process 3.3.1. Direct estimator The direct estimate of the total ←∗∗ , as expressed by formula (3.5), is given by: (3.6) � ←∗∗ + � ←∗ + (1 − ) � ∗∗ = � ←∗∗ , ← ∗ ∗ ← ℰ ∗∗ ∗ ← ← in which � ←∗ , is the generalized weight share method (GWSM) (Lavallé 2007) estimator of ←∗ � ← , and the total ←∗ obtained with the data collected by the longitudinal sample ← ∗ , ∗∗ ∗ ← � ←∗∗ are the GWSM estimates of the totals ∗ and ℰ ∗∗ ∗ computed with the data of the ℰ ∗∗ ∗ ← ← ← sample ← ∗∗ , with 0 ≤ ≤ 1. The parameter can either be fixed in advance or calculated from the survey data. Further discussion on the choice of is provided in formula (3.15) below. � ←∗ is given by: The GWSM estimator ∗ ← ←∗ , � ←∗ = � (3.7) � , ←∗ , ∗ ← =1 =1 where ←∗ , is the GWSM weight, given by 1 ∗,ℓ 1 ← ∗ (3.8) ←∗ , = ∗ � � ,ℓ . ← ℓ∈∗ =1 ∗,ℓ ∗ The enumerators can collect the value of ← during the survey, gathering the information on how many members of the household were present in the country at the time ∗ . Furthermore, we note that all the households in sample ← ∗ have a value of ,← ∗ , which equals 1. � ←∗∗ are given by: � ←∗∗ and The GWSM estimators ∗ ℰ ∗∗ ∗ ← ← ←∗∗ , � ←∗∗ = � (3.9) � , ,← ∗ ←∗∗ , , ∗ ← =1 =1 ←∗∗ , � ← (3.10) ∗∗ =� � , �1 − ,← ∗ � ←∗∗ , , ℰ ∗∗ ←∗ =1 =1 where ←∗∗ , is the GWSM weight, given by 1 ∗∗ ,ℓ 1 ←∗∗ (3.11) ←∗∗ , = ← ∗∗ � � ,ℓ . ℓ∈∗∗ =1 ∗∗ ,ℓ ∗∗ The enumerators can collect the value of ← during the survey, gathering the information on how many members of the household were present in the country at time ∗∗ . For the households collected in sample ← ∗∗ , the value of ,← ∗ may be either equal to 0 or 1. If ,← ∗ = 0, then ℰ, ∗∗← ∗ = 1. The enumerators can collect the value of ,← ∗ during the survey, asking all the people in household if: 1. they immigrated to the country after time ∗ , or 2. they moved outside the perimeter established for tracking the sample after time ∗ . If all the members of household give a positive answer to question 1, or question 2, then the value of ,← ∗ equals 0. Let 12 (3.12) (←) = ← ∗ ∪ ← ∗∗ be the longitudinal sample-union at current time , obtained by joining together the two longitudinal samples ← ∗ and ← ∗∗ . The sample (←) has (←) = ← ∗ + ← ∗∗ households, and (←) = ← ∗ + ← ∗∗ individuals. � ∗∗ in the standard form as a weighted sum of the data in We may express the estimator ← the sample union as: , � ∗∗ = � (3.13) � , (←), , ← ∈ (←) =1 where (←), is the direct sample weight of household of sample (←) . With straightforward algebraic manipulation, starting from the above formula (3.6), … , (3.11) we have ←∗ , if ∈ ← ∗ (3.14) (←), = �(1 − )←∗∗ , if ∈ ← ∗∗ ∧ ,← ∗ = 1 , ←∗∗ , if ∈ ← ∗∗ ∧ ,← ∗ = 0 where the expressions for ←∗ , and ←∗∗ , are given in formula (3.8) and (3.11) respectively. In the following, we denote (←), as the base weights of the sample-union. As of the definition of the value, Singh and Mecatti (2011) provide an in-depth illustration of the different approaches proposed in literature for fixing the optimal value of in the context of multiple frame surveys. Hartley (1962, 1974) proposes choosing to minimize the variance � ∗∗ . Unfortunately, the solution depends on the variable and may be negative. Hartley of ← (1974) suggests opting for a simpler alternative expression which is always positive, even if it depends on . We suggest here an even simpler solution which does not suffer from the above drawback and closely approximates the Hartley’s solution ← ∗ (3.15) = . ← ∗ + ← ∗∗ If ← ∗ ≅ ← ∗∗ , as in the case illustrated in Section 5 below, then = 0.5. The reciprocal of is known as factor of multiplicity. In the case illustrated in Section 5 below, this factor equals 2. 3.3.2. Calibration estimator The direct estimates � ∗∗ (see 3.13) obtained with the direct final weights , may not be ← (←) accurate at least for three reasons, thus requiring adjustment. First, � ∗∗ does not represent ← the aggregate ℰ←∗∗ and the estimates could under-cover the actual population for that segment. Second, non-response may introduce bias in the estimates, as the nonresponding population may have different characteristics from those who agree to participate in the survey. Third, there may be differences in the sample estimates with the known totals of some auxiliary variables available on the cross-sectional population . High values of the differences are often a sign of inaccuracy, resulting from either the abovementioned under-coverage problem or if some subpopulations exhibited higher non-response rates than others. The direct estimates can be adjusted jointly for the three issues above using the calibration estimator (Deville and Särndal 1992; Singh and Mohl 1996). This estimator defines calibrated 13 weights (←), to be used for the estimation, as these are as close as possible to the (←), direct weights and allow for reproducing the known totals of some auxiliary variables. The calibration estimator of the total is then defined as: , �, = � (3.16) � , (←), , ∈ (←) =1 or if the variable refers to the households, as �, = � (3.16) , (←), . ∈ (←) As Särndal and Lundström (2015) argued, the calibration approach can easily leverage the wealth of information available on non-response; if the auxiliary variables are explicative of the non-response, the calibration estimator reduces the no-response bias. In this way, the estimates jointly smooth the non-response bias and calibrate the known population totals. This approach has three advantages for the method often adopted for producing survey estimates, which is based on performing preliminary adjustments for non-response based on no-response propensity scores and which produces weights defined at the individual level, implying that people in the same household may have different sampling weights. The first advantage is that the sampling variance estimation is better founded and more transparent. Calibration estimators behave asymptotically, as they converge in probability to the regression estimator. They produce robust inferences with sound statistical properties for both the sampling design and the statistical model. The second benefit is that fewer adjustment steps are needed, making the process flow for the estimation more straightforward. Särndal and Lundström (2005, pg. 88) note that if the information on non-response is available only at the sample level, the calibration approach can be carried out in two steps, where the first step calibrates on the known totals at the selected sample level, which includes both respondents and non-respondents. The two-step approach improves the accuracy of the estimates when the reasons for attrition are different from the reasons for sampling under-coverage. Thus, two calibrations are implemented: the first calibration adjusts the base weights for panel attrition and the second calibration uses the adjusted weights to produce the final weights to address the under-coverage. Finally, the third advantage is that the calibration method produces estimates that are defined at the household level. That means all individuals in the same household have the same final weight, which significantly facilitates the consistency of estimates with variables defined at the individual level and those constructed with variables at the household level. ′ To introduce this estimator, let = �X,1 , … , X, , … , X, � be a column vector of auxiliary known totals for the population from administrative data, the latest census, or demographic ′ statistics. Let , = �X,1, , … , X,, , … , X,, � be a vector of auxiliary variables of the household k such that � , = . =1 Let X,,, be the auxiliary variable defined for the -th individual of the -th household. There are two possible situations regarding the relationship between the auxiliary variables defined at the household level, X,, , and those defined at the individual level, X,,, . Suppose the variable X,, can only be defined at the household level (e.g., the household's state of poverty). In that case, all individuals in the household will have the same value of the household variable, 14 being X,, = X,,, , and in this case, the variable X,, is obtained as the average of the X,,, values. If, on the other hand, the variable X,, is a count variable (such as the number of females in a particular age group), then it can be obtained as the sum of the X,,, variables at the individual level where X,,, = 1 if the -th person in household has the given characteristic. Considering the above example again, X,,, = 1 if person is a female of that age group and, X,,, = 0 otherwise. Let us suppose that the vector is known for the whole population and that the vectors , are known for the sample households. The calibrated weights (←), are obtained as a solution of the following minimum constraint problem ⎧� �(←), , (←), � = min ⎪ ∈(←) (3.17) , ⎨ �∈( ) , (←), = ← ⎪ ⎩ (←) , ≤ (←) , ≤ (←) , where: �(←), , (←) , � is a truncated distance function between (←), , and (←), , ; 0 ≤ ≤ 1; and ≥ 1. The truncated distance functions (Singh and Mohl 1996) ensures that the calibrated weights are bounded in the interval �(←), , (←) , �, thus obtaining a solution without outlier or negative weights. The problem (3.17) can be solved, among other solutions, with the open-source software Regenesees 2 which allows for the use of two different truncated functions: the truncated linear and the truncated logistic (Zardetto 2015). The calibration estimator, �, , with weights , , obtained as solution of (3.17), has the (←) following positive characteristics. The system (3.17) defines weights (←), at the household level and we can use them for estimating parameters of both individuals and households. Thus, they ensure the coherence of the household and individual estimates. The estimates of the auxiliary variables , are benchmarked to the known totals (or estimated by a large and accurate survey) , defined at country level. Therefore, the weights (←), guarantee the coherence of the sample estimates with the system of statistics at the country level. The use of truncated distance functions entails that the weights (←), are always positive, where the outliers of the weights have a limited impact on the final estimates. The calibration estimator of the total 2.5 is given by � ∗ = ∆ � � , − ← , ∗ , . ⃖� ∗ , as defined by formula 2.6, , The calibrated estimate of the gross change for the individuals, ← , may be obtained simply as the weighted sum of the flow variables ⃖← ∗, defined in expression 2.7. ⃖� ,∗ , as defined by formula The calibrated estimate of the gross change for the households, ← 2.10 may be obtained simply as the weighted sum of the flow variables ⃖� ,∗ defined in ← , expression 2.11. 2 https://www.istat.it/en/methods-and-tools/methods-and-it-tools/process/processing-tools/regenesees , accessed September 2019. 15 4. DATA COLLECTION 4.1. Tracking rules Statistical offices define the tracking rules to deal with the potential bias and sample size reduction caused by movers. The definition of the rules must consider the trade-off between improving the accuracy of the estimates and the cost of following the movers throughout the country. This trade-off usually leads to a compromise solution where movers are followed within a limited area. In this case, individuals are followed when they move within the area of the first observation. To carry out the tracking, the survey needs to define a tracking protocol based on: - The tracking objectives. To describe the procedure in concrete terms, let us consider the three waves ∗ , ∗∗ , and already introduced in Section 4.1. At the start of the fieldwork for the current time , we have the sample of individuals and households originally selected at ∗ and ∗∗ or the individuals incorporated in the households in the successive waves. For example, as a tracking rule, we can specify that the survey plan applies the tracking protocol at only to the individuals selected in the initial samples ∗ and ∗∗ . - The delimitation of the area in which the tracking is carried out (i.e., Enumeration Area, District, Parish, etc.). - The questionnaire definition. For instance, in the event that some individuals have already decided to move, it might be useful to collect information that would facilitate re-contacting them in the next survey (e.g., whether the respondent is thinking of moving in the next wave of the panel, the reason for the move, and the telephone number of the potential mover). - The protocol for the proxy interviews that defines the minimum set of variables to be collected for non-respondents by doorstep interview and for movers by proxy interview. - The definition of the data collection procedure for movers, which considers the area of interview (for instance, the data collection may be carried out through a face-to-face interview if the mover stays in the sample area or by phone interview if the mover moves to an area outside the sample). 4.2. Linking and other auxiliary variables for computing base weights We refer to the base weight as the weight calculated according to the Generalized Weight Share Method (Lavalleé, 2007) divided by the multiplicity factor (Section 3.3.1). These weights are defined at household level and each individual has the base weight of the household to which he or she belongs. As explained in section 3.3.1, the GWSM requires (i) the variable linking the individual to the initial sample to which they belong (in the previous example we have two original samples at ∗ and ∗∗ ) and (ii) the variable linking the individual to the target population, i.e., the year of their first appearance in the target population (for newborns and immigrants). The process of calculating the base weights includes the multiplicity factor, which represents the number of opportunities for a household to be selected in the initial samples. Each updated sample offers one potential opportunity for the units to be selected. As in section 4.15, if 16 ← ∗ ≅ ← ∗∗ , we assign the multiplicity factor to the household observed at time according to the two conditions below: Condition 1. If the household has at least one component that belongs to the target population in wave ∗ and is not a mover, the multiplicity factor is 2. Condition 2. If no component of the household satisfies condition 1, then the multiplicity factor is 1. Note that the conditions 1) and 2) introduce the concept of fixed individual at time ∗ (non- traceable mover with respect to time ∗ ) and fixed individual at time ∗∗ (mover or immigrant with respect to time ∗ and fixed with respect to time ∗∗ ). A fixed individual at ∗ has individual multiplicity factor equal to 2; a fixed individual at ∗∗ has individual multiplicity factor equal to 1. Consequently, in wave the following household multiplicity values may occur: 1. The multiplicity factor equals 2 if at least one component was selected in the original sample at wave ∗ . 2. The multiplicity factor equals 2 if at least a component of the household was selected at wave ∗∗ and variable ,← ∗ of the household equals 1. 3. The multiplicity factor equals 1 if at least a component of the household was selected at wave ∗∗ and variable ,← ∗ of the household equals 0. Finally, the response indicator variables of the individuals in the current wave and in the first wave (for the individuals selected in the original samples) complete the information to build the base weights. To facilitate operational understanding of the steps involved in constructing base weights at the time t, we list the variables involved in this process and the values they can take in table 4.1. 17 Table 4.1. Linking and other auxiliary variables t to compute the base weights at the time . Example: two original samples (first sample, ∗ , and refresh sample, ∗∗ ) Variable Symbol Description Values Personal Code. 1 Unique code over time. identification code Time of the original sample including Original sample of the individual. Missing when the Time: ∗ , ∗∗ ; missing. 2 selection individuals is incorporated in a sample household. Code of the individual’s household in Household the original sample. Unique code over identification code in 3 Code; missing. time. Missing when the individual is the original sample incorporated in a sample household. Inclusion probability of the individual in 4 = , the original sample. Zero when the Numeric: constant in the Inclusion probability (with equals to ∗ individual is incorporated in a sample household. or ∗∗ ) household. Household Code of the household in the current identification code in 5 wave. Code. the current wave 1 (when the individual is from the target population of the Individual membership to the target Population original sample); 6 population related to sample (in membership* 0 (when the individual is not relationship with 2 variable). in the target population of the original sample). 2 (when the individual is a fixed individual with respect Multiplicity factor of the individual to the population ∗ ); Individual 7 selected in an original sample with 1 (when the individual is a multiplicity factor in respect to the population ∗ or ∗∗ (built fixed individual with respect the current wave () considering the 2 variable). to the population ∗∗ ); Missing (not in the original samples). 1 Household 8 = Household multiplicity factor. Consider 2 (when at list an individual of multiplicity factor in the max of the 7 values for the the household has 7 =2). ( defined in the current wave individuals in the household. 1 (otherwise). Formula 4.15) Response indicator in the original 1 (respondent); Response indicator of the individual in sample (only for 9 0 (non-respondent or not in the original sample. estimate gross- the original sample) change parameters) Response indicator Response indicator of the individual in 1 (respondent); 10 in the current wave the current wave. 0 (non-respondent). *Example: in wave the household has at least one individual selected in the original sample at ∗ (target population is the population at ∗ ): Value=1 when the individual is selected at ∗ (individual belongs to the target population); Value=1 when the incorporated individual in the household was in the target population at ∗ ; Value=0 when the incorporated individual in the household was not in the target population at ∗ (i.e, newborn or immigrant after time ∗ ). Based on the variables listed in the above table, the base weight introduced in Section 3.3.1 of the household may be computed as: 1 1 1 = � �� 10 ��. 8 ∑∈ 6 ∈ 4 where for each individual ∈ is 5 = 5 , being 5 the household code. Finally, we note that, following the dynamics of the households, the household code, 5 , can change over time, with the exception of certain households for which the codes do not change. 18 4.3. Defining the minimal set of variables to be collected on non-respondents by doorstep interviews and on movers by proxy interviews Sample size reduction must improve the accuracy of cross-sectional and longitudinal estimates. Attrition (people dropping out and people moving) can bias estimates if those who drop out or move are systematically different from those who remain in the survey. For example, suppose that people drop out when they become employed, because they no longer have time to participate in the survey. In this case, the panel will produce downward biased estimates of the number of people in employment. Panel updating, tracking, and calibration estimation address the loss of sampled units. To reduce attrition bias, it may be helpful to collect additional information on non-respondents, which can be used as auxiliary variables in a two-step calibration approach, where the known totals at the selected sample level include variables derived from respondents along with information collected on non-respondents. For this, two calibrations will be required: the first adjusts the basic weights for panel attrition and the second calibration deals with under- coverage. As mentioned above, the two-step approach improves the accuracy of the estimates if the reasons for attrition are different from the reasons for the sample under-coverage. However, the collection of this information must be designed and implemented with great care; its utility and accuracy should be assessed by empirical studies before introducing its collection into the current statistical process. The additional variables can be collected through a short form questionnaire module that captures demographic variables (e.g., age, gender, etc.), social variables (employment status, literacy level, etc.) and, for movers, a variable on the reason for leaving the enumeration area (e.g., economic reason, personal reason, etc.). Finally, the telephone number of the movers should be recorded to facilitate phone interviews with individuals who the survey is unable to locate. Doorstep interview. The doorstep interview is an abbreviated version of the interview used to obtain key information on the characteristics of non-respondents, who are often related to target phenomena of interest. For example, the occurrence of literacy-related non-response is likely to be largely concentrated among migrants with low literacy in the official survey languages, and perhaps with low literacy in general. However, the extent of this concentration is currently unknown due to a lack of further information on these groups. The doorstep interview is a short and simple interview conducted with households (or individuals) who are not willing to participate in the main survey and is designed to obtain this basic information for the non-respondent. As mentioned above, demographic, literacy, and employment questions are simple questions that could be administered to non-respondents. Proxy interview. The proxy interview can be used in two different contexts: i) if one person in the household is a mover, then another household member can answer the short questionnaire and ii) if the whole household moves, then the neighbours (or the community) can answer the short questionnaire. In both cases, obtaining the telephone numbers of the movers may allow for a telephone interview if tracking is not allowed by the tracking rules. As with the doorstep interview, the purpose of the proxy interview is to ascertain the concentration of specific characteristics in the sample of movers. A module can be administered to the proxy respondent. Two other questions are then added to the short interview: the telephone number of the mover and the reason for moving out of the household. 19 4.4. Planning the multi-mode data collection The operational costs of the survey limit the tracking rules; thus, movers who leave the sampled area (i.e. enumeration area, the parish, district, etc.) cannot be tracked in all cases. To obtain synthetic information on the variables of interest for movers and to reduce panel attrition due to movers, a cost-effective strategy is to conduct a telephone interview. In this case, a shortened questionnaire is preferable or, in more sophisticated cases, the survey design may include a CATI (Computer Assisted Telephone Interview) interview and the panel survey will rely on multi-mode data collection. Finally, the standard questionnaire needs to ask for the telephone number to avoid non- response of movers in future waves. 5. SHORT DESCRIPTION OF THE LSMS-ISA PANEL SURVEYS We test this methodological framework through a simulation using real data from the Living Standards Measurement Study – Integrated Surveys on Agriculture (LSMS-ISA), conducted with the support of the World Bank. The purpose of the simulation is to verify the properties and usability of the proposed estimator and to compare its estimates with those currently produced in a specific application case. The LSMS–ISA surveys are quite heterogeneous when considering the entire estimation process, including the sampling plan, the tracking protocol, the target parameters, the set of auxiliary variables, and the estimators used. For the sake of simplicity, the proposed estimator does not include all the specific issues and technicalities affecting the survey used in the simulation. Instead, we focus on the main steps of the process that are common to all longitudinal surveys to draw general conclusions that are not survey specific. LSMS-ISA surveys are nationally representative, longitudinal household surveys designed to understand the relationship between living standards and agriculture in Africa. Since its launch in 2008, the LSMS-ISA project has worked with statistical offices in eight partner countries in Sub-Saharan Africa, providing technical assistance to design and implement multi-purpose household panel surveys with a strong focus on agriculture, while promoting innovation and efficiency in data collection methods. Table 5.1 summarizes the partner countries, the panel surveys supported, and the years of survey implementation. Table 5.1 LSMS-ISA Surveys Country Survey Year Burkina Faso Burkina Faso Enquête Harmonisée sur le 2018/19, 2022 Conditions de Vie des Ménages (EHCVM) Ethiopia Ethiopia Socioeconomic Survey (ERSS)  2011/2012, 2013/2014, 2015/2016, 2018/2019, 2021/2022 Malawi Third Integrated Household Survey (IHS3) 2010 Integrated Household Panel Survey (IHPS)  2013, 2016/2017, 2019/2020 Mali Enquête Agricole de Conjoncture Intégrée 2014/2017 (EAC-I) Niger National Survey on Household Living 2011/2014 Conditions and Agriculture (ECVM/A) Nigeria General Household Survey (GHS) 2010/2011, 2012/2013, 2015/2016, 2018/2019 Tanzania Tanzania National Panel Survey (TZNPS) 2008/2009, 2010/2011, 2012/2013, 2014/2015, 2020/2021 Uganda Uganda National Panel Survey 2009/2010, 2010/2011, 2011/2012, 2013/2014, 2015/2016, 2018/2019, 2017/2018, 2019/2020 20 In general, the LSMS-ISA longitudinal samples coincide totally or partially (i.e., as a subsample) with an existing agricultural or household sample survey. For instance, the Ethiopia Socioeconomic Survey (ESS) interviewed a subset of agricultural households from the existing Agricultural Sample Survey (AgSS), complementing its exclusively rural sample with a sample of urban EAs. The Tanzania National Panel Survey (NPS) and the Uganda National Panel Survey (UNPS) are composed of a subset of EAs drawn from household budget surveys, namely and respectively the Tanzania Household Budget Survey (THBS) and the Uganda National Household Survey (UNHS). In Malawi, the Integrated Household Panel Survey (IHPS) tracked and reinterviewed a subsample of households from the Third Integrated Household Survey (IHS3). In Nigeria, the General Household Survey-Panel (GHS-Panel) is a subsample of the GHS core cross-sectional survey. Finally, in Niger, the longitudinal study followed the entire sample of the National Survey on Household Living Conditions and Agriculture (ECVM/A). The LSMS-ISA surveys consist of two-stage probability samples which use the general population census for their sampling frame. In most samples of the LSMS-ISA, enumeration areas (EAs) are selected as primary sampling units with probability proportional to size. A sample of households is then randomly chosen from the complete listing of households in the selected EAs. Thus, the LSMS-ISA sample constitutes a random sample of EAs, households, and individuals. The LSMS-ISA samples are meant to be nationally representative of households and of individuals. Ideally, longitudinal studies preserve representativeness over time, indicating that the sample should represent both the current population at each survey occasion and the dynamics over time of the initial population. To maintain both types of representativeness, longitudinal surveys follow up with people interviewed in previous survey rounds and add new individuals to ensure that new members of the population such as migrants and newborns are included (Glewwe and Jacoby 2000). To this end, panel surveys establish rules to define interview targets in follow-up rounds and create specific protocols to track households and individuals that move away from their original location. Moreover, they adopt refreshed samples (i.e., rotating panel or split panel) and protocols for the inclusion in the survey of individuals not previously involved in the study. When discussing tracking in panel surveys, Witoelar (2011) introduced the concept of interview targets and tracking targets. Interview targets are those that need to be interviewed; tracking targets are those individuals or households that should be tracked if they have moved and, if found, should be interviewed. Similarly, the author distinguished between following-up rules and tracking rules in defining the criteria for individuals or households to be interviewed depending on whether they are found in the same location or are movers. To some extent, the LSMS-ISA panel surveys can be seen as a panel of both households and individuals. All households and individuals found and interviewed during the baseline are interview targets across the entire history of the panel sample. The LSMS-ISA tracking approaches are more heterogeneous and mainly diverge in terms of: (i) targeted tracking units, (ii) characteristics of the tracking targets, and (iii) geographical scope of the tracking. Decisions across these three elements will impact representativeness and cost. The LSMS-ISA are heterogenous in terms of the unit targeted for tracking and reinterviewing. Usually, these panel surveys have tracked dwellings, households, or individuals, with each option having its own distinct implications. In 2022, the Burkina Faso EHCVM revisited the dwellings previously visited in 2018/19. Nigeria and Ethiopia track and reinterview those households participating in the first wave of data collection but not individuals that split from the household. The Malawi IHPS, Tanzania NPS, and Uganda UNPS track individuals. 21 Tracking dwellings instead of households is a simpler option. Following up with households over time requires defining which household is the successor of the original household, which is not a trivial task, as households may split or regroup over time. As noted by Witoelar (2011) household membership and composition quickly change, household structure and boundaries are often unclear, and the definition of household often varies according to the objective of the survey. Continuity rules to define which household is regarded to be the continuation of the one observed during the previous survey are proposed and discussed in Falorsi (2009). The LSMS-ISA panel surveys have established practical rules to define which household to follow. The Ethiopia ESS, for instance, identifies the group of household members still living in the original dwelling as the original household. If the household moved away as a whole, it is tracked and re-interviewed if it is still residing within the Ethiopian border. If all household members left the country, the unit is dropped from the sample. If the household split, the portion of the household that moved with the member identified as the head of the household in the previous wave is tracked and interviewed. Similarly, the Nigeria GHS-Panel interviews household members found in the original dwelling and tracks households that moved outside of the dwelling but are still located in Nigeria. If the household splits, the original household is defined as the group of household members still residing with the head of the household from the previous round. If the original household head is not found, the original household is defined as the one comprising the largest number of original household members. Panels of dwellings and households have the disadvantage that members who moved out of their baseline household are not part of the sample anymore, despite still being part of the population. The sample of the remaining members therefore will be less representative of the population at the present time as well as of the population evolving over time. However, as we will detail later in the paper, this lack of representativeness is mitigated because all LSMS-ISA surveys incorporate new individuals who have joined interviewed (and tracked) households, such as newborns or immigrants. The loss of representativeness over time can be addressed by tracking original individuals. The extent to which individual tracking can mitigate representativeness issues depends on decisions regarding the characteristics of the tracking targets and the geographical scope of the tracking. Conceptually, deciding which individuals to track depends on the characteristics of the population that need to be represented over time. Aspirations of studies that focus on a specific subpopulation are different from LSMS-ISA panel surveys, which intend to describe the national population. However, tracking all individual movers can be very expensive. The more ambitious the tracking goals (e.g., the greater the distance away from their original home that an individual must be tracked), the more that costs will increase. Generally, LSMS-ISA individual tracking targets are defined based on the individual’s age as well as on their relationship to the household head. Malawi, Tanzania, and Uganda track and follow up with movers that are biologically related to the household head, excluding therefore servants or guests at the time of the previous survey. Moreover, all three surveys impose a lower bound under which individuals moving out of the households are not tracked (12 years old in Malawi and 15 years old in Tanzania and Uganda). The geographical scope of the tracking may also impact the representativeness of the panel survey in future rounds, if those individuals or households that move the farthest are also very different from others. How far to track is generally established based on administrative boundaries, geographic boundaries, or distance (Witoelar, 2011). Panel surveys that rely on local tracking only search for households in the same EAs or nearby. Distant tracking allows for tracking movers far away the original dwelling. Regardless of whether they track households 22 or individuals, all LSMS-ISA surveys use distant tracking to track and interview movers out of the original EA, as long as they still live within the country. In the Malawi, Tanzania, and Uganda panel surveys, once the individual forms a new household, all household members residing with the individual enter the sample and become tracking targets. Integrating the joint household into the sample accounts for new populations who may be otherwise underrepresented and captures the evolving dynamics of the population at large. However, including the mover’s new household increases data collection costs. An alternative and less costly way to enhance representativeness in longitudinal studies are the adoption of rotating panels or sample refreshes, which have been used by some LSMS-ISA surveys. For instance, the Uganda UNPS 2013/14 has rotated out and replaced one-third of the EAs from its initial sample. The Tanzania NPS sample was refreshed in 2014/15, but the subsequent 2020/21 round of NPS data collection only follows up with this refresh sample, marking a discontinuity in the longitudinal representativeness of the population from initial rounds. In 2022, the Ethiopia ESS sample was fully refreshed and the Burkina Faso Enquête Harmonisée sur le Conditions de Vie des Ménages (EHCVM) followed up with its 2018/19 households, complementing the sample with a set of new EAs selected from a more recent sampling frame. Finally, Nigeria also partially refreshed its survey sample in 2018/19. 6. EMPIRICAL EVALUATION OF THE UGANDA DATA As a case study, we apply the estimator methods discussed above to the Uganda National Panel Survey (UNPS). The UNPS is a multi-purpose household panel survey implemented by the Uganda National Bureau of Statistics (UBOS) with the technical support of the World Bank LSMS-ISA project. Started in 2009/10 as a follow up of the Uganda National Household Survey 2005/6, its primary aim is to inform policy makers in budgetary decisions and policy interventions and to monitor major national policies and programs. Implemented on an annual basis, the UNPS also provides representative information on income dynamics at the household level and consumption expenditure estimates to monitor poverty during intervals between other national survey efforts. Moreover, the UNPS collects high-quality agricultural data integrated within a multi-topic framework, which allows for understanding the linkages between agriculture, welfare, and other sociodemographic characteristics. Since its launch in 2009, the UNPS conducted 8 waves of data collection on a sample of approximately 3,000 households. The UNPS 2009/10 was followed by additional rounds in 2010/11, 2011/12, 2013/14, 2015/16, 2017/18, 2018/19, and 2019/20. Its sample is representative at the national, urban-rural, and regional levels. This study uses data from the UNPS 2009/10 (wave 1), the UNPS 2013/14 (wave 4), and the UNPS 2015/16 (wave 5). The UNPS 2009/10 comprises a sample of 2,975 households, the UNPS 2013/14 counts 3,118 households, and the UNPS 2015/16 interviewed 3,304 households. After data cleaning and data preparation for the analysis, our final dataset comprises 18,313 individuals from wave 1, 17,377 individuals from wave 4, and 15,905 individuals from wave 5. Figure 6.1 shows the change in status of the surveyed individuals across the 3 panel waves of data collection used in this work. The 2009/10 level represents the 18,313 individuals interviewed in the first wave. Of these, 45% (8,211 individuals) were found and reinterviewed in 2013/14, 15% were not found, and 40% were rotated out. The sample refresher of 2013/14 brought in 5,387 new individuals, or around 30% of the original sample. New individuals in the non-mover and split-off households accounted for 3,779 new units, equal to the 21 % of the original sample. Between 2013/14 and 2015/16, t31% of the 2013/14 wave sample was lost, while 85% was found and reinterviewed. The figure shows the dynamics across years of the old and refresher samples as well. For instance, we can see that 87% of the refresher sample was 23 interviewed in the 2015/16 round vis-à-vis 82% of the old sample interviewed in the 2013/14 wave. For the purposes of this paper, we focus on the 2009/10-2015/16 and 2013/14-2015/16 panels. In each of the panels, only the individuals present in both years are considered in the analysis. Since no individuals entered the sample in wave 5 after leaving in wave 3, the final number of observations for the first and second panels is 7,215 and 4,688, respectively. Figure 6.1. Individual dynamics in the UNPS 2009/10 - 2013/14 - 2015/16 panel sample In addition to the sample refresh in 2013/14, Figure 6.1 is the result of complex protocols that define the tracking rules. The UNPS tracking scheme considers the mobility of the target population by following i) households that moved away from their initial location as a whole, ii) households where some members moved to another location and others went elsewhere 24 and became split-offs, and iii) individuals that moved out their original household to form new split-off households. Tracking rules have changed over the course of the panel rounds for all three tracking targets mentioned above. In the first three waves, the UNPS tracked all the original households that moved away from their 2005/06 original location to any other location, although not all households of the initial sample were targeted for individual tracking in cases where any of their members moved out of the household. Individuals and the split-off households they formed were only tracked for a subsample of the original households, namely 2 households randomly selected per EA (20 percent of the original household sample). This individual tracking was meant to compensate for the losses to the sample due to attrition. This 20 percent of households from which the individuals were tracked were referred to as “split-offs tracking targets”. Moreover, of the split- offs tracking targets, only individuals 15 years and above who were biologically related to the household head were tracked. These individuals and split-off households were found and interviewed, even if they moved beyond their original EA/parish. Once interviewed, the split- off individuals and all members of their new household became part of the UNPS sample and were interviewed and tracked in all subsequent UNPS waves. Starting in UNPS 2013/14, the scope of the split-off target tracking was expanded to include all households that were part of the sample as long as they were still within Uganda, regardless of distance from the original household location and whether they were original or split-offs. In those households, only members older than 15 years and identified in the previous wave as the head, the spouse, or the child of the household head were eligible for tracking. Other members were interviewed only if they were living with one of these core members. This means that if no core member is found in the last known location, the household was not interviewed even if other previous household members still lived there. In wave 4, the sample of the UNPS was refreshed. One-third of the original sampled households were rotated out as part of the panel refresh and were no longer tracked or interviewed. In the current UNPS setting, tracking individuals requires the completion of an individual tracking form which contains all contact information for the split-offs and/or the individual movers. The information on their new location needed for the full tracking is generally gathered from their previous household members or any other knowledgeable person. For each core member that had moved away, a tracking form is completed. Based on the information filled in this form, the mover individuals are contacted and interviewed. Although the tracking target sample comprises only the core members of each household, all persons living with these core members are interviewed and become part of the UNPS sample. Finally, if these individuals are core members of the new split-off household, they are interviewed in the subsequent waves of the UNPS, even if they move to different locations. 6.1. Empirical evaluation of the cross-sectional estimates We computed the UNPS 2015/16 estimates. Data from the previous waves are included in the analysis to identify the household dynamics (e.g., movers, immigrants, and newborns). We calibrated the base weights (expressed in Formula 3.8) to the known sex by age class population totals using UBOS official projections for 2015 and based on the population census of 2014. The projections used to calibrate the weights are presented in Table 6.1. Hereafter, we denote these weights as calibrated GWSM base weights. We applied the calibrated GWSM base weights to a set of variables from UNPS 2015/16 data and compared them to Uganda official statistics to assess the functioning of the weights vis-à- 25 vis the UNPS 2015/16 sampling weights. The base weights are those defined in Section 3 at the household level. The UNPS sampling weights are those defined at the individual level and are currently used for the survey estimates by UBOS (2011). They are obtained according to a complex process, based on performing preliminary adjustments for non-response based on no- response propensity scores. Specifically, the final individual-level weights are obtained through the following six methodological steps: i) definition of the base weights as the reciprocal of the probability of inclusion, ii) calculation of the fair share correction factor, iii) pooling of the weights defined in step 3, iv) computation of the correction factors for attrition, v) trimming of the weights defined up to step 4, and vi) post-stratification. Table 6.1. Population projection by age and gender (2015) Age group Male Female 0-4 3,220,300 3,028,800 5-9 2,870,800 2,730,500 10-14 2,528,000 2,500,100 15-19 2,008,700 2,097,500 20-24 1,509,200 1,790,100 25-29 1,181,200 1,403,800 30-34 938,700 1,082,800 35-39 743,900 836,500 40-44 630,600 677,500 45-49 470,800 486,800 50-54 382,300 443,200 55-59 241,700 277,900 60-64 194,600 241,600 65-69 140,400 171,100 70-74 114,900 158,600 75-79 71,500 90,700 80+ 96,400 140,600 Total 17,344,000 18,158,100 Note: UBOS official projections for 2015 and based on the population census of 2014 available at https://www.ubos.org/wp- content/uploads/statistics/Population_Projections_2018.xlsx, downloaded June 28, 2021. We use published data and reports from the National Population and Housing Census (NPHC) 2014 (UBOS 2016; UBOS 2017) and the Uganda National Household Survey (UNHS) 2016/17 as main sources of official statistics. The former is conducted by UBOS about every 10 years with the aim of collecting benchmark demographic and socio-economic data of the Uganda population. The latter is the sixth follow-up survey of the UNHS, a cross-sectional survey implemented by UBOS starting in 1999/20, which aims to collect data on demographic and socioeconomic characteristics, with a sample of 15,636 households. The UNPS is a follow-up to its 2005/06 survey and largely implements the same methodology. The NPHC 2014 and UNHS 2016/17 are the official source of data with the closest collection period to the UNPS 2015/16 we found. The reference period for the UNPS 2015/16 is March 26 2015 to March 2016, whereas data collection for NPHC 2014 was pursued in August/September and the UNHS 2016/17 data collection was implemented between June 2016 and June 2017. Table 6.2 presents the comparison estimate indicators on individual and household characteristics at the national level. The first three columns show the indicators from the UNPS 2015/16 without using sampling weights (unweighted estimator), using the original UNPS weights (current UNPS estimator), and using the calibrated GWSM base weights (calibrated GWSM base estimator). The column of unweighted estimates is helpful because it indicates the different effects by which various weighting strategies act on the final estimates, thus changing the unweighted estimates. In the last two columns, the table presents the official statistics and their source. In general terms, estimates are more accurate using the calibrated GWSM base estimator rather than the current UNPS estimator for indicators at the individual level. The share of female population and the share of children below 18 years old weighted using the calibrated base GWSM weights closely approximate the official statistics. This result is expected as the weights are calibrated using the population totals projection based on the 2014 census that we are using as a benchmark. Regardless of the weight we apply, the UNPS 2015/16 overestimates the official “true” value of the literate population above 10 years. Although the three estimates are consistent with each other, the differences reported in the table are statistically significant because they are based on a sample of about 16,000 individuals. For such a sample size, considering a deft value(Kish, 1966) ranging from 2 to 3 (implying the deff value ranges from 4 to 9), the sampling standard deviation of a proportion varies around values between 0.007 and 0.010. The unweighted statistic, which has a much higher value (80.7%) than the official statistics (72.2%), suggests that the non-response affects the non-literate population. For such reason, this subpopulation is under-represented in the panel. However, the observed difference could be an issue of comparability in how this information was asked in the questionnaires. The different weighting systems seem unable to deal with this problem. Finally, the calibrated GWSM base weight estimator appears to be more accurate for two key indicators of the survey, namely the working population and the poverty head count at the national level. At the household level, the current UNPS and the calibrated GWSM base estimators seem more comparable, even though the former produces more accurate statistics, particularly for the variables on share of urban households and share of agricultural households. In a few cases – namely, access to electricity and ownership of the dwelling – the UNPS 2019/20 produces inaccurate estimates of the official statistics, regardless of the weight used. 27 Table 6.2. Individual and household estimates by the unweighted, current UNPS and calibrated GWSM base estimators Calibrated Unweighted UNPS GWSM Official Statistics base Mean Mean Mean Mean Source Individuals Female 51.1 50.8 51.1 51.0 Census (2014) % children below 18 years old 51.2 52.1 55.0 55.0 Census (2014) Literate population (+10 yrs) 80.7 80.8 81.7 72.2 Census (2014) Working population 76.8 76.1 80.1 78.8 UNHS (2016) Poverty headcount 20.4 19.3 19.9 21.4 UNHS (2016) Households Share of urban households 25.3 25.1 22.9 25.0 Census (2014) Size of the household 4.8 5.0 4.8 4.7 Census (2014) Share of agricultural households 78.7 80.1 79.4 80.0 Census (2014) Number of rooms in the 2.2 2.3 2.1 2.4 UNHS (2016) household Household owns a cellphone 74.0 72.9 75.4 74.3 UNHS (2016) Household has access to 15.4 15.1 15.6 21.0 Census (2014) electricity Household has access to safe 75.2 72.2 71.7 72.0 Census (2014) water Household owns the dwelling 81.9 82.9 81.1 71.8 UNHS (2016) House has brick walls 67.2 63.8 64.2 66.6 UNHS (2016) Table 6.3 breaks down the share of working population by area of residence. The calibrated GWSM base estimator (82.0%) seems closer to the official statistics than the current UNPS estimates (78.8%) in representing the official statistics in rural areas (82.6%). As indicated earlier, these differences, although small, are statistically significant. More controversial are the estimates in the urban areas, where we do not see a best estimator. Table 6.3. Employed population by area of residence Calibrated GWSM Unweighted UNPS Official Statistics base Mean Mean Mean Mean Year Source Rural 79.6 78.8 82.0 82.6 2016 UNHS Urban 67.5 67.4 72.7 69.0 2016 UNHS Having concluded the analysis of statistics at the national level, we now turn to the study of poverty by area of residence (Table 6.4), which does not show concrete differences among the estimators. The incidence of poverty is much higher in the rural areas than urban areas. Table 6.4. Poverty head-count ratio by area of residence Unweighted UNPS Calibrated Official Statistics GWSM base 28 Mean Mean Mean Mean Year Source Rural 23.8 22.2 22.3 25.0 2016 UNHS Urban 9.0 9.6 10.2 9.6 2016 UNHS 6.2. Empirical evaluation of the longitudinal estimates Panel surveys produce net change and gross change estimates (Section 2.2.2). Gross change offers valuable insights into net change, so it is important that the cross-sectional estimates achieved by transition matrices are consistent with the estimates obtained with the overall UNPS 2015/16 sample. Table 6.5 shows the dynamics of employment across the 2009/ 2015 and 2013/2015 panels. In particular, the first three columns report the employment status in 2015 with respect to the status in 2009. Only individuals present in both 2009 and 2015 are represented in the table. The last three columns show the same statistics for those individuals in 2015 who also responded to the survey in 2013. Focusing on the 2009/2015 estimates, we note that all estimators produce upward estimates with respect to the estimates presented in table 6.2. This result suggests that the individuals who left across the years – characterizing most of the attrition – were largely unemployed. The three estimators indicated that the panel comprised 78.6%, 77.6%, and 81.4% employed persons. The refresh operation rebalanced the panel. Focusing on the cross-sectional estimates given by the 2013/2015 panel, 77.4% of individuals were employed according to the unweighted estimator, 76.7% for the current UNPS estimator, and 80.8% for the calibrated GW- SM base weight estimator. We can state that the last estimator produces more stable and consistent estimates when considering different sub-samples, i.e., the 2009/2015 and the 2013/2015 panel and the entire sample observed in the 2015 with a percentage relative variance (ratio of standard error to mean) among the the three estimates equal to 0.8%, The UNPS estimator has 1.0% of relative error and the unweighted estimator 1.2% of standard error. Table 6.5. Employment transition matrix estimates by unweighted, current UNPS, and calibrated GWSM base estimators 2009/15 2013/15 Calibrated Calibrated Unweighted UNPS Unweighted UNPS GWSM GWSM base base Unemployed in 2009 (or 2013) and 2015 5.1 5.4 3.7 11.6 12.1 9.3 Unemployed in 2009 (or 2013) and employed in 2015 6.0 5.6 5.8 8.5 8.1 7.6 Employed in 2009 (or 2013) and 2015 72.6 72.0 75.6 68.9 68.6 73.2 Employed in 2009 (or 2013) and unemployed in 2015 16.3 16.9 14.9 11.0 11.2 9.9 29 The breakdown of employment dynamics in the 2009/2015 and 2013/2015 panels by area of residence as seen in Table 6.6 below demonstrates that panel attrition is concentrated on the unemployed in urban areas. Consistently with what we see in table 6.3 the Calibrated GWSM are more accurate than the UNPS panel weights in estimating employment in rural area in the 2009/2015 sample. Results for the the 2013/15 panel show that the refresher has improved accurancy. In urban area, all methods gave upward estimates and in particular for the 2009/15 panel sample. The findings indicate that individuals leaving the UNPS are mostly unemployed in urban areas, suggesting the need to improve tracking rules to reduce the potential bias of the estimates. Table 6.6. Employment by area of residence transition matrix estimates by the unweighted, current UNPS, and calibrated GWSM base estimators 2009/15 2013/15 Unweighted UNPS Calibrated Unweighted UNPS Calibrated GWSM GWSM base base Unemployed in 2009 Rural 3.8 4.2 3.1 8.9 9.2 7.4 (or 2013) and 2015 Urban 9.5 9.7 6.9 20.3 21.3 16.8 Unemployed in 2009 Rural 4.4 4.0 4.3 8.1 7.7 7.1 (or 2013) and Urban employed in 2015 11.9 11.3 13.3 10.0 9.4 9.7 Employed in 2009 (or Rural 75.4 74.6 77.3 71.8 71.3 75.2 2013) and 2015 Urban 62.5 63.1 66.8 59.5 59.9 65.5 Employed in 2009 (or Rural 16.4 17.3 15.3 11.3 11.8 10.4 2013) and Urban unemployed in 2015 16.1 15.8 13.0 10.1 9.3 8.1 We carried out a similar analysis for the poverty headcount ratio (table 6.7). Estimates from the 2009/2015 panel are all around the official statistics: we have respectively 21.9%, 21.1%, and 21.4% for the unweighted, UNPS and the calibrated GWSM base estimator. The refresh operation (2013/2015 sample) reduced the frequencies to 20.5%, 19.3%, and 20.0%. When we compare these estimates with the official statistics (21.4%), it does not appear that one estimator surpasses the others in terms of quality. Nevertheless, comparing the percent relative variance among the three estimates of the poverty headcount ratio, two from the longitudinal estimates and one from the cross-sectional estimator, we note that the UNPS estimators has 5.2% percent relative variance while the calibrated GWSM estimator has 4.1% of percent relative variance. The unweighted estimator has 4.0% of percent relative variance. 30 Table 6.7. Poverty rate transition matrix estimates by the unweighted, current UNPS, and calibrated GWSM base estimators 2009/15 2013/15 Calibrate Calibrated Unweighte Unweighted UNPS UNPS d GWSM GWSM base d base In non-poor HH in 2009 (or 2013) and 2015 63.1 64.8 63.6 65.0 66.5 65.4 In non-poor HH in 2009 (or 2013) and poor HH 2015 12.3 12.1 13.0 8.3 8.1 8.3 In poor HH in 2009 (or 2013) and poor HH 2015 9.6 9.0 8.4 12.2 11.2 11.7 In poor HH in 2009 (or 2013) and non-poor HH 2015 15.0 14.2 15.1 14.4 14.1 14.5 Table 6.8 shows the transition matrix for the area of residence, for which the 2009/2015 panel estimates are greater than 2013/2015 estimates. The overestimations of poor people suggest that non-poor individuals, especially in rural areas, are more dynamic and tend to move out of the UNPS sample more frequently than poor individuals. In the 20013/2015 panels, figures stabilize. All statistics are consistent with the national-level estimates from the UNPS 2015 cross-sectional sample. The refresh of the sample in 2013/14 seems to have mitigated the attrition of non-poor individuals. Rural and urban areas of residence show the same trends. Comparing the percent relative variance among the three estimates of the poverty headcount ratio, two from the longitudinal estimates and one from the cross-sectional estimator, we note that for the rural sub-population, the UNPS estimators has 5.3% of percent relative variance whilethe calibrated GWSM estimator has 3.5%. The estimates for the urban sub-population show the UNPS estimators has 0.6% percent relative variance, the calibrated GWSM estimator has 3.0%, while the unweighted has 5.4%. So in this case we are not able to indicate which one is the more stable estimator. 31 Table 6.8. Poverty rate by area of residence transition matrix estimates by the unweighted, current UNPS, and calibrated GWSM base estimators 2009/15 2013/15 Calibrated Calibrated Unweighted UNPS GWSM Unweighted UNPS GWSM base base In non-poor HH in 2009 Rural 58.1 59.4 59.9 59.4 61.4 61.0 (or 2013) and 2015 Urban 81.1 83.5 81.6 84.2 83.8 83.4 In non-poor HH in 2009 Rural 13.8 13.7 14.0 9.2 8.8 9.1 (or 2013) and in poor HH in 2015 Urban 7.0 6.5 7.7 5.5 5.6 5.0 In poor HH in 2009 (or Rural 11.4 10.6 9.7 14.7 13.4 13.2 2013) and 2015 Urban 3.0 3.1 2.1 3.8 4.1 5.4 In poor HH in 2009 (or Rural 16.7 16.3 16.3 16.7 16.4 16.6 2013) and non-poor HH in 2015 Urban 8.8 6.9 8.6 6.5 6.5 6.2 6.3. Summary of the empirical results The comparison of the estimators has been carried out upon two levels: their statistical properties and their practical implementation. As far their statistical properties, section 4 introduced the properties of the new calibrated estimator and section 6 provided concrete results on real data. Given the complexities involved in claiming the superiority of one estimator to another, we used benchmarking statistics to conduct this evaluation. The results are summarized as follows: - The calibrated GWSM base estimator seems to produce more accurate individual-level statistics than the current UNPS estimator. - The two estimators produce equally accurate statistics at the household level, suggesting that the GWSM base estimator can be further improved by considering household known totals. - While the cross-sectional estimates based on the transition matrix generally do not reproduce the exact cross-sectional estimates based on the entire UNPS sample, the sample refresh operation mitigates this problem. This suggests that both estimators can only partially deal with the dynamic nature of the population, making it important to i) coherently implement a periodic sample refresh (i.e., rotate the panel) and ii) improve the tracking rules, as panel attrition affects specific subpopulations of interest. - The calibrated GWSM base cross-sectional estimates on the transition matrix appear generally more stable when changing the sample (i.e., 2009/2015 or 2013/2015) than the current UPNS estimates. While the scope of the simulation has been to obtain general conclusions, as far as the practical implementation level is concerned, considering the Uganda case to be concrete, we can give the following indication, defined to change the current process as little as possible: - According to Table 5.1, Uganda panel data stores the variables, 1 , 3 and 5 . The variable 2 is not explicitly stored, since a non-respondent in the original sample (wave 1) is not clearly distinguished from a new individual that was not included in the original sample. - We artificially created the population membership variable, 6 , using other stored variables and making some accurate assumptions. The variable should be directly asked 32 in the new version of the questionnaire (i.e., “Given that the household has at least one component belonged to the original sample selected from a population, does the individual belong to the same population or are they new (newborn or immigrant)?”). - We artificially created the individual multiplicity factor, 7 , using other stored variables and making some accurate assumptions. This variable should be directly asked in the new version of the questionnaire (i.e., “Does the individual of the original sample change parish from 2009 to 2013 (years of the original samples?)”). - Based on the values of 7 , we computed the household multiplicity factor 8 . - We artificially rebuilt the response indicator variable in each wave, 9 and 10 , using other stored variables and making some accurate assumptions. - We stored the inclusion probability, 4 , for each individual selected in the original sample (not for the incorporated individuals), even if the individual did not respond (i.e., in the current panel data, the weight is missing). - We computed the GWSM weight that does not rely on the concept of the parent household, which can be a complex operation in practice. - We computed the final weights using the standard calibration estimator and applied these for the cross-sectional and longitudinal estimates, creating a unique vector of sampling weights that simplifies the coherence of the individual and household estimates. 7. CONCLUSIONS Given the complexity of the topics covered, we now retrace our main steps to provide a unified picture of the various results. In this paper, we explored the problem of enhancing the quality of cross-sectional and longitudinal estimates from surveys that are obtained by combining samples chosen at previous times with given rules to track panel members as they move away from the location of their initial survey interview. We have seen that the issue is complicated and that action needs to be taken on several fronts, which refer to related but different scientific fields. The first aspect is defining the quantities one wants to know, considering the nature of the population, which has both a cross-sectional and longitudinal aspects. That leads to appropriate consideration of the definition of the statistical units of interest over time and at the various points at which attention is focused for constructing cross-sectional and longitudinal estimates. While the solutions for simple statistical units (such as individuals) are relatively uncontroversial, for composite units (such as households), there are multiple possible solutions, and each solution has pros and cons in terms of being representative of what is happening in the reality of natural populations and of the observational ease of the phenomena of interest. We proposed the many-to-many approach for defining the continuity of a household over time, meaning that one household from an initial time may generate many families at the next time. Conversely, a household at the current time may be derived from several households from previous times. With that approach, there is a perfect correspondence between the longitudinal population of people and that of households. Moreover, the many-to-many approach includes the one-to-one continuity rule as a particular case. Next, we considered the observational processes that make theoretically defined population quantities observable. We have seen that it is necessary to take into due consideration the timeline of repeated observations to avoid distortions that derive from considering the same 33 observations several times or, on the contrary, failing to represent in any way some sub- populations that may have a particular interest from the economic and social point of view, such as the movers or newborns or those who enter the population at a given moment. Moving now to the estimation methodology, we have seen how the GWSME estimator offers a method that allows us to provide unbiased estimates and is simple to implement. The calibration of the estimates with the correction of the weights at the household level presents several theoretical and applicative advantages. The variance estimation has a well-established asymptotic behavior and is far more transparent and well-founded. Because the estimation is based on a small number of adjustment stages, the process flow is simpler. Lastly, the approach greatly improves the consistency between estimates created with household-level variables and those utilizing individual-level definitions of the variables. Finally, the last aspect examined concerns data collection and the different strategies that can be implemented to mitigate the bias resulting from non-response and panel attrition. In conclusion, we used data from the Uganda National Panel Survey (UNPS) to experimentally evaluate the suggested technique. In summary, we found that the calibrated GWSM base estimator yields individual-level statistics that appear to be more accurate than those produced by the current UNPS estimator. Additionally, the calibrated GWSM base cross-sectional estimates on the transition matrix show a generally higher degree of stability when the sample is changed compared to the current UNPS estimates. We stress that the above results are obtained using less auxiliary information of the current UNPS estimator, both because some of this information is not clearly documented and because one of the objectives is to propose a simplified estimator with respect to the current one, using fewer computational steps and based on a robust sampling theory well known in the scientific literature. REFERENCES Deville, J. C. and Särndal C.E. (1992). Calibration Estimator in Survey Sampling. Journal of American Statistical Association. Vol.87. N° 418. FAO (2015). Integrated Survey Framework, Guidelines, Rome. http://www.gsars.org/wp- content/uploads/2015/05/ISF-Guidelines_12_05_2015-WEB.pdf. Accessed on 16 December 2015. FAO (2014). The Global Strategy to Improve Agricultural and Rural Statistics. Technical Report on the Integrated Survey Framework, Technical Report Series GO-02-2014, http://gsars.org/wp-content/uploads/2014/07/Technical_report_on-ISF-Final.pdf. Accessed on 1 December 2014. Groves, R.M., Don A. Dillman, J. L. Eltinge, R- J. A. Little. (2001). Survey No-Response, Wiley. New York. Falorsi P. D., P. Righi, P. Lavallée. (2019). Optimal Sampling for the Integrated Observation of Different Populations. Survey methodology, Vol. 45, No. 3, pp. 485-511. Statistics Canada, Catalogue No. 12-001-X. Falorsi, F, Russo A. (2009). Popolazioni e parametri di interesse nelle indagini ripetute nel tempo. In Alleva G., Falorsi P. Indicatori e modelli statistici per la valutazione degli squilibri territoriali. Franco Angeli, Milano. ISBN: 9788856812077. 34 Glewwe, P. and Jacoby, H. (2000). “Recommendations for collecting panel data”, in M. Grosh and P. Glewwe (eds.), Designing Household Survey Questionnaires for Developing Countries: Lessons from 15 years of the Living Standards Measurement Study. Washington D.C., World Bank. Horvitz, D.G. and D.L. Thompson. (1952). A generalisation of sampling without replacement from finite-universe. J. Amer. Statist. Assoc., 47,663-685. Kish, L. (1965), Survey Sampling, New York: Wiley. ISBN 978-0471109495. Lavallée, P. (2007). Indirect Sampling. Springer: Ottawa. Lynn, P., (2009), Methodology of Panel Surveys, Wyley. Wiley and Sons, Ltd, Publication. Mecatti, F. (2007). A single frame multiplicity estimator for multiple frame surveys. Survey Methodology, 33(2): 151-157. Narain, R.D. 1951. On sampling without replacement with varying probabilities. J. Ind. Soc. Agril. Statist., 3,169-174. Särndal C.E., and Lundström S. (2015). Estimation in Surveys with Nonresponse. Wiley. New York. Singh, A.C. & Mecatti, F. (2011). Generalized Multiplicity-Adjusted Horvitz-Thompson Estimation as a Unified Approach to Multiple Frame Surveys. Journal of Official Statistics, 27(4): 633–650. Uganda Bureau of Statistics (2011), The Uganda National Panel Survey (UNPS) 2009/10 – Main Report, Kampala, Uganda. Uganda Bureau of Statistics (2017), The National Population and Housing Census 2014 – Education in the Thematic Report Series, Kampala, Uganda. Uganda Bureau of Statistics (2016), The National Population and Housing Census 2014 – Main Report, Kampala, Uganda. University of Essex. (2016), British Household Panel Survey, https://www.iser.essex.ac.uk/bhps Witoelar, Firman. (2011). Tracking in Longitudinal Household Surveys. World Bank’s Living Standards Measurement Study-Integrated Surveys on Agriculture (LSMS-ISA). World Bank. World Bank. (2020), Basic Information Document - The Uganda National Panel Survey (UNPS) 2015/2016. http://surveys.worldbank.org/lsms/programs/integrated-surveys-agriculture- ISA/uganda#bootstrap-panel. Zardetto, D. (2015). ReGenesees: An advanced R system for calibration, estimation and sampling error assessment in complex sample surveys. Journal of Official Statistics, 31 (2), 177. 35