Public Infrastructure and Structural Transformation

This study argues that public infrastructure is an important though previously neglected driving mechanism of the structural transformation process. To assess its significance quantitatively, this study first develops a multisector neoclassical growth model with heterogeneous firms, where public infrastructure contributes to firms' production and mitigates the barriers to firms' entry. The model is calibrated using data from Brazil, a country that has significantly expanded its infrastructure in recent decades, yet remains in deep need of further infrastructure improvements. The accumulation of infrastructure accelerates the structural transformation through generating higher returns and lowering entry costs in sectors with greater public capital intensity. In the simulations, public capital formation explains about 15 percent of the process. The paper also shows the effects of different barriers to public capital formation on the structural transformation and GDP per capita.


Introduction
Economists argue that a rich variety of human motivations shape outcomes in important ways. Some categories (e.g., pro…t-maximization, altruism, inequity aversion, maximin preferences, or warm glow) can be readily handled using standard tools, most notably classical game theory. However, many other important sentiments which involve what we will call "belief-dependent motivation" defy standard analysis. A broader mathematical framework called "psychological game theory"(PGT), pioneered by Geanakoplos, Pearce & Stacchetti (1989) (GP&S) and further developed by Battigalli & Dufwenberg (2009) (B&D), provides adequate modeling tools. 1 We explain how and why.
Let us provide early exposure to key ideas via three examples that relate to the following prominent categories of belief-dependent motivations: emotions, including guilt, disappointment, elation, regret, joy, frustration, anger, anxiety, suspense, shame, and fear; reciprocity, or the inclination to respond to kindness with kindness and to be unkind to whoever is unkind; image concerns, e.g., when someone wants others to believe that he is smart, altruistic, or honest.
Example 1: Guilt & tipping This example involves the emotions category. Psychologists Baumeister, Stillwell & Heatherton (1994) argue that "the prototypical cause of guilt would be the in ‡iction of harm, loss, or distress on a relationship partner" and that if "people feel guilt for hurting their partners ... and for failing to live up to their expectations, they will alter their behavior (to avoid guilt) in ways that seem likely to maintain and strengthen the relationship"(see p. 245; cf. Tangney 1995). That outlook is re ‡ected in the following vignette: Tipper feels guilty if she lets others down. When she travels to foreign countries, and takes a cab from the airport, this in ‡uences the gratuity she gives. Tipper gives exactly what she believes the driver expects to get, to avoid the pang of guilt that would plague her if she gave less.
To model this, consider game form G 1 where Tipper (player 2) chooses tip t 2 f0; 1; :::; M g and M > 0 is the amount of money in her wallet. The driver (player 1) is not active, and there is no future interaction. His (material) payo¤ is t. Choice t thus pins down an end-node. Tipper's material payo¤ is (M t). However, her utility equals (M t) 2 [ t] + , where is 1's expectation of t (which Tipper can only guess) and 2 0 is a sensitivity parameter measuring how much Tipper su¤ers when she lets 1 down. (Also, [ t] + := maxf t; 0g.) In words, Tipper's utility equals own money minus a pang of guilt which is proportional to how much less the driver gets than what he expects to get. Tipper's behavior in the vignette is expected utility maximizing if 2 > 1. The presence of in her utility makes it beliefdependent, leading to what is called a "psychological game" (p-game) here given by G 1 . The key characteristic is that utility at an end-node depends on beliefs, in this case that 2's utility at t depends on 1's beliefs (via ). Had we had a traditional game, utilities would be de…ned on end-nodes independently of beliefs and Tipper's best choice would be independent of her guess of .
[G 1 and G 1 ] Terminology Example 1 illustrates key concepts we rely on throughout: A game form speci…es the structure of a strategic situation (the "rules of the game"): the players, how they can choose, and the material consequences (typically money) of players'actions. We reserve the term payo¤s for material consequences. Unless players are expected payo¤ maximizers, payo¤s do not represent preferences over end-nodes. 2 These are instead given by utility functions (or utilities). Whereas in traditional game theory utilities are de…ned on end-nodes only, in PGT they also depend on features of beliefs about behavior, like in Example 1, and higher-order beliefs. Such beliefs are determined by the strategic analysis, i.e., they are endogenous. We use the term motivations to distinguish conceptually di¤erent parts that may a¤ect utility. Tipper is a¤ected by two motivations: own money and guilt. We use the term belief-dependent to describe whether or not a motivation or utility depends on beliefs. Tipper's …rst motivation (own money) is 1285), who compares the (boxing, boxing) and (opera, boxing) equilibria, highlighting a non-standard aspect of his model: In the natural sense, both of the equilibria ... are strict: each player strictly prefers to play his strategy given the equilibrium. In the equilibrium (boxing, boxing), player 1 strictly prefers playing boxing to opera. In the equilibrium (opera, boxing) player 1 strictly prefers opera to boxing. No matter what payo¤s are chosen, these statements would be contradictory if payo¤s depended solely on the actions taken.
Example 3: Status & conformity Our next example illustrates an image concern as modeled by Bernheim (1994). A special case goes as follows: Agents in a population are uniformly distributed on T = [0; 2], where t 2 T is an agent's "type" of preference for "brightness of clothing". Each agent simultaneously chooses a (garment) color c 2 T to wear. All agents observe these choices, and form beliefs about the type of the chooser conditional on the choice. Let t c denote the expected type of an agent who chose c. An agent of type t's utility equals (t c) 2 (1 t c ) 2 . In words, he su¤ers quadratic losses of (i) letting his chosen color deviate from the one he favors, and of (ii) status by being perceived as having an expected type that deviates from 1 (the "fashion standard").
Focusing on the case where agents'types are private information, Bernheim analyzes this model as a signaling game. He looks for equilibria where the "sender" (the only active player) maximizes expected utility given the way beliefs are formed, while beliefs are formed consistently with Bayes'rule given how choices depend on types. He argues that a plausible class of equilibria involve pooling at c = 1. Under our parameterization, such pooling can be universal, if out-of-equilibrium inferences-which cannot be pinned down by Bayes'rule-satisfy (e.g.) that t c 2 f0; 2g, for all c 6 = 1. 4 The example is interesting to us for two reasons. First, aspect (ii), mentioned above, makes utility belief-dependent, 5 and so creates a p-game G 3 . Second, consider a modi…ed version of G 3 , call it G 3 , where agents' types 4 If an agent of type t stays with the proposed equilibrium he gets utility (t 1) 2 0. If he deviates, the best way to do so would be to choose c = t, in which case he would get utility 0 (1 0) 2 (or 0 (1 2) 2 ), hence he cannot gain by deviating. 5 Note that t c is a feature of an endogenous belief, because it is derived from an initial belief about types and choices by conditioning on the observed choice. are observed ex post. Obviously, an agent of type t will be believed to have type t, regardless of his choice c. In the unique equilibrium, he will rely on a dominant strategy: c = t, so the prediction will di¤er from Bernheim's. A striking observation, from a game-theoretic point of view, is that the difference between G 3 and G 3 concerns information across end-nodes. It is imperfect in G 3 but perfect in G 3 . In traditional game theory, information across end-nodes never a¤ects predictions, and is therefore not even speci-…ed. That this property does not extend to p-games shows that information at end-nodes (and more generally the information players have when they are inactive) should be carefully speci…ed.
What' s next?
The above examples illustrate di¤erent belief-dependent motivations that lead to belief-based utility and p-games. They are suggestive of how one needs to move beyond traditional game theory for modeling purposes, and how the framework of PGT can do this. PGT provides a useful intellectual umbrella, under which many trends in psychology and economics can be understood, related, and synthesized via appropriate p-games. Awareness of and interest in PGT is on the rise, yet far from universal. In this article, we explain what PGT is and what motivations can be modeled, highlighting a variety of idiosyncratic features. We present old insights and speculate about new ones that PGT may deliver. We discuss basic theory, experimental tests, and applied work. Although we cite a lot of papers, our primary goal is to highlight the structure and potential of various forms of work involving PGT. Our style is semi-formal, presenting some notions verbally rather than mathematically. Readers who wish to dig deeper should compare with relevant passages of GP&S, B&D, and other articles. This includes, in particular, the recent methodological article by Battigalli, Corrao & Dufwenberg (2019) (BC&D), a text we frequently draw connection to.
Our discussion mostly consists of showing how to functionally represent the belief-dependent motivations in question and highlighting the ensuing best-reply behavior. Sometimes we analyze strategic reasoning either by iterated elimination of non-best replies, or informally applying an equilibrium concept. For a broader discussion of solution concepts see B&D and BC&D, as well as our brief critical remarks in Section 9.
In writing this text we faced a pedagogical challenge. On one hand, notions of belief-dependent motivation are exciting and intuitive, although likely novel and unusual to many. With this, we wish to allow our read-ers to develop a feel quickly. On the other hand, serious study involving PGT requires that one develops a precise mathematical framework. We also wish to explain how to do that. Had this been a textbook, we would have presented the formal framework before exploring all the notions of beliefdependent motivation that can be analyzed. However, some of our referees convinced us that such an approach might scare o¤ potentially interested readers, who might stop reading before they get to the sections where all the exciting forms of motivation are described. So, we switched the order. In the next four sections (2)(3)(4)(5) we elucidate the palette of sentiments that PGT can explore (starting with the three prominent categories of motivations mentioned above). This is doable, because we mainly focus on simple examples for which we can provide needed technical side-comments as we go. The heavy-lifting is postponed until Section 6, which presents the formal framework underlying the analysis of the preceding sections. We encourage readers who like formal analysis to read Section 6 before the preceding ones.
A crucial building block of the analysis concerns player i's kindness to j, denoted ij ( ). 7 It is the di¤erence between the payo¤ (i.e., the material/monetary reward) i believes j gets (given i's choice) and a comparison payo¤ C that i computes as follows: C is the average of the minimum and the maximum payo¤ that i believe j could get, for other choices of i. 8 In G 4 , if 1 believes there is probability p that 2 would choose take, we get Note that i's kindness to j has the dimension of the (expected, material) payo¤ of j, it ranges from negative to positive, and it may depend on i's beliefs (as it does for 1 in G 4 ). Player i is taken to maximize (the expectation of) a utility that depends on actions and beliefs according to a functional form of the following kind: where i ( ) is i's (material) payo¤ function and parameter i 0 re ‡ects i's reciprocity sensitivity. The desire to reciprocate kindness, as intuitively described in the Introduction, is captured via "sign-matching;" i ij ( ) ji ( ) is positive only if the signs of ij ( ) and ji ( ) match. 9 To illustrate in G 4 : if 2 is high enough, 2 wants to "surprise" 1, i.e., 2 chooses take if p < 1 2 and give if p > 1 2 . We make several PGT-related observations: (i) Player 2 chooses between end-nodes. So, in traditional game theory, her optimal choice would be independent of beliefs. This is not the case with reciprocity. In G 4 , 2's optimal choice depends on p, 1's belief. This illustrates that G 4 , when played by agents motivated by reciprocity, is a p-game.
(ii) Relatedly, backward induction can not be used to …nd 2's optimal choice independently of beliefs. Player 2 must consult her beliefs about p to compute his expected-utility-maximizing action.
(iii) In traditional game theory, …nite perfect-information games have equilibria (justi…able by backward induction) where players rely on degenerate, deterministic plans (intended choices). This is not the case in G 4 , for high values of 2 . We have not de…ned equilibrium here, but suppose we have a notion of equilibrium that requires 1 to correctly anticipate 2's plan (and that plans are carried out), and for 2 to anticipate that 1 will do so." (Dufwenberg & Kirchsteiger's equilibrium has that property.) If 2 plans to choose take, and 1 anticipates that 2 plans to choose take, then p = 1. But, if 1 anticipates that, then (as explained above) 2's best response would be give, not take. An analogous argument rules out an equilibrium where 2 plans to choose give.
Our next example, the Mini-ultimatum Game form G 5 , gives further insights regarding reciprocity, and will be used for later comparisons as well: Reasoning as before (with p now 1's belief about reject), 12 (greedy; p) is strictly negative for all p. 10 If 2 is large enough, the utility maximizing plan for 2 is reject. Suppose this case is at hand. What should 1 do? If 1 = 0, meaning that 1 is sel…sh, then 1 would choose fair (since 5 > 0). If instead 1 is large (enough), then there are two possibilities. The …rst one is that 1 chooses fair. To see why, suppose that (at the root, i.e., before the start of play) 1 believes that 2 believes that 1 plans to choose fair. Then 1 believes that 2 believes that 2 is not (as evaluated at the root) a¤ecting 1's payo¤. That is, at the root, it holds that 21 ( ) = 0, implying that, to maximize his utility, 1 should act as if sel…sh and choose fair (since 5 > 0). The second, very di¤erent, possibility is that 1 chooses greedy, despite the anticipation that 2 will choose reject. This is a "street …ght" outcome, with negative reciprocity manifesting along the path of play. To get the intuition, suppose 1 believes that 2 believes (at the root) that 1 is going to choose greedy. Then 1 believes that 2 is planning to generate a payo¤ of 0 rather than 9 for player 1. In this case, 2 would be unkind to 1. Since 1 is large, 1 reciprocates (in anticipation!) choosing greedy thereby generating a payo¤ of 0 rather than 5 for player 2.
The analysis here re ‡ects a key feature of the approach, namely that players'kindness is re-evaluated at each history. For example, 2's kindness to 1 at the root may be zero (if 2 believes 1 plans to choose fair) and yet 2's kindness after 1 chooses greedy would at that time not be zero. 11 Dealing with "bombs" The account of reciprocity theory just given glosses over a subtle issue which we now ‡ag. To illustrate, let G X 5 be a modi…cation of G 5 such that player 1 has a third choice at the root-Xwhich explodes a bomb, leaving each player with a material payo¤ of 100.
Recall how we de…ned i's kindness to j as the di¤erence between the payo¤ i believes j gets and the average of the minimum and maximum payo¤ that i believes j could get. G X 5 can illustrate how, in some game forms, absurd implications follow unless the calculation of "the minimum payo¤ i believes j could get" is modi…ed to not consider choices that hurt both i and j. In G 5 we concluded that 1's kindness when choosing greedy was negative ( 12 (greedy; p) = 2 p 2 , as noted in a footnote). Reasoning analogously, in G X 5 we would get that 1's kindness of choice greedy is instead positive. 12 Arguably, this is implausible; while hurting everyone would surely be unkind, not doing so shouldn't automatically render other choices kind. Arguably (for a given p) the kindness of choice greedy should rather be the same in G X 5 as in G 5 . Dufwenberg & Kirchsteiger (2004), as well as Rabin, propose kindness de…nitions that achieve such an objective, by calculating "the minimum pay-o¤ j could get"without regard to so-called "ine¢ cient strategies"that hurt both i and j. Their approaches, while to a large degree similar in spirit, di¤er in details. The (somewhat contentious) issues involved are too subtle to warrant coverage here. We refer to Dufwenberg  which allows him to address broader notions of "attribution" and "procedural concerns". Sohn  As PGT-based models gain popularity they will be increasingly used to do applied economics. Most such work to date is based on reciprocity theory (and in particular Dufwenberg & Kirchsteiger's 2004 model). Topics explored include wage setting, voting, framing e¤ects, hold-up, bargaining, gift exchange, insolvency in banking, mechanism design, trade disputes, public goods, randomized control trials, memoranda of understanding, climate negotiations, communication, and performance-based contracts. 14

Emotions
For a long time, neither psychologists nor economists paid much attention to emotions and how they shape behavior. We recommend Keltner & Lerner's (2010) handbook chapter which explains how while "founding …gures in psychology"(in particular Charles Darwin and William James) paid signi…cant attention to emotions, during most of the 20 th century and "the heyday of behaviorism ... emotions resided ... outside the purview of observable mea- 13 So do Rabin (1993Rabin ( , p. 1298) as well as Charness & Rabin (2002) in appendix-versions of their social preference models. These models and the references in the main text are PGT-based. Levine (1998) surement"and were considered "undeserving of scienti…c inquiry"(p. 317). 15 Furthermore, Elster (1996Elster ( , 1998 forcefully argues that economists by and large have neglected to study the emotions, despite that the topic is potentially of great importance. In his 1996 text he goes so far as to note that "all human satisfaction comes in the form of emotional experiences" (p. 1368). He argues that by failing to recognize such an important source of utility economists are potentially failing to get a correct grip on how decisions are formed.
That view is corroborated by the development in more recent psychology. According to Keltner & Lerner, not only has (since 1980) "a robust science of emotion ... emerged"(p. 317), but it has indicated that a large variety of emotions, each one in distinct ways, impacts well-being and behavior. The causalities are complex and hardly fully understood, but a key idea that is often stressed involves what since Lerner & Keltner (2000, 2001 has been called "appraisal-tendency." Lerner, Li, Valdesolo & Kassam (2014) discuss in particular the implications for decision making and how "appraisal tendencies are goal-directed processes through which emotions exert e¤ects on judgments and decisions" (p. 479). The themes they bring up include how emotions a¤ect content and depth of thought, goal activation, and interpersonal assessments. 16 Reading these psychological discussions is highly inspiring, and we encourage economists to do so. Yet, at times, getting a full grip can be frustrating as the concepts and connections tend to be, not only overwhelmingly plentiful, but also informal. We suspect and hope that some complementary clarity can be brought to the table by invoking analytical methods. PGT provides an adequate set of tools. 17 In his previous article in this Journal, Elster (1998) argued that emotions "are triggered by beliefs" (p. 49) and that they can have important economic consequences. How "can emotions help us explain behavior for which good explanations seem to be lacking?" he asked (p. 48). While he lamented economists'dearth of attention to the issue, PGT has subsequently been put to such use, and there is more to do. 15 Keltner & Lerner quote Skinner (1948) who wrote that emotions "are the …ctional causes to which we ascribe behavior"and "useless and bad for our peace of mind and out blood pressure." 16 See also Keltner & Lerner's Table 9.3 and the related discussion of attention, certainty, control coping, pleasantness, responsibility, legitimacy, and anticipated e¤ort. 17 See also Chang & Smith (2015) who elaborate on this theme.

Guilt
Among the emotions, guilt has been explored the most using PGT. 18 Motivated by work in psychology (e.g., Baumeister et al. and Tangney, cited in the Introduction), Battigalli & Dufwenberg (2007) develop a model allowing exploration of how (two versions of) guilt shapes strategic interaction in a general class of extensive game forms. While most follow-up work has been experimental (see Section 7), a few applied theory papers explored how guilt in ‡uences marriage & divorce, corruption, deception, framing, tax evasion, public goods, embezzlement, and expert advice. 19 We provide (BC&D's account of) Battigalli & Dufwenberg's (2007) notion of "simple guilt:"Player i experiences guilt when he believes that the payo¤ j gets ( j ( )) is lower than the payo¤ j initially expected given j's …rstorder beliefs j . This expectation is denoted E[ j ; j ], and it depends on j's beliefs about (own and others') actions. 20 Speci…cally, i 6 = j maximizes (the expectation of) a utility of the form where z is the sequence of chosen actions (terminal history, path, or end-node). Again, i 0 is a sensitivity parameter. As seen in the Introduction, Tipper's behavior in G 1 is captured if 2 > 1. We now discuss also a trust game form G 6 . 21 Assume that 1 = 0 and 2 > 0 to get the p-game G 6 , displayed alongside, where^ 1 = E[ 1 ; 1 ] 2 [0; 10] denotes 1's expected payo¤. 22 [G 6 and G 6 ] 18 Reciprocity, which we do not count as an emotion, has been explored even more than guilt. See Azar (2019) for a statistical analysis of the bibliometric impact of PGT-based reciprocity and guilt theory. 19 20 The authors actually assume that i su¤ers only to the extent that he causes j to get a lower payo¤ than j initially expected. Stating that precisely leads to a more complicated utility than the one seen here. However, best responses are identical, so we opt for the simpler version here. 21 Compare, e.g., Huang & Wu (1994), Dufwenberg (2002), Dufwenberg & Gneezy (2000), Bacharach, Guerra & Zizzo (2004), and Charness & Dufwenberg (2006). 22 Note that [^ 1 1 (trust; grab)] + =^ 1 because^ 1 0 = 1 (trust; grab). G 6 is a p-game, because of the presence of^ 1 , an expectation derived from 1's beliefs. One may think of 2's utility as re ‡ecting a form of "statedependent"preference, i.e., what 2 would prefer if he knew^ 1 . To maximize her utility, 2 must consult her beliefs about^ 1 . 23 In some strategic settings, powerful predictions may obtain if players reason about each other's reasoning. This may be relevant in p-games, and the emotion of guilt, as modeled in G 6 , can illustrate this in a stark way: If 10 > 14 2^ 1 , then 2 prefers share over grab, and vice versa. No matter how high 2 is, if^ 1 is low enough 2 prefers grab over share. Nevertheless, 2 may reason that if 1 chose trust then^ 1 5, since otherwise 1 would not be rational. If 2 > 4 5 , player 2 will then prefer share over grab, and if 1 believes that 2 will reason that way, he should choose trust. 24 As argued by Charness & Dufwenberg (2006), simple guilt can help explain why communication can foster trust and cooperation. Suppose G 6 =G 6 is augmented with a pre-play communication opportunity and that 2 promises 1 to choose share. If 1 believes this, and if 2 believes that 1 believes this, then simple guilt makes 2 live up to her promise. A promise by 2 feeds a self-ful…lling circle of beliefs about beliefs that share will be chosen. We emphasize that guilt, per se, does not imply such a positive e¤ect of communication (nor does it rule out a negative e¤ect), but it is consistent with it. 25 Let us …nally discuss the following three guilt-related distinct topics: Counterfactual emotions In G 6 , if 2 chooses share to avoid guilt, then 2 will (along the realized path) not experience guilt. Nevertheless, guilt has shaped the outcome. This illustrates a more general phenomenon: An 23 Early work on guilt (e.g., Dufwenberg 2002) plugged that second-order belief (rather than^ 1 ) into u 2 . As explained by B&D, the two approaches are equivalent. We prefer our chosen one. The shape of 2's utility is kept simpler with only …rst-order belief in its domain (see Section 6 where we only consider such beliefs). 24 Dufwenberg (2002) calls this line of reasoning "psychological forward induction." See B&D, BC&D, and Battigalli et al. (2020) for more discussion and formalization via extensive-form rationalizability. 25 In other game forms, one may argue that if a vulnerable party, say player i, were afraid that a guilt averse player j would take an action that could hurt i, then i might wish to tell j either that he had "high expectations" or that (for given expectations) the loss due to the hurtful action would be large. These are other ideas that link guilt aversion and communication, which have been explored by Cardella (2016)  emotion (it could also be, e.g., disappointment or regret, as we'll see in coming sections) need not actually realize in order to a¤ect economic outcomes.
This observation marks a di¤erence, to a degree, between what is the natural focus of economists and psychologists. For economists it is obvious that a counterfactual emotional experience is important, if it in ‡uences behavior and who gets what. Psychologists'discussions, by contrast, tend to focus on the impact of guilt when it actually occurs. The quote from Baumeister et al., regarding guilt, which we included in the Introduction, is exceptional.
Expecting too much? Battigalli & Dufwenberg's (2007) model does not distinguish whether or not a belief by j is "reasonable," as regards whether or not guilt of i can be triggered. This assumption was made in order to keep things simple, and it could be unrealistic. For example, in G 1 , if M is large and the driver expected Tipper to give away all she has then she might plausibly …nd the driver obnoxious, and enjoy giving nothing! Balafoutas & Fornwanger (2017) and Danilov, Khalmetski & Sliwka (2019) discuss such "limits of guilt".
Guilt vs. reciprocity With reference back to Section 2, the following points of comparison are noteworthy. First, in G 6 , the incorporation guilt or reciprocity has opposite connections between q and 2's preference. Under simple guilt (i.e., in G 6 ), the higher is q the more inclined 2 will be to choose share (see (iii)). However, the higher is q the less kind is 1 (reasoning as in Section 2), so if 2 were motivated by reciprocity a higher q would spell less inclination to choose share. 26 Second, under simple guilt, a single utility function, that depends on initial payo¤ expectations and on which end-node is reached, can be applied at each history where a player moves. By contrast, to capture reciprocity motivation one must describe and re-evaluate each player's kindness at each history. 27 26 For more on this, see Attanasi, Battigalli & Nagel (2013). 27 Herein lies two di¤erences: First, a new utility function is needed for each history; see Dufwenberg & Kirchsteiger (2004) for more on this feature, which we have not illustrated very clearly since players moved once in the games we considered. Second, since kindness depends on (foregone) choice options, game-form details matter in a way that lacks counterparts with simple guilt. See BC&D for a detailed discussion of this distinction, concerning "game-form free" vs. "game-form dependent" preferences.
Third, recall our above remark regarding how, in G 6 =G 6 , if guilt may make 2 chooses share then 2 will not experience guilt. By contrast, if 2 were instead motivated by reciprocity, her belief-dependent motivation might be felt as she chooses share; at that time she perceives 1 as kind (in inverse proportion to q) which in ‡uences her utility as she chooses.

Disappointment
Dufwenberg (2008) gives the following example which illustrates a critical role of prior expectations: I just failed to win a million dollars, and I am not at all disappointed, which however I clearly would be if I were playing poker and knew I would win a million dollars unless my opponent got lucky drawing to an inside straight, and then he hit his card.
Belief-dependent disappointment was …rst modeled by Bell (1985) and Loomes & Sugden (1986). More recent work by K½ oszegi & Rabin (2006Rabin ( , 2007Rabin ( , 2009) and also Shalev (2000) is technically closely related, but since it is di¤erently motivated we write about it under the separate heading of "Belief-dependent loss aversion"in Section 5 below. Gill & Prowse (2011) argue that disappointment may help explain behavior in tournaments for "promotions; bonuses; professional partnerships; elected positions; social status; and sporting trophies"(p. 495).
The needed modeling machinery was in part present already in the part on guilt of Section 3. Factor [E[ j ; j ] j (z)] + , seen in eq. (3), captures j's disappointment, although in (3) it was used for the purpose of modeling i's guilt. 28 To let i's utility re ‡ect disappointment we can instead look at where k 0. In words, i's utility equals money minus a pang of disappointment which is linked to his prior expectation. Note that k = 0 incorporates disappointment in the most straightforward way, while if k > 0 then i discounts "small"disappointments so that they have no e¤ect on utility. 29 Below, we consider a case with k > 0 to make a technical point. Utility (4) looks deceptively similar to (3) but is crucially di¤erent in that i's utility depends (in part) on i's plan, that is, the part of i representing i's beliefs about the actions he is going to take (more on that in Section 6). Such "own-plan dependence,"where i's beliefs about his choices impacts the utility of his choices, can lead to subtle complications as we now highlight (and see BC&D for more).
While (4) is applicable to any game form, and hence can shape strategic interaction generally, the clearest way to exhibit the essence of disappointment is to use a one-player game form with chance moves, like G 7 . Assume that 0 < x < 1 while 0 k minfx; 1 xg.

[G 7 ]
Can stay be an optimal plan for 1 in G 7 (with utility given by (4))? This requires x |{z} utility of stay after planning stay utility of bet after planning stay Similarly, bet is an optimal plan if utility of bet after planning bet utility of stay after planning bet () x 2 + 1 1 k 2 + 2 1 .
(6) First, assume that k = 0. Inspecting (5) and (6) one sees that if x 2 [ 2 2+ 1 ; 2+ 1 2+2 1 ] then either stay or bet can be an optimal plan. If x 2 ( 2 2+ 1 ; 2+ 1 2+2 1 ) then 1 incurs a loss if he deviates from the plan. Such multiplicity of strictly optimal plans could never happen without own-plan dependent utility. 30 In 29 Disappointment aversion may violate …rst-order stochastic dominance. For example, if k in eq. (4) is 0 and i > 1, then i prefers a sure payo¤ x > 0 to the lottery that yields x and 2x with 50% chance. The axiomatization of Gul (1991) rules this out. Cerreia-Vioglio, Dillenberger & Ortoleva (2018) derive an explicit representation of preferences à la Gul (1991). 30 This statement is true if there is perfect recall; otherwise similar complications occur as, again, dynamically inconsistent preferences may appear, and the conditional expected utility of actions may depend on the planned probability of choosing "earlier" actions. See, e.g., Piccione & Rubinstein (1997), which is the lead article in a special issue devoted to imperfect recall. the standard case, multiplicity of optimal plans is possible only if there is indi¤erence.
An interesting variation arises if k > 0. Could it be that neither stay nor bet is an optimal plan? If so, then neither (5) nor (6) would hold, and we would get To see that this is possible, pick a case that is easy to compute: assume that x = k = 1 2 , and study (7) as 1 increases. The leftmost term exceeds 1 2 for any 1 0, while the rightmost term is lower than 1 2 for high enough 1 (it decreases from 1 to 1 4 as 1 goes from 0 to in…nity). All in all, for a high enough value of 1 , (7) must hold.
We round up with two more remarks: Elation This emotion, discussed by Bell (1985) and Loomes & Sugden (1986), is a sort of opposite of disappointment. It can be modeled by substituting [ ] for [ ] + in (4) which then leads to p-games. 31 Reference point Bell (1985) and Loomes & Sugden (1986) di¤er from us in the way they de…ne rational choice/planning: We assume that realized payo¤ is compared to the ex ante (before choice) expected payo¤, which depends on the agent's pre-determined plan. They instead assume that the term of comparison (reference point) depends on the actual (irreversible) choice. Since this relates to how K½ oszegi & Rabin (2006Rabin ( , 2007Rabin ( , 2009) model belief-dependent loss aversion, we postpone the discussion to Section 5.

Frustration & anger
Psychologists argue that people get frustrated when they are unexpectedly denied things they care about. That sounds like disappointment! However, while disappointment is mainly discussed in regards to pangs incurred and anticipated, frustration is more often discussed for how it in ‡uences decision 31 Elation is not discussed nearly as often as disappointment, and seems to be less often regarded as empirically relevant. In line with that, Gill & Prowse (2011) report results indicating "that winners are elated while losers are disappointed, and that disappointment is the stronger emotion" (p. 495). making going forward. In particular, there is the "frustration-aggression hypothesis,"originally proposed by Dollard et al. (1939) (see also, e.g., Averill 1982, Berkowitz 1978, Potegal, Spielberger & Stemmler 2010, whereby frustration breeds aggression towards others. We limit our discussion of frustration to its role in that context, which, we argue, suggests a di¤erence in how to model frustration and disappointment. Anger and aggression can have profound economic impact, though few economists studied the topic. Battigalli, Dufwenberg & Smith (2019) propose a broadly applicable model. They do not develop applications, but mention pricing, domestic violence, riots, recessions, contracting, arbitration, terrorism, road rage, support for populist politicians, and bank bail-outs as potentially interesting ones. 32 We sketch key features of the approach, and start with an example from the authors-G 8 -designed to make a technical point about frustration and how it compares with disappointment: Suppose that if 2 is frustrated she will consider 1 an attractive target of aggression. What would she do if 1 chooses forward? The answer may seem intuitively obvious, but consider what would happen if frustration were modeled as disappointment (more disappointment giving higher inclination to aggression). Building on eq. (4), there would be multiple optimal plans for 2, following the logic of (ii) in Section 3.2. If 2 plans to choose havoc, and if she believes 1 will choose down, then she would be disappointed after forward, hence choose havoc in order to hurt 1.
With outcome (2; 2) available, this seems psychologically implausible. Battigalli, Dufwenberg & Smith resolve the issue by requiring players to focus on what has happened and what they can achieve in the future.
Maybe she will be frustrated and end up meting out a costly punishment, but that should be a reaction to, rather than a cause of, her frustration. This consideration leads to the following de…nition of i's frustration at history h: 32 As the authors discuss, some of these topics have been analyzed by others empirically or using models that feature anger which however is not modeled using PGT. See, e.g., Rotemberg (2005,  where E[ i j(h; a i ); i ] is the material payo¤ that i expects to get, according to his …rst-order beliefs i , given history h and action a i . Applied to G 8 , let p be the probability 2 initially assigns to forward while q is the probability with which 2 plans to choose bliss (thus, 2 is described by p and q). We get F 2 (forward; 2 ) = [(1 p) 1 + p q 2 2] + = 0. Zero frustration breeds no aggression, so 2 will choose bliss.
While eq. (8) di¤ers from the disappointment-part of (4), it is still a feature that brings own-plan dependence and belief-dependent p-game utilities. Having de…ned frustration, the next step is to model how that breeds anger and frustration. We avoid going into technical details-see the article for that-and here just highlight some key themes. Number one is that one must now theorize about blame. Consider G 9 (where players payo¤s are listed in alphabetical order, and Don is a dummy player): Battigalli, Dufwenberg & Smith assume that a frustrated player (which in G 9 could only be Penny because (8) must equal 0 at the root) becomes inclined to hurt those deemed blameworthy. They develop three models based on di¤erent blame notions. We indicate how they play out in G 9 : Simple anger: All co-players are blamed independently of how they have behaved. 33 In G 9 , if Penny's anger sensitivity P is high enough, she would choose d, going after Don whom she is most e¢ cient at punishing.
Anger from blaming behavior: i's co-players are blamed to the extent that they could have averted i's frustration had they chosen di¤erently. In G 9 , with P high, Penny would choose b, going after Ben, since Don is no longer blameworthy (he had no choice!), and Penny is more e¢ cient at beating up Ben than Abe.
Anger from blaming intentions: i's co-players are blamed to the extent that i believes they intended to cause i's frustration. In G 9 , with P high, 33 Some psychologists argue that frustrated people tend to be unsophisticated and inclined to blame in such a way; see Marcus-Newhall et al. (2000) for a discussion. It seems to us that how and why people blame is an interesting empirical issue, which may depend on, e.g., how tired a person is, or on whether he or she has drunk a lot of beer.
Penny would choose a, going after Abe, since also Ben is no longer blameworthy (while he could have averted Penny's dismay, he had no rational way of correctly …guring out chance's actual choice, and thus can't have had bad intentions). This third category, because Penny cares about others'intentions, injects a second form of belief-dependence in players'utilities.
Finally, a comment about how these models apply to the Mini-ultimatum Game form, G 5 . A comparison with reciprocity theory is of interest, since both approaches can help explain the prevalence of fair o¤ers and rejections. In both cases (anger and reciprocity), 2 may rationally plan to choose reject (if 2 is high enough, and, in the case of anger, if 2's initial belief that 1 will choose fair is strong enough). However, whereas in Dufwenberg & Kirchsteiger's theory it is possible that 1 chooses greedy even if he expects 2 to choose reject (since 1 then views 2 as unkind, and so may want to retaliate), this could never happen in (any of the three versions of) Battigalli, Dufwenberg & Smith's theory. As hinted at in the previous paragraph, at the root a player cannot be frustrated and he must therefore maximize his expected material payo¤.

Regret
Despite Édith Piaf's assertion, regret can be a powerful feeling. To appreciate this, ask anyone who didn't sell stock while the coronavirus was ravaging China and Italy, but had not yet hit the US where stock prices remained close to all-time high. Zeelenberg & Pieters (2007) discuss other examples and synthesize much evidence from psychology. 34 Research on regret starts with theoretical work by Bell (1982) and Loomes & Sugden (1982), who focus on pairwise choice. Quiggin (1994) proposes an extension for general choice sets. These authors restrict attention to single decision maker settings, but regret makes equal sense with strategic interaction. B&D, BC&D, and Dufwenberg & Lin (2019) formulate relevant de…nitions. We explain why (unlike in the case with disappointment) PGT is not needed for handling the decision theorists'settings, and why nevertheless PGT is crucial for analyzing general game forms. 35 Consider the following version of Quiggin's approach: Let and A be (…nite) sets of states (chosen by chance, or nature) and actions of the decision maker (= 1). The payo¤ function 1 : A ! R has a …nite range C R of monetary consequences. Function v 1 : C ! R describe 1's "choiceless utility" (Loomes & Sugden's terminology) of consequences. However, after 1 chooses a 2 A, chance's choice ! 2 is revealed and 1 now ruminates on what could have been. His regret-adjusted utility, which is what he wants to maximize, is a function u 1 : A ! R de…ned by where f : R + ! R + is strictly increasing. In words, i's overall utility involves pangs of regret that re ‡ect i's evaluation of how much better o¤ he could have been had he chosen di¤erently. For our purposes it is useful to re-formulate this as a one-player game form with a chance-move, with perfect information at end-nodes: Chance makes a choice from . Player 1 is not informed of chance's choice, and chooses a 2 A. Then end-node (terminal history) (!; a) is reached and revealed to 1, whose utility is computed using (9). Note that this is a traditional game, because 1's utility is uniquely determined at each end-node. However, if one generalizes the above steps to apply to any game form, then one arrives at a p-game: To see this, …x an extensive game form, focus on player i, and try to compute his regret-adjusted utility at end-node z (and at the associated terminal information set; here is one more instance where terminal information will in ‡uence the analysis). To do that, one needs to …gure out what would have happened had i chosen di¤erently. This, in turn, depends both on what choices i's co-players actually made, and which ones they would have made at any history in the game tree that i could have made play reach had he chosen di¤erently than he did. And that computation, of course, will reach a di¤erent answer dependent on which choices the coplayers are assumed to make. In contrast to the single-player example of the previous paragraph, i's regret-adjusted utility will not be uniquely de…ned. If i regret-adjusts based on his beliefs about what would have happened had he chosen di¤erently, we would get a p-game. The belief-dependence of i's utility involves his own beliefs at end-nodes (and associated information sets) regarding co-players'choices.
For example, consider G 4 . Would 1 experience regret if he chose stay, and if so how much? The answer depends on p, the probability with which 1 believes that 2 would choose take had 1 chosen reach. Analogous remarks apply to, e.g., G 5 , G 6 , and G 8 .

Anticipatory feelings
So far we considered either the action tendencies caused by emotions, as in the frustration/aggression hypothesis, or how actions cause emotions (own or of others) with positive or negative valence and how players take this into account in their choice, as with guilt, disappointment or regret. Behavior of the second kind is explained by the anticipation of future feelings under di¤erent courses of action. Now we consider how uncertainty about the future can cause "anticipatory feelings" with negative or positive valence in the present (cf. Loewenstein, Hsee, Weber, & Welch 2001). Of course, the anticipation of such anticipatory feelings can drive behavior in earlier periods.
Timing is essential to model anticipatory feelings. The simplest setting for a meaningful discussion is one with two periods t 2 f1; 2g between three dates 0, 1, and 2. Each period t is between dates t 1 and t. Action pro…le a t is selected in period t. To make the problem interesting, player i-the decision maker under consideration-has to be active in period 1 and another player (typically, chance) has to be active in period 2.
Anxiety is an anticipatory feeling with negative valence caused by uncertainty about future material outcomes (e.g., health, or consumption). Huang (2020) argues that anxiety has major welfare consequences during pandemics. Drawing on earlier work by Kreps & Porteus (1978) on preferences for the temporal resolution of uncertainty, Caplin & Leahy (2001) put forward an axiomatic model of utility of "temporal lotteries"and consider speci…c functional forms. As one example, they analyze portfolio choice. Using our notation, they consider the following utility where V is the variance operator, V i and E i 0 are sensitivity parameters, and v 2 i is the period-2 utility of the realized material outcome. The higher is the variance exhibited by i's beliefs about his material payo¤, the lower is his utility. The theory helps explain the risk-free rate puzzle and the equitypremium puzzle: when buying safe assets an agent is "paying for his peace of mind".
Caplin & Leahy (2001) also brie ‡y mention how their general theory can be adapted to model suspense, i.e., the pleasure experienced immediately prior to the anticipated resolution of uncertainty. This theme is explored in depth by Ely, Frankel, & Kamenica (2015). Finally, Caplin & Leahy (2004) draw on their (2001) theory to study interaction between, e.g., an anxious patient and his caring doctor, who decides whether or not to reveal information a¤ecting the patient's anticipatory feelings.

Wrap-up remarks on emotions
We round up this section by collecting three remarks on distinct topics: Valence and action-tendency Emotions have many characteristics, two important ones being valence, meaning the (material or psychological) costs or rewards associated with an emotion, and action-tendency, or how an emotion's occurrence incites new behavior. When modeling emotions using PGT one may want to choose which aspect to highlight, or abstract from. For example, Battigalli & Dufwenberg's (2007) models of guilt (cf. Section 3.1) are all about valence, abstracting away from action-tendency. This could well be restrictive; see, e.g., Silfver (2007) for a discussion of "repair behavior," which could be thought of in terms of an action-tendency of guilt. Similar remarks apply concerning the approaches to disappointment and regret presented in Sections 3.2 and 3.4. By contrast, Battigalli, Dufwenberg & Smith's models of frustration and anger are all about action-tendency, as frustration has no valence in their models. Again, this may be a restrictive abstraction. Frustration may, e.g., plausibly have similar valence as disappointment).
Is reciprocity an emotion? A referee raised the issue. Judging by similarity of mathematical styles (in Sections 2 and 3), perhaps the answer could be yes. However, scholars working on reciprocity rarely describe what they model as involving emotions, and reciprocity usually does not …gure in the list of emotions. We have chosen to structure our presentation accordingly.
Elster' s and Keltner & Lerner' s lists While we have covered several emotions, and highlighted their connections with PGT, we have not been exhaustive. Elster (1998) discusses anger, hatred, guilt, shame, pride, admiration, regret, rejoicing, disappointment, elation, fear, hope, joy, grief, envy, malice, indignation, jealousy, surprise, boredom, sexual desire, enjoyment, worry, and frustration. Keltner & Lerner (2010) o¤er another list (p. 330), which overlaps to a large degree but also adds contempt, disgust, embarrassment, contentment, enthusiasm, love, compassion, gratitude, awe, interest, amusement, and relief. We suspect that many of the additional sentiments listed here involve belief-dependent motivation that could be explored using PGT. However, rather than pursue these topics we propose that they hold promise for rewarding research to come.

Image concerns
Introspection and empirical and experimental evidence suggest that people are willing to give up some material payo¤s to improve the opinion of others about them. Evidence about deception can be explained by a trade-o¤ between monetary payo¤ and a reduction of the perceived extent of cheating or lying (see the example below). Other models instead assume that agents try to signal that they have "good traits" such as being altruistic or fair (e.g., Bénabou  , which may explain behavior in the Dictator Game, or why people seldom give anonymously to charities, while they are happy to give non-anonymously (as shown by Glazer & Konrad 1996). Several other articles explore various forms of image concerns explaining, e.g., conformity, job-seeking e¤ort, randomized survey-response, shame avoidance, peer evaluations, and pricing distortions. 36 The aforementioned examples suggest two broad kinds of image about which people are concerned: others'(terminal) beliefs about (i) imperfectly observed bad/good actions, and (ii) imperfectly observed bad/good traits. Both are modeled by psychological utility functions.

Opinions about bad/good actions
Play in a game form is represented by a terminal history, or path z 2 Z of actions taken by the players (including chance, when relevant). Suppose for simplicity that, according to some standard, paths in Z B i (resp. Z G i ) are such that player i behaved in a bad (resp. good) way. Some paths may be neutral, e.g., because i did not play. For example, in a deception game form Z B i could be the set of paths where i lies; in a Trust Minigame (e.g., G 6 above and G 11 below) i is the trustee and Z B i (resp. Z G i ) contains the paths where he grabs (resp. shares). 37 Let j be an observer who obtains possibly imperfect information about the realized path z, and let p B j;i (z; j ) (resp. p G j;i (z; j )) denote the observer's ex post probability of bad (resp. good) deeds conditional on what he observed, given z and j's system of beliefs about actions j . An image concern related to bad/good deeds can be captured by a simple functional form like More generally, one can assume that intrinsic motivations-besides image concerns-also play a role (i (dis)likes good (bad) deeds as in Gneezy, Kajackaite & Sobel 2018 and Khalmetski & Sliwka 2019), or that i cares about the perceived distance from the standard rather than mere compliance. We expand on the second theme presenting the model of cheating by , which is a useful illustrative example.
Perceived cheating aversion A large recent literature explores humans' reluctance to lie or cheat using an experimental "die-roll paradigm" introduced by Fischbacher & Föllmi-Heusi (2013). 38   propose a PGT-based account of this behavior. We draw on their work to illustrate how a concern with others'opinions regarding chosen actions can be modeled.
A subject is asked to roll a six-sided die in private and to report the outcome, but the report is non-veri…able and can be submitted with impunity. The subject is paid in proportion to the reported number, with one exception: reporting six yields a payout of zero. We will refer to a six as a "zero". 37 Note that paths record the behavior of every active player, hence we can accomodate norms such as behaving (or not) like the majority. 38  Formally, chance (player 0) draws x 2 f0; :::; 5g from a uniform distribution (x = 0 corresponding to rolling a six). Player 1 observes x and then chooses a report y 2 f0; :::; 5g after which he is paid y. 39 Choice y, but not realization x, is observed by player 2, who is an "audience". In applications the audience might be a fellow citizen, but in the lab it could be the experimenter or an observer "imagined" by player 1. Player 2 has no (active) choice, but forms beliefs about x after observing y. The associated game form is G 10 : The analysis will not depend on 2's payo¤s, which are therefore not speci…ed. The dotted lines depict information sets across end-nodes. This is a feature rarely made explicit in traditional game-theoretic analysis, but here it will be critical. In G 10 , these sets re ‡ect player 2's end-of-play information. Consider the following preference: Player 1 feels bad to the extent that player 2 believes that 1 cheats. Measure actual cheating at end-node (x; y) as [y x] + , i.e., cheating involves reporting a higher number than the roll and downward lies do not count as cheating. Player 2 cannot observe x, but draws inferences about x conditional on y. Let 2 (x 0 jy) 2 [0; 1] be the probability 2 assigns to chance event x = x 0 given report y, with P x 0 2 (x 0 jy) = 1, so 2's expectation of 1's cheating equals P x 0 2 (x 0 jy)[y x 0 ] + . Player 1's utility of (x; y) given 2 is where 1 0 measures 1's sensitivity to 2's expectation of 1's cheating. Note that (12) is independent of x. This re ‡ects the fact that 1 cares about his image, not about cheating per se. Also, 1 may feel bad even if he does not lie, if the audience believes that he cheats. Appending utility function (12) to game form G 10 , we obtain a p-game because 2 (x 0 jy) is an endogenous belief, i.e., it has to be derived by strategic reasoning. Adopting the traditional equilibrium approach, the strategic analysis of this p-game is tractable and delivers testable predictions. In this game form, the relevant beliefs of player 1 about actions describe 1's plan (or behavior strategy), so 1 (yjx) is the probability that 1 assigns to y after 1 observes x. 40 Dufwenberg & Dufwenberg solve for equilibria such that 1 maximizes (12) given 2's beliefs, and 2 (x 0 jy) is computed as a conditional probability using correct initial beliefs, that is, 1 (x) = 1=6 and 2 (yjx) = 1 (yjx). 41 It can be shown that an equilibrium always exists. However, if 1's concern for his image is strong enough ( 1 > 2), neither honesty ( 1 (xjx) = 1 for all x) nor sel…sh choice ( 1 (5jx) = 1 for all x) is an equilibrium. The striking implication: if 1 > 2 then equilibrium play involves partial lies (in expectation).
Walking through a sketch of the proof is helpful to get intuition for why this result holds: If honesty were expected by 2 then 2 (xjx) = 1 for all x, so cheating by 1 to y = 5 > x would raise no suspicion, hence be 1's best response, ruling out an honest equilibrium (for any value of 1 0). If sel…sh play ( 1 (5jx) = 1 for all x) were expected then 2's expectation of 1's cheating would equal P x 1 6 [5 x] + = 2:5; if 1 > 2 player 1 could then increase his utility by deviating to y = 0 (so that perceived cheating = 0).
The analysis just conducted depends critically on the information across the end-nodes. To see this, consider what would happen if those informations sets were split into singletons. That is, assume that 2 is told about both x and y, i.e., which path (x; y) occurred. At (x; y), player 2 would form beliefs such that 2 (xjy) = 1, implying that perceived and actual cheating coincide. If 1 > 1 then 1's choices would be honest ( 1 (xjx) = 1 for all x); if 1 < 1 then 1's choices would be sel…sh ( 1 (5jx) = 1 for all x). The partiallies prediction evaporates. This illustrates a feature, reminiscent also of the earlier comparison of G 3 and G 3 , that is unique to p-games. In traditional game theory, utilities are not a¤ected by information across end-nodes, which therefore has no impact on the strategic analysis. 42 40 Player 1's initial beliefs about chance moves -exogenously given by the uniform distribution -are irrelevant because he chooses after observing x. However, 1 takes into account that 2 knows the chance probabilities. 41 Formally, Our use of equilibrium analysis in this example does not mean that we endorse it in general; see the last part of Section 9. 42 This explains why in traditional game-theoretic analysis information sets over endnodes are usually not drawn, even if such information is objectively determined by the rules of interaction.

Opinions about bad/good traits
The second kind of image concern starts from intrinsic motivation. People have heterogeneous intrinsic motivations to do good deeds and avoid bad ones, and are imperfectly informed about the motivations of others. This expands the domain of uncertainty: now we have to consider systems of …rst-order beliefs about both (paths of) actions and traits. Suppose, just for the sake of simplicity, that actions are perfectly monitored ex post. Then, after the realization of any path of play (terminal history) z, each player j holds an endogenous conditional belief j ( jz) about the traits of others j . 43 Intrinsic motivation of i is measured by parameter I i 0, and ibesides liking material payo¤ and being intrinsically motivated-also cares about his reputation, that is, j's ex post estimate of I i . For example, i's psychological utility could be denotes the net intensity of i's good deeds in path z. 44 More generally, j's ex post belief about I i may be conditional on possibly imperfect information about the realized path of play. This allows for comparing non-anonymous and anonymous donations, or to consider the possibility that i has imperfect recall and is his own observer (j = i), as in the work Bénabou & Tirole (2002, 2006. 45 Utility functions like (13) introduce a familiar element of signaling into the strategic analysis: even if i's intrinsic motivation to do good ( I i ) is low, he may be willing to pay a material cost to make j believe that I i is high, hence that i is a "good guy". The simplest models of this kind are signaling games where only the sender is active and the receiver is a mere observer. 46 43 Formally, we are considering beliefs in games with incomplete information; see Section 5 below (under the heading "Private sensitivities") for a discussion. 44 In the binary case considered at the beginning of this section, 45 See below. Note that Bénabou & Tirole (2006) consider a model similar to the one in the main text (j 6 = i), but also put forward a reinterpretation with imperfect recall where j is a future self of i. 46 Readers may fail to recognize that Bénabou & Tirole's (2006) model is a signaling game, because they choose not to frame it explicitly as such, making it seem more like a decision problem. But they indirectly hint (in footnote 17) at the fact that they are considering a re…nement of signaling equilibria.
A noteworthy application of this approach concerns privacy. Depending on available technology and regulation, what we do may be monitored even when it does not a¤ect the material payo¤ of anybody else. Many people seem to care about this and in Western countries there is a consensus that privacy should be protected. Gradwohl & Smorodinsky (2017) model this by considering functional forms such that (as in eq. 13), for each (z; i ), u i (z; j ; i ) depends on the ex post belief of the "audience"j about i . In particular they assume that-other things being equal-the agent either wants j's posterior to be the same as the prior, 47 or dislikes being identi…ed. 48 Focusing on the simple case where i is the only active agent and actions are observable by j (lack of privacy), they analyze the pooling and separating (Bayesian perfect) equilibria of the resulting signaling game. Pooling distorts actions from the …rst best that would obtain under perfect privacy. Separation in ‡icts a psychological utility loss due to identi…cation.
Another application concerns identity. According to Hupkau & Maniquet (2018), one's own personal traits are part of an agent's identity, and he can su¤er from the discrepancy between his true identity and others'perceptions of it. 49 This in turn is a¤ected by actions via signaling, possibly causing ine¢ ciencies. For example, high-type agents may refrain from requesting useful service from a provider to avoid being pooled with low-type agents. Bénabou & Tirole (2011) instead consider forgetful agents who "care about who they are". Their actions depend on such self-perception, which may be forgotten later on. Thus, actions are also identity signals for the future self, introducing an identity-investment concern in the choice of the current self. 50

More motivations
The previous three sections focused on the three categories of motivation mentioned in the Introduction. We now complement that presentation by 47 That is, is the exogenous and known prior of j about i . 48 That is, i 2 arg max j 2 ( i) u i z; j ; i for each (z; i ), where i is the degenerate (Dirac) marginal belief that assigns probability 1 to i . 49 In this case, the third element in eq. (13) is replaced by` , wherè : R + ! R + is an increasing (e.g., quadratic) loss function. 50 Bénabou & Tirole's model is a p-game, because utility depends on an endogenuos belief, but-unlike Hupkau & Maniquet-they do not make the link to PGT explicit. discussing additional forms of belief-dependent motivations, and broader related issues.
Opposites Sometimes a meaningful belief-dependent motivation takes an "opposite"form of another sentiment, switching a sign or replacing [ ] + with [ ] in the utility formula. We already saw examples in Sections 3.2 and 3.5, where, respectively, elation was compared to disappointment, and suspense was related to anxiety.
Another example involves an opposite to guilt. Ru-e (1997) and Khalmetski, Ockenfels & Werner (2015) consider situations in which player i enjoys surprising j, in particular so that j gets a higher material payo¤ than j expected. See also Dhami (3). Is surprising others this way another "emotion"? Maybe yes; obviously it is a kind of joy, and joy is often listed as an emotion.
The desire to surprise actually has venerable PGT-ancestry. GP&S explored the idea in their verbally presented opening example, although a different variety than the work cited above. GP&S's example does not require that the co-player is surprised in terms of material payo¤. 51 Here is the quote (from p. 62), illustrating the sentiment and a feature idiosyncratic to p-games: Think of a two-person game in which only player 1 moves. Player 1 has two options: she can send player 2 ‡owers, or she can send chocolates. She knows that 2 likes either gift, but she enjoys surprising him. Consequently, if she thinks player 2 is expecting ‡owers (or that he thinks ‡owers more likely than chocolates), she sends chocolates, and vice versa. No equilibrium in pure strategies exists. In the unique mixed strategy equilibrium, player 1 sends each gift with equal probability. Note that in a traditional …nite game with only one active player, there is always a pure strategy Nash equilibrium. That this is untrue in psychological games demonstrates the impossibility of analyzing such situations merely by modifying the payo¤s associated with various outcomes: any modi…cation will yield a game with at least one pure strategy equilibrium.
Belief-dependent loss aversion When we discussed disappointment, in Section 3, we mentioned how that sentiments is closely related to ideas explored by K½ oszegi & Rabin (2006Rabin ( , 2007Rabin ( , 2009) and by Shalev. The goal of these authors, however, is not to model disappointment, but rather to tie in with Kahneman & Tversky's (1979) work on prospect theory. K½ oszegi & Rabin model prospect theory's central notion of a "reference level"as a decision maker's initially expected outcome. When he gets less than he expects he experiences loss, e¤ectively much like in disappointment theory. K½ oszegi & Rabin allow for losses in many dimensions, e.g., in n + 1 dimension if there are n goods as well as money. To capture that, we would have to augment the framework in Section 6, allowing i's outcome function i to be vector-valued.
The key features we highlighted in regards to disappointment in Section 3.2 have counterparts in the work of K½ oszegi & Rabin. Most notably the feature of own-plan dependent utility is there, and it may lead to multiplicity of non-equivalent rational plans, as well as non-existence of degenerate rational plans. Beyond those technical similarities, details di¤er quite a lot. The exact way in which K½ oszegi & Rabin de…ne belief-dependent loss is di¤erent from the way that Bell (1985) and Loomes & Sugden (1986) (and we) de…ne disappointment, and they also consider more notions of rational planning than we did when we discussed disappointment. The recent and penetrating survey on "Reference-Dependent Preferences" by O'Donoghue & Sprenger (2018) discusses all of these aspect in depth, so we refer to their text (and especially their Sections 5-7) for further details. Here we only mention one aspect. The notion of rational plan that we illustrated in Section 3.2 corresponds to K½ oszegi & Rabin's concept of "personal equilibrium." They also consider (i) a re…nement, "preferred personal equilibrium," that-in case of non-trivial multiplicity-selects the personal equilibrium most favorable to the initial self, and (ii) another concept, "choice-acclimating personal equilibrium," whereby the referent to which realized outcomes are compared is determined by the actual (irreversible) choice of the agent rather than his ex ante plan, i.e., it is the expected outcome conditional on the agent's action.
Consistently with K½ oszegi & Rabin's explicit reference to di¤erent time frames, we can accommodate such distinctions in the PGT framework by explicitly introducing time periods, which in turn may comprise multiple stages (see Section 6 and BC&D): Endgame utility is the sum of the utilities of di¤erent periods, the referent for one-period gain-loss utility is determined by beginning-of-period beliefs. With this, personal equilibrium refers to rational planning in one-period decision problems, whereas choice-acclimating equilibrium applies to two-period situations where the agent chooses in the …rst period and uncertainty realizes in the second period. Except for differences concerning the exact de…nition of the referent, the approach of Bell (1985) and Loomes & Sugden (1986) …ts the choice-acclimating equilibrium concept.
As regards applied work, K½ oszegi & Rabin discuss consumption, riskpreferences, and savings. O'Donoghue & Sprenger discuss other papers that deal with endowment e¤ects, labor supply, job search, pricing, and mechanism design.
Self-esteem Self-esteem re ‡ects an individual's overall subjective emotional evaluation of his own worth. It is "the positive or negative evaluations of the self" and "how we feel about it" (Smith & Mackie, 2007). We can model self-esteem by assuming that a valuable personal trait 0;i of player i is imperfectly known by i. Such trait could be general intelligence, or ability. Player i's utility is increasing in his ex post estimate of 0;i conditional on the path of play z, 52 as in function where the "ego-utility" v e i is increasing, and we allow material payo¤ i to depend on parameter vector because traits such as ability typically a¤ect material outcomes. For example, Mannahan (2019) shows that if i is observed ex post and v e i is concave, i may decide to handicap himself ensuring a bad outcome (e.g., by not sleeping before an exam) rather exposing himself to the risk of discovering that his ability is low. 53 Also, better informed players may engage in signaling to a¤ect i's selfesteem: Does a teacher want to reveal to a student how bad his performance was? Better information may allow for a better allocation of the student's time (more study, less leisure), but it may also be detrimental: by decreasing the student's estimate of his ability it can bring it in a range where ego-utility is more concave and cause the self-handicapping e¤ect described above.
A few economic studies of self-esteem model utility in line with our description here, although (unlike Mannahan) they do not make the PGT- Higher-order belief-dependence The framework presented in Section 6 restricts the domain of a player's utility to depend on beliefs (own and others') up to only the …rst order. 55 This is enough to handle almost all forms of motivation that to date have been modeled using PGT. 56 The main exception is Battigalli & Dufwenberg's (2007) model of guilt-from-blame (but see also B&D, p. 14). We now indicate how that sentiment works in an example designed to provide a contrast with simple guilt (as presented in Section 3). Guilt-from-blame plugs a third-order belief into the domain of a player's utility, so we leave the framework of Section 6. We sketch the approach without going into all formal details: First, for each end-node z in a game, measure how disappointed j (z)] + ; this is how much j would blame i if j knew he were at z. Finally, i su¤ers guilt-from-blame in proportion to j's blame, and i's utility trades o¤ avoidance of that pang against i's material payo¤.
Battigalli & Dufwenberg (2007; see Observation 1) prove that simple guilt and guilt-from-blame sometimes have similar implications. However, this is not true in general. To illustrate, consider G 11 , a modi…ed version of G 6 in which even if 2 chooses share there is a 1 6 probability that 1 gets a material payo¤ of 0. Moreover, if 1 gets 0 then 1 is not informed of 2's choice. As in 54 The aforementioned work of Bénabou & Tirole may be interpreted as modeling selfesteem, but their approach relies on imperfect recall and self-signaling. 55 Player i may still have to consider his second-order beliefs, if his utility depends on j's …rst-order beliefs (as it did in our presentation of reciprocity, guilt, anger from blaming intentions, and image concerns). Since i does not know j's beliefs, he has to form beliefs about them to calcuate a best response. 56 This includes reciprocity, if formulated as in Section 2. (As we noted in a footnote there, others use a di¤erent formulation with utilities that depend on second-order beliefs.) Section 3, 1 believes that there is probability q that 2 would choose share.
Everything we said about simple guilt and (3) in Section 3.1 we could have formulated vis-à-vis G 11 rather than G 6 . We used G 6 merely because it is more spare, but Charness & Dufwenberg (2006) (cited in Section 3.1) actually used G 11 rather than G 6 . 57 If player 2 is instead motivated by guilt-from-blame then the implications are very di¤erent in G 6 and G 11 . In G 6 , following trust, if player 2's secondorder beliefs assign probability 1 to q = 1, then for a high enough 2 player 2's best response is share. This is true just as it would be also under simple guilt. In G 11 , however, following trust, if player 2's second-order beliefs assign probability 1 to q = 1, then player 2's best response is grab regardless of how high 2 is! To appreciate why, note that if 2 believes that q = 1 then 2 believes that 1 will not blame 2 if 2 chooses grab. Therefore, 2 can grab with impunity. 58 Game form G 11 , with guilt-from-blame appended to it, joins models of image concern such as G 10 in illustrating the critical role information across end-nodes can play in p-games. Modify G 11 such that 2's doubleton information set is broken up into two singletons. That is, if 1 gets 0 then 1 is informed of 2's choice. 59 The logic of the previous paragraph no longer applies, and in the modi…ed version of G 11 guilt-from-blame and simple guilt again work similarly.  57 Their reason is conceptual; from a contract-theoretic viewpoint G 11 may be seen to incorporate an element of "moral hazard" which is absent in G 6 . See Charness & Dufwenberg (2006, p. 1582). 58 The logic is similar to that we illustrated in regards to G 10 in Section 4.1. 59 Tadelis compares behavior in experimental treatments that resemble G 11 as well as the variation that we are describing here. concern a player's conception of "the right thing to do" and a proclivity to do what others believe is the right thing to do, especially if there is consensus about this (which would then be an ideal case of a social norm). Departing from the social norm entails an element of disappointing the expectations of others, and the authors explore the idea that decision makers are averse to doing so. In this regard, the motivation resembles guilt, as modeled in Section 3.1. However, d'Adda et al. consider players'expectations regarding how one ought to behave, rather than regarding how one will actually behave. This marks a way that the approach is not formally captured by p-games, as we have described them in this paper.
Many scholars have written papers about social norms, but few proposed formal models, in particular models that can be generally applied. 60 There is work to do in this arena, and we suggest that it should involve (some possibly extended version of) PGT. 61 Punishing transgressors Several motivations that we discussed incorporate some form of desire to punish those who, somehow, "misbehave." Negative reciprocity (Section 2) and anger (Section 3.3) have such features built in, and other notions may quite naturally be extended in that direction. For example, Sebald & Walzl (2015) explore the idea that player i may wish to be unkind (as in reciprocity theory) to j if j produced information (e.g., a performance review) that reduced i's self-esteem. Another example could be if someone is motivated to punish those who violate a social norm.
A recent paper by Molnar, Chaudhry & Loewenstein (2020) highlights an intriguing aspect of such punishment, one that would add a novel feature of belief-dependent motivation (addressable via suitable modi…cations of each of the motivations we just mentioned). Namely, the punisher may care about whether transgressors are able to …gure out that they are being sanctioned. The PGT-connection would be that the punisher's utility depends on the transgressor's belief regarding the punishers choice (and maybe also his intention). 60 López-Pérez (2008) is an important exception. His model is not PGT-based however. 61 We do not expect the topic to be easy to address. There are many subtle issues. Is a norm a strategy or a strategy pro…le (or, possibly, a set thereof)? If people like to follow norms, what exactly is the nature of the preference involved? Is the cost of breaking a norm dependent on whether and how many others do so?
Emotion carriers In most of the models we discussed, the belief-dependent part of a player's utility was built up with reference to particular material payo¤s. For example, following Battigalli & Dufwenberg (2007), player 2's guilt in G 4 has the dimension of (expected) material payo¤ of player 1. And in Battigalli, Dufwenberg & Smith's model, player i's frustration has the dimension of (expected) material payo¤ of i. This is not a necessary feature of belief-based utility, and alternatives have been considered. Attanasi, Rimbaud, Villeval (2019) consider "situations where donors need intermediaries to transfer their donations to recipients and where donations can be embezzled before they reach the recipients". They discuss how intermediaries may experience guilt if they do not meet the owner's expectation, although the associated material cost would be incurred by the recipient rather than the donor. And Battigalli, Dufwenberg & Smith (in their Section 5) mention how in principle frustration may depend on regret of a previous decision, unexpected perceived unfairness, or negative shocks to self-esteem.
Unawareness Almost all game-theoretic analysis assumes that the game form is commonly known between the players. Casual observations of reality suggest that this assumption may be too strong. It seems plausible that before the "Fosbury Flop"and the "V-style"were popularized by, respectively, Dick Fosbury and Jan Boklöv, many high jumpers and ski jumpers were not aware of these techniques, or at least of their lucrative payo¤ consequences. Or, awareness of the possibility and expected nature of hi-jackings may have been altered by the 9/11 events.
There are formal models of unawareness-see Fagin & Halpern (1988) for a pioneering e¤ort-and recent literature develops related techniques for modeling unawareness in games. Heifetz, Meier & Schipper (2006) is a key contribution here (and Burkhard Schipper provides an "Unawareness bibliography"with further references on his homepage at UC Davis). It is natural to imagine that belief-dependent motivation interacts with unawareness. For example, negative surprises that reveal previously unforeseen danger could instill fear; so, if 9/11 involved unawareness, then the occurrence of that event might have consequences for subsequent demand for air travel or supply of airport security.
Exploring unawareness using PGT seems potentially interesting, but we only know one paper that is devoted to the topic: Nielsen & Sebald (2017). We quote the informal example with which they open their paper (pp. [2][3] as it illustrates nicely how unawareness may interact with a belief-dependent motivation (namely, guilt): 62 Assume it is Bob's birthday, he is planning a party and would be very happy, if Ann could come. Unfortunately Bob's birthday coincides with the date of Ann's …nal exam at university. She can either decide to take the exam the morning after Bob's party or two weeks later at a second date. Ann is certain that Bob would feel let down, if she were to cancel his party without having a very good excuse. Quite intuitively, although Ann would really like to get over her exam as soon as possible, she might anticipate feeling guilty from letting down Bob if she canceled his party to take the exam the following morning. As a consequence, Ann might choose the second date to avoid letting Bob down. In contrast, consider now the following variant of the same example: Ann knows that Bob is unaware of the second date. In this situation Ann might choose to take the exam on the …rst date and not feel guilty. Since Bob is unaware of the second date and the …nal exam is a good excuse, he does not expect Ann to come. Ann knows this and, hence, does not feel guilty as Bob is not let down. In fact, if she were certain that Bob would never become aware of the second date, she probably had an emotional incentive to leave him unaware in order not to raise his expectations.

Motivated beliefs The 2016 summer issue of the Journal of Economic
Perspectives contains an interesting symposium on "Motivated Beliefs,"with an introduction by Epley & Gilovich (who credit George Loewenstein for taking "the leading role in stimulating and organizing the papers") and contributions by Bénabou & Tirole; Golman, Loewenstein, Moene & Zarri; and Gino, Norton & Weber. 63 The idea is this: Beliefs a¤ect people's well-being. This, in turn, a¤ects how they reason, control information, and gather & evaluate evidence. To some extent, it is argued, they may even choose their 62 We notice, however, that equilibrium concepts that do not capture strategic reasoning, such as self-con…rming equilibrium, are consistent with some forms of unawareness. For example, players' whose utility depend on their action and a state variable aggregating others'actions may hold conjectures (con…rmed in equilibrium) about such state variable without being aware of the actions of others. See Battigalli, Panebianco, & Pin (2018). 63 See also Loewenstein & Molnar (2018) who discuss related themes.
beliefs, although such choice may be unconscious and the ability to do so is hampered by reality-checks and various costs of having faulty beliefs. Epley & Gilovich mention how the topic has "a long history in psychological science"(p. 139). A particularly important reference would seem to be Kunda (1990), who wrote a highly in ‡uential paper on how motivation in ‡uences reasoning.
PGT is obviously useful for describing how beliefs a¤ect well-being; such links are embodied in almost every example of belief-dependent utilities that we have exhibited. Second, relatively little work in the literature on motivated beliefs has been formal, and PGT may provide relevant tools for scholars who want to develop theory. Third, PGT is well equipped to deal with how belief-dependent motivation may impact how people control information, and how they gather evidence. These aspects concern choices that presumably can be straightforwardly described in carefully selected game forms. To see this more clearly, note that PGT models the (rational) choice of an agent as a process that takes as given his system of conditional beliefs, but the actual beliefs held on the realized path may well depend on his actions (as well as actions of others and exogenous shocks). 64 For example, an agent with imperfect recall may store and recall, possibly at a cost, the ‡ow of information he receives, thus manipulating what he is able to remember and his beliefs. 65 As regards addressing other cognitive phenomena that the literature on motivated beliefs has discussed (e.g., modes of reasoning, evaluating evidence, and unconscious manipulation of beliefs) it seems less clear how PGT may provide useful tools. However, we are optimists and conjecture that PGT might prove useful for approaching those topics as well.

Formal framework
The previous exposition of PGT relied on intuition, examples, and some formulas involving game forms and beliefs that could be understood in the context of speci…c models. But a better appreciation of PGT requires an 64 Indeed, we touch on examples of this sort, e.g. in Section 5 on self-esteem, and also where we discussed the impact of di¤erent information structures (Section 4.1 and the part on "higher-order belief-dependence" of this section). 65  understanding of its formal framework, which is necessarily abstract. Here we focus on a simple class of games that covers all the examples of this paper except G 3 and G 3 in the Introduction: we consider p-games obtained from …nite multistage game forms with monetary outcomes, in which players may move simultaneously at some stage and perfectly observe past moves (including chance moves) when they have to make a choice. 66 However, we allow for the possibility of imperfect terminal information, which-as highlighted in the previous sections-may matter for psychological reasons. 67 The key feature of the analysis is the representation of players' beliefs about how the game form is played, and their beliefs about beliefs, as such beliefs a¤ect the (psychological) utility of end-nodes and expected utility calculations at non-terminal nodes. We mostly assume common knowledge of the rules of the game form and of players'utility functions, i.e., complete information. Yet, as illustrated in Sections 4 and 5, incomplete information has to be addressed when we analyze speci…c motivations such as image concerns and self-esteem, where utility depends on terminal beliefs about unknown personal traits.
Our conceptual perspective mostly relies on B&D, rather than the seminal work of GP&S. The reason is that GP&S only encompasses utilities that depend on players'initial hierarchical beliefs, since at the time of their writing (i) a formal analysis of hierarchical conditional beliefs had yet to be developed, and (ii) the importance of letting utility depend on updated beliefs had not been underscored in applications. B&D instead could leverage on the recently developed theory of hierarchical conditional beliefs (Battigalli & Siniscalchi 1999) and a wealth of applications where updated beliefs enter the utility function. Motivated by conceptual arguments as well as applications, B&D substantially generalize GP&S in several ways. We will brie ‡y point out the di¤erences. Finally, our formalism relies on the recent methodological article by BC&D, which simpli…es the analysis by putting only …rst-order 66 G 3 and G 3 involve game forms with a continuum of actions. G 9 is covered only if modi…ed (or interpreted) such that chance and Ben move simultaneously. 67 We further simplify in two ways: First, we do not explicitly describe players' nonterminal information when they are not active, which might be relevant for some anticipatory feelings (3.5). Our analysis works "as is" under the assumption that non-active players have the coarsest information consistent with perfect recall. Second, we assume that material consequences accrue at end-nodes only. See BC&D for a more general and explicit analysis of time, in which the game may last for one or more periods, which may have multiple stages, and consequences accrue after each period.
beliefs of all players in the domain of utility (so that expected utility depends only on second-order beliefs), but sharpens other aspects, such as the representation and role of players'plans.
Game form Formally, we start with a game form G = I; H; ; p 0 ; (P i ; i ) i2I with the following elements: I is the set of players not including chance, who is player 0; the set of personal players plus chance is H is a …nite set of possible sequences of action pro…les, or histories h = a k k =1 (for di¤erent values of`, and possibly including actions of chance) with a tree structure: every pre…x of a sequence in H (including the empty sequence ?) belongs to H as well. Thus, histories in H correspond to nodes of the game tree and ? is the root. Set H is partitioned into the set of non-terminal histories/nodes H and terminal histories (paths, end-nodes) Z.

For each h 2 H, (h)
I 0 is the set of active players, who perfectly observe h. With this, H i = fh 2 H : i 2 (h)g denotes the set of histories where i is active, and the set of feasible action pro…les is where A i (h) denotes the set of feasible actions of i 2 (h).
p 0 is the chance probability function, which speci…es a (discrete) probability density function p 0 ( jh) 2 (A 0 (h)) for each h 2 H 0 .
Beliefs We model the …rst-order beliefs of (personal) player i as a system i = ( i ( jh)) h2H[P i of conditional probabilities about paths of play z 2 Z. We are not assuming that i observes h when he is not active at h (h 2 HnH i ). In this case we interpret i ( jh) as a "virtual" conditional belief. We assume that: (i) i is consistent with p 0 , (ii) the chain rule holds, and (iii) i's beliefs about simultaneous or past and unobserved actions of other players do not depend on i's chosen action. 68 The latter implies that, for each h 2 H, i's conditional beliefs about the continuation can be obtained by multiplication from i's plan (behavior strategy) i;i 2 h2H i (A i (h)) and i's conjecture i; i 2 h2H i A (h)nfig (h) about co-players. Note that i's plan is part of his …rst-order beliefs. For example, i's initially expected material payo¤ E[ i ; i ] (which may a¤ect his utility via disappointment or frustration) depends on both i;i and i; i . As we further explain below, the interpretation is that i plans his contingent choices given his conjecture and thus ends up with an overall system of beliefs about paths.
Let 1 i denote i's space of …rst-order beliefs. We model second-order beliefs as systems i = ( i ( jh)) h2H[P i of conditional probabilities about both paths of play z 2 Z and co-players …rst-order beliefs i 2 j2Infig 1 j such that: (i) the marginal beliefs about paths form a …rst-order belief system in 1 i (hence they are also consistent with p 0 ), (ii) the chain rule holds, and (iii) i's beliefs about i and simultaneous or past and unobserved actions of other players do not depend on i's chosen action. We let 2 i denote the set of second-order beliefs systems of i.
To summarize, i 2 1 i denotes i's (…rst-order) beliefs about sequences of actions, or paths, whereas i 2 2 i denotes i's (second-order) overall beliefs about paths and co-players'(…rst-order) beliefs. In formulas with two-level hierarchies ( i ; i ), we maintain the coherence assumption that i is the marginal of i .
We point out two conceptually relevant di¤erences with B&D: (i) There, we represented behavior (what players have …rst-order beliefs about) as a complete description of the actions that players would take at each history where they are active, that is, a (pure) strategy pro…le rather than a path of play. (ii) In B&D, we explicitly represented …rst-order beliefs as beliefs about the strategies of others. Our explicit interpretation in B&D was that each player knows his (pure) plan and there is a necessary coincidence between each player's plan and the objective description of how he would behave whenever active, and that such coincidence is transparent to all players (see B&D, p. 11). Here instead we follow BC&D in modeling players' beliefs about paths, hence beliefs about the behavior of everybody. 69 Beliefs about own behavior are interpreted as (possibly non-deterministic) plans, which need not coincide with actual behavior. For example, if i is initially certain that j's plan is j;j and then observes a deviation from j;j , he may still believe that j' plan was indeed j;j but that he took an unplanned action by mistake (a kind of "tremble" as in Selten 1975). The analysis of B&D instead rules this out: every observed action is necessarily interpreted as a planned choice (cf. our discussion of guilt aversion in Section 3.1). In sum, our framework is su¢ ciently expressive to model players' intentions, their perceptions of the intentions of others, and how such perceptions are a¤ected by observing actions. This is important in standard games to elucidate the di¤erence between, say, forward-and backward-induction reasoning. 70 It is even more important when players care intrinsically for the intentions of others, as with many forms of belief-dependent preferences.
Standard utility Before we describe the belief-based utilities that are characteristic of p-games, it may be helpful to recall how utilities are de-…ned in traditional game theory. Namely, player i's utility has the general form u i : Z ! R. This does not imply that i is "sel…sh". Caring only about own material reward is a special case (u i (z) = i (z) for all z 2 Z), but i could alternatively be motivated by a host of "social preferences" including altruism, inequity aversion, maximin preferences, or warm glow. 71 However, the forms of motivation that we discussed are ruled out as they require a richer notion of utility.
Psychological utility and p-games As argued by BC&D, most forms of belief-dependent motivations for a given player i can be modeled by assuming that, for some terminal history z, i's utility for reaching z depends on the …rstorder beliefs pro…le ( j ) j2I . Thus, we have utility functions with the general 69 The set of strategy pro…les is exponentially more complex than the set of paths. Hence, beliefs about paths are simpler. 70 1 j ! R. These typically involve both the material payo¤s and some features of own or others'initial, interim, or terminal …rstorder beliefs. For example, in the cheating model of Section 4.1, player 1's utility at terminal history (x; y) depends on his monetary payo¤ 1 (x; y) = y and on 2's terminal belief about die roll x given report y. In this case, utility depends on the terminal …rst-order beliefs of someone else. If instead i (besides liking money) dislikes disappointing j, then his utility for reaching z is decreasing j's disappointment [E[ j ; j ] j (z)] + , which depends on j's payo¤ and his initial belief (cf. Section 3.1 on guilt aversion). In both cases, i's utility of terminal histories depends on payo¤s and the (unknown) …rstorder beliefs of another player. This is like a standard state-dependent utility function. As noted by B&D, the maximization of its expected value can be analyzed with standard techniques leveraging on the dynamic consistency of subjective expected utility maximizers.
For other motivations like aversion to disappointment (Section 3.2), or belief-dependent loss aversion (Section 5), i's utility depends on his expectations (e.g., on the initially expected material payo¤ E[ i ; i ]), hence on his own plan i;i . We showed in Section 3.2 that such forms of own-plan dependence yield dynamic inconsistency of preferences, which implies that some care is required in de…ning what it means to be subjectively "rational". Similar considerations apply to emotions like frustration and anger (Section 3.3) and to anticipatory feelings with negative or positive valence like anxiety, or suspense (Section 3.5). Essentially, i's plan i;i must form an "intra-personal equilibrium" given his overall belief system ( i ; i ) (cf., e.g., K½ oszegi 2010 and the relevant references therein). Next, we explain this in detail.
The combination of a game form and psychological utilities for all players gives a p-game. We consider p-games where the belief-dependence of utility is limited to …rst-order beliefs. The part of Section 5 on "higher-order beliefdependence"mentions exceptions.
Subjective rationality Fix a second-order belief i 2 2 i with marginal …rst-order belief i 2 1 i including i's plan i;i . For every non-terminal or terminal history h 0 2 H, we can determine the expectation of u i conditional on h 0 , written E[u i jh 0 ; i ]. Now consider a history at which i is active, viz. h 2 H i . Each action a i 2 A i (h) yields expected utility Belief system ( i ; i ) satis…es rational planning if every action that i expects to take with positive probability is a local best reply, that is, . Subjective rationality requires that player i plans rationally given his beliefs and carries out his plan when given the opportunity. Thus, from our perspective, consistency between plan and behavior is not a necessity, but rather a rationality condition. A player's plan of how to play cannot constrain his behavior in the game form, it can only guide it. Consistency between plan and behavior should be a rationality condition. This is the perspective adopted here and in more recent work on p-games. 72 When u i (z; ) does not depend on i , or-more generally-does not depend on i's plan i;i , then rational planning is equivalent to the standard sequential rationality condition 73 and i's rational plan can be non-deterministic (not a pure strategy) if and only if i is always indi¤erent between the pure strategies in the "support"of i;i (cf. Remark 1 in BC&D). If instead u i (z; ) depends on i;i , …rst, it may be impossible to satisfy the standard sequential rationality condition, second, deterministic rational plans may not exist. The discussion of disappointment in game form G 7 (Section 3.2) provides a simple illustrative example.
Let us here note that a feature of early work on p-games and, most speci…cally, on guilt was the tacit assumption that a player's behavior is necessarily consistent with his plan, independently of his being rational or not. B&D, in fact, explicitly assume that behavior is necessarily consistent with plan and that this is transparent to the players. We can illustrate this with reference back to the discussion of guilt in Section 3.1). In G 6 , when 2 observes trust, he takes for granted that^ 1 = 10 q where q denotes the probability subjectively assigned by 1 to share (given trust), because 1 must have planned to trust. With this, the relevant psychological utility of 2 given (trust,grab) can be written as 14 2 10 q. This makes a di¤erence for the de…nition and implications of some solution concepts, like sequential equilibrium, as explained in detail by BC&D in their Section 7.6. 72 See B&D and also Battigalli & De Vito (2018) and . 73 The strategy of i is ex ante optimal, and the continuation strategy is optimal starting from every h 2 H i .
Local utilities, incomplete information, and time Solution concepts for p-games can be de…ned and analyzed starting from the "local"expected utility functions u i;h : To model some belief-dependent action tendencies such as the desire to reciprocate (un)kind behavior (un)kindly (Section 2), or the desire to vent one's own frustration by harming others (3.3), it is convenient to work directly with such historydependent utility functions, without deriving them from utilities of terminal histories.
A realistic analysis of strategic thinking may have to account for uncertainty about personality traits, i.e., incomplete information. This can be achieved by parameterizing such traits with some vector and letting players'…rst-order beliefs concern the unknown part of as well as behavior. We argued that beliefs about personal traits are also essential to model some motivations such as image concerns and self-esteem (see Sections 4 and 5).
Finally, as in several game-theoretic models in standard economics and behavioral economics, it may be necessary to introduce the role of time explicitly, not only to model preferences over streams of outcomes, but also to model (i) preferences over streams of emotional states (e.g., Ely et al. 2015), (ii) changing belief-dependent referents (e.g., K½ oszegi & Rabin 2007Rabin , 2009, and (iii) decay of emotional states, such as anger, and the related actions tendencies (e.g., Gneezy & Imas 2014). To do this, one has to distinguish between stages and periods. Actions are taken within stages and information accrues to players between stages. Outcomes occur within periods, which may comprise multiple stages, and decay or discounting concern di¤erent periods. For example, we mentioned in Section 5 how K½ oszegi & Rabin's concepts of Personal Equilibrium and Choice-acclimating Personal Equilibrium can be accommodated within our framework by applying the aforementioned notion of rational planning to di¤erent time frames.
For a general analysis of the relationship between "global" and "local" utility functions, of incomplete information, and of the role of time see BC&D and the relevant references therein.
Di¤erences with GP&S Our perspective and formal analysis di¤ers from that of GP&S in several ways. Let us …rst address the least important one: unlike GP&S (and B&D) we stop at beliefs of the second order. To our knowledge, this is enough to encompass the overwhelming majority of applications. Thus, we focus on the case where only (…rst-and) second-order beliefs matter for expected utility calculations. With this, GP&S consider only initial beliefs about the behavior and the initial beliefs of others. In particular, in game forms with simultaneous moves (where Z = A := A (?)) GP&S consider utilities of the following form:û i a; ?
i; i , where ? i; i 2 (A i ( j6 =i (A j ))) denotes i's initial belief about the behavior and the (…rst-order) beliefs of co-players. We obtain such functional forms in the special case where only initial beliefs about others matter (see B&D for details). The approach of GP&S has three important limitations. First, it rules out models where utility depends on updated beliefs, such as models of sequential reciprocity (Section 2), image concerns (4), deception (4.1), and self-esteem (5). Second, it rules out own-plan-dependent utility as in models with belief-based reference-dependence (Sections 3.2, 3.3, and 5) and anticipatory feelings (3.5). Third, even if utility depends only on initial beliefs, GP&S'framework restricts the toolbox of strategic analysis to (extensions of) traditional equilibrium concepts whereby players have correct beliefs about the (initial) beliefs of others, which therefore never change as play unfolds, on or o¤ the equilibrium path. Indeed, if this were not the case (as in appropriate versions of rationalizability, see BC&D), it would be necessary to address the issue of how players update their beliefs concerning what they care about, i.e., others'beliefs.

Experiments
In developing theory, our favored approach is to focus on interesting assumptions about forms of belief-dependent motivation, and to explore what they imply. Our main goal is not to explain data (although we are not saying that doing so cannot make sense). With this outlook, once a theory is formulated, a natural next step is to be inquisitive as regards its empirical support. Theories formulated using PGT can be tested for empirical relevance in lab experiments. Our main goal in this section is to give a fairly exhaustive review of aspects of experimental design that are of particular relevance to tests of PGT-based theory. We do not try to give an exhaustive account of results, although we brie ‡y describe some prominent …ndings.
We cover a series of themes we deem important. For clarity, we give each a separate heading. The …rst themes are closely related. We list them next to each other. Later on, the themes are less closely related. We apologize that the transitions then may feel choppy.
Belief elicitation Models formulated using PGT suggest ways that particular beliefs impact preferences and play. To conduct lab tests it is often helpful to elicit those beliefs. The very …rst experiment speci…cally designed to test a PGT-based prediction was built around that insight. Dufwenberg & Gneezy (2000) considered versions of G 1 (recall: player 2 chooses t 2 f0; :::; M g) as well as Trust Games where 1 could take an outside option (choose out) or choose in and let 2 choose in a subgame structured like G 1 ("Lost Wallet Games"). They measured 1's …rst order-belief (FOB = expectation of t) by asking 1 to guess t (with rewards for accuracy). And they measured 2's second-order belief (SOB = the conditional expectation of 1's FOB) by asking 2 to guess 1's guess (again with rewards for accuracy). 74 The test for guilt checks whether for subjects in the position of player 2 there is positive correlation between t and those guess-guesses. Dufwenberg & Gneezy performed such tests and, by and large, found support for the theory.
There is a large follow-up literature testing for the empirical relevance of guilt in various game forms, and which often elicits beliefs. Most commonly, binary Trust Games like G 6 are explored. See Cartwright (2019) for a survey. 75 Some studies elicit beliefs in order to study other forms of motivation than guilt, again to a large degree reporting support. The pioneer to do this for reciprocity theory is Dhaene & Bouckaert (2010), who carefully elicit precisely the belief-data that the task demands (including particular conditional beliefs). 76 And a few recent studies testing aspects of Battigalli, Dufwenberg 74 The description is precise as regards G 1 . In the Lost Wallet Games, 2 was actually asked about the average guess of all the subjects in the role of 1 who chose in. This is crucial to eliciting the right belief, namely 2's belief conditional on 1 choosing in. 75 Many scholars report continued support for the theory, though this is not universal. See also Guerra  & Smith's models of frustration and anger also do it. 77 Some scholars we met, mainly of decision-theoretic bent, seemed skeptical on grounds of principle to the idea of belief-dependent motivation, and in the sequel to the idea of eliciting beliefs. This would be people who revere "revealed preference", who argue that beliefs are not real, or at least not observable. Beliefs are merely a theory feature, something that should be viewed only as part of a preference "representation". Conceivably, it is then argued that belief elicitation is pointless, as one then measures something that is not real. In our view, this position has little merit. It militates against introspection, against the rationale involved in humans'use of language, and against some of the experimental evidence we cited. Much like psychologists have by and large given up on their analogous "behaviorist" approach (favored in much of the twentieth century), many calls for revealed preference in economics seem obsolete to us. That said, however, one has to admit that there are many thorny methodological issues surrounding how to best measure subjects'beliefs. 78 Di¤erent PGT-related papers take di¤erent approaches and some (e.g., Cartwright) discuss pros & cons. See Schotter & Trevino (2014) for a (broader than just PGT) critical survey of the literature on belief elicitation in laboratory experimental economics.
Belief disclosure Charness & Dufwenberg (2006) point out that the guilt hypothesis just discussed is confounded by a form of "false consensus," if 2's choice (done for whatever reason) shapes her SOB such that she believes others believe she made that choice. This would imply that a subject's choice drives his SOB, rather than the other way around (as the guilt story has it). Ellingsen, Johannesson, Tjøtta & Torsvik (2008) propose a clever alternative design, which avoids that issue but which has another problem.
indicate that many, but not all, subjects conform with the theory. 77 See Aina, Battigalli & Gamba (2020) and Dufwenberg, Li & Smith (2018a,b). Also Persson (2018) performs such a test, although he does so without eliciting beliefs. With the exception of Persson not documenting support for "simple anger," much of this evidence is supportive. See also Battigalli, Dufwenberg & Smith's (Section 5) discussion of several other older experiments which were not designed to test the theory and yet may be viewed in such a light ex post. 78 For example, should guesses be done before or after choices are made; refer to probabilities of a particular co-player's choices or frequencies of choices among a set of subjects one might be matched with; be incentivized or not, and if so how? These questions often have no obvious answers (for example, a quadratic scoring rule may provide precise incentives to reveal a particular expectation, but may also be harder for a subject to understand).
Rather than elicit 2's SOB they elicit 1's FOB, which they then disclosed to 2 before she made her choice. This procedure induces 2's SOB without the risk of false consensus. The drawback, however, is a potential loss of control. In Ellingsen et al.'s design 2 is informed that 1 was not informed that his elicited belief would be handed down to 2. This design feature is important, because if 1 knew then he would have had an incentive to lie (if he believed 2 would believe him). The problem is that when 2 learns that some design information is withheld from the players she may wonder if possibly there are other design aspects that are withheld from her. Perhaps that would a¤ect her behavior. 79 No elicitation It is not always necessary to elicit beliefs to meaningfully test PGT-based hypotheses. Sometimes patterns of behavior are idiosyncratic enough to a speci…c theory that clear conclusions can be drawn by observing only choice data. To illustrate, recall the perceived cheating aversion theory of Section 4.1. In their die-roll experiment, using a design matching G 10 , Fischbacher & Föllmi-Heusi found reporting frequencies fell in between what would obtain with honest choices (16.7% for each y) and sel…sh reporting (100% y = 5). One does not need to elicit any beliefs to see that this is in line with the theoretical prediction we described. 80 A further striking insight concerns treatments that manipulate player 2's information. Recall our discussion of how (with perceived cheating aversion) such a change is predicted to undermine the partial lies prediction. Gneezy et al. There is also a large experimental literature which discusses and tests (and often …nds support for) more informally formulated notions of reciprocity. Typically, no explicit connection to PGT is made. We will not discuss details, but see Fehr & Gächter for an early highly in ‡uential discussion.
Which theory wins? That's the wrong question! It is true that to the extent that we have so far cited experimental results, these have mostly been supportive. Someone may wonder: can really each theory, whether it concerns guilt or reciprocity or anger, be supported? And shouldn't we …gure out which one is more relevant? Our answers are yes and a quali…ed no. We …nd it plausible that all (or most) of the many motivations we have discussed matter (including, e.g., all the emotions list cited at the end of Section 3.6). Why else would humans have come up with words to describe them?
As regards experiments, the style we focus on in this section is not trying to …nd a "best"theory for explaining data. Rather, we explore the empirical relevance of propositions that are mostly not mutually exclusive. Do we believe all of these motivations to always be relevant in all situations at the same time? No we certainly don't, and we would like to propose that an important research area of high (but seemingly largely untapped) potential is to …gure out which situational cues trigger what kind of motivations.
Communication Charness & Dufwenberg (2006) argue that guilt can help explain why communication, and in particular promises, can foster trust & cooperation. Recall our discussion in Section 3.1. They designed an experiment to test that hypothesis, using methods similar to those of Dufwenberg & Gneezy described above. Vanberg (2008) argued that the results are confounded by another "commitment-based theory," i.e., that decision makers have a belief-independent preference not to break a promise they made. To test his theory, Vanberg came up with an ingenious design, based on a "switching feature". Any subject to whom a pre-play promise were issued was "switched" and replaced by another subject who would play with the person who issued the promise. If there were a switch, the promisor was told but the promisee was not. The key idea is that promisors would su¤er expectations-based guilt independently of whether or not a switch occurred, whereas any cost of breaking a promise would apply only if no switch took place. The commitment-based theory is not PGT-based. However, discussions of it typically involve comparisons with Charness & Dufwenberg's belief-based account, so it is important for PGT-scholars to know about Vanberg's work.
The studies cited in the previous paragraph report support for the story they set out to test, and lots of follow-up work has attempted to evaluate which one has stronger pull on data. It is fair to say that the some studies support one while others support the other, and that a debate is ongoing. Exogeneity & causal inference Vanberg's approach is important also for the following methodological reason: Testing for belief-dependent preferences by comparing subjects who self-report di¤erent beliefs, as Charness & Dufwenberg did, has the drawback of not relying on exogenously created variation. Subjects are not randomly assigned to their (so-to-say, homegrown) beliefs. This weakens the force with which valid causal evidence can be drawn.
Similarly, if subjects can choose which message to send, then they are not randomly assigned to their messages. Vanberg overcame this last issue via his switching mechanism, creating exogenous variation is whether or not a subject had sent a promise to the player he eventually interacted with. Vanberg did not attempt to create exogenous variation in subjects SOB though, so his design is not ideal for reconsidering Charness & Dufwenberg's hypotheses.
Ederer & Stremitzer (2017) developed a design that involves exogenous variation in subjects' SOB's, and Di Bartolomeo et al. developed a design that features exogenous variation in both SOB's and promises. We refer to these studies for more information, while noting that exogenous variation and causal inference has become of high importance in this literature.
Avoidance For certain PGT-related testing purposes it may be useful to employ designs that allow a subject to avoid making another subject aware of a game being played. This would presumably be useful if one were to test ideas that directly involved unawareness, like in Nielsen & Sebald's "party example" of section 5, but it can also be useful for testing whether subjects care about image as described in section 4. For example, consider the design of Dana, Cain & Dawes (2006): Subject i were given a choice whether to "exit"a $10 Dictator Game form (like G 1 , with M = 10) and take $9 instead, knowing that the exit option would leave the receiver j nothing and ensure that j never knew that a dictator game form could have been played. Dana et al.'s (2006) design provides a test whether i cares for his image. The idea is that by exiting i may enjoy a (rather) high payo¤ without su¤ering a bad image. 81 Alternative designs can test similar hypotheses without leaving players unaware of strategic possibilities, by allowing a player to avoid revealing a choice. For example, the design of Andreoni & Bernheim (2009) examines "an extended version of the Dictator Game in which (i) chance sometimes intervenes, choosing an unfavorable outcome for the recipient, and (ii) the recipient cannot observe whether chance intervened" (p. 1609). 82 The idea is that by choosing directly the "unfavorable outcome" that chance might implement, a subject can avoid a bad image.
Method of play PGT provides a perspective to elucidate the costs and bene…ts of the methods of play used in experiments about sequential games and decision problems: the direct-response method, whereby subjects play sequentially (thus generating limited and unbalanced data about responses to di¤erent actions of early movers), and the strategy method, whereby subjects commit in advance (but covertly) on their contingent choices, i.e., they play the game in strategic form. 83 When and how should we expect the strategy method to distort behavior compared to what would occur in a sequential situation? This depends on the game form and the motivations that are supposed to drive behavior.
From a theoretical viewpoint, playing with the strategy method should be equivalent to playing with the direct response method when preferences are dynamically consistent. If instead, preferences are dynamically inconsistent, the ability to "tie the hands of one's later selves" by committing to a conditional response may signi…cantly a¤ect behavior in some situations. PGT underscores a wealth of psychological reasons for dynamic inconsistency, and 81 An exit choice gives $-payo¤ combination (9; 0) to i and j, whereby i reveals a preference for that outcome over each of the $-payo¤ combinations (10; 0), (9; 1), and ( suggests when they may be relevant. To illustrate, if subjects play an Ultimatum Game, anger and inequity aversion (a non-belief-dependent motivation) both provide potential explanations for rejections. According to the theory of frustration and anger of Battigalli, Dufwenberg & Smith, anger can only arise when a responder actually observes a greedy o¤er; instead, the mere contemplation of such a possibility with the strategy method is unlikely to engender enough frustration and anger to make him reject. 84 By contrast, if the only reason for rejections were inequity aversion, a consequentialist model of distributional preferences, 85 then the strategy method should be equivalent to the direct response method. For example, in the Ultimatum mini-game G 5 , the commitment to reject the greedy o¤er makes a di¤erence for the distribution of monetary payo¤s only if the proposer indeed makes the greedy o¤er; therefore, the ex ante preference between committing to accept or reject coincides with the post-o¤er preference between the payo¤ pairs (0; 0) (reject) and (9; 1) (accept, with $1 for the responder). 86 In the Trust mini-games G 6 and G 11 , second movers cannot be negatively surprised when they have to choose. Thus, even if they are prone to anger, such emotion cannot a¤ect behavior. Guilt aversion and image concerns, instead, are likely drivers of behavior. But, as explained in Section 6, in such models psychological utility does not depend on the agent's plan, and the resulting belief-dependent preferences are dynamically consistent. Thus, the strategy method should be equivalent to the direct response method. 87 Other forms of data It may be useful to consider other kinds of data than choices and elicited beliefs to test PGT-based hypotheses. For example, brain imaging data (e.g., fMRI), emotion self-reports ("please rate how strongly you feel emotion X on a scale..".), electrodermal activity, or facereader data may be useful. Chang 86 Aina et al. report that the rejection rate in the Ultimatum mini-game is indeed higher with the direct response method, consistently with the hypothesis that anger is a driver of behavior. 87  emotion self-reports, in a way that was mindful of the possibility that pangs of guilt might be counterfactual and yet crucial (compare observation (x) in the guilt part of Section 3 above). 88 We do not know of any face-reader study which was conducted with an explicit PGT-connection in mind, but van Leeuwen, Noussair, O¤erman, Suetens, van Veelen & van de Ven (2018) use the technology to explore anger and Battigalli, Dufwenberg & Smith cite their results when motivating their own theory.

Applications
We hope to inspire applied work. One may think of applications on two levels. The broader one would be to formulate within PGT a model of some beliefdependent motivation that can be used to analyze some class of game forms. We already discussed this topic, for models involving reciprocity, guilt, regret, and anger. Our focus in this section will instead be to applications of a more targeted form, exploring some particular economic setting or question. Our …rst and third examples of the introduction provide two speci…c examples, tipping and fashion choice being the economic phenomena scrutinized. Most applied work involves reciprocity, guilt, belief-dependent loss aversion, and image concerns. We listed relevant references in previous sections, but space constraints make it impossible to describe details. Instead, in this section, we attempt to provide some perspective by commenting on possible angles that applied research may follow.
First, one may …nd settings in which some form of belief-dependent motivation is plausibly operational, but which economists gave scant attention. Likely, economists may not have been used to exploring the sentiment in question, and therefore didn't look at the overall scenario. An example of such research could be Caplin & Leahy's (2004) exploration of how a caring doctor should disclose (or not) information to a patient who is about to undergo an operation and who is a¤ected by anxiety.
Second, one may take a well-known classical model and explore whether predictions are robust to the incorporation of some particular form of beliefdependent motivation which might be plausible in that setting. The paper by Patel & Smith (2019) can exemplify. They consider a classic setting of public goods provision and ask how the equilibrium set is a¤ected when players are in ‡uenced by guilt.
Third, one may explore whether comparative statics that hold under classical analysis are relevant also if some form of belief-dependent motivation is at play. Dufwenberg et al.'s (2013) study of "the hold-up problem" can exemplify. The analysis involves a form of reciprocity theory where players only reciprocate negative kindness, and involves the game form G 12 : The parameter ! takes di¤erent values depending on how details of a holdup problem are described, 89 but under classical assumption that does not matter. If utility equals (own) money, then the backward induction solution is independent of !! However, if player 1 is inclined to take revenge, applying the theory, he will wish to choose reject if ! is low enough, so the hold-up problem is mitigated in these cases. The authors discuss how this may have implications for how to organize …rms, a topic discussed in the academic discipline called Strategy.
Fourth, one may observe some pattern of behavior that looks like a puzzle from a classical viewpoint, and explore whether some form of beliefdependent motivation could plausibly be involved and resolve the puzzle. One example of such work could be the study by Conconi et al. (2017), which starts with the arguably puzzling observation that trade disputes are often initiated by incumbent political candidate shortly before elections (e.g., by Obama in 2012). Conconi et al. show that if the electorate is motivated by reciprocity then the observed pattern may be predicted. Another example is the demonstration by Herweg, Muller & Weinschenk (2010) that if agents in a contract-theoretic setting are in ‡uenced by belief-dependent loss aversion à la K½ oszegi & Rabin, then under some circumstances optimal incentive 89 The authors use the following story to derive the special case of ! = 2: "An artist (player 1) has been asked by a presumptive buyer (player 2) to paint a 'beautiful portrait of 2.' 1 may disagree or agree. In the former case, 1 and 2 go separate ways. In the latter case, 1 spends $2; 000 worth of his/her time on the painting, and a contract says 2 should subsequently pay $5; 000 to 1. The value to 2 is $8; 000, but 2 may complain and claim (falsely) that the portrait is 'rather ugly'and attempt to renegotiate o¤ering a newprice of $1; 000. Given the ambiguity of what constitutes beauty, 1 cannot enforce the $5; 000 payment and will have to accept or reject the new o¤er. 1 knows that no person other than 2 would pay to acquire the painting." In another story, where the fruits of the e¤ort becomes human capital of player 2 (which 1 cannot deny 2 access to), ! = 10. schemes will have a simple form, involving only a …xed wage and a bonus. This addresses the puzzling observation that many real-world contracts look that way, whereas traditional contract theory predicts more complexity and nuance in regards to how payment depends on performance.
Fifth, one may demonstrate how theoretical tools that are known to have certain properties in traditional games may somehow have di¤erent implications when a belief-dependent motivation is at play. Bernheim (1994) can demonstrate what we have in mind. Besides shedding light on an image concern and conformity, his study also reveals some new insights solutions on signaling games. Bernheim applies a particular re…nement (viz, the "D1 criterion"), argues that it has proven "hostile to pooling in a variety of contexts" (p. 855), and emphasizes how special and telling it is that he nevertheless gets pooling/conformity in his model.
Sixth, we have emphasized several times how in traditional games predictions do not depend on information across end-nodes, but how this ceases to be true in some p-games. One may look for interesting economic settings where this matters. An example is provided by Dufwenberg & Nordblom's (2018) study of tax evasion and guilt, in which it is shown that whether tax returns are private or public in ‡uences information across end-nodes and strategic play.
Seventh, experiments may have revealed insights regarding the impact of some belief-dependent preferences. The documentation of e¤ects may, however, have been con…ned to rather special lab games, and one may explore how the ideas apply in naturally occurring economic settings. For example, consider Charness & Dufwenberg's (2006) evidence that pre-play communication, via mechanisms that depend on guilt, may foster trust and cooperation. They derive that insight in a simple binary Trust Game form. One could consider exploring whether the pattern extends also to a setting with cartelists who meet informally to discuss price-…xing.
Finally, as we wrap up this section, it is natural to wonder where the meat will be going forward. What new speci…c topics should one look at? It is beyond the scope of our text to exhaustively answer that question. Our goal is to inspire e¤orts in that direction rather than conduct new studies ourselves. Yet we cannot resist pointing in two particular directions which we feel o¤er fertile grounds for timely and original contributions: (i) Starting from the perspective of a particular emotion, we propose that there is scope for interesting work to be done involving anger. In Section 3.3, we mentioned many settings including road rage and support for populist candidates. Take the last of these. Suppose that (two or many) competing non-populist candidates (or parties), somehow, make promises that, for one reason or the other, they do not live up to once elected (or at least some folks feel that way). So the elected politicians end up frustrating many voters. Now comes Donald Trump (or Matteo Salvini) and says he'll "drain the swamp," which presumably would be a policy that is annoying for the incumbent. That (populist) candidate may then get the votes of those frustrated voters just because they want to hurt the incumbent! Such a prediction would seem consistent with the model of Battigalli, Dufwenberg & Smith, although one would have to develop the application to say with more con…dence.
(ii) Starting from the perspective of a particular societal topic, we believe PGT has much to o¤er for analyzing behavior and economic outcomes during pandemics. As a matter of fact, we are not the …rst to argue this point. It is the core idea of a recent paper by Peter Huang (2020). He "advocates building rigorous, theoretical models to analyze how leaders and leadership communications in pandemics can reduce anger, anxiety, and frustration, prevent panic, inhibit complacency, and foster compliance with NPIs"(i.e., nonpharmaceutical interventions, like social distancing and self-quarantining), and goes on to argue that PGT provides relevant tools. We couldn't agree more, and strongly recommend his text for inspiration.

Comments on methodology
We now address a collection of methodological topics concerning how to model economic settings and how to generate predictions.
Fictitious auxiliary games Can and should we study the psychological phenomena we have described using traditional game theory instead of PGT? Sometimes p-games, most notably perhaps those involving image concerns as described in Section 4, can be turned into "strategically equivalent"traditional games by endowing the observer with a …ctitious action space whereby he reports a belief, or estimate of I i and is rewarded with an incentive compatible scoring rule. The receiver's belief-or estimate-is then replaced by his action/report in the sender's utility function. For example, the (pseudo-) utility of observer j is and E h e I i jP j (z) ; j i is replaced by a j in (13). This works because j maximizes the expected value of (15) by letting a j equal the conditional expectation of e I i . As long as i believes in j's rationality, the strategic analysis of the p-game and such an associated traditional game are equivalent.
As in many …elds of pure and applied math, transforming a problem into an "equivalent"one may give access to the application of known techniques and results. 90 However, the possibility of such transformations has also engendered the claim that PGT is, after all, not needed: choosing di¤erent "weird" assumptions about utility 91 one can go back to good, old, familiar game theory, making everybody feel at home. We are very critical of such attitudes. They confuse formalism with reality. The reality is described by the true game form (something that can be designed and controlled in the lab) and the true utility (which-in so far as it exists-one can try to elicit under appropriate auxiliary assumptions). If player j is passive in reality, coming up with a false representation of reality to claim representability with an old framework can be misleading. 92 For example, the suggestion that traditional game theory can model everything (possibly after some transformation), even if correct (which we doubt), distracts from important phenomena that cannot be explained when the true game is not a traditional one, such as the dependence of behavior on the information another player gets when he is inactive. More generally, we propose that adopting indirect techniques referring to known di¤erent …elds may prevent the conceptually and mathematically correct understanding of the object of analysis, without giving signi…cant bene…ts over the use of a direct approach. One of the problems is that the theorist often does not realize that he is relying on assumptions that may be wrong/unappealing in some contexts. We illustrate this with a PGT example. GP&S give indirect 90 To mention non-obvious ones, results about forward-induction reasoning and rationalizability in a class of in…nite dynamic games (Battigalli & Tebaldi 2019) can be applied to p-games with image concerns. 91 As one of our colleagues put it. 92 Furthermore, nobody has shown that all interesting forms of p-games can be turned into "equivalent" standard games. Considering claims made at seminars we have given, we suspect this is not for lack of trying. The only article we know of dealing with the topic is that of Kolpin (1992). He limits attention to the class of p-games considered by GP&S. In our view, while his exercise is pioneering and useful as an attempt at proof-of-concept, the speci…c assumptions he engages are too convoluted to be practical. de…nitions of di¤erent kinds of equilibria in p-games. 93 They …x a pro…le of (initial) belief hierarchies, then look at equilibria of the resulting traditional game and the induced degenerate belief hierarchies whereby beliefs of all orders are correct, and …nally de…ne psychological equilibria as …xed points of the resulting correspondence from the space of pro…les of belief hierarchies to itself. This is a clever …rst step, but it is also convoluted. Indeed, B&D show that a direct de…nition is both feasible and more transparent. Furthermore, when GP&S consider (trembling-hand) perfect equilibria, their indirect approach induces, in our view, a conceptual error: They "forget"that-in the spirit of Selten (1975)-a perfect equilibrium of the p-game should be the limit of equilibria of perturbed p-games whereby players are constrained to "tremble" with strictly positive, although vanishing, minimal probabilities. Using this conceptually correct de…nition, one can show that a perfect equilibrium always exists under continuity of utility in beliefs. 94 GP&S instead obtain with their indirect de…nition a di¤erent equilibrium concept and show by example that it may fail to exist (p. 73). What went wrong? Using their indirect approach that considers equilibria of auxiliary traditional games, GP&S did not realize that trembles in actions must induce corresponding perturbations in beliefs, hence in the utility of terminal nodes. Once this is taken into account, their counterexample vanishes.
Fictitious auxiliary p-games A convenience-in-modeling argument may cut in the other direction as well: Models with an intrinsic concern for beliefdependent reputation can sometimes o¤er a compact reduced-form approach to modeling repeated interaction in settings where players are not assumed to have any belief-dependent motivation. That is, a p-game can in such a case be useful modeling tool. For examples that take such an approach, see Morris (2001) and Ottaviani & Sørensen (2006).
Solution concepts PGT-analysis involves two key steps: (i) modeling belief-dependent utility, and (ii) applying a solution concept.
Step (i) is unique to p-games, step (ii) is relevant also for traditional game theory. Our main goal has been to emphasize what is unique to p-games, so we have focused mostly on step (i). However, since we feel strongly about step (ii), 93 Of the restricted class they analyze. 94 Essentially, B&D prove the existence of perfect equilibrium, to obtain existence of sequential equilibrium, a slightly weaker concept. let us explain our view.
Economists frequently take for granted that ad hoc notions of equilibrium (whereby players are assumed to have correct beliefs) meaningfully describe strategic interaction. Often such assumption are not well justi…ed. In oneshot play settings, if players reason about each other's rationality and beliefs, inferences should concern steps of deletion of non-best-replies (possibly all the way to a form of rationalizability). If learning is allowed by looking at recurrent interaction, the appropriate solution is (some version of) self-con…rming equilibrium, in which beliefs may be incorrect, although consistent with evidence. In neither case is the most commonly applied solutionsequential equilibrium-generally implied. 95 Only in rare cases is it justi…ed to assume that a sequential equilibrium will be played, most notably when this solution concept yields the same prediction as rationalizability, or self-con…rming equilibrium. 96 Since a proper discussion would call for its own article, we have not gone there. Our approach has mainly been consistent with our favored view as we focused on steps of deletion of non-best-replies. But since previous scholarship (including ours) often referred to more traditional notions of equilibrium, we made a few related references when recalling such work.
We hope future work will take the appropriateness and relevance of solution concepts more seriously than has been done in the past.
Transformations of the game form A small literature studies transformations of the game form (with monetary payo¤s) 97 that do not change the reduced normal form (see in particular Thompson 1952, Dalkey 1953, and Elms & Reny 1994. Considering only those that preserve perfect recall, 95 B&D extend Kreps & Wilson's (1982) classic notion of sequential equilibrium to pgames. See BC&D for relevant p-games de…nitions of all solution concepts mentioned above. See Battigalli, Corrao & Sanna (2019) and Jagau & Perea (2018) for epistemic foundations of (versions of) rationalizability. 96 A more special circumstance may apply in G 10 as analyzed in Section 4.1, given the interpretation that player 2 is player 1's "imagined" audience (as hinted at). If 1 is, so-to-say, "his own audience," we have a single active player, so "forming equilibrium expectations should be easy" (Dufwenberg & Dufwenberg, p. 262). 97 The literature refers to "extensive-form games,"which include a speci…cation of players'payo¤s at terminal nodes. We interpret such payo¤s as monetary one, hence we have what we call "game forms". they are: interchanging essentially simultaneous moves (INT), 98 coalescing sequential moves (COA; inverse: sequential agent splitting), and addition of a super ‡uous move (ADD). Any solution concept that just requires knowledge of the normal form, such as Nash equilibrium, or iterated admissibility, is obviously invariant to such transformations (and their inverses). Yet, such invariance cannot be a dogma. Note that the most used solution concept, sequential equilibrium, is only invariant to INT. Versions of rationalizability for extensive-form games are invariant to INT and COA, but not ADD. 99 Our general view is that it is interesting to study the invariance properties of independently motivated solution concepts, without viewing violations of some form of invariance as a ‡aw. We have just one caveat: If simultaneity of moves is represented indirectly, by letting players move in an arbitrary sequence and de…ning information sets so that the choices of early movers are not observed, then invariance w.r.t. INT is a must. 100 How does this relate to the analysis of p-games? We have a two-tiered answer: (i) In general p-games, players'utility depends on the temporal sequence of beliefs. This means that an accurate representation must explicitly account for time and distinguish between periods and stages within periods, as-say-in models of bargaining protocols. Transformations involving moves across periods should be expected to have an impact on behavior. For example, as shown by Gneezy & Imas (2014), angry agents "cool o¤" as time goes by and are therefore less willing to harm others (see the discussion of fast vs. slow play in the working paper version of Battigalli, Dufwenberg & Smith). Consider a game form with time in which player i in period t (when he may be frustrated) decides to stop (S) or continue (C). If C is chosen, the play moves to period t + 1 when i can harm j (H) or not (N ). Since i in period t + 1 may have cooled o¤, a COA transformation making i choose either S, or C&H, or C&N in period t, makes it more likely that i harms j. Thus, it is important to use rich and accurate representations of the rules of the game (see BC&D for a …rst step in this direction). The rules of games played in the …eld and in the lab, by necessity, imply that information accrues to both active and inactive players. Since information a¤ects beliefs, 98 Note that the formalism initially adopted to describe games indirectly represented simultaneous moves in an arbitrary sequential order without ‡ows of information to late movers. 99 See the discussion and characterization by Battigalli, Leonetti & Maccheroni (2020). 100 See the characterization of INT-invariance by Bonanno (1992).
we should formally represent the information of inactive players. Also, the rules of the game only specify information ‡ows, not how much information is retained by players. To keep a sharp separation between game form and players'personal features, the former should specify information ‡ows as, for example, in Myerson (1986), or in the literature on repeated games with imperfect monitoring (e.g., Mailath & Samuelson 2006). The use of traditional information partitions is acceptable under the presumption that players have perfect recall (and this is common knowledge).
(ii) Compared to traditional game theory, invariance with respect to a particular transformation depends also on the form of psychological utility functions, not only on the solution concept. For example, if utility depends only on the initial, or the terminal beliefs of others, sequential equilibrium is invariant w.r.t. INT (which, however, may not be applicable if simultaneity is represented directly as we do), whereas extensive-form rationalizability is invariant to both INT and COA. Yet, utility may depend on beliefs in ways that prevent any form of invariance.
Private sensitivities and incomplete information In preceding sections i often denoted player i's "sensitivity" with respect to some psychological concerns (e.g. guilt or reciprocity). It makes sense to assume that player j is not informed of i , and has to form beliefs about it in order to predict i's behavior. In this case the analysts must consider beliefs in p-games with incomplete information. We already discussed this topic, in Section 4, where, however, we focused on image concerns, i.e. how i might care about j's beliefs about i . However, even absent image concerns, incomplete information is very natural and important. If i measures i's sensitivity to feel, e.g., guilt, reciprocity, or anger, then j's beliefs about i may matter not because i cares about those beliefs but because they will impact j's choices. 101 Most applied work using PGT otherwise either assumes complete information, or (legitimately) ignores the issue by focusing on the shape of the best reply correspondences, without using solution concepts. The analysis of psychological games with incomplete information is developed in Attanasi, Battigalli & Manzoni (2016), Bjorndahl, Halpern & Pass (2015), and BC&D, which is more general and systematic. Incomplete information is the lack of common knowledge of some features of the game form (feasible sets, information structure, material payo¤ functions) or of players' preferences. As explained in BC&D, if we rely on solution concepts with an epistemic foundation (versions of rationalizability) or a learning foundation (versions of self-con…rming equilibrium), incompleteness of information can be addressed directly by means of relatively straightforward extensions of such solution concepts. 102 The situation is di¤erent if the analyst maintains the traditional equilibrium assumption that players have correct conjectures about the decision rules associating co-players' private information with their behavior. In this case, following the seminal contribution by Harsanyi (1967-68), the traditional approach proceeds in two steps. It …rst posits an implicit representation of players'possible interactive beliefs about the unknown parameter vector by means of "types" encoding both private knowledge about and hierarchical exogenous beliefs; this yields a "Bayesian game". Then one proceeds to analyze the Bayesian perfect (or sequential) equilibria of such a game, that is, pro…les of decision rules whereby every type of every player carries out a sequential best reply to the co-players'decision rules given the beliefs of this type. The key to connecting Harsanyi's approach to PGT is that, in a Bayesian equilibrium, each type of each player is associated to a hierarchy of (exogenous and endogenous) beliefs, which-in a p-game-enters the utility functions. This allows us to obtain all the known "traditional" equilibrium concepts (those that postulate correct beliefs, as in GP&S and B&D) as special cases of Bayesian perfect equilibrium. Harsanyi's approach gives an additional degree of ‡exibility seldom noticed by non-specialists: a Bayesian game may feature multiple types of the same player with the same exogenous hierarchy of beliefs, even when there is complete information! This allows for equilibria where types with the same exogenous belief hierarchy have di¤erent plans and di¤erent beliefs about co-players. Since observed actions signal types, in equilibrium players may update their higher-order beliefs about the beliefs (and plans) of co-players, hence their intentions, which is instead prevented by the equilibrium concepts of GP&S and B&D. 103 Bounded rationality All the work on PGT that we know of assumes that players are rational. Clearly, however, the idea of bounded rationality makes as much sense in the context of p-games as in standard games. We propose that exploring related topics might be interesting, although we mainly leave exploring it for future e¤orts. We see at least one interesting potential interaction between bounded rationality and PGT: While the latter models emotions as part of players'utility, it is known that some emotions such as anxiety (Rauh & Seccia 2006) and anger (Gneezy & Imas 2014) can hamper rational cognition, and this is factored in by early movers who can trigger such emotions. PGT in its current form is not equipped to model such e¤ects.

Concluding remarks
Decisions are driven by a plethora of desires. Yet economists' approaches traditionally took a narrow view, focusing mainly on concern for own income (or consumption). When richer models were proposed, it was often taken as an advantage if the deviations from the tradition were limited. For example, much of the literature on "social preferences"considers it a success if data sets can be explained using utilities de…ned on distributions of material payo¤s according to simple formulas. 104 Being spare is not necessarily a virtue. If human psychology is rich and multi-faceted, one cannot know the e¤ect of the involved sentiments unless one dives in and explores how and why that plays out in economic contexts. Many interesting desires that shape behavior in important ways take the form of belief-dependent motivation. This includes reciprocity, emotions, image concerns, belief-dependent loss aversion, and self-esteem. We have argued that the mathematical framework of psychological game theory (PGT) is useful and needed for modeling such sentiments, and we have tried to show why & how.
Working with PGT is exciting, and interest is growing. For example, the Journal of Economic Behavior and Organization recently published a special issue on PGT, comprising 15 articles (cited above); see Dufwenberg & Patel (2019) for the guest editors' introduction. We derive utility from our hope (=item #12 in Elster's list from Section 3.6) to inspire others to follow suit.