{"id":1354,"date":"2017-02-23T08:25:20","date_gmt":"2017-02-23T13:25:20","guid":{"rendered":"http:\/\/datacolada.org\/?p=1354"},"modified":"2022-07-28T00:49:19","modified_gmt":"2022-07-28T04:49:19","slug":"57-interactions-in-logit-regressions-why-positive-may-mean-negative","status":"publish","type":"post","link":"https:\/\/datacolada.org\/57","title":{"rendered":"[57] Interactions in Logit Regressions: Why Positive May Mean Negative"},"content":{"rendered":"

Of all economics papers published this century, the 10th<\/sup> most cited appeared in Economics Letters <\/em>, a journal with an impact factor of 0.5. \u00a0It makes an inconvenient and counterintuitive point: the sign of the estimate (b\u0302) of an interaction in a logit\/probit regression, need not correspond to the sign of its effect<\/em> on the dependent variable (Ai & Norton 2003, .pdf<\/a>; 1467 cites).<\/p>\n

That is to say, if you run a logit regression like y=logit(b1<\/sub>x1<\/sub>+b2<\/sub>x2<\/sub>+b3<\/sub>x1<\/sub>x2<\/sub><\/span>), and get b\u03023<\/sub>=\u00a0.5<\/span>, a positive<\/span> interaction estimate, it is possible (and quite likely) that for many xs, the impact of the interaction on the dependent variable is negative<\/span>; that is, that as x1<\/sub> gets larger, the impact of x2<\/sub> on y gets smaller <\/em>[1<\/a>].<\/p>\n

This post provides an intuition for that reversal, and discusses when it actually matters.<\/p>\n

side note: Many economists run “linear probability models” (OLS) instead of logits, to avoid this problem. But that does not fix<\/strong><\/span> this problem, it just hides<\/strong><\/span> it. I may write about that in a future post<\/span>.<\/span><\/p>\n

Buying a house (no math)<\/strong>
\nLet\u2019s say your decision to buy a house depends on two independent factors: (i) how much you like it (ii) how good an investment it is.<\/p>\n

Unbounded scale. <\/em>If the house decision were on an unbounded scale, say how much to pay for it, liking and investment value would remain independent. If you like the house enough to pay $200k, and in addition it would give you $50k in profits, you\u2019d pay $250k; if the profits were $80k instead of $50k, then pay $280k. Two main effects, no interaction [2<\/a>].<\/p>\n

Bounded scale. <\/em>Now consider, instead of $ paid, measuring how probable it is that you buy the house; a bounded dependent variable (0-1).\u00a0 Imagine you love<\/em> the house (Point C<\/span> in figure below). Given that enthusiasm, a small increase or drop in how good an investment it is, doesn\u2019t affect the probability much. If you felt lukewarm, in contrast (Point B<\/span>), a moderate increase in the investment quality could make a difference. And in Point A<\/span>, moderate changes again don\u2019t matter much.<\/p>\n

<\/a><\/p>\n

Key intuition:<\/u> when the dependent variable is bounded [3<\/a>]<\/span>, the impact of every independent variable moves it closer\/further from that bound, and hence, impacts how flat the curve is, how sensitive the dependent variable it is to changes in any <\/em>other variable. Every<\/em> variable, then, has an interactive effect on all<\/em> variables, even if they are not meaningfully related to one another and even if interaction effects are not included in the regression equation.<\/span><\/p>\n

Mechanical vs conceptual interactions<\/strong>
\nI call interactions that arise from the non-linearity of the model, mechanical interactions<\/span>, and those that arise from variables actually influencing each other, conceptual interactions<\/u>.<\/p>\n

In life, most conceptual interactions are zero: how much you like the color of the kitchen in a house does not affect how much you care about roomy closets, the natural light in the living room, or the age of the AC system. But, in logit regressions, EVERY mechanical interaction is \u22600; if you love the kitchen enough that you really want to buy the house, you are far to the right in the figure above and so all other attributes now matter less: closets, AC system and natural light all now have less detectable effects on your decision.<\/p>\n

In a logit regression, the b\u0302s one estimates, only capture conceptual interactions. When one computes \u201cmarginal effects\u201d, when one goes beyond the b\u0302 to ask how much the dependent variable changes as we change a predictor, one adds<\/u><\/em> the mechanical interaction effect.<\/p>\n

Ai and Norton\u2019s point, then, is that the coefficient may be positive, b\u0302>0, conceptual interaction positive, but the marginal effect negative, conceptual+mechanical negative.<\/p>\n

Let\u2019s take this to logit land<\/strong>
\nLet
\ny: probability of buying the house
\nx1<\/sub>: how much you like it
\nx2<\/sub>: how good an investment it is<\/p>\n

and,
\ny= logit(b1<\/sub>x1<\/sub>+b2<\/sub>x2<\/sub>)\u00a0 [4<\/a>]\n(note: there is no interaction in the true<\/strong> model, no x1<\/sub>x2<\/sub> term)<\/p>\n

Below I plot that true model, y on x2<\/sub>, keeping x1<\/sub> constant at x1<\/sub>=0 (R Code<\/a> for all plots in post).<\/p>\n

<\/a>
\nWe are interested in the interaction of x1<\/sub> with x2<\/sub>. On how x2<\/sub> affects the impact of x1<\/sub> on y. Let\u2019s add a new line to the figure, keeping x1<\/sub> fixed at x1<\/sub>=1 instead of x1<\/sub>=0.<\/p>\n

<\/a>
\nFor any given investment value, say x2<\/sub>=0, you are more likely to buy the house if you like it more (dashed red vs solid black line). The vertical distance between lines is the impact of x1<\/sub>=1 vs x1<\/sub>=0; one can already see that around the extremes the gap is smaller, so the effect of x1<\/sub> gets smaller when x2<\/sub> is very big or very small.<\/p>\n

Below I add arrows that quantify the vertical gaps at specific x2<\/sub> values. For example, when x2<\/sub>=-2, going from x1<\/sub>=0 to x1<\/sub>=1 increases the probability of purchase by 15%, and by 23% when x2<\/sub>=-1 [5<\/a>]\n

<\/a><\/p>\n

The difference across arrows captures how the impact of x1<\/sub> changes as we change x2<\/sub>; the interaction. The bottom chart, under the brackets shows the results. \u00a0Recall there is no conceptual interaction here, model is y=x1<\/sub>+x2<\/sub>, so those interactions, +.08 and -.08 respectively, are purely mechanical.<\/p>\n

<\/a><\/p>\n

Now: the sign reversal
\n<\/strong>So far we assumed x1<\/sub> and x2<\/sub> were not conceptually related. The figure below shows what happens when they are: y=logit(x1<\/sub>+x2<\/sub>+0.25x1<\/sub>x2<\/sub>). Despite the conceptual interaction being b=.25 > 0, the total effect of the interaction is negative for high values of x2 <\/sub>(e.g., from x2<\/sub>=1 to x2<\/sub>=2, it is -.08); the mechanical interaction dominates.<\/p>\n

<\/a>
\nWhat to do about this?<\/strong>
\nAi & Norton propose not focusing on point estimates at all, not focusing on\u00a0b\u03023<\/sub>=.25. To instead compute how much the dependent variable changes with a change of the underlying variables, the marginal effect of the interaction, the one that combines conceptual and mechanical. To do that for every data-point, and reporting the average.<\/p>\n

In another Econ Letters <\/em>paper, Greene (2010; .pdf<\/a>) [6<\/a>] argues averaging the interaction is kind of meaningless. He has a point, ask yourself how informative it is to tell a reader that the average interaction effect depicted above, +.11 and -.08, is +.015. He suggests plotting<\/em> the marginal effect for every value instead.<\/p>\n

But, such graphs will combine conceptual and mechanical interactions. Do we actually want to do that? It depends on whether we have a basic-research or applied-research question.<\/p>\n

What is the research question?<\/strong>
\nImagine a researcher examining the benefits of text-messaging parents of students who miss a homework and that the researcher is interested on whether messages are less beneficial for high GPA student (so on the interaction: message*GPA).<\/p>\n

An applied research question may be:<\/p>\n

How likely is a student to get an A i<\/span>n this class if we text message his parents when missing a homework?”<\/p><\/blockquote>\n

For that question, yes, we need to include the mechanical interaction to be accurate. If high GPA students were going to get an A anyway, then the text-message will not increase the probability for them. The ceiling effect is real and should be taken into account. So we need the marginal effect.<\/p>\n

A (slightly more) basic-research question may be:<\/p>\n

How likely is a student to get more academically involved<\/span> in this class if we text message his parents when missing a homework?<\/p><\/blockquote>\n

Here grades are just a proxy, a proxy for involvement; if high GPA students were getting an A anyway, but thanks to the text-message will become more involved, we want to know that. We do not<\/span><\/em><\/strong> want the marginal effect on grades, we want the conceptual interaction, we want b\u0302.<\/p>\n

In sum: When asking conceptual or basic-research questions, if b\u0302 and the marginal effects disagree, go with\u00a0b\u0302.
\n<\/span>
\n\"Wide<\/p>\n

\n

Authors feedback.<\/strong>
\nOur policy is to contact authors whose work we discuss, asking to suggest changes and reply within our blog if they wish. I shared a draft with Chunrong Ai & Edward Norton. Edward replied indicating he appreciated the post and suggested I tell readers about another article of his, further delving into this issue (.pdf<\/a><\/strong>)<\/span><\/p>\n

Subscribe to Blog via Email<\/h2>\n\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t\t

Enter your email address to subscribe to this blog and receive notifications of new posts by email.<\/p>\n<\/div>\n\t\t\t\t\t\t\t\t\t\t

\n\t\t\t\t\t\t

\n\t\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t