Responsible reporting and use of data is one of my favorite topics (and the lack of it is one of my pet peeves). As I was writing about another topic and researching accordingly, I had reason to go look up the (externally available) demographics of American Mensa. The page is very interesting, and I wanted to share some of my observations about the age section, its data, and the implications of the way it is reported.
First, for the page overall, there’s no source or date for the data. At the bottom, a source of March 2011 data is cited for the list of the largest local groups, and the “American Mensan facts” section cites a survey of 53K members. I have no idea how recently any of this was updated, nor does anyone who looks at this page. That’s a problem, and I’ll explain why it’s a problem as we go on. For now, just consider that basic data reporting guidelines dictate that the date and source of the data should be provided.
Now, look at the section titled, “Mensa members by age” (quoted with line numbers added for easy reference later on):
Mensa members by age
(1) The youngest Mensan is 2 years old; the oldest is 102 years old.
(2) Approximately 38 percent are Baby Boomers between the ages of 51 and 68.
(3) Thirty-one percent are Gen-Xers between the ages of 27 and 48.
(4) More than 2,600 members are under the age of 18.
Line (1): While near meaningless in terms of data, this sort of trivia is what reporters like to have when writing interest pieces about Mensa, so I’m fine with it. I’ll also use it a bit later on, so just keep it in mind for now.
Lines (2)-(4): Most of the rest of this will talk about these lines. The membership is segmented by age, with percent make-up given for each segment. The first segment is “Baby Boomers” ages 51-68, spanning 17 years. The second mentioned is “Gen-Xers” 27-48, spanning 21 years. The third is “under the age of 18”, spanning 17 years if you use a strict reading where that is not inclusive of 18-year-olds. Let’s first try to use this data to figure out the freshness of the data they’re using.
Backing into data freshness using generations
Gen X is most often cited as being defined by a birthdate in the range 1961-1981. That’s the date range given by both Wikipedia and Mensa’s own Gen-X SIG. That’s a 20 year span, but the Mensa representation given is 21 years (ages 27-48). Let’s suppose, though, that in giving ages they fudged a year because birthdate could come into play based on the date the data was created. If a person was born in 1981, they’d be 33 or 34 now in 2015 depending on exact birthday. So, the data isn’t from now. Instead, let’s take the youngest age given and add it to the youngest birth year (27 + 1981), which yields 2008 as the likely date of the data. Wow. A bit out of date, don’t you think?
However, I ran the same comparison for Baby Boomers, who are defined by Wikipedia as being born between 1946 and 1964. (The Baby Boomer generation is defined much more strictly than Gen X or Gen Y, for what it’s worth.) Mensa is only showing 17 years, and the generation spans 18. (Note that here they’ve shaved off a year, whereas in the other data, they added a year. Tuck that away for later consideration.) If we use the same method here, let’s figure the data is from 2008 and subtract the youngest age in the segment (51): 1957. Well, that’s in the range, but it’s far from the youngest that should be included. In fact, if I had started with the Baby Boomer age ranges, the ages given would suggest that the data was from 2014-2015. So, either Mensa has used a very different date range to calculate or the data shown here is from two different data sets.
Or, more likely, the ascription to generational titles (GenX and Baby Boomer) is arbitrary and in no way aligns to the actual generations. For now, note the inconsistency we’re seeing, because it’s a piece to the puzzle I’ll be working through as we go.
Apples to oranges?
At this point in history, it’s reasonable to assume that if you’re reading this, you’ve probably taken a survey or two at some point in your life. That survey has probably requested demographic data like your age (or year of birth which they then used to calculate your age). Have you ever seen a survey ask for your data in the specific segments used on this page? Let’s imagine that for a moment:
Doesn’t that seem like a weirdly tiny gap? And don’t those ranges overall seem arbitrary? You’re probably far more used to seeing something that breaks the ages into around 9-10 year segments, or into more general segments that roughly correspond to “child”, “college or grad student or recent graduate”, “young adult”, “middle age”, “retired or close”, etc.
So, I think it’s reasonable to assume that they didn’t use these segments when gathering the data. (As a matter of fact, I know they didn’t, because I know, as a member, that they know the ages of every member, but I wanted to approach this as if I didn’t have that information, since it’s not presented as part of the page. In fact, let me just take this opportunity to note that I’m fully aware that the real data is available to membership officers and leaders in Mensa. It’s the decision of how to present it on this externally visible page that I’m discussing, not whether the data exists.) Thus, they chose these segments specifically.
Let’s consider some reasons they might have chosen these bands and why they may have excluded ages 49-50 from the group. As I already demonstrated when trying to figure out the age of the data based on the generational segments, it certainly wasn’t out of alignment to the generations as they’re commonly defined. Maybe these were the most prominent groups? Perhaps membership clusters in these regions? But, even if that were the case, it seems unlikely that you’d throw out age 49-50, especially since that’s not likely to be a zero (far from it).
Let’s also consider that they chose to line these up as if it’s a fair comparison of three groups, when, in fact, they’re not accurate to compare at all. For the “under 18” group, by providing a number rather than the percentile, it implies that the group is of a significant size. The total membership count is on a separate page, so you don’t have a ready way to see that the count given implies less than 5% of the membership total.
Apples to apples comparison
So…being an overachiever, I decided to infer out and add data to make this a more fair comparison of age segments in American Mensa.
Mensa Demographics (Apples to Apples Comparison)
|Number of Members||Percent of Membership||Age Range (inclusive)||Years Included in Range||Average Members Per Year of Age Segment||Ratio vs. Mensa Age Deciles (values >1 indicate over-representation)||Ratio vs. US Population (using % of pop. in age range, values >1 indicate over-representation)|
Before you leave this section, go back up and look at how the Mensa demographics page presented this data and compare against this table. Does it seem like they presented a fair accounting of the composition of American Mensa?
What does this tell us?
To paraphrase Joan Calamezzo, it’s time to speculate wildly!
What we can see here is that the “baby boomer” generation is over-represented in Mensa, both relative to the US population and relative to the average population you’d expect for a segment made of up that many years of age. However, Mensa went to a lot of trouble to hide that or make that harder to figure out. For example, they added a year to Gen-X and they took away a year from Baby Boomer (I told you to remember that for later). Consider that they skipped two years that, most likely, should have been included in Baby Boomers (ages 49-50). Were I a betting woman, I’d put my bet on that including ages 49-50 meant that the percentages would have gapped much farther, probably pushing the upper segment above 40% and thus making it stand out as being over-represented even in the minimal data presented, whereas the ages they used makes the original text look far more balanced. Even as is, once you normalize vs. the US population or average Mensa age deciles, the older “baby boomer” band is significantly more prominent than the “Gen X” band. But…they didn’t normalize it. They tried to cast this as parallel data, when in fact, it is not.
This is a sales pitch, so, of course they’re going to present the data in a way that sends a message. I’m not faulting them for that, per se. This is an exercise in examining data critically. This is also an exercise in reading between the lines when data is presented.
American Mensa, or, at least, the author(s) of this page, clearly wanted to give the message that the membership of Mensa was much more balanced in age than it actually is. That’s interesting, not because it’s necessarily unusual. A lot of commmunity/social organizations skew to older/retired folks, unless they’re explicitly defined by age or life stage (e.g., Girl Scouts). It’s just a matter of nature for people to be more likely to seek out that kind of organization as they are out of school, have moved to different regions or have reached a point where work-based socialization is less feasible…and all of those things become more common with age.
Consider who is likely to consult this page:
- Press contacts writing a story about American Mensa
- Prospective members wanting to understand the group before they test and/or join
In both these cases, why would American Mensa want to be deceptive about their composition? In the former (members of the press), perhaps it’s to seem more relevant or representational. American Mensa wants to be perceived as an organization where the only unifying factor is intelligence, when, in reality, it’s more likely to represent a specific subgroup of the population of intelligent Americans (specifically older, white males, but that’s really a digression…if people are interested, I’ll do a similar deep dive on the other data presented here to spell that point out, but if you’d rather not see how the sausagefest is made, just trust me that the data on the Demographics page backs it up). In the latter (and to some extent the former), it’s probably because they think people are more likely to join if the group seems younger or more balanced.
However, I’d argue that actually does American Mensa a disservice and probably increases member churn. If I’m considering membership and one of the things I care about is whether my generation (or any other demographic quality) is represented, it is not going to take me long to figure out that this page was deceptive. At best, maybe I come away with a perception that it’s just my local group that is out of whack vs. the whole org, but more likely, I come away disappointed and possibly upset that I wasted money on a group that wasn’t what I wanted.
In short, selling a product by lying about it is never a good idea.
What should be done?
It’s popular these days for companies and orgs to release their demographic data and own up to their disparities. American Mensa should follow that trend. American Mensa should use comparative data expressed in the same units of measurement and show more common segments or complete segments. If the concern about the org being “boomer-heavy” is real, it’d be reasonable to link to programs for younger members, to illustrate that the org offers value despite under-representation. It’d be even better to have a statement written by the current chair that is linked, iterating current actions and plans in place to address disparities. Consider how Google is presenting its workforce demographic data as a model to follow. Wouldn’t it be great to see that level of transparency and ownership coming from American Mensa?