I’m reviving this thread based on comments from here. This revival will be focused on the ethics of the approach rather than the technical aspects. For those uninterested in this aspect of the discussion, please feel free to unwatch/unsubscribe (and also feel free to DM me if you don’t know how to do that).
I don’t think it comes down to different readings. Rather, I think the author is responding to situations (like conferences) where a survey would be just as easy, yield better/more accurate data, and not have all of the problems this approach has. I agree that the author would likely extend the argument to other use-cases, but doesn’t really address them, and as a consequence, I find it less persuasive.
For example, two of the three primary objections don’t really seem to apply for the use-case I’m primarily interested in, or at least the case wasn’t made. Reason (1) that it’s “inaccurate” isn’t applicable because (a) no measument in science is perfect, and one can assess and account for error (b) the author erroneously talks about the databases as coming only from the top 100 names, but most that I’ve seen come from things like socal security records than include all names (I think as long as they show up at least 2 times in a given year) (c) I can show how it compares (favorably) to other more laborious methods such as manually looking at social media profiles and looking at self-stated gender (d) the biases w/r/t things like Asian names being under represented or impossible to infer from especially after romanization, are real and should be acknowledged, but even if we expect gender trends in science publishing to be radically different between those that are represented in the database and those that aren’t, pointing out disparities only in the communities that are represented still seems worthwhile.
The third objection that it’s usually unnecessary only addresses the conference situation, and no solution for my use case is offered. Yes, surveys of academics are possible, but far more costly and time consuming, and suffer from their own problems of bias. I’m any case, I would be unable to do this due to time and budget constraints. So one could argue that the work isn’t worth doing, given the other objections. Or one could argue that there’s a better way to do it, given time and budget constraints, but these arguments weren’t made.
The final objection, that it’s morally horrifying, I just find unconvincing. Quoting from the piece here:
The voids in these datasets don’t cover everyone evenly. Rather, they often fall straight down lines of race, culture and ethnicity.
Totally agree.
Accordingly, names that are largely unique to non-white groups are far more likely to be excluded from a top-N dataset than common names used by white people, for the simple reason that there are fewer non-white people .
This has a glimmer of truth but not for the reasons started. As I said above, most of the software I’ve seen (and this package) use datasets that are far more inclusive. There are lots of non-white babies born in the US, and so lots of non-white makes included in the datasets. Add to that, there are plenty of non-white babies named David or Sara.
That said, there are clearly biases. I’ve already mentioned a general problem of Asian names, especially when given romanized spelling, but there’s also the problem of non-ASCII characters being excluded or paved over in a way that obscures/changes the implied gender, plus the fact that the entire continents if Africa and South America tend not to publish such data sets.
All stipulated.
Congratulations: your methodology is racially biased.
So, this is true, but that’s different than saying it’s racist.
The result is, invariably, that you end up with a model that underrepresents people of colour, be they from European/North American contexts or elsewhere. Both are vital, non-excludable populations to consider in even the most half-hearted inclusion initiative.
I agree with all of this, but all models are wrong. Some are useful. This author seems to be arguing that, unless you can get a census-like count, there’s no value in assessing the gender make up of anything, ever. From earlier in the post:
there’s considerable variation in the data: ambiguous names that you can at best probabilistically tie to a binary gender. “Sam” could be of any gender or none.
Again, stipulated. This point is written as if it’s some kind of scandal. In some situations, a probabilistic model can provide valuable, if imperfect insight. At least, I think so. I’m open to being persuaded.
The issue of erasure of trans/non-binary people is the part I’m most conflicted about, though I find this person’s arguments really unconvincing, eg
Frankly, claiming that birth name maps immutably to gender in the first place is the kind of essentialist TERFy nonsense that has no place in inclusion efforts.
This feels like fighting a straw man - I would certainly never claim such a thing. I won’t go through them all, but many other points strike me to way - as if my algorithm would say that Sam is 80% likely to be male, so whenever I met someone named Sam I’ll treat them like a stereotype of a man and refuse any other information.
There’s no argument that addresses what to do if I have a dataset of 100 Sams, 100 Sallys, 100 Roberts and 100 Yishans. Should I say that, because the Sams and the Yishans are ambiguous, and that some of the Roberts and Sallys might be trans or non-binary people that I have absolutely no information?
But even getting past all of that, it could be possible that simply trying to study the inclusion of women, without addressing trans people, is too exclusionary. Does this mean that we can’t talk about the gender pay gap without taking about figuring out whether there’s a trans pay gap? Can we talk about how women take on more childcare responsibilities in general, and how this is being exacerbated by COVID with children being out of school, if we can’t also assess the impacts of childcare responsibilities on non-binary people?
I 100% believe that trans and non-binary people have very different experiences, are generally more marginalized, and deserve to be acknowledged and taken seriously. And the same for POC. If I were on a committee that oversees grant proposals, I would absolutely look to fund efforts trying to assess their contributions in science publishing, which could obviously not be done with this method. But none of that send to indicate that trying to address the role of women (even if it mostly only applies to cis women) is bad.
I don’t mind at all 
. You’re right - this was not the best way to refer to this person. In the heat of the moment, I was responding to someone that I felt was accusing me if the equivalent of “I have a friend who’s black, so I can’t be racist.” I don’t think that having a non-binary friend means that I can’t have bias, but I sought out that person’s opinion because they are non-binary and also spend their life and career in trans-activism. My response was intemperate, to be sure, but I don’t think the author of that piece necessarily had more credibility than my friend.
This is an excellent paper! I think I will need to re-read a couple of times (currently in vacation, so didn’t do a deep dive), but I think that there’s a lot of stuff that’s relevant to my research and to this package.
I will note though that these authors took a similar approach for one of their papers, and are not arguing against the practice categorically. They offer a number of suggestions to make such research more aware of and cognizant of these biases to be sure - and I think this is super valuable.