Humans are biased. Generative AI is even worse | Hacker News Reader

Humans are biased. Generative AI is even worse

bloomberg.com

73 points1991_years_and90 comments

If you ask Stable Diffusion for a picture of a cat it always seems to produce images of healthy looking domestic cats. For the prompt "cat" to be unbiased Stable Diffusion would need to occasionally generate images of dead white tigers since this would also fit under the label of "cat".

I'm being absurd of course, but my point is even if you assume the results are biased we also need to consider that generally people want the biased result.

For example in the UK our media vastly over represents black individuals. But we do this purposefully because companies want to be inclusive so even if it's not representative of reality we still want to include a variety of races.

So is it even bad that Stable Diffusion seems to have a bias towards images of healthy cats rather than sick or dead cats?

The answer here if anything is to reduce undesirable human bias and allow RLHF to do its job.

Exactly this. Generative AI shows that most people doing technical work are men? It probably also shows that most construction workers are men, most social workers are women, etc.. Guess what, that reflects reality.

If you want something else? You can ask for it. "Picture of a woman welding." "Picture of a black, male social worker." You'll get it, no problem.

Your comment about UK media is spot on: DEI fans want to bend reality to match their ideals. It's annoying and arguably counterproductive.

The article compared stable diffusion to job statistics. It was bad even if you controlled for that.

I think this might be representative of some of the pushback to DEI. People can feel like minorities are being pushed on them even when they aren’t.

Does is always show the same breed?

Healthy vs sick/dead isn't biased.

The difference of course is that while we roughly mean to ask for a “normal instance,” (and only occasionally for a “representative” instance strictly speaking), there are some circumstances where we disagree about what “normal” is or (crucially) should be. When we ask each other questions, we expect not only a factually but a normatively correct answer as well. And norms are inherently open to dispute as part of the normal course of conversation. But not so with generative ai. Yes, you can correct the ai during your session. But when I disagree with you in conversation I hope that my disagreement (if we resolve the matter) will outlive our interaction and you will no longer say the thing I objected to when talking to others, or if you do, you’ll do it in a different manner with a different tone.

I think the correct position on AI bias is that it is important to have biases, to the extent possible, known, disclosed, and vetted for appropriateness to the application domain (including mitigation in other parts of the system than the model in which the bias conpared to the desired state exists), not to eliminate them, which is impossible, becauase bias is always relative to a particular reference, and there is no one true authoritative reference for evaluating bias (which is why to document bias, you have to document what it is biased compared to.)

So to inject preferred biases instead of whatever is embedded in the data.

All biases are injected by humans, whether in curation of training data or at other points; training data isn’t a naturally occurring collection without human influence (not that it would really matter if it was.) As with any tool, the issue is fitness for purpose, and as with anything being supplied by a party with potentially different interests than the user/consumer, and important part of that is disclosure/transparency of relevant information that may not be apparent on the surface.

This is an obvious trap as biases are individual but you are proposing (or at least hinting at) injecting biases into a universally used model. Or should the ultimate goal be injecting hyper local biases and then selecting the right bias group based on the location of the prompter (or even their race/gender/etc)?

Try google "white couple" in google images. Over half of results are not "white couple".

Is it actually true that AI is biased? Surely, an AI cannot "be biased", it just looks at the input data and outputs it's own. This is not bias. If anything the regulation "hardcoded" into the models is the only bias which exists, and it is purposefully put there to combat things like what people in this article are upset about.

If someone finds AI output problematic, you should look at the input data. If the input data such as the world in general is causing "problematic" output data, it might be time to reconsider what you think of as problematic.

> If someone finds AI output problematic, you should look at the input data. If the input data such as the world in general is causing "problematic" output data, it might be time to reconsider what you think of as problematic.

Well, that's just it, aint it? Folks aren't saying the AI isn't aligned with itself. They're saying the input data isn't aligned with something. That seems pretty reasonable. Does anybody think scraping random internet data is going to spit out viewpoints that "the world in general" can all agree on?

Is that "problematic"? Frankly, yes, as in it's almost guarenteed to cause problems. That doesn't mean we need to censor or remove it, it means we should discuss what "problematic" means in these contexts.

> If the input data such as the world in general is causing "problematic" output data, it might be time to reconsider what you think of as problematic.

The data in the article is arguing against that kind of pat assumption. For example it says that: "Women made up a tiny fraction of the images generated for the keyword 'judge' — about 3% — when in reality 34% of US judges are women, according to the National Association of Women Judges and the Federal Judicial Center."

So part of the concern is that the input data is not a representative sample of "the world in general," but relies on stock photos, celebrity media images, sensationalist news reports, items which by design do not accurately reflect the real world.

> when in reality 34% of US judges are women, according to the National Association of Women Judges and the Federal Judicial Center

What percentages of judges are women is a different question than what percentage of judges depicted in visual media are women... and that is an even different question than what percentage of judge images used to train stable diffusion were women?

Yes, that's the point.

The dataset's proportions don't match reality's proportions, which leaves the AI with an inaccurate perception of reality.

This presents as a phenomeonon one might describe as "bias", even though the AI itself has no motivations.

> The dataset's proportions don't match reality's proportions, which leaves the AI with an inaccurate perception of reality.

People have this problem as well.

The bias is in the input data! That is the very problem. AI takes human bias and perpetuates it.

Look at the issue of predictive policing via AI: police are biased, therefore the data is biased, and thus the model is biased.

The data is data - bias enters the scene when silly humans assume that the data is necessarily representative, and then proceed to assert false claims based on their misunderstanding, which other humans store as "knowledge" (because distinguishing between knowledge and belief is "pedantic", "Sea-lioning", etc), and then inject new derivative false truths back into the system conversationally, through their actions, etc.

Meme world.

> If the input data such as the world in general is causing "problematic" output data, it might be time to reconsider what you think of as problematic.

It might be time to consider the possibility that some of what's out in the world in general is problematic. Ask "the world" their opinion on Jews or atheists or germ theory or quantum superposition and you'll get interesting answers AI shouldn't necessarily consider to be accurate.

Even trippier: ask "right thinking" people about the nature of "people"[1] who have opinions on Jews.

Humanity is so weird.

[1] Which they will construct using their imagination, and not realize...the very thing they are mocking others for doing (though, I am being reductive here).

Nah, I've had this "you can't ever know anything about someone else's beliefs" discussion with you before. It's no more compelling than last time. https://news.ycombinator.com/item?id=33791880

That your mind read missed this badly, and you cannot care, is a demonstration of the truth of my point...I predicted your behavior in advance, without even knowing you were going to reply.

All ML-based AI is biased. It has to be. Otherwise the data presented to the learning algorithm would be random noise and nothing could be learned.

Bias in learning data is essential to detecting patterns, by AI or by humans. ML's dependency on being fed biases (AKA as stereotyping) is an Achilles heel that affects every kind of learning, but it creates real problems when learning sophisticated patterns (like those learned by LLMs) where such biases can cause misbehavior and misunderstanding.

Unsurprisingly, fans of ML simply have avoided publicly addressing this limitation because it's essentially unsolvable by automation. The only solution is to manually acknowledge every exception to the rule -- the way children have to to learn every irregular verb. But until we do this, AI will remained fundamentally driven by its inbred biases and stereotypes.

“Anyone who disagrees with me is biased”

> Is it actually true that AI is biased? Surely, an AI cannot "be biased", it just looks at the input data and outputs it's own. This is not bias.

By that logic, nothing is biased. That racist cop? Just looking at her input data and outputting her own.

(I'm ignoring the OP article, which I haven't read.)

No, that doesn't follow at all.

Bias is structural ignorance of input data that contradicts with an entity's ideology that they want to protect.

A racist cop who came to the job with an internalized ideology that black people are bad and executes on that ideology despite experience and data to the contrary is biased. But a cop whose lived experience on the job is that (at least within their jurisdiction) black people are more likely to commit crimes and be violent, and therefore warrant a higher degree of wariness isn't biased, but operating rationally based on their input data. If that rational conclusion morphs into racist ideology that is inflexible to change given new input data, then it becomes bias. It's also possible they misinterpreted the input data and keyed in on race rather than poverty as the root cause, but that also isn't necessarily bias, just bad processing or incomplete input data.

The process of science is the attempt to follow the input data to the conclusion without letting theoretical ideologies bias us away from discovering unknown truths.

I disagree. I think you can be both rational and biased by operating with incomplete information.

Potentially less polarizing: Aristotle when developing his theories of violent vs natural motion. I would not call him unbiased for theorizing without looking outside.

You can't claim to be unbiased from just individual experiences, you need to attempt to understand the whole picture.

Major question:

What’s the difference between bias and knowledge? Science changes etc

If AI had been invented in Galileo’s time it would have been “aligned” to espouse the wonders of the Catholic Church, and would happily assert the virtues of creationism.

We have similar religion-like dogma today, it has just evolved.

https://en.wikipedia.org/wiki/Bias_(statistics)

That’s not the type of bias that gets people upset. For example companies want their hiring process to be statistically biased in favor of selecting competent candidates.

Actually it is the same bias. People care about bias when it's deemed unfair to people or animals, but not when bias affects inanimate objects, like buying shoes or choosing what to eat. But bias is at the center of every choice or opinion, nevertheless.

Paging Dr Pangloss, paging Dr Pangloss...

We have no problem personifying/anthropomorphizing A.I., touting it's "brilliancy", and "creativity" and even going at great lengths to create new definitions of those terms so they can be applied to AI...until it threatens AI, then all of a sudden it's "simply a function of it's inputs!"

Which one is it?

Just inject random 'diverse' keywords in the prompts with some probabilities to make journalists happy. For an online generator you could probably take some data from the user's profile to 'align' the outputs to their preferences.

Obviously generative models will be biased based on their training set. And if the world is "progress[ing] toward greater equality in representation" then they will not be representative of current year because of historical data.

I don’t think Stable Diffusion is made to produce statistically accurate depictions of groups. Asking for a judge 300 times won’t give you 34% female judge images, it will give you individual photos of the demographic most likely to be a judge, which is white male. I do agree this statistically results in exaggerated stereotypes though, I just don’t know how you would get generative images to produce while remembering what they’ve produced in the past and making it representative of the whole population of inputted images.

I don’t think you need to remember what you’ve produced in the past to create a representative sample. There’s no global counter of male/female babies making sure we have a balanced population either.

Wouldn’t something like isMale*P(male=.66) work fine?

I wonder where you would put that in the prompt?

I wonder how you would generalize this?

The issue is the command "draw a judge" shows a white guy 100% of the time.

if you automate prompt generation to be "draw a {gender} judge", with as you suggest the term gender has 66% chance of being "male", sure.

But how do you extend that automated prompt generation to include "draw a {skin color} judge" or "draw a {gender}, {skin color} judge".

or to occasionally generate a character from the Book of Judges.

Or Judge Dread.

I guess my curiosity is, what happens if you asked 300 individual artists(representative of the population) to produce a judge?

> Wouldn’t something like isMale*P(male=.66) work fine?

It doesn't think like that.

If it did, they could've just done `P(hasFiveFingersPerHand)=0.99999`.

But it doesn't even necessarily draw what you ask it to. Instead, it generally adopts a set of de-noising transforms that it's been trained to believe would tend to lead to what the prompt sounds like.. then whatever those transforms produce would, hopefully, be sorta like what was requested.

Custom loss functions absolutely work and work basically the way described above.

https://colab.research.google.com/drive/1dlgggNa5Mz8sEAGU0wF...

You can see them define a custom color loss and apply it simultaneously with the regular diffusion loss. I've actually expanded this notebook to allow regional specification of the custom loss.

It's quite difficult to define a function that detects if an individual has 5 fingers or not. That's the real issue.

The comment I'd responded to seemed to have thought that StableDiffusion picked what the sex of a person would be according to some internal odds that could be modified.

My point was that it doesn't actually think like that. For example, prompting StableDiffusion for a picture of a doctor doesn't necessarily get it to draw a human at all, much less a doctor of a pre-determined sex; instead, StableDiffusion de-noises the image until the result emerges, where that result would (ideally) contain a doctor of whatever sex it happened to come up with.

That said, you're right that we can add more code to try to guide things.

We could even just brute-force it by just re-generating images over-and-over, or tweaking them after generation, until they match exactly what we wanted. (Realistically, something like branch-and-bound would probably be preferred to blindly guess-and-check-ing.)

My point was more that you can add these guardrails without having to keep track of what the model had previously generated.

And I think if you used a perfectly balanced dataset for training, you’d get these guardrails for free because the right probabilities would be baked into the model’s weights.

Yeah, the idea to use random-selection instead of keeping track of generation-history seems reasonable. The idea of guardrails from perfect-balancing seems less obvious to me.

For example, say someone wants to generate a "US President" -- what would the ideal range of outputs be?

The article checked for just two things: sex (male or female) and skin-tone (I, II, III, IV, V, or VI). To date, all US Presidents have been male, and they were probably mostly skin-tones I or II (not bothering to check), except for Obama who was probably.. like IV or something (still not bothering to check).

So if we run StableDiffusion for a "US President", what would a "perfectly balanced" output look like? Should there be any women? What about the skin-tone distribution?

Also, Obama was a 2-term President, so.. if his skin-tone should somehow affect the distribution, should it have a stronger effect because he was in office for longer than average? Or should all US Presidents have the same effect regardless of their time in office? And either way, why?

I think the real problem is that human beings are biased and have a very strong negative reaction to things which are actually unbiased.

There are no such things as things which are “actually unbiased”, since bias is relative to a reference scope and there is no one objectively-favored reference scope.

There are things where the bias aligns with someone's intended use, but that will differ from evaluator to evaluator.

Is "the fine-grained truth is technically not known" biased?

Yeah this reminds me of the sentiment that a newspaper that manages to regularly upset both the political left and right is probably pretty unbiased, haha.

Bias toward political centrism is an ideological bias (which, despite sharing a name, is not the same thing as a representational bias of the type being discussed, though either can explain the other in some cases.)

I personally think research and improved methods to produce input data is where AI will improve the most in the near term. Mistral AI sent shockwaves in the community as their 7B model seemed to outperform Llama 2’s 13B model and I think there is still much to do here. Both in terms of bias but also discovering what is the optimal representation of input data to produce the desired results from the AI in an optimal model size. This pressure will only grow as the enterprise takes on AI and wants to fine tune their own models in order to not rely on the cloud as they need to fulfill privacy regulations and deal with sensitive documents. And in this regard, I think we have been underestimating just how much the input data influences the result.

Seeing some comments which align with but misunderstand what I suspect the point is.

Generative AI outputs the median of the probability distribution.

Yes, that's an issue with the training data (as people comment), but it is more than that. The training data will have some observations away from the median, but it will still only have one median.

To take my observation to the extreme (where it will fail), if the data is 49.999% X and 50.001% Y, then Y is the most likely output and you'll only every see Y from generative AI.

[yes, I already said that extreme was wrong because the model samples around the median not exactly the median]

This is flat out not true. Generative AI is meant to approximate the distribution. It’s biased from the data, yes, but if it was just the median it would only spit out one face, one gender, etc.

Being aware of this bias is important -- in the same way that being aware of bias in all media is important. I don't think that there's all that much blame to be assigned. If anything, it's an interesting lens into the biases that exist in the training material.

But the goal shouldn't necessarily be to twist the model into knots to avoid any hint of harmful stereotypes. The goal should be awareness for those using them, and development of tools to mitigate when needed.

Any recommendations on modern philosophy/literature that goes deeply into this stuff? "Bias" is of course a woefully under-defined and weaponized word.

Princeton philosophy professor Thomas Kelly wrote a book called "Bias: A Philosophical Study" in which his central thesis is based on the principle that "bias involves a systematic departure from a norm or standard of correctness." He's not talking (specifically) about political correctness, but about any standard by which bias can be evaluated, which might be accuracy, or legal justice, or morality, etc. Anyway, it covers a lot of ground but is fairly accessible. There isn't much jargon, and he doesn't expect the reader to have familiarity with the literature.

https://global.oup.com/academic/product/bias-9780192842954

That's something I was wondering myself. I like to read philosophy, but I don't know any references that cover the subject to a satisfying degree. Even if I go to Stanford Encyclopedia of Philosophy (a good starting point for your research with lots of pointers towards other articles and books) there is an article for Implicit Bias, but there is no article for Bias itself; this concept seems to be taken for granted and as nonproblematic.

The idea of bias is quite simple in the context of instrumental rationality: you have an objective and you optimize for it. For example, you may want to determine whether something is a face or not a face. In such case face pareidolia is a manifestation of bias.

But what is bias if we don't talk about instrumental rationality and goal-oriented behavior? What is bias in terms of so-called value rationality? There is a whole bag of those weird abstract concepts that exist as a foundation for fashionable contemporary mainstream philosophy and even beyond that as the Zeitgeist of our age. But those concepts seem to often just be asserted as an obvious starting point you have to agree with before something like (quite ironically called considering the context) Critical Theory would make sense to you.

https://www.google.com/search?q=semiotics+Stanford+Encyclope...

That will give you lots to keep you busy, though it is a bit deeper in the stack than most people in modern culture can tolerate.

Not sure how semiotics is relevant.

You used one of the trickiest, most misleading symbols out there: "is". This one word causes massive amounts of confusion about "reality" (yet another very tricky symbol), and yet "no one" "thinks" (yet another) twice about how they deploy it.

That's semiotics.

The true story here is "AI reflects the same bias as traditional media." Not that surprising as traditional media is the training set.

If you're writing an article about Stable Diffusion involving "ethics" or "bias" and you're not writing about the GIANT ELEPHANT in the room (waifus), you're doing a disservice to your readers.

Forget SD with a generic prompt generating something I don't like. What about the fact that much of the custom SD content is porn, or merges or porn models. This is to the point that "innocent" loras such as pastelmix dramatically increase its tendency for any female gen to be topless or similarly NSFW.

Despite Civit.ai being the best example of the male gaze on the internet, everyone just wants to talk about how SD assumes "rich africans" have big huts.

Bias, disposition, tendency, whatever terminology you would like to give, is baked into rationale and induction. You cannot have the latter without the former. This is even more true with LLM. Without bias, LLM is inaccurate and will become useless.

This is the true 500 pound gorilla in the room. Somehow, as we all become ever more dependent on AI, we also need to become ever more aware of the biases built into that AI. Without constant vigilance, we risk becoming obedient stereotypes ourselves, oblivious to hidden biases and agendas in the smart tools that we come to blindly trust.

I wish mathematicians or philosophers or biologists or neurologists or psychologists would outline a definition of what intelligence is, outside of machine machine learning or ai, so that we could learn something.

For example a gorilla was able to learn sign language. Domestic animals can recognize objects or words. Orcas are social and intelligent.

I'm a bit curious why computer scientists don't try to map or examine real neural structures.

So because things in the real world are getting better and stable diffusion trains on photographs taken before last year, comparing the gender and skin tone of CEOs and janitors creates a sample set 'more biased' than reality in the current year?

I mean... not surprising but that's a feelgood story right?

Yeah all this is solved in DallE 3. It automatically injects race into the prompt probabilistically.

That's why if you ask it to make comics it will randomly include a race https://x.com/neilkli/status/1709450248186167715?s=46&t=YWmQ...

Eventually we'll just have an LLM modify our prompts so that we create representative output.

Bloomberg is always either behind the times or making junk content. I like people who read it because they are proper suckers like all the HN users who were acting like the SMCI story was real because they like to cosplay as Mr Robot.

I took your money. And I'm going to take your money again when you read Bloomberg news on public companies because you guys worship the EMH and I know the alpha is in watching suckers sucker themselves. Give me your money, suckers. Give it to me. You want to.

Humans are human. What is with the obsession to make us into machinery.

Machines are just tools to enhance human function. A hammer enhances my ability to hit with my arm, a bicycle enhances my legs ability to move my body. AI is just a tool to enhance my brain. The obsession with using tools to enhance our limits is what propels humanity forward.

If it wasn't biased the output would just be white noise no matter what the prompt.

If anything it gives a quantifiable lens into what everyone perceives based on media and validated by their own experience

What is the goal of this observation? Patch it for “representation” to inspire toddlers?

From reading the article it looks like the first step they'd call for is to retrain the model to line up with reality? If 34% of Judges are female you'd expect 34% of the generated images to be perceived as female. Inspiration would be skewing it in the other direction

> If 34% of Judges are female you'd expect 34% of the generated images to be perceived as female.

That's not what i'd expect, no. I'd expect as it trains it would begin to trend towards a median idea of a single judge, an ur-judge if you will. It will represent the median judge. And then with fine-tuning training it would stop making ambiguous characteristics, which would lead it down the most likely path every time. You'd have to intentionally inject some randomness to prevent this. If anything i'm a little surprised they got as much diversity as they did - I can only assume there is some light randomness applied behind the scenes.

Note I'm not commenting on "should", but rather what would likely happen given an training dataset that reflects reality

Something that behaves like that would be a useless model. These models are not deterministic, llms for example have a literal temperature dial you can turn to increase randomness. If it narrows onto the idea of a single judge then what's the point of the AI? You could just grab a picture of some old white guy (your median judge) and use Photoshop and call it a day. Or just use stock photos if all the generated judges are the same.

If there is >50% of something, I wouldn't be surprised if a model chooses it 100% of time.

I know this is an old thread but also just remembered - what folks are describing in this thread (where it just picks the mode) is actually explicitly one of the reasons we use diffusion instead of GANs - it's called mode collapse and it's something people spend a lot of time and effort trying to prevent: https://proceedings.neurips.cc/paper/2017/file/44a2e0804995f...

I"m really confused by all these responses in this vein. If the model just chooses the most likely thing 100% of the time that's an entirely useless piece of software! Just use a stock photo or a bar chart and pick the mode!

Whether it is useless or not depends on your needs.

No because if it always produces a deterministic image it's just lookup table. The whole point is that it's generative ai, not a way to burn all the computational power to generate some summary statistics

The thing about learning in general is that it inherently introduces biases. I imagine you could train up a genocidal maniac LLM with the right data sets.

When are we going to get tired of journalists of middling intelligence speaking as if there's something profound about identifying bias? These stories just aren't useful. Every system has some form of bias, more or less, and a system that has less of a functional bias than another system isn't necessarily a better one. A system might be made to be relatively "unbiased", yet be useless to most people as a result. At worst, civilization continues to coerce humans into servitude of menial tasks rather than free up everyone's time if we decide that systems that are biased or aren't God-level perfect are considered harmful.

About the time when SWEs with a middling grasp of perspective of the consequences of their actions stop handwaving away all criticism of their work.

(Is there some kind of universal law that you're citing, that claims that it's impossible to build a useful, unbiased system?)

Edit: Oh, right, this goes right into a rant about how women choose to not engage. There are no additional barriers that we put in front of them, I'm sure...

> Edit: Oh, right, this goes right into a rant about how women choose to not engage. There are no additional barriers that we put in front of them, I'm sure...

I removed what I added there because I realized it was a distraction from the primary point I was making.

And I'm glad I did, because your response reminds me of how so few people take the time to understand what others are saying in explicit language. I'm not trying to be rude, but things like this are upsetting. Not once did I say that there are "no additional barriers", and I even said explicitly that there are.

If journalists are unhappy with the output of a private SWE labor, they can learn to code and generate a product to match their exact specifications.

If SWEs are unhappy with journalists writing about them, they can learn to write, start a newspaper, reach a few million readers, and publish their own investigative stories/op-eds that will match their exact specifications and agendas.

Ironically most journalists have a job BECAUSE of bias. Otherwise there would be just 1 news source kinda like Reuters, reporting completely unbiased factual events about the world and that's it.

What are the consequences of biases in Stable Diffusion?

SD? Not a lot, it's a toy.

'AI-driven' fraud detection/'predictive policing'/facial recognition/whatever other kind of Kafka-esque hell you may fall into? Quite a lot.

Sure, but none of those ML applications are generative AI.

> (Is there some kind of universal law that you're citing, that claims that it's impossible to build a useful, unbiased system?)

Usually it'd be a trade-off, where you'd give up some quality at doing the main job to gain whatever the secondary objective would be, e.g. producing results according to whatever desired distribution.

That said, the bigger issue would be defining what "unbiased" means. There're some mutually-exclusive notions of what it'd take to be unbiased, so if "unbiased" requires not being biased according to any perspective, then, yeah, it wouldn't generally be possible.

For a quick example, say that we want to be unbiased in our representation of men-vs.-women in a profession. Then, is the proper ratio: (1) 50% men and 50% women; (2) population-weighted ratios (because men and women aren't generally equally represented in the population); (3) profession-weighted ratios (because men and women aren't generally equally represented in a profession); (4) observer-subjective ratio (this is, a realistic ratio for how often a viewer might actually see men-vs.-women of that profession); (5) something else? What about intersexed people -- should they be represented, and if so, at what ratio, and how should men/women be adjusted to account for them?

Then, what if the photo would be served by having a person of a certain height, to best fill the space while not covering up more -- then, should the algorithm be biased toward a man or a woman based on typical heights, to accurately represent sex-specific height-distributions? Or, should it deviate from that, and present men and women as having the same height-distribution? And if we do deviate from that to pretend that men and women have the same height-distribution, but that height-distribution more closely matches the actual height-distribution for one sex than the other, then is that itself a form of bias?

Then, same as above, but for weight. Then for body size.

Then, what if men and women have different fashions in the relevant culture, and one fashion would better fit the scene -- can that be a factor? And if so, what should be the logic for picking the relevant culture?

Then, how should interactions be handled? For example, if we're generating pictures of doctors who're also mothers, then should men be included in that, or is it okay to only have female doctors in that context? Or what if we're generating pictures of doctors at a conference for female doctors in specific but has some male attendees -- then what ratio would be correct?

Point being that, for someone designing algorithms that would be "unbiased", they'd presumably want to know what folks would accept as a valid solution to that objective. Without a clear definition of what's desired, it'd seem like any particular solution would be open to criticism from other perspectives.

Reply on news.ycombinator.com