{"id":1145,"date":"2025-09-16T06:41:51","date_gmt":"2025-09-16T06:41:51","guid":{"rendered":"https:\/\/ambedkarsociety.org\/assa\/?p=1145"},"modified":"2025-10-06T06:51:07","modified_gmt":"2025-10-06T06:51:07","slug":"india-ai-is-learning-caste-bias-in-india-who-will-audit-it-for-discriminations","status":"publish","type":"post","link":"https:\/\/ambedkarsociety.org\/assa\/india-ai-is-learning-caste-bias-in-india-who-will-audit-it-for-discriminations\/","title":{"rendered":"INDIA: AI is learning caste bias in India. Who will audit it for discriminations?"},"content":{"rendered":"
\n
\"\"<\/figure><\/div>\n\n\n

A young recruiter in Bengaluru told me that the algorithm doesn\u2019t discriminate. \u201cIt just looks at data.\u201d I wanted to believe him. <\/p>\n\n\n\n

As a content and brand marketer who\u2019s embraced tech, I have seen the upside and the ugly. The upside is speed, scope, and new forms of storytelling. But the same tools can quietly reproduce the old biases that my people from marginalised castes already fight every day. In short, the machine promises neutrality, but the data it eats often isn\u2019t neutral. That\u2019s not abstract\u2014there\u2019s mounting evidence from India and global research that algorithms trained on skewed data can silence, misclassify, misrepresent, and financially exclude Dalits at scale. The more I see how these systems operate, the more I worry that AI may not free us from prejudice. It may fossilise it. <\/p>\n\n\n\n

We often assume technology is neutral, a clean break from the messiness of human bias. Yet, algorithms are only as fair as the data they are trained on. And in India, that data is soaked in centuries of caste hierarchy. CVs from upper-caste names dominate elite institutions. Dalit and Muslim-majority districts are tagged as \u201chigh risk\u201d because poverty is entrenched there. Content moderation systems, primed by majoritarian sentiment, routinely flag Ambedkarite assertion as \u201chate speech\u201d while casteist slurs (bhimta, chamar, chandal, ricebag, <\/em>etc) slip through unchecked. AI is faithfully learning the old prejudices. <\/p>\n\n\n\n

Discrimination hiding behind objectivity<\/strong><\/p>\n\n\n\n

Generative AI image\/video models trained on massive web datasets frequently reproduce racial and socioeconomic stereotypes. Prompts for \u201chigh-paying jobs\u201d produce lighter-skinned, \u2018upper-caste\u2019 looking images; prompts for low-status work produce darker-skinned figures or visual cues of poverty. Recent academic work specifically quantifies India-centric biases in image output and shows models equate \u201cIndianness\u201d with higher-caste visual markers while depicting Dalit identities with poverty markers. It matters for brand creatives, art directors and producers who increasingly use AI tools for mood boards, casting mockups, and campaign images.<\/p>\n\n\n\n

In practice, this means that a Dalit actor or protagonist often ends up depicted in stereotyped ways in AI mockups used to pitch a series\u2014 and brand creatives who rely on them amplify harmful frames without intent. <\/p>\n\n\n\n

Recommendation engines power who gets seen on YouTube, Instagram reels, Spotify playlists and even publisher homepages. If the training signals (likes, comments, reshares) favour mainstream and majoritarian content, Dalit storytelling (nuanced caste critique, oral histories, counter-narratives) struggles to break the \u201cengagement\u201d loop that amplifies content. Investigations of platform dynamics in India show majoritarian patterns persist online, and experts warn that algorithmic curation can entrench these patterns. <\/p>\n\n\n\n

A Dalit writer\u2019s long-form explainer or a producer\u2019s short documentary gets poor recommendations, not because of quality, but because the model\u2019s engagement priors undervalue its audience or mislabel its tone. <\/p>\n\n\n\n

The real danger is not just discrimination, but that it hides behind a facade of objectivity. A recruiter can shrug: the system rejected the candidate. A bank can say: the algorithm flagged the loan. <\/em>Responsibility gets outsourced to the machine, while prejudice is quietly legitimised by the authority of mathematics. <\/p>\n\n\n\n

This \u201ctech washing\u201d is especially dangerous in India, where digital systems are becoming the backbone of everyday life, eg, Aadhaar for identity, UPI for payments, predictive analytics for welfare and even policing. A single biased recruiter may block a few careers; a biased algorithm can exclude millions at once, invisibly. <\/p>\n\n\n\n

This is not a distant risk. A neo-Buddhist entrepreneur in Uttar Pradesh may be denied credit because his pin code is flagged as high-risk. On social media, Dalit voices celebrating BR Ambedkar can be silenced by moderation systems that normalise dominance but punish assertion. Each decision feels technical, but together they map a bleak picture: the future being shaped by machines that think exactly like the casteist society we live in. <\/p>\n\n\n\n

What this looks like in my work sphere <\/strong><\/p>\n\n\n\n