Extra Credit: Coded Bias Viewing

[albertkun]

Extra Credit Assignment due 4/22

Watch the film Coded Bias either on PBS or Netflix and write a short comment here answering the following questions:

• What were the highlights of the documentary for you?
• What could be improved?
• How does it relate to web development and civil society?

Stream on:

PBS

https://www.pbs.org/independentlens/videos/coded-bias-full-film/

Netflix

https://www.netflix.com/title/81328723?s=i

Submission

Post your comment below in this discussion.

[annyrviloria]

I appreciated this documentary as it succinctly captures all the dimensions of our lives affected by algorithmic biases. I was familiar with many of the issues mentioned but was still taken aback by the depth of this issue. Some of the highlights for me:

• I cannot stop thinking about the fact that it only took 14 hours for the microsoft twitter bot to learn to be a racist a*shole. It drove home the point that the biases in our technology are our own biases, and not simple accidents devoid of context. Or the issue with the use of AI to scan job applications at google that resulted in ZERO women being recommended for hire... I mean, wow.
• I appreciated the different examples of civil advocacy exemplified by the work of Joy with the NYC tenants or by Big Brother Watch in the UK. It is imperative that we inform, empower, and work for the benefit of the communities most targeted by these algorithms. The point someone made about technological tools of surveillance first reaching marginalized communities given the low expectation of respect for civil rights is one we should sit with and take into consideration as young professionals in a technocratic world.
• I also liked the comment made regarding facial recognition tech in China and the US: the only real difference between these two countries is how overt China is about their data collection practices, whereas in the US it's all conducted under a veil. I often hear folks talk about surveillance in China as something to fear we might be headed into... when in reality the US is at the same level. That type of comment, IMO, simply advances the anti-China rhetoric US politicians love to tout.
• My favorite scene is when Joy testifies at the congressional hearing. I am in awe of her endurance and brilliance in the work she's leading. I can understand the hesitation anyone in the field of technology might have regarding coming forward against these biased algorithms yet she's out there leading this important conversation at a national scale. Props to her and to all the other badass women featured in the film!
• This is unrelated but my favorite map which I mentioned during my introduction in week 1 of our course is (briefly) featured in the documentary! You'll see it on the floor of the main room of the Apartheid Museum in Cape Town. It shows what the original community of the area where the museum was built before Black communities were evicted and segregated. The map was drawn by the original residents of the area and oral histories continue to be added to this day. Let me know if you spot it too!

I don't have any thoughts about how it can be improved. I suppose it could have gone deeper into each of the points that it raised, paid more attention to how the US weaponizes AI to fight wars abroad, or addressed the anti-Muslim sentiment behind the rise of the hyper-surveillance state in urban spaces -- but that would have made for a super, super long and depressing film.

As for how this relates to web development and civil society, after Mariah Tso's presentation, I believe we all have a clear sense of the how power plays a central role in the uses/consequences of a given technology (maps, for example). I would extend that idea to web dev and reflect on the ways in which the web products we develop serve to advance power: whose power, power for what, and how. These questions should not be an afterthought, but rather, they should be quite literally built into the code that generates our products to ensure we aren't replicating the power imbalances of our own contexts. Community oriented projects such as the one we are developing for this course are one example of the ways our web dev endeavors can very intentionally give power to communities/activists/advocates to both create applications with a social mission AND control the data that can be collected without compromising the welfare of the communities accessing the applications we build.

[albertkun]

* It only took 14 hours for the microsoft twitter bot to learn to be a racist a*shole.

I love this comment.

It's great that your favorite map had a cameo in the film!!

Your entire response regarding the impact of algorithms and all those highlighted scenes were meaningful for me as well! I'm glad to see that the film expanded on the problematic issues that you already knew about machine learning/AI. Your comment on how, who, and why we should wield power for succinctly summarized the objectives of the course AND tied it with the film!! 💯 Great post!

[brillianz]

What were the highlights of the documentary for you?
The horizontal relationship that was established by the researcher at MIT alongside the Brooklyn residents was the highlight of the documentary for me. I appreciate anytime a researcher is able to “translate” and empower communities that they identify with in a personal setting. Whether through qualitative, quantitative, or a combination of both methodological approaches, I am intent on implementing methods for empowerment for a community-based approach. The poem that she read at the end was a direct way of acknowledging their work as just as important as that of the researchers.
What could be improved?
Throughout the documentary, I thought about the role of consumers and immediate steps that we can take on an everyday level. Being that many of Netflix’s consumers are not researchers or politicians, I am curious about some of the immediate placements that can be used in order to transform the predatory nature of machine learning and algorithms.
How does it relate to web development and civil society?
The documentary speaks to the importance and responsibility of creating an ethical base that is conscious of human bias in the development of mediums for profit, leadership, and justice.

[rachan2023]

What were the highlights of the documentary for you?

1. I enjoyed seeing Buolamwini speak and listen to the Brooklyn tenants’ concerns about the biometrics security system that their landlord implemented then later amplifying the tenants’ concerns in front of Congress. It was empowering to see that she saw in her research that algorithms were biased and oppressive and even went to talk to people that were directly affected by this technology in her community. She gave them a voice and contributed to the efforts in banning facial recognition technologies in cities such as San Francisco and Oakland. Although there are no regulations on algorithms at the moment, Buolamwini and the tenants made a lot of a progress in sharing with politicians that these algorithms are detrimental. This gives me hope that people can fight these algorithms.

2. Another highlight was seeing other forms of resistance used to combat these algorithms, such the man who covered his face to prevent the surveillance camera from identifying him as well as the protests in Hong Kong who used lasers and spray paint to desensitize the surveillance cameras. This further emphasizes the point that people do have power and can find ways to counter these algorithms.

3. I thought it was personally terrifying that big corporations (specifically only 9 companies are at the forefront of these algorithms of oppression??) build and use these algorithms, but they do not understand the full extent of harm that these algorithms can cause. In particular, England continues to use surveillance cameras as a method for catching criminals even though their facial recognition software misidentifies people 98% of the time. This is especially dangerous for people of color. The fact that these tools are still implemented also reveals a lot about the government's biases as well.

What could be improved?
Overall, the documentary did a great job explaining in detail about how these algorithms are biased and the consequences they have on citizens. However, one aspect that can be improved is going into more depth about other ways in which regular citizens can combat these algorithms in their daily lives. For example, Buolamwini went to speak with the Brooklyn tenants and tried to understand from their perspective how these facial recognition systems are violating their rights. For people that don’t have the opportunity to talk to someone who can help them, what are other ways that they can protect themselves from these algorithms?

How does it relate to web development and civil society?
This relates to society because it shows the consequences of just taking data from communities and using them to create technologies that will “benefit” society. When people’s data are used with the sole intent of solving a problem, people can create bias algorithms and technologies that do more harm than good. As a result, people have to stop and ask themselves if this is what the community wants and needs; the community is the sole focus. Furthermore, this relates to today’s lecture when Nina Kin was talking about how we have to be conscious about our own individual biases and privileges and ensure that we are not projecting those biases into the technology that we are creating. It is important to actively listen and understand what the local communities’ needs are and find sustainable as well as ethical ways to address their problems.

[Wny-Duong]

What were the highlights of the documentary for you?

The ending was probably one of the best points of the documentary itself, after the hearing and when they were showing the faces of the people who were targeted by facial recognition technology and its unethical usage and usage by police. It's nice to draw attention to the violation of civil rights and highlight discrimination where it occurs as a result of computer vision algorithms, but more optimistic stuff like how such things can help enrich the lives of disenfranchised groups or victims of those violations paints a better, more hopeful message. In this case, it was by encouraging those groups to foster an interest in technology and coming to their conclusions about how such tech should be developed and deployed. Personally, I didn't think the call to action to promote legislation against the use of algorithms seemed realistic, especially due to the sheer presence of them in the fields of big tech, various branches of research, and more. But the next best thing is definitely drawing awareness to the shortcomings of just focusing on the technology and how they negatively effect people as corporations try to strive for efficiency at the cost of the societies they exist and rely on to thrive.

What could be improved?

Personally, I thought the documentary went out of its way to try and make the technical components of invasive data collection and facial recognition more digestible to the users. I know I should be focusing more on the issues relevant to civil society and personal freedoms brought up by the technology, but a thorough understanding of how threats to those things are created feels pretty valuable to me. Stuff like interviews with engineers and developers who worked on these unethical system's would've shined some interesting light on how such models were even allowed to be created. I did notice that there was a brief scene with a member of Google's Ethical AI team when they were discussing the Amazon blog post trying to defend their technology, and was kind of disappointed that they didn't go into how/if such companies did practice some oversight to their machine learning technology. Also, I would've loved it if they tried to get an in-depth look at the data collection process and explore avenues of problems there. The documentary mentioned the collection of state driver's license photos as a way to build the facial recognition databases used for policing. Some scenes trying to trace that back to origin (to whatever government body approved for the collection of the data) would've been really eye-opening to see.

How does it relate to web development and civil society?

In regards to web development, there are open source libraries out there that fledgling developers can use to deploy machine learning and artificial intelligence to their personal projects or to projects they're employed to develop on. It's important, in this case, for the people creating new software to be mindful of how they're using such things and to ensure that they're developing in an ethical manner. Skepticism and scrutiny of libraries and methods of development are required to ensure that potentially good projects don't end up being in a position to present a harm to society.

In regards to civil society, the glamor of high-tech innovations and the speed in which things such as deep learning, prediction algorithms, etc. develop ends up clouding the technology in mystery to people who aren't kept up to date with it. Some people get upset, harmed, and confused when algorithms are applied to them without knowing that they're able to try and rally against those harmful instances. Informing the general public through documentaries such as Coded Bias allows people to recognize when and where their personal data can be used against them and help to promote discourse on the matter to ensure that they can protect themselves from malicious usage.

[vcta99]

What were the highlights of the documentary for you?
I really appreciated how thoroughly they covered coded bias in so many different fields: education, job applications, probation sentencing, etc. Doing that really instilled a sense of danger in terms of how pervasive and world-altering this technology could be. Perhaps not a "highlight", but it did make me wonder how my life and my decisions have been affected by coded bias. Like many others in the class, I essentially grew up in the wild west of the internet (I think I've had my FaceBook account since I was 10 years old). With news of Mark Zuckerberg rolling out an "Instagram for Kids", I wonder how younger and younger populations will become vulnerable to this type of exploitation.

Also, like others above, I also enjoyed how the film highlighted the community organizing of the tenants being affected in Brooklyn (I particularly noticed how Joy asked them, "How can I support you?", as community engagement and listening has been a big theme in the class so far.)

What could be improved?
A relatively small critique, but I was a but turned offed by the way they presented surveillance systems in Hong Kong/China. Especially during the later half of the documentary when Cathy O'Neil was critiquing the social credit system. I think O'Neil provided a fair argument, but it was strange to hear her voicing over, explaining a problem that doesn't affect her, while b-roll of Chinese people using technology played. I especially felt that they did the Wei Su (the Chinese citizen) a bit dirty with her interview. Su is a proponent of the social credit system, but the ominous music they played during her interview sort of framed her as "sheep"? It was like, "hey, look at this naive product of the system who doesn't understand the detriment surveillance has to society". If they wanted to contrast viewpoints, I felt like it would have been more fair to find a Chinese citizen (who's identity was kept anonymous or who has recently emigrated out of China) to share their perspective against the social credit system. Again, I agree with the critique, just not necessarily how it was presented.

How does it relate to web development and civil society?
I think it has everything to do with web development and civil society. I'm not sure who exactly in the documentary used the phrase "human cost" (it was towards the end when they were preparing to meet with Congress), but I thought that was a really effective way of framing the harm web development creates. Companies can remove personhood in order to achieve profit, but the very real effects of loss of liberty, loss of opportunity, etc. are felt by humans. What's made more difficult is that civil society operates by pushing against defined problems and offering, when possible, actionable solutions. But when the problems are wrapped up in blackboxes, it obscures what the source of inequality is, making it that much more difficult to fight against it.