Welcome to CS 5342, Trust and Safety: Platforms, Policies, Products. Trust and safety is an emerging field that focuses on reducing the harm from interpersonal abuse in digital spaces. The abuse types involved - harassment, misinformation, unwanted sexual content - are often "lawful but awful," requiring developers to build their own socio-technical frameworks of what is appropriate behavior in their platform. In this course, we will look at digital abuse through an analysis of historical incidents. We will study how the field developed standards across algorithmic response, product design and manual removals. Students will join teams to describe an emerging online abuse type, develop appropriate moderation pipelines (e.g., using modern machine learning such as LLMs), and detail associated policies, all in an environment mimicking the realities seen in practice.
Classroom: Bloomberg 165 (B165)
Lectures: Tu/Th 11:40 AM - 12:55 PM
Instructors: Alexios Mantzarlis (https://www.mantzarlis.com/) and Tom Ristenpart (https://rist.tech.cornell.edu)
Office hours: By appointment
TAs: Armin Namavari ([email protected])
Office hours: TBA
-
Course website (Github): This Github repository will include general information about the course and will be updated to include lecture material and slides throughout the semester.
-
Course schedule (Google spreadsheet): A preliminary schedule is available. The schedule may evolve, but we will try to stick to it particularly with respect to the homeworks.
-
Course management system (Canvas): We will be using Cornell's Canvas for homeworks.
Programming skills and an introductory course on discrete structures and / or algorithms (e.g., CS 2800 or CS 4820). Homeworks will require good comfort with Python programming.
If you are in doubt, talk to instructor.
This course will require some interaction with harmful digital content in order to best understand the correct prevention and mitigation strategies. We define "harmful digital content" as websites, apps and other types of content that engages in or encourages illegal or almost illegal behavior (e.g., sale of drugs, glorification of terrorism, non consensual nude generators). Here are a few things to know and some advice to follow. University policies:
- Do not break any laws: Cornell University requires people who use its information technology resources to do so in a responsible manner, abiding by all applicable laws, policies, and regulations (Policy 5.1)
- Your traffic is not monitored, with exceptions: Cornell University does not monitor the content of network traffic except for legal, policy, or contractual compliance; in the case of a health or safety emergency; or for the maintenance and technical security of the network (Policy 5.9) Advice for this course:
- Follow safety protocols: Harmful digital content is typically hosted in locations that do not comply with the highest security protocols. Be especially careful about what you click on, do not share any personal information, and be careful about what you download / upload.
- Minimize: You should seek to minimize your time on sites that promote harmful content and behaviors, but you are allowed to visit them if it is germane to course preparation or activities (e.g. to read the Terms of Use a deepfake pornography service)
- Document: document the reason you visited any harmful sites in a logbook for your future reference. Should any flags be raised, it will be helpful to have documentation explaining behavior.
- Observe but don’t engage: For the most part, you are strongly discouraged from doing anything other than inspecting sites that might be hateful, violent, or otherwise problematic. Do not post content, share information, or otherwise use their services unless directly instructed to do so by a member of faculty
- Report unwanted behavior: If we discover unexpected harmful content on platforms where it is against policy, please report it to an instructor and or report to the developer through the feedback mechanisms available.
The class will involve a combination of lectures, homeworks, participation credit, and opportunities for extra credit. You'll be graded according to the following:
Students will choose a digital service and conduct research into the reporting mechanisms, moderation policies, algorithmic solutions and product interventions aimed at reducing interpersonal harms. With this information they will create a comprehensive overview of the T&S setup at that service that includes a diagram that pieces all the elements together.
Students will develop a policy document that describes a Trust & Safety intervention on the platform BlueSky. This will include a definition of the type of content they want to encourage or discourage, a threat modeling assessment, and a full set of prevention and mitigation measures including product design, algorithmic interventions and manual enforcement guidelines.
Students will build a labeler on BlueSky that automatically labels certain types of accounts / posts.
Students will be expected to prepare substantive questions for guest lecturers that connect to themes discussed in class. There will also be a handful of other light-weight assignments, most notably readings that you will be asked to discuss at the beginning of most classes.
Grades will not be curved, so everyone could get an A in the class if everyone does well.
We are planning 3 larger assignments. A preliminary schedule of assignments and due dates appear on the course schedule linked above. Homeworks are due on the due date by 11:59:59pm EST. You can use in total 3 late days throughout the semester.
The bulk of material will be delivered via lecture, as such you won't get much out of the class if you skip lectures and won't be prepared for homeworks. We will make slides available, but do not plan to record lectures.
Participation scores will be based on contributing questions ahead of guest lectures, as well as contributing to in-class discussions.