Description

As AI and machine learning systems become integral to products and services across industries, it is critical to identify and mitigate the risks associated with their design, deployment, and operation. This course examines the evolving landscape of AI governance, exploring both technical and organizational strategies for developing trustworthy and responsible AI systems.

In 2026, the course expands to cover responsible development and governance of Agentic AI—systems capable of autonomous reasoning, planning, and collaboration. Students explore governance strategies across the AI lifecycle, including model alignment (RLHF, RLAIF), fairness, differential privacy, explainability, interpretability, and AI red teaming. The course integrates evolving policy and regulatory frameworks such as the EU AI Act, NIST’s AI Risk Management Framework, ISO/IEC 42001, and OECD guidelines. Case studies examine responsible AI practices in foundation models, generative systems, and agent-based ecosystems.

The course combines technical, policy, and management perspectives to equip students with the tools and frameworks needed to assess and mitigate AI-related risks.

Objective

This course is designed for advanced undergraduates and graduate students preparing to design, develop, deploy, or oversee AI-based systems. It introduces key principles, methodologies, technologies, and best practices for responsible AI and risk mitigation.

Students will:

• Understand the governance implications of next-generation AI systems, including multi-agent and Agentic AI architectures.
• Learn technical and organizational approaches for ensuring transparency, accountability, security, privacy, fairness, robustness, and safety.
• Gain hands-on experience analyzing governance frameworks and applying responsible AI techniques and tools such as red teaming, differential privacy, and interpretability audits.
• Examine regulatory, ethical, and policy issues shaping AI practice across sectors.

Prerequisites

No deep technical knowledge of AI or machine learning is required. A basic understanding of probability and statistics is expected. The course is designed to accommodate students from diverse technical and non-technical backgrounds, including engineering, computer science, policy, design, and management.

Students interested in AI engineering, product management, law and policy, design, or risk management will find the course particularly relevant. Selected sessions provide intuitive introductions to modern AI safety techniques—such as RLHF, adversarial testing, and explainability methods—and their governance implications.

Format

Weekly sessions combine lectures, discussions, and project work. All lectures will be given during the first half of the semester. Students interested in taking the class for just Mini 3 can do so for either 6 units or 9 units (the latter requires completion of a team project). Students interested in continuing to work on their team projects and working on a 3rd assignment can do so in mini 4 for an additional 3 units (total of 12 units).

6-Unit vs 9 vs 12-Unit Sections & Grading

6-Unit Section:

• Midterm: 25%
• Final: 25%
• Homework Assignments: 2 × 25% each

9-Unit Section:

• Midterm: 20%
• Final: 20%
• Homework Assignments: 2 × 15% each
• Team Project: 30%

Projects may focus on applying regulatory or technical governance frameworks—such as the EU AI Act, NIST AI RMF, or ISO/IEC 42001—or on assessing and mitigating real-world AI risks through red teaming, interpretability analysis, or privacy-preserving design.

12-Unit Section:

Additional 3 units for work in mini 4.

• Midterm: 15%
• Final: 15%
• Homework Assignments: 3 × 10% each
• Team Project: 40%

Lecture Topics (Spring 2026)

1. Introduction: The Expanding AI Ecosystem - Overview of AI lifecycle risks, governance frameworks, and Agentic AI
2. Ethical Principles and Foundations of Responsible AI - Global ethical frameworks, values alignment, and trustworthiness
3. Governance, Data, and Privacy - Differential privacy, data governance, and regulatory compliance
4. Fairness and Bias in AI Systems - Bias auditing, algorithmic fairness, and equitable model design
5. Transparency, Explainability, and Interpretability - SHAP, LIME, causal interpretability, and documentation frameworks
6. Model Alignment and Oversight - Reinforcement Learning with Human and AI Feedback (RLHF, RLAIF); human-in-the-loop governance
7. AI Red Teaming and Adversarial Evaluation - Probing model behavior, safety testing, and governance integration
8. AI Security and Robustness - Adversarial attacks, model extraction, and resilience strategies
9. Legal and Regulatory Landscape - EU AI Act 2025, U.S. Executive Orders, ISO/IEC 42001, and international harmonization
10. Organizational Governance and Compliance Practices - Risk management systems, accountability structures, and assurance tools (incl. NIST AI RMF)
11. Governance of Agentic and Multi-Agent AI Systems - Oversight of autonomous systems and emergent behavior
12. AI Safety and Societal Risks - Applications in critical systems: autonomous driving, healthcare, defense
13. Copyright, Intellectual Property, and Model Use Policies - Generative AI, content provenance, and copyright compliance
14. Project Poster Fair - Award certificate(s) for best project(s)

Faculty and instructors who have taught this course in the past

Norman Sadeh