In the next decade, we may witness the arrival of superintelligence. These AI systems would have vast capabilities and could be hugely beneficial, but also potentially pose large risks. Today, we use reinforcement learning from human feedback (RLHF) to ensure AI systems are safe. However, aligning future superhuman AI systems will pose fundamentally new and qualitatively different technical challenges.
Superhuman AI systems will be capable of complex and creative behaviours that humans cannot fully understand. Existing alignment techniques like RLHF that rely on human supervision may no longer be sufficient. This leads to the fundamental challenge: how can humans steer and trust AI systems much smarter than them?
This is one of the most important unsolved technical problems in the world, and OpenAI is offering a solution. They are offering $100K–$2M grants for academic labs, nonprofits, and individual researchers. For graduate students, they are sponsoring a one-year $150K OpenAI Superalignment Fellowship: $75K in stipend and $75K in compute and research funding. The deadline for applications is February 18, 2024.
With these grants, OpenAI is particularly interested in funding research directions such as weak-to-strong generalization, interpretability, scalable oversight, honesty, chain-of-thought faithfulness, adversarial robustness, evals and testbeds, and more. They expect to be able to make grants to individuals and institutions in most other countries as well, barring legal restrictions. No prior experience working on alignment is required; they are actively looking to support researchers who are excited to work on alignment for the first time.
OpenAI believes that the challenge of aligning superhuman AI systems is solvable with a concerted effort. There are many promising approaches and exciting directions, with lots of low-hanging fruit. They see an enormous opportunity for the ML research community and individual researchers to make major progress on this problem today.
Interested and qualified? Visit the OpenAI website on airtable.com to apply. The application process is simple, and they’ll get back to you within four weeks of applications closing. Join OpenAI in bridging the gap between humans and superhuman AI systems.
