Open Philanthropy recommended a grant of $1,045,620 to Princeton University to support a project to develop a benchmark for evaluating the performance of Large Language Model (LLM) agents in software engineering tasks, led by Assistant Professor Karthik Narasimhan.
This grant was funded via a request for proposals for projects benchmarking LLM agents on consequential real-world tasks and falls within Open Philanthropy’s focus area of potential risks from advanced artificial intelligence.