The era of AI regulation
Plus, Trustible launches its new policy analyzer, Anthropic claims to explain AI, and more
Hi there! The news has once again been a buzz with everything from new AI laws to controversies brewing at Open AI. We are also excited to launch our new AI Policy Analyzer. And if you plan to attend, join us next week in Brussels at IAPP AIGG 2024! (5 minute read)
The Era of AI Regulations
Trusible launches AI Policy Analyzer
Anthropic’s Explainable AI
Cl-ai-mate Change
Open AI’s Turning Point?
1. The Era of AI Regulations
After months of discussion and re-negotiation, AI regulations across the globe are moving from committee rooms to enacted laws. On May 21, 2024, the EU’s Council of Ministers formally approved the EU AI Act, which was the final procedural hurdle for the world’s first comprehensive AI law. The act is not slated to enter into force 20 days after it is published in the Official Journal of the European Union, which is expected to occur sometime in June. While most obligations will not take effect for a couple of years, the prohibition for certain AI systems will likely take effect by the end of 2024.
Back in the U.S., Colorado Governor Jared Polis signed SB 205 into law on May 17, 2024. SB 205 is the first comprehensive state AI law and requires that AI used to support ‘consequential decisions’ for certain use cases should be treated as ‘high risk’ and will be subject to a range of risk management and reporting requirements. Lawmakers in California have also advanced a bill out of committee in the state senate that would regulate large frontier models by requiring certain safeguards before training and deploying those models, as well as incident reporting obligations.
Our Take: While we provide regular updates on where there is AI regulatory movement, this update feels different. At a national level, the EU AI Act’s entry into force makes the complex law a reality when just months ago there was speculation that the law might not pass. Likewise, in the U.S., the first domino has fallen and states can now follow Colorado’s regulatory model or pave their own way while not having the “first-mover” pressure. Either way, the legal foundation has been set for the ecosystem of AI regulations.
2. Trustible launches AI Policy Analyzer
Many organizations have only recently drafted an organization-wide AI policy to help them establish the foundation of their AI governance strategy. This may be as humble as a short, general AI tool use policy, or as complex as an AI risk management policy reviewed and approved by a Board of Directors. There are not yet a lot of standards for what should be covered in an AI policy, and as mentioned above, new regulations and standards are getting quickly introduced. This can cause policies to become out of date quickly, and as companies prepare for a more regulated AI environment, having a clear, compliant AI policy is going to become essential.
To assist with this challenge, Trustible launched a new feature, the AI Policy Analyzer, which examines a customer’s existing AI policy and identifies gaps in the policy relative to a framework, standard, or regulation. The tool, which itself is powered by a large language model (LLM), leverages Trustible’s domain expertise to identify what key elements may be missing from a policy, and can make suggestions on what sections ought to be changed. As new regulations emerge, the AI Policy Analyzer can quickly identify if any policy changes are necessary to comply with the new requirements.
Full Announcement: https://www.trustible.ai/post/product-launch-trustible-s-ai-policy-analyzer
3. Anthropic’s Explainable AI
In a new study, researchers at Anthropic demonstrated a technique for identifying patterns of activation in LLMs that correspond to certain human concepts, like “unsafe code” and “the Golden Gate Bridge.” For context, LLMs consist of billions of parameters, but not all of them are used to process every input; patterns of activations refer to groups of the parameters that are used by the model to process similar inputs. While this research can help build safer models by flagging activation of certain “bad concepts,” it does not make LLMs significantly more “interpretable” for end users for several reasons.
Consider a case where an LLM is used for resume screening. Using this research, it may be possible to discover that a concept related to “women’s colleges” or even “gender bias in professions” were activated, but it does not explain how these concepts interacted or how the model used them to arrive at a specific decision. In fact, there are likely a large number of additional concepts that were not identified or that do not map to an easy-to-label concept. Furthermore, this methodology requires access to the model’s weights; if Anthropic released an explainability-related feature, it would be limited to concepts that they labeled.
Overall, this work focuses on global interpretability (i.e. understanding overall model behavior) not local interpretability (i.e. showing how a model arrived at a specific decision); the latter can be more helpful for building users' trust in models. Looking at abstract features will only provide hints to the mechanics of a specific decision. Effective techniques for local interpretability may focus on highlighting examples from the training data that contributed to the decision or providing counterfactual explanations. For a deeper look into the components of interpretability, check out this free textbook.
Key Takeaway: Anthropic’s latest research won’t directly address the ‘explainability’ aspects many regulators seek, especially when it comes to automated decision making.
4. Cl-ai-mate Change
Microsoft recently released its 2024 Environmental Sustainability Report, and buried in the report is a significant 29% increase in data center driven emissions, likely caused by Microsoft’s aggressive moves in the AI space. While Microsoft has pledged to still meet its target of being carbon neutral by 2030, and sees AI as a potentially key tool in assisting them, this increase underscores one of the core ‘societal’ risks of AI and the AI arms race: accelerated environmental impacts.
LLMs take weeks to train using thousands of servers, and then run the models for inference using similar hardware that is distributed around the world. While we cannot calculate the exact amount of energy usage from leading LLMs models, as few models share information about the number of parameters and requisite hardware, some early academic work by HuggingFace and Carnegie Mellon identified that generative AI, and generative image/video in particular, use substantial amounts of energy for inference. To support these estimates, in a recent podcast interview, Mark Zuckerberg suggested training Llama-3 took the full energy output of a nuclear power plant. The energy requirements lead to high carbon emissions from power generation, additional water usage for cooling, and more hardware creation and the associated material extraction. While many countries had previously been working to build consensus on combating climate change, the economic and geopolitical advantages of AI may put some of those goals at risk, as well as pose massive challenges for mitigating the societal impacts from the AI race.
Key Takeaways: AI training and inference can use huge amounts of energy, and the corporate and geopolitical AI race could put global climate goals at risk.
5. Open AI’s Turning Point?
To say that Open AI had a rough couple of weeks may be an understatement. It started with two high-profile departures, Open AI co-founder Ilya Sutskever and Jan Leike. The two men co-led Open AI’s “superalignment” team, which attempted to foresee and mitigate longer-term AI risks. The shake-up comes at a time when AI technology is facing more scrutiny from policymakers over AI risks and safety concerns. Adding fuel to the controversy, it was reported shortly after Sutskever and Leike’s exit that Open AI may have leveraged former employees’ vested equity to force them into signing a lifetime non-disparagement agreement.
Open AI seemingly knocked the bad news off the front page when it released ChatGPT-4o, which allows users to engage in humanlike conversations with the model. However, shortly after ChatGPT-4o’s release accusations began to swirl around Sky, one of the model’s five voices, which soundly uncannily like actress Scarlett Johansson. The controversy deepened when it was revealed that Sam Altman asked Johansson to be a voice actor for ChatGPT, which she declined. While Open AI has denied that Sky is Johansson’s voice, it refused to name the voice actress for privacy reasons and has since pulled Sky offline.
Our Take: The string of controversies rocking Open AI raises legitimate questions around the company’s mission. Leike publicly criticized the company’s safety culture, noting it has taken a “backseat to shiny products.” While Open AI was founded as a non-profit committed to ethical AI development, we have seen the “Do No Evil” storyline before and it caused more issues than it solved.
*********
As always, we welcome your feedback on content and how to improve this newsletter!
AI Responsibly,
- Trustible team