Our reading this week is AI centric, from a new paper about the accuracy of LLM generated summaries to reporting on models' insatiable demands for training data and the steps tech companies are taking to procure it. And with broad relevance towards AI regulation and beyond, don't miss Pitt Cyber Founding Director David Hickton's op-ed on the need for policymaking centered around preparedness.
From pandemic preparedness to cybersecurity, technological advances can only deliver on their promise if we embrace a proactive, rather than reactive, policymaking stance.
Artificial Intelligence and Democratic Values | Center for AI and Digital Policy
Clocking in at 1600+ plus pages, it is safe to say that no one is going to read it cover to cover, but Center for AI and Development Policy's latest AI and Democratic Values Index represents an amazing resource detailing the state of AI regulation in 80+ countries and recapping various multilateral initiatives underway.
AI Could Actually Help Rebuild the Middle Class | Noema
As follow up the 'AI and the Workforce' edition we did a couple of weeks ago, we wanted to flag this optimistic hypothesis from MIT economist David Autor. Arguing that "Artificial Intelligence is inversion technology," Autor reasons that AI has the capacity to enable "a larger set of non-elite workers to engage in high-stakes decision-making. It would simultaneously temper the monopoly power that doctors hold over medical care, lawyers over document production, software engineers over computer code, professors over undergraduate education, etc." His argument falls in the category of AI augmentation (versus automation), with a particular focus on the potential gains for less skilled workers who have been hurt by previous waves of automation.
We're pleased to see the U.S. Departments of Education, Health and Human Services, Homeland Security, Housing and Urban Development, and Labor sign on this commitment to use their existing authorities to ensure that anti-discrimination law is upheld when AI in integrated into decision making processes.
How Tech Giants Cut Corners to Harvest Data for A.I. - The New York Times
Excellent investigative reporting from the New York Times digging into LLM training datasets. With AI developers set to "run through the high-quality data on the internet as soon as 2026 … The companies are using the data faster than it is being produced." As a result, companies have engaged in a range of legally questionable behaviors, from transcribing YouTube videos in likely violation of the platforms terms & conditions to bending the rules in the effort to use "people’s publicly available content in Google Docs, Google Sheets and related apps."
Evaluating Faithfulness and Content Selection in Book-length Summarization | Arxiv
We stumbled on this paper through Gary Marcus' Substack and it's a fascinating one. Many use LLMs to summarize long articles. But are the LLM generated summaries accurate? This paper compares the 'faithfulness' (accuracy) of book summaries across models: Claude 3 Opus (Anthropic) scores the highest, with 90% of its summaries deemed faithful to the original text. ChatGPT-3.5 performs relatively poorly at 72%, although Chat-GPT4 improves to 78%.