THE TEXAS TRIBUNE – When the Texas Workforce Commission became inundated with jobless claims in March 2020, it turned to artificial intelligence.
Affectionately named for the agency’s former head Larry Temple, who had died a year earlier, “Larry” the chatbot was designed to help Texans sign up for unemployment benefits.
Like a next generation FAQ page, Larry would field user-generated questions about unemployment cases. Using AI language processing, the bot would determine which answer prewritten by human staff would best fit the user’s unique phrasing of the question. The chatbot answered more than 21 million questions before being replaced by Larry 2.0 last March.
Larry is one example of the ways artificial intelligence has been used by state agencies. Adaptation of the technology in state government has grown in recent years. But that acceleration has also sparked fears of unintended consequences like bias, loss of privacy or losing control of the technology. This year, the Legislature committed to taking a more active role in monitoring how the state is using AI.
“This is going to totally revolutionize the way we do government,” said state Rep. Giovanni Capriglione, R-Southlake, who wrote a bill aimed at helping the state make better use of AI technology.
In June, Gov. Greg Abbott signed that bill, House Bill 2060, into law, creating an AI advisory council to study and take inventory of the ways state agencies currently utilize AI and assess whether the state needs a code of ethics for AI. The council’s role in monitoring what the state is doing with AI does not involve writing final policy.
Artificial intelligence describes a class of technology that emulates and builds upon human reasoning through computer systems. The chatbot uses language processing to understand users' questions and match it to predetermined answers. New tools such as ChatGPT are categorized as generative AI because the technology generates a unique answer based on a user prompt. AI is also capable of analyzing large data sets and using that information to automate tasks previously performed by humans. Automated decision making is at the center of HB 2060.
More than one third of Texas state agencies are already utilizing some form of artificial intelligence, according to a 2022 report from the Texas Department of Information Resources. The workforce commission also has an AI tool for job seekers that provides customized recommendations of job openings. Various agencies are using AI for translating languages into English and call center tools such as speech-to-text. AI is also used to enhance cybersecurity and fraud detection.
Automation is also used for time-consuming work in order to “increase work output and efficiency,” according to a statement from the Department of Information Resources. One example of this could be tracking budget expenses and invoices. In 2020, DIR launched an AI Center for Excellence aimed at helping state agencies implement more AI technology. Participation in DIR’s center is voluntary, and each agency typically has its own technology team, so the extent of automation and AI deployment at state agencies is not closely tracked.
Right now, Texas state agencies have to verify that the technology they use meets safety requirements set by state law, but there are no specific disclosure requirements on the types of technology or how they are used. HB 2060 will require each agency to provide that information to the AI advisory council by July 2024.
“We want agencies to be creative,” Capriglione said. He favors finding more use cases for AI that go well beyond chat bots, but recognizes there are concerns around poor data quality stopping the system from working as intended: “We’re gonna have to set some rules.”
As adoption of AI has grown, so have worries around the ethics and functionality of the technology. The AI advisory council is the first step toward oversight of how the technology is being deployed. The seven-member council will include a member of the state House and the Senate, an executive director and four individuals appointed by the governor with expertise in AI, ethics, law enforcement and constitutional law.
Samantha Shorey is an assistant professor at the University of Texas at Austin who has studied the social implications of artificial intelligence, particularly the kind designed for increased automation. She is concerned that if technology is empowered to make more decisions, it will replicate and exacerbate social inequality: “It might move us towards the end goal more quickly. But is it moving us towards an end goal that we want?”
Proponents of using more AI view automation as a way to make government work more efficiently. Harnessing the latest technology could help speed up case management for social services, provide immediate summaries of lengthy policy analysis or streamline the hiring and training process for new government employees.
However, Shorey is cautious about the possibility of artificial intelligence being brought into decision-making processes such as determining who qualifies for social service benefits, or how long someone should be on parole. Earlier this year, the U.S. Justice Department began investigating allegations that a Pennsylvania county’s AI model intended to help improve child welfare was discriminating against parents with disabilities and resulting in their children being taken away.
AI systems “tend to absorb whatever biases there are in the past data,” said Suresh Venkatasubramanian, director of the Center for Technology Responsibility at Brown University. Artificial intelligence that is trained on data that includes any kind of gender, religious, race or other bias is at risk of learning to discriminate.
In addition to the problem of flawed data reproducing social inequality, there are also privacy concerns around the technology’s dependence on collecting large amounts of data. What the AI could be doing with that data over time is also driving fears that humans will lose some control over the technology.
“As AI gets more and more complicated, it's very hard to understand how these systems are working, and why they're making decisions the way they do,” Venkatasubramanian said.
That fear is shared by Jason Green-Lowe, executive director at the Center for AI Policy, a group that has lobbied for stricter AI safety in Washington DC. With the accelerating pace of technology and a lack of regulatory oversight, Green-Lowe said, “soon we might find ourselves in a world where AI is mostly steering. … And the world starts to reorient itself to serve the AI's interests rather than human interest.”
Some technical experts, however, are more confident that humans will remain in the driver’s seat of increasing AI deployment. Alex Dimakis, a professor of electrical engineering and computer science at the University of Texas at Austin, worked on the artificial intelligence commission for the U.S. Chamber of Commerce.
In Dimakis’ view, AI systems should be transparent and subject to independent evaluation known as red teaming, a process in which the underlying data and decision-making process of the technology are scrutinized by multiple experts to determine if more robust safety measures are necessary.
“You cannot hide behind AI,” Dimakis said. Beyond transparency and evaluation, Dimakis said the state should enforce existing laws against whoever created the AI in any case where the technology produces an outcome that violates the law: “apply the existing laws without being confused that an AI system is in the middle.”
The AI advisory council will submit its findings and recommendations to the Legislature by December 2024. In the meantime, interest is growing in deploying AI at all levels of government. DIR operates an artificial intelligence user group made up of representatives from state agencies, higher education and local government interested in implementing AI.
Interest in the user group is growing by the day, according to a DIR spokesperson. The group has more than 300 members representing more than 85 different entities.
Disclosure: University of Texas at Austin and US Chamber of Commerce have been financial supporters of The Texas Tribune, a nonprofit, nonpartisan news organization that is funded in part by donations from members, foundations and corporate sponsors. Financial supporters play no role in the Tribune's journalism. Find a complete list of them here.
This article originally appeared in The Texas Tribune.