AT&T cut the costs of coding and some other advanced artificial intelligence tasks by as much as 56% by using tools that route employees’ queries to cheaper models when appropriate, The Information reported Thursday (Aug. 20).
When doing so, the quality of the performance of the AI declined by only 2%, according to the report, which cited an interview with Mark Austin, an AT&T vice president who oversees AI used by the telecommunication company’s employees.
The tools AT&T used in this instance were LiteLLM model routers, which determine the complexity of the task and then decide if it can be sent to a cheaper AI model, the report said.
Shifting to Open Source
AT&T aims to keep its employees spending on models from Anthropic and OpenAI flat by making greater use of open-source or open-weight models. The company aims to increase the share of employees’ queries that are powered by open-source models from the current 40% to between 60% and 70% in the coming years, per the report.
AT&T is using open-source or open-weight models such as Nvidia’s Nemotron, Meta’s Llama and Google’s Gemma. The company is not using open-source models from the Chinese firms DeepSeek and Moonshot but is evaluating the potential risks of using them, according to the report.
Related: CFTC to Draft Rules on Compute Derivatives Contracts
Austin said that while the capabilities of open-source models have generally been six to 10 months behind those of frontier models, the gap is narrowing and the open-source models are “just as good or better” than older models from Anthropic and OpenAI.
Cost Control Measures
It was reported in June that companies are looking to better manage their use of AI after seeing the costs of the technology rise. The rising costs have been driven by the shift from chatbots to agents, which consume more computing power, as well as the AI labs’ move from flat subscriptions to token-based billing.
PYMNTS reported in July that new tools are being launched to address the issue of AI costs and that the era of “tokenmaxxing,” or pushing employees toward the biggest AI models and heaviest usage, is ending after two years of unchecked growth.
For the average employee, the switch to model routers means the system will handle routine requests with lighter models, reserving the heavy-duty computing for complex tasks. This change forces IT teams to balance the budget against the potential for occasional errors in simpler queries, a trade-off that companies are now trying to quantify.
