Want smarter insights in your inbox? Sign up for our weekly newsletters to get only what matters to enterprise AI, data, and security leaders. Subscribe Now
Deciding on AI models is as much of a technical decision and it is a strategic one. But choosing open, closed or hybrid models all have trade-offs.
While speaking at this year’s VB Transform, model architecture experts from General Motors, Zoom and IBM discussed how their companies and customers consider AI model selection.
Barak Turovsky, who in March became GM’s first chief AI officer, said there’s a lot of noise with every new model release and every time the leaderboard changes. Long before leaderboards were a mainstream debate, Turovsky helped launch the first large language model (LLM) and recalled the ways open-sourcing AI model weights and training data led to major breakthroughs.
“That was frankly probably one of the biggest breakthroughs that helped OpenAI and others to start launching,” Turovsky said. “So it’s actually a funny anecdote: Open-source actually helped create something that went closed and now maybe is back to being open.”
Factors for decisions vary and include cost, performance, trust and safety. Turovsky said enterprises sometimes prefer a mixed strategy — using an open model for internal use and a closed model for production and customer facing or vice versa.
IBM’s AI strategy
Armand Ruiz, IBM’s VP of AI platform, said IBM initially started its platform with its own LLMs, but then realized that wouldn’t be enough — especially as more powerful models arrived on the market. The company then expanded to offer integrations with platforms like Hugging Face so customers could pick any open-source model. (The company recently debuted a new model gateway that gives enterprises an API for switching between LLMs.)
More enterprises are choosing to buy more models from multiple vendors. When Andreessen Horowitz surveyed 100 CIOs, 37% of respondents said they were using 5 or more models. Last year, only 29% were using the same amount.
Choice is key, but sometimes too much choice creates confusion, said Ruiz. To help customers with their approach, IBM doesn’t worry too much about which LLM they’re using during the proof of concept or pilot phase; the main goal is feasibility. Only later they begin to look at whether to distill a model or customize one based on a customer’s needs.
“First we try to simplify all that analysis paralysis with all those options and focus on the use case,” Ruiz said. “Then we figure out what is the best path for production.”
How Zoom approaches AI
Zoom’s customers can choose between two configurations for its AI Companion, said Zoom CTO Xuedong Huang. One involves federating the company’s own LLM with other larger foundation models. Another configuration allows customers concerned about using too many models to use just Zoom’s model. (The company also recently partnered with Google Cloud to adopt an agent-to-agent protocol for AI Companion for enterprise workflows.)
The company made its own small language model (SLM) without using customer data, Huang said. At 2 billion parameters, the LLM is actually very small, but it can still outperform other industry-specific models. The SLM works best on complex tasks when working alongside a larger model.
“This is really the power of a hybrid approach,” Huang said. “Our philosophy is very straightforward. Our company is leading the way very much like Mickey Mouse and the elephant dancing together. The small model will perform a very specific task. We are not saying a small model will be good enough…The Mickey Mouse and elephant will be working together as one team.”
