Small language models bring big business benefits

A recent survey has found that most global IT leaders are concerned their companies will be “left behind” if they do not adopt artificial intelligence (AI). Over half of these leaders also say that pressure from customers is a crucial driver for AI adoption and that AI is pivotal for enhancing efficiency and customer service within the business. Most companies view AI adoption as a requirement to maintain a competitive advantage.

Despite the enthusiasm, many companies have concerns about implementation costs, employee misuse, and potential compliance issues. The August 2024 State of Intelligent Automation Report by ABBY shows how these concerns rank among survey respondents.

The report also highlights that IT leaders have a high level of trust in small language models (SLMs) compared to other types of AI. One key finding is that the manufacturing sector exhibits highest trust in SLMs at 92%, closely followed by financial services and IT at 91%.

What is a small language model?

A small language model (SLM) is a neural network designed to generate natural language content but with fewer parameters than large language models (LLMs). We’ll come back to the meaning of parameters, but here’s a quick look at the differences between SLMs and LLMs:

Purpose/Use-cases:

• SLMs: Domain-specific tasks, edge computing, resource-constrained environments. Training is focused on domain-specific datasets. SLMs often provide faster responses/inferences and lower latency. Domain and endpoint deployments are better suited for handling sensitive data because the data remains local.
• LLMs: General-purpose language tasks, complex reasoning. Training is based on vast, diverse datasets, providing them with broader knowledge and greater flexibility. Large language models are also better at complex tasks than the leaner, domain-focused SLMs, but LLMs may also require sensitive data to be sent to the cloud for processing.

Operational Requirements:

• SLMs: Lower computational power, less memory, and suitable to deploy on-premises or on edge devices. SLMs are almost always more cost-effective to train and run in production.

• LLMs: High computational power, large memory requirements, and higher operational and training costs.

Now, let’s get back to parameters. These are numerical values that determine how an SLM or LLM processes input and generates output. There are outliers, but SLMs typically have fewer than 100 million parameters, whereas LLMs usually have billions or trillions. A simple way to illustrate the relationship between a parameter and a language model is to use the example of a library. A medical or law library may have hundreds or thousands of books directly relevant to its particular field. A large library with resources on every subject will have more books (parameters), but they are not all relevant to your interests. The larger library requires more resources to manage, but it can also provide information on more topics.

The parameters are the ‘knowledge’ the language model learned during its training. If your company needs AI technology that can perform a limited set of tasks very well, the small language model with fewer parameters may suit your needs.

Data security and transparency

Because SLMs are trained on limited data and can be deployed to edge devices, these models may be more palatable to companies concerned about security and compliance. Data is processed locally, which makes it easier to audit, control, and monitor the decision-making processes of the model. The AI regulatory environment is changing rapidly, and many governments have already implemented transparency regulations. For example:

The European Union (EU) AI Act (2024) requires users to be informed when they are interacting with AI systems in certain applications. It also requires companies operating high-risk AI systems to provide documentation on certain aspects of those systems.

Utah, Colorado, and California are among the first in the United States (U.S.) to develop regulations around the transparency of AI systems and usage. These regulations may require disclosure of the use of AI, risk-management policies, and protection against biases in the AI systems.  

Technology vendors and associations have published their own guidelines on AI governance and ethics, which may include transparency as a foundational element to adoption.

This push for transparency does cause a different type of concern for developers and companies working with AI. Proprietary small or large language models may be considered intellectual property (IP) and a competitive advantage. Companies normally do not want to disclose the details of these assets. There is also a legitimate security concern around providing too much information about a language model. Threat actors might use this information to attack or misuse the model.

Other concerns about regulating transparency include the complexity of the models, which makes it difficult to explain the required information to someone who doesn’t have a background in the technology. This complexity and the lack of universally accepted standards for AI leave many concerned that compliance with transparency regulations may become a blocker to innovation and deployments.

Edge computing

Edge computing is growing at a ridiculous rate, largely due to Industry 4.0 initiatives and the proliferation of internet-connected devices and controllers in manufacturing, energy, and transportation sectors. Advancements in 5G technology and the benefits of real-time processing on remote devices have also contributed to this growth. The COVID-19 pandemic accelerated the adoption of edge computing to support remote work, but this factor is much less significant than the growth in the Internet of Things (IoT) and Industrial Internet of Things (IIoT).  

Small language models are a near-perfect solution for edge computing devices, and edge AI keeps improving. Still, there are still some limitations to consider. Edge device SLMs often require more frequent updates and fine-tuning, which can be challenging with devices with limited connectivity. SLMs also reach their performance limits faster when data processing requirements increase. And, although SLMs generally offer greater privacy, data transmitted from the edge may be exposed to the cloud.

Continued growth for SLMs

There’s no question that business adoption of small language models will continue to grow, and it’s not just driven by edge AI and IIoT. Customer service automation, language translation, sentiment analysis, and other specific use cases will contribute to this growth. Microsoft, Google, and other AI vendors believe that SLMs offer a “more accurate result at a much lower cost,” and a shift toward a portfolio of models that allows companies to choose the best fit for their scenarios.

If you’d like to learn more about SLMs and how they work, these sites can help:

IBM: Are bigger language models always better?

Salesforce: Tiny Titans: How Small Language Models Outperform LLMs for Less

HatchWorksAI: How to Use Small Language Models for Niche Needs in 2024

Microsoft: Tiny but mighty: The Phi-3 small language models with big potential