Название: Observability for Large Language Models: Understanding and Improving Your Use of LLMs
Автор: Phillip Carter
Издательство: O’Reilly Media, Inc.
Год: 2023-09-28
Язык: английский
Формат: pdf, mobi, epub
Размер: 10.2 MB

Artificial Intelligence (AI) has revolutionized numerous industries, enabling organizations to accomplish tasks and solve complex problems with unprecedented efficiency. In particular, large language models (LLMs) have emerged as powerful tools, demonstrating exceptional language-processing capabilities and fueling a surge in their adoption across a wide range of applications. From chatbots and language translation to content generation and data analysis, LLMs are being adopted by companies of all sizes and across all industries.

As organizations eagerly embrace the potential of LLMs, the need to understand their behavior in production and use that understanding to improve development with them has become apparent. While the initial excitement surrounding LLMs often centers on accessing their remarkable capabilities with only a small up-front investment, it is crucial to acknowledge the significant problems that can arise after their initial implementation into a product. By introducing open-ended inputs in a product, organizations expose themselves to user behavior they’ve likely never seen before (and cannot possibly predict). LLMs are nondeterministic, meaning that the same inputs don’t always yield the same outputs, yet end users generally expect a degree of predictability in outputs. Organizations that lack good tools and data to understand systems in production may find themselves ill-prepared to tackle the challenges posed by a feature that uses LLMs.

One answer to solving these problems lies in software observability. Observability refers to the ability to understand an application’s behavior based on the data that it generates at runtime, called telemetry. The rise of observability in modern software comes from a need to understand the constantly changing, often nondeterministic behavior of applications. The nature of interconnected cloud services with ever-changing infrastructure and application code across several teams makes traditional debugging impossible, so unreliability increases significantly without a different set of tools. As it turns out, the problems posed by modern software systems are very similar to those of LLMs. As such, the tools and practices of observability are a good fit for taming the complexity and unreliability of product features using LLMs.

LLMs represent a step change in the capability and accessibility of machine learning (ML) models for organizations. Every product has problems to solve for its users where there is no single solution but rather a set of solutions lying on some spectrum of “correct” or “right.” Traditionally, companies turned to AI to solve these problems, but at great cost. Now, much of that cost has evaporated, and any product engineering team—even if they have no experience with ML—can solve problems using LLMs through a simple API.




ОТСУТСТВУЕТ ССЫЛКА/ НЕ РАБОЧАЯ ССЫЛКА ЕСТЬ РЕШЕНИЕ, ПИШИМ СЮДА!