Название: Deep Learning at Scale: At the Intersection of Hardware, Software, and Data (Final Release)
Автор: Suneeta Mall
Издательство: O’Reilly Media, Inc.
Год: 2024
Страниц: 448
Язык: английский
Формат: True/Retail PDF, True EPUB (Retail Copy)
Размер: 35.6 MB

Bringing a deep-learning project into production at scale is quite challenging. To successfully scale your project, a foundational understanding of full stack deep learning, including the knowledge that lies at the intersection of hardware, software, data, and algorithms, is required.

This book illustrates complex concepts of full stack deep learning and reinforces them through hands-on exercises to arm you with tools and techniques to scale your project. A scaling effort is only beneficial when it's effective and efficient. To that end, this guide explains the intricate concepts and techniques that will help you scale effectively and efficiently.

Deep learning and scaling are correlated. Deep learning is capable of scaling your objectives from single task to multitask, from one modality to multimodality, from one class to thousands of classes. Anything is possible, provided you have scalable hardware and a large volume of data and write software that can efficiently scale to utilize all the resources available to you.

Scaling is complex, and thus not free. Developing a deep learning–based system requires a large number of layers, a large volume of data, and hardware capable of handling computationally intensive workloads. Scaling requires understanding the elasticity of your entire system—not just your model but your entire Deep Learning stack—and adapting to situations where elasticity nears a breaking point. Therein lies the secondary motivation of this book: to enable you to gain a deeper understanding of your system and when it might break, and how you can avoid unnecessary breaks.

You'll gain a thorough understanding of:

How data flows through the deep-learning network and the role the computation graphs play in building your model
How accelerated computing speeds up your training and how best you can utilize the resources at your disposal
How to train your model using distributed training paradigms, i.e., data, model, and pipeline parallelism
How to leverage PyTorch ecosystems in conjunction with NVIDIA libraries and Triton to scale your model training
Debugging, monitoring, and investigating the undesirable bottlenecks that slow down your model training
How to expedite the training lifecycle and streamline your feedback loop to iterate model development
A set of data tricks and techniques and how to apply them to scale your training model
How to select the right tools and techniques for your deep-learning project
Options for managing the compute infrastructure when running at scale

Who This Book Is For:
This book aims to help you develop a deeper knowledge of the Deep Learning stack—specifically, how Deep Learning interfaces with hardware, software, and data. It will serve as a valuable resource when you want to scale your deep learning model, either by expanding the hardware resources or by adding larger volumes of data or increasing the capacity of the model itself. Efficiency is a key part of any scaling operation. For this reason, consideration of efficiency is weaved in throughout the book, to provide you with the knowledge and resources you need to scale effectively.

This book is written for machine learning practitioners from all walks of life: engineers, data engineers, MLOps, Deep Learning scientists, Machine Learning engineers, and others interested in learning about model development at scale. It assumes that the reader already has a fundamental knowledge of Deep Learning concepts such as optimizers, learning objectives and loss functions, and model assembly and compilation, as well as some experience with model development. Familiarity with Python and PyTorch is also essential for the practical sections of the book.




ОТСУТСТВУЕТ ССЫЛКА/ НЕ РАБОЧАЯ ССЫЛКА ЕСТЬ РЕШЕНИЕ, ПИШИМ СЮДА!