Название: Scaling Python with Dask: From Data Science to Machine Learning (Fifth Early Release)
Автор: Holden Karau, Mika Kimmins
Издательство: O’Reilly Media, Inc.
Год: 2023-01-23
Страниц: 111
Язык: английский
Формат: pdf, epub (true), mobi
Размер: 10.1 MB

Dask is a free and open source library for parallel computing in Python that helps you scale your Data Science and Machine Learning workflows. With this quick but thorough resource, data scientists and Python programmers will learn how Dask provides APIs that make it easy to parallelize PyData libraries like NumPy, Pandas, and Scikit-learn.

Author Holden Karau shows you how you can use Dask computations in local systems and then scale to the cloud for heavier workloads. This practical book explains why Dask is popular among industry experts and academics and used by organizations that include Walmart, Capital One, Harvard Medical School, and NASA.

Dask is a framework for parallelized computing with Python that scales from multiple cores on one machine to data centers with thousands of machines. It has both low-level task APIs and higher-level data-focused APIs. The low-level task APIs power Dask’s integration with a wide variety of Python libraries. Having public APIs has allowed an ecosystem of tools to grow around Dask for various use cases.

Continuum Analytics, now known as Anaconda Inc, started the open-source DARPA funded BLAZE project, which has evolved into Dask. Continuum has participated in developing many essential libraries and even conferences in the Python data analytics space. Dask remains an open-source project, with much of its development now being supported by Coiled.

Dask is unique in the distributed computing ecosystem, by integrating popular data science, parallel, and scientific computing libraries. Dask’s integration of different libraries allows developers to re-use much of their existing knowledge at scale. You can also frequently re-use some of your code with minimal changes.

Why Do You Need Dask? Dask simplifies scaling analytics and ML code written in Python, allowing you to handle larger and more complex data and problems. Dask aims to fill the space where your existing tools, like pandas DataFrames, or your sci-kit machine learning pipelines start to become too slow (or do not succeed). While the term “big data” is perhaps less in vogue now than a few years ago, the data size of the problems has not gotten smaller, and the complexity of the computation and models have not gotten simpler. Dask allows you to primarily use the existing interfaces that you are used to (such as pandas and multiprocessing) while going beyond the scale of a single core or even a single machine.

With this book, you'll learn about
What is Dask is, where you can use it, and how it compares to other tools
Batch data parallel processing
Key distributed system concepts for Dask users
Higher-level APIs and building blocks
Integrated libraries, such as scikit-learn, pandas, and PyTorch