Название: Vision Language Models: Building VLMs with Hugging Face (Early Release)
Автор: Merve Noyan, Miquel Farré, Andrés Marafioti, Orr Zohar
Издательство: O’Reilly Media, Inc.
Год: 2025-06-10
Язык: английский
Формат: epub
Размер: 10.1 MB

Vision-language models (VLMs) combine computer vision and natural language processing (NLP) to create powerful systems that can interpret, generate, and respond in multimodal contexts. Vision Language Models is a hands-on guide to building real-world VLMs using the most up-to-date stack of Machine Learning tools from Hugging Face, Meta (PyTorch), Nvidia (cuda), OpenAI (Clip), and others, written by leading researchers and practitioners Merve Noyan, Miquel Farras, Andres Marafioti, and Orr Zohar. From image captioning and document understanding to advanced zero-shot inference and retrieval-augmented generation (RAG), this book covers the full VLM application and development lifecycle.

A typical Machine Learning workflow requires either training a model from scratch or taking a pre-trained model and fine-tuning it on a dataset with low-level PyTorch implementation of model architectures. For multimodal use cases, using a pre-trained model is preferred as training from scratch is too costly. But where do you find these pre-trained models, and how hard is it to fine-tune them?

The Hugging Face ecosystem makes it easy for the community to fine-tune and share models. The success of the Transfer Learning paradigm is mostly owed to the open-source libraries of Hugging Face. Hugging Face ecosystem consists of two main components: a hub and a set of libraries that work together to simplify access to models, datasets, and demos with an easy-to-use code layer.






ОТСУТСТВУЕТ ССЫЛКА/ НЕ РАБОЧАЯ ССЫЛКА ЕСТЬ РЕШЕНИЕ, ПИШЕМ СЮДА!