Название: Parallel And Distributed Computing
Автор: Ajit Singh
Издательство: Independently published
Год: 2021
Страниц: 269
Язык: английский
Формат: pdf, epub
Размер: 10.2 MB

This text is an introduction to the complex and emerging world of the Parallel and Distributed Computing. It helps you understand the principles and acquire the practical skills of MPI programming using the C/FORTAN programming language. My aim is for you to gain sufficient knowledge and experience to perform simple useful programming tasks using the best up-to-date techniques and so we hope for it to be the easiest book from which you can learn the basics of MPI programming. It helps you understand the principles, algorithm & implementation of Parallel and Distributed Computing.

This book is emphatically focused on the concept. Understanding the fundamental ideas, principles, and techniques is the essence of a good programmer. Only well-designed code has a chance of becoming part of a correct, reliable, and maintainable parallel system. Through this book, I hope that you will see the absolute necessity of understanding Parallel and Distributed Computing. I have taken a top-down approach, addressing the issues to be resolved in the design of parallel and distributed systems and describing successful approaches in the form of abstract models, algorithms and detailed case studies of widely used systems.

Parallel computing has been a subject of interest in the computing community over the last few decades. Ever-growing size of databases and increasing complexity of the new problems are putting great stress on the even the super-fast modern single processor computers. Now the entire computer science community all over the world is looking for some computing environment where current computational capacity can be enhanced by a factor in order of thousands. The most obvious solution is the introduction of multiple processors working in tandem i.e. the introduction of parallel computing. Parallel computing is a form of computation in which many calculations are carried out simultaneously, operating on the principle that large problems can often be divided into smaller ones, which are then solved concurrently ("in parallel").

Parallel computing is the simultaneous execution of the same task, split into subtasks, on multiple processors in order to obtain results faster. The idea is based on the fact that the process of solving a problem can usually be divided into smaller tasks, which maybe solved out simultaneously with some coordination mechanisms. Before going into the details of parallel computing, we shall discuss some basic concepts frequently used in parallel computing. Then we shall explain why we require parallel computing and what the levels of parallel processing are. We shall see how flow of data occurs in parallel processing. We shall conclude this unit with a discussion of role the of parallel processing in some fields like science and engineering, database queries and artificial intelligence.

Traditionally, computer software has been written for serial computation. To solve a problem, an algorithm is constructed and implemented as a serial stream of instructions. These instructions are executed on a central processing unit on one computer. Only one instruction may execute at a time - after that instruction is finished, the next is executed. Parallel computing, on the other hand, uses multiple processing elements simultaneously to solve a problem. This is accomplished by breaking the problem into independent parts so that each processing element can execute its part of the algorithm simultaneously with the others. The processing elements can be diverse and include resources such as a single computer with multiple processors, several networked computers, specialized hardware, or any combination of the above.

The book aims to provide an understanding of the principles on which the parallel and distributed computing are based; their architecture, algorithms and design; and how it meets the demands of contemporary distributed applications. I began with a set of chapters that together cover the building blocks for a study of parallel and distributed systems. The first few chapters provide a conceptual overview of the subject, outlining the characteristics of parallel and distributed systems and the challenges that must be addressed in their design: scalability, heterogeneity, security and failure handling being the most significant. These chapters also develop abstract models for understanding process interaction, failure and security.

About the author:
Ajit Singh - PhD Council Member - INSTITUT de DIPLOMATIE PUBLIQUE, United Kingdom, London.