Streaming Data Mesh

Streaming Data Mesh Book in PDF, ePub and Kindle version is available to download in english. Read online anytime anywhere directly from your device. Click on the download button below to get a free pdf file of Streaming Data Mesh book. This book definitely worth reading, it is an incredibly well-written.

Streaming Data Mesh

Hubert Dulay,Stephen Mooney
Streaming Data Mesh Book PDF

Author : Hubert Dulay,Stephen Mooney
Publisher : "O'Reilly Media, Inc."
Page : 230 pages
File Size : 44,8 Mb
Release : 2023-05-11
Category : Computers
ISBN : 9781098130688

Get Book

Streaming Data Mesh by Hubert Dulay,Stephen Mooney Pdf

Data lakes and warehouses have become increasingly fragile, costly, and difficult to maintain as data gets bigger and moves faster. Data meshes can help your organization decentralize data, giving ownership back to the engineers who produced it. This book provides a concise yet comprehensive overview of data mesh patterns for streaming and real-time data services. Authors Hubert Dulay and Stephen Mooney examine the vast differences between streaming and batch data meshes. Data engineers, architects, data product owners, and those in DevOps and MLOps roles will learn steps for implementing a streaming data mesh, from defining a data domain to building a good data product. Through the course of the book, you'll create a complete self-service data platform and devise a data governance system that enables your mesh to work seamlessly. With this book, you will: Design a streaming data mesh using Kafka Learn how to identify a domain Build your first data product using self-service tools Apply data governance to the data products you create Learn the differences between synchronous and asynchronous data services Implement self-services that support decentralized data

Streaming Data Mesh

Hubert Dulay,Stephen Mooney
Streaming Data Mesh Book PDF

Author : Hubert Dulay,Stephen Mooney
Publisher : "O'Reilly Media, Inc."
Page : 226 pages
File Size : 43,6 Mb
Release : 2023-05-11
Category : Computers
ISBN : 9781098130695

Get Book

Streaming Data Mesh by Hubert Dulay,Stephen Mooney Pdf

Data lakes and warehouses have become increasingly fragile, costly, and difficult to maintain as data gets bigger and moves faster. Data meshes can help your organization decentralize data, giving ownership back to the engineers who produced it. This book provides a concise yet comprehensive overview of data mesh patterns for streaming and real-time data services. Authors Hubert Dulay and Stephen Mooney examine the vast differences between streaming and batch data meshes. Data engineers, architects, data product owners, and those in DevOps and MLOps roles will learn steps for implementing a streaming data mesh, from defining a data domain to building a good data product. Through the course of the book, you'll create a complete self-service data platform and devise a data governance system that enables your mesh to work seamlessly. With this book, you will: Design a streaming data mesh using Kafka Learn how to identify a domain Build your first data product using self-service tools Apply data governance to the data products you create Learn the differences between synchronous and asynchronous data services Implement self-services that support decentralized data

Data Mesh

Zhamak Dehghani
Data Mesh Book PDF

Author : Zhamak Dehghani
Publisher : "O'Reilly Media, Inc."
Page : 387 pages
File Size : 42,9 Mb
Release : 2022-03-08
Category : Computers
ISBN : 9781492092360

Get Book

Data Mesh by Zhamak Dehghani Pdf

Many enterprises are investing in a next-generation data lake, hoping to democratize data at scale to provide business insights and ultimately make automated intelligent decisions. In this practical book, author Zhamak Dehghani reveals that, despite the time, money, and effort poured into them, data warehouses and data lakes fail when applied at the scale and speed of today's organizations. A distributed data mesh is a better choice. Dehghani guides architects, technical leaders, and decision makers on their journey from monolithic big data architecture to a sociotechnical paradigm that draws from modern distributed architecture. A data mesh considers domains as a first-class concern, applies platform thinking to create self-serve data infrastructure, treats data as a product, and introduces a federated and computational model of data governance. This book shows you why and how. Examine the current data landscape from the perspective of business and organizational needs, environmental challenges, and existing architectures Analyze the landscape's underlying characteristics and failure modes Get a complete introduction to data mesh principles and its constituents Learn how to design a data mesh architecture Move beyond a monolithic data lake to a distributed data mesh.

Building Event-Driven Microservices

Adam Bellemare
Building Event Driven Microservices Book PDF

Author : Adam Bellemare
Publisher : "O'Reilly Media, Inc."
Page : 347 pages
File Size : 43,7 Mb
Release : 2020-07-02
Category : Computers
ISBN : 9781492057840

Get Book

Building Event-Driven Microservices by Adam Bellemare Pdf

Organizations today often struggle to balance business requirements with ever-increasing volumes of data. Additionally, the demand for leveraging large-scale, real-time data is growing rapidly among the most competitive digital industries. Conventional system architectures may not be up to the task. With this practical guide, you’ll learn how to leverage large-scale data usage across the business units in your organization using the principles of event-driven microservices. Author Adam Bellemare takes you through the process of building an event-driven microservice-powered organization. You’ll reconsider how data is produced, accessed, and propagated across your organization. Learn powerful yet simple patterns for unlocking the value of this data. Incorporate event-driven design and architectural principles into your own systems. And completely rethink how your organization delivers value by unlocking near-real-time access to data at scale. You’ll learn: How to leverage event-driven architectures to deliver exceptional business value The role of microservices in supporting event-driven designs Architectural patterns to ensure success both within and between teams in your organization Application patterns for developing powerful event-driven microservices Components and tooling required to get your microservice ecosystem off the ground

Engineering Data Mesh in Azure Cloud

Aniruddha Deswandikar
Engineering Data Mesh in Azure Cloud Book PDF

Author : Aniruddha Deswandikar
Publisher : Packt Publishing Ltd
Page : 314 pages
File Size : 50,8 Mb
Release : 2024-03-29
Category : Computers
ISBN : 9781805128946

Get Book

Engineering Data Mesh in Azure Cloud by Aniruddha Deswandikar Pdf

Overcome data mesh adoption challenges using the cloud-scale analytics framework and make your data analytics landscape agile and efficient by using standard architecture patterns for diverse analytical workloads Key Features Delve into core data mesh concepts and apply them to real-world situations Safely reassess and redesign your framework for seamless data mesh integration Conquer practical challenges, from domain organization to building data contracts Purchase of the print or Kindle book includes a free PDF eBook Book DescriptionDecentralizing data and centralizing governance are practical, scalable, and modern approaches to data analytics. However, implementing a data mesh can feel like changing the engine of a moving car. Most organizations struggle to start and get caught up in the concept of data domains, spending months trying to organize domains. This is where Engineering Data Mesh in Azure Cloud can help. The book starts by assessing your existing framework before helping you architect a practical design. As you progress, you’ll focus on the Microsoft Cloud Adoption Framework for Azure and the cloud-scale analytics framework, which will help you quickly set up a landing zone for your data mesh in the cloud. The book also resolves common challenges related to the adoption and implementation of a data mesh faced by real customers. It touches on the concepts of data contracts and helps you build practical data contracts that work for your organization. The last part of the book covers some common architecture patterns used for modern analytics frameworks such as artificial intelligence (AI). By the end of this book, you’ll be able to transform existing analytics frameworks into a streamlined data mesh using Microsoft Azure, thereby navigating challenges and implementing advanced architecture patterns for modern analytics workloads.What you will learn Build a strategy to implement a data mesh in Azure Cloud Plan your data mesh journey to build a collaborative analytics platform Address challenges in designing, building, and managing data contracts Get to grips with monitoring and governing a data mesh Understand how to build a self-service portal for analytics Design and implement a secure data mesh architecture Resolve practical challenges related to data mesh adoption Who this book is for This book is for chief data officers and data architects of large and medium-size organizations who are struggling to maintain silos of data and analytics projects. Data architects and data engineers looking to understand data mesh and how it can help their organizations democratize data and analytics will also benefit from this book. Prior knowledge of managing centralized analytical systems, as well as experience with building data lakes, data warehouses, data pipelines, data integrations, and transformations is needed to get the most out of this book.

Building an Event-Driven Data Mesh

Adam Bellemare
Building an Event Driven Data Mesh Book PDF

Author : Adam Bellemare
Publisher : "O'Reilly Media, Inc."
Page : 260 pages
File Size : 45,7 Mb
Release : 2023-04-04
Category : Computers
ISBN : 9781098127565

Get Book

Building an Event-Driven Data Mesh by Adam Bellemare Pdf

The exponential growth of data combined with the need to derive real-time business value is a critical issue today. An event-driven data mesh can power real-time operational and analytical workloads, all from a single set of data product streams. With practical real-world examples, this book shows you how to successfully design and build an event-driven data mesh. Building an Event-Driven Data Mesh provides: Practical tips for iteratively building your own event-driven data mesh, including hurdles you'll experience, possible solutions, and how to obtain real value as soon as possible Solutions to pitfalls you may encounter when moving your organization from monoliths to event-driven architectures A clear understanding of how events relate to systems and other events in the same stream and across streams A realistic look at event modeling options, such as fact, delta, and command type events, including how these choices will impact your data products Best practices for handling events at scale, privacy, and regulatory compliance Advice on asynchronous communication and handling eventual consistency

Data Mesh in Action

Jacek Majchrzak,Sven Balnojan,Marian Siwiak
Data Mesh in Action Book PDF

Author : Jacek Majchrzak,Sven Balnojan,Marian Siwiak
Publisher : Simon and Schuster
Page : 326 pages
File Size : 47,9 Mb
Release : 2023-02-14
Category : Computers
ISBN : 9781633439979

Get Book

Data Mesh in Action by Jacek Majchrzak,Sven Balnojan,Marian Siwiak Pdf

Revolutionize the way your organization approaches data with a data mesh! This new decentralized architecture outpaces monolithic lakes and warehouses and can work for a company of any size. Data Mesh in Action teaches you to establish a data mesh in organizations of any size. The book avoids a dogmatic one-size-fits-all approach and utilizes flexible "sliders" to adjust a data mesh to your company's specific needs. You'll learn processes and facilitative leadership techniques that will help change the way your colleagues think about data. Data Mesh in Action reveals how this groundbreaking architecture looks for both small startups and large enterprises. You'll see a data mesh in action as you explore both an extended case study and multiple real-world examples. As you go, you'll be expertly guided through discussions around Socio-Technical Architecture and Domain- Driven Design with the goal of building a sleek data-as-a-product system. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications.

Streaming Data

Andrew Psaltis
Streaming Data Book PDF

Author : Andrew Psaltis
Publisher : Simon and Schuster
Page : 314 pages
File Size : 41,9 Mb
Release : 2017-05-31
Category : Computers
ISBN : 9781638357247

Get Book

Streaming Data by Andrew Psaltis Pdf

Summary Streaming Data introduces the concepts and requirements of streaming and real-time data systems. The book is an idea-rich tutorial that teaches you to think about how to efficiently interact with fast-flowing data. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. About the Technology As humans, we're constantly filtering and deciphering the information streaming toward us. In the same way, streaming data applications can accomplish amazing tasks like reading live location data to recommend nearby services, tracking faults with machinery in real time, and sending digital receipts before your customers leave the shop. Recent advances in streaming data technology and techniques make it possible for any developer to build these applications if they have the right mindset. This book will let you join them. About the Book Streaming Data is an idea-rich tutorial that teaches you to think about efficiently interacting with fast-flowing data. Through relevant examples and illustrated use cases, you'll explore designs for applications that read, analyze, share, and store streaming data. Along the way, you'll discover the roles of key technologies like Spark, Storm, Kafka, Flink, RabbitMQ, and more. This book offers the perfect balance between big-picture thinking and implementation details. What's Inside The right way to collect real-time data Architecting a streaming pipeline Analyzing the data Which technologies to use and when About the Reader Written for developers familiar with relational database concepts. No experience with streaming or real-time applications required. About the Author Andrew Psaltis is a software engineer focused on massively scalable real-time analytics. Table of Contents PART 1 - A NEW HOLISTIC APPROACH Introducing streaming data Getting data from clients: data ingestion Transporting the data from collection tier: decoupling the data pipeline Analyzing streaming data Algorithms for data analysis Storing the analyzed or collected data Making the data available Consumer device capabilities and limitations accessing the data PART 2 - TAKING IT REAL WORLD Analyzing Meetup RSVPs in real time

Data Management at Scale

Piethein Strengholt
Data Management at Scale Book PDF

Author : Piethein Strengholt
Publisher : "O'Reilly Media, Inc."
Page : 404 pages
File Size : 54,8 Mb
Release : 2020-07-29
Category : Computers
ISBN : 9781492054733

Get Book

Data Management at Scale by Piethein Strengholt Pdf

As data management and integration continue to evolve rapidly, storing all your data in one place, such as a data warehouse, is no longer scalable. In the very near future, data will need to be distributed and available for several technological solutions. With this practical book, you’ll learnhow to migrate your enterprise from a complex and tightly coupled data landscape to a more flexible architecture ready for the modern world of data consumption. Executives, data architects, analytics teams, and compliance and governance staff will learn how to build a modern scalable data landscape using the Scaled Architecture, which you can introduce incrementally without a large upfront investment. Author Piethein Strengholt provides blueprints, principles, observations, best practices, and patterns to get you up to speed. Examine data management trends, including technological developments, regulatory requirements, and privacy concerns Go deep into the Scaled Architecture and learn how the pieces fit together Explore data governance and data security, master data management, self-service data marketplaces, and the importance of metadata

Kafka: The Definitive Guide

Neha Narkhede,Gwen Shapira,Todd Palino
Kafka The Definitive Guide Book PDF

Author : Neha Narkhede,Gwen Shapira,Todd Palino
Publisher : "O'Reilly Media, Inc."
Page : 374 pages
File Size : 47,5 Mb
Release : 2017-08-31
Category : Computers
ISBN : 9781491936115

Get Book

Kafka: The Definitive Guide by Neha Narkhede,Gwen Shapira,Todd Palino Pdf

Every enterprise application creates data, whether it’s log messages, metrics, user activity, outgoing messages, or something else. And how to move all of this data becomes nearly as important as the data itself. If you’re an application architect, developer, or production engineer new to Apache Kafka, this practical guide shows you how to use this open source streaming platform to handle real-time data feeds. Engineers from Confluent and LinkedIn who are responsible for developing Kafka explain how to deploy production Kafka clusters, write reliable event-driven microservices, and build scalable stream-processing applications with this platform. Through detailed examples, you’ll learn Kafka’s design principles, reliability guarantees, key APIs, and architecture details, including the replication protocol, the controller, and the storage layer. Understand publish-subscribe messaging and how it fits in the big data ecosystem. Explore Kafka producers and consumers for writing and reading messages Understand Kafka patterns and use-case requirements to ensure reliable data delivery Get best practices for building data pipelines and applications with Kafka Manage Kafka in production, and learn to perform monitoring, tuning, and maintenance tasks Learn the most critical metrics among Kafka’s operational measurements Explore how Kafka’s stream delivery capabilities make it a perfect source for stream processing systems

I Heart Logs

Jay Kreps
I Heart Logs Book PDF

Author : Jay Kreps
Publisher : "O'Reilly Media, Inc."
Page : 60 pages
File Size : 53,7 Mb
Release : 2014-09-23
Category : Computers
ISBN : 9781491909331

Get Book

I Heart Logs by Jay Kreps Pdf

Why a book about logs? That’s easy: the humble log is an abstraction that lies at the heart of many systems, from NoSQL databases to cryptocurrencies. Even though most engineers don’t think much about them, this short book shows you why logs are worthy of your attention. Based on his popular blog posts, LinkedIn principal engineer Jay Kreps shows you how logs work in distributed systems, and then delivers practical applications of these concepts in a variety of common uses—data integration, enterprise architecture, real-time stream processing, data system design, and abstract computing models. Go ahead and take the plunge with logs; you’re going love them. Learn how logs are used for programmatic access in databases and distributed systems Discover solutions to the huge data integration problem when more data of more varieties meet more systems Understand why logs are at the heart of real-time stream processing Learn the role of a log in the internals of online data systems Explore how Jay Kreps applies these ideas to his own work on data infrastructure systems at LinkedIn

Kafka Streams in Action

Bill Bejeck
Kafka Streams in Action Book PDF

Author : Bill Bejeck
Publisher : Simon and Schuster
Page : 410 pages
File Size : 43,7 Mb
Release : 2018-08-29
Category : Computers
ISBN : 9781638356028

Get Book

Kafka Streams in Action by Bill Bejeck Pdf

Summary Kafka Streams in Action teaches you everything you need to know to implement stream processing on data flowing into your Kafka platform, allowing you to focus on getting more from your data without sacrificing time or effort. Foreword by Neha Narkhede, Cocreator of Apache Kafka Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. About the Technology Not all stream-based applications require a dedicated processing cluster. The lightweight Kafka Streams library provides exactly the power and simplicity you need for message handling in microservices and real-time event processing. With the Kafka Streams API, you filter and transform data streams with just Kafka and your application. About the Book Kafka Streams in Action teaches you to implement stream processing within the Kafka platform. In this easy-to-follow book, you'll explore real-world examples to collect, transform, and aggregate data, work with multiple processors, and handle real-time events. You'll even dive into streaming SQL with KSQL! Practical to the very end, it finishes with testing and operational aspects, such as monitoring and debugging. What's inside Using the KStreams API Filtering, transforming, and splitting data Working with the Processor API Integrating with external systems About the Reader Assumes some experience with distributed systems. No knowledge of Kafka or streaming applications required. About the Author Bill Bejeck is a Kafka Streams contributor and Confluent engineer with over 15 years of software development experience. Table of Contents PART 1 - GETTING STARTED WITH KAFKA STREAMS Welcome to Kafka Streams Kafka quicklyPART 2 - KAFKA STREAMS DEVELOPMENT Developing Kafka Streams Streams and state The KTable API The Processor APIPART 3 - ADMINISTERING KAFKA STREAMS Monitoring and performance Testing a Kafka Streams applicationPART 4 - ADVANCED CONCEPTS WITH KAFKA STREAMS Advanced applications with Kafka StreamsAPPENDIXES Appendix A - Additional configuration information Appendix B - Exactly once semantics

Multi-Cloud Strategy for Cloud Architects

Jeroen Mulder
Multi Cloud Strategy for Cloud Architects Book PDF

Author : Jeroen Mulder
Publisher : Packt Publishing Ltd
Page : 471 pages
File Size : 48,6 Mb
Release : 2023-04-27
Category : Computers
ISBN : 9781804616093

Get Book

Multi-Cloud Strategy for Cloud Architects by Jeroen Mulder Pdf

Solve the complexity of running a business in a multi-cloud environment with practical guidelines backed by industry experience. Purchase of the print or Kindle book includes a free eBook in PDF format. Key Features Explore the benefits of the major cloud providers to make better informed decisions Accelerate digital transformation with multi-cloud, including the use of PaaS and SaaS concepts Get the best out of multi-cloud by exploring relevant use cases for data platforms and IoT Unlock insights into top 5 cloud providers in one book - Azure, AWS, GCP, OCI, and Alibaba Cloud Book Description Are you ready to unlock the full potential of your enterprise with the transformative power of multi-cloud adoption? As a cloud architect, you understand the challenges of navigating the vast array of cloud services and moving data and applications to public clouds. But with 'Multi-Cloud Strategy for Cloud Architects, Second Edition', you'll gain the confidence to tackle these complexities head-on. This edition delves into the latest concepts of BaseOps, FinOps, and DevSecOps, including the use of the DevSecOps Maturity Model. You'll learn how to optimize costs and maximize security using the major public clouds - Azure, AWS, and Google Cloud. Examples of solutions by the increasingly popular Oracle Cloud Infrastructure (OCI) and Alibaba Cloud have been added in this edition. Plus, you will discover cutting-edge ideas like AIOps and GreenOps. With practical use cases, including IoT, data mining, Web3, and financial management, this book empowers you with the skills needed to develop, release, and manage products and services in a multi-cloud environment. By the end of this book, you'll have mastered the intricacies of multi-cloud operations, financial management, and security. Don't miss your chance to revolutionize your enterprise with multi-cloud adoption. What you will learn Choose the right cloud platform with the help of use cases Master multi-cloud concepts, including IaC, SaaS, PaaS, and CaC Use the techniques and tools offered by Azure, AWS, and GCP to integrate security Maximize cloud potential with Azure, AWS, and GCP frameworks for enterprise architecture Use FinOps to define cost models and optimize cloud costs with showback and chargeback Who this book is for Cloud architects, solutions architects, enterprise architects, and cloud consultants will find this book valuable. Basic knowledge of any one of the major public clouds (Azure, AWS, or GCP) will be helpful.

Data Mesh in Action

Jacek Majchrzak,Sven Balnojan,Marian Siwiak
Data Mesh in Action Book PDF

Author : Jacek Majchrzak,Sven Balnojan,Marian Siwiak
Publisher : Simon and Schuster
Page : 326 pages
File Size : 51,8 Mb
Release : 2023-03-21
Category : Computers
ISBN : 9781638351849

Get Book

Data Mesh in Action by Jacek Majchrzak,Sven Balnojan,Marian Siwiak Pdf

Revolutionize the way your organization approaches data with a data mesh! This new decentralized architecture outpaces monolithic lakes and warehouses and can work for a company of any size. In Data Mesh in Action you will learn how to: Implement a data mesh in your organization Turn data into a data product Move from your current data architecture to a data mesh Identify data domains, and decompose an organization into smaller, manageable domains Set up the central governance and local governance levels over data Balance responsibilities between the two levels of governance Establish a platform that allows efficient connection of distributed data products and automated governance Data Mesh in Action reveals how this groundbreaking architecture looks for both small startups and large enterprises. You won’t need any new technology—this book shows you how to start implementing a data mesh with flexible processes and organizational change. You’ll explore both an extended case study and multiple real-world examples. As you go, you’ll be expertly guided through discussions around Socio-Technical Architecture and Domain-Driven Design with the goal of building a sleek data-as-a-product system. Plus, dozens of workshop techniques for both in-person and remote meetings help you onboard colleagues and drive a successful transition. About the technology Business increasingly relies on efficiently storing and accessing large volumes of data. The data mesh is a new way to decentralize data management that radically improves security and discoverability. A well-designed data mesh simplifies self-service data consumption and reduces the bottlenecks created by monolithic data architectures. About the book Data Mesh in Action teaches you pragmatic ways to decentralize your data and organize it into an effective data mesh. You’ll start by building a minimum viable data product, which you’ll expand into a self-service data platform, chapter-by-chapter. You’ll love the book’s unique “sliders” that adjust the mesh to meet your specific needs. You’ll also learn processes and leadership techniques that will change the way you and your colleagues think about data. What's inside Decompose an organization into manageable domains Turn data into a data product Set up central and local governance levels Build a fit-for-purpose data platform Improve management, initiation, and support techniques About the reader For data professionals. Requires no specific programming stack or data platform. About the author Jacek Majchrzak is a hands-on lead data architect. Dr. Sven Balnojan manages data products and teams. Dr. Marian Siwiak is a data scientist and a management consultant for IT, scientific, and technical projects. Table of Contents PART 1 FOUNDATIONS 1 The what and why of the data mesh 2 Is a data mesh right for you? 3 Kickstart your data mesh MVP in a month PART 2 THE FOUR PRINCIPLES IN PRACTICE 4 Domain ownership 5 Data as a product 6 Federated computational governance 7 The self-serve data platform PART 3 INFRASTRUCTURE AND TECHNICAL ARCHITECTURE 8 Comparing self-serve data platforms 9 Solution architecture design

Spark in Action

Jean-Georges Perrin
Spark in Action Book PDF

Author : Jean-Georges Perrin
Publisher : Simon and Schuster
Page : 574 pages
File Size : 52,6 Mb
Release : 2020-05-12
Category : Computers
ISBN : 9781638351306

Get Book

Spark in Action by Jean-Georges Perrin Pdf

Summary The Spark distributed data processing platform provides an easy-to-implement tool for ingesting, streaming, and processing data from any source. In Spark in Action, Second Edition, you’ll learn to take advantage of Spark’s core features and incredible processing speed, with applications including real-time computation, delayed evaluation, and machine learning. Spark skills are a hot commodity in enterprises worldwide, and with Spark’s powerful and flexible Java APIs, you can reap all the benefits without first learning Scala or Hadoop. Foreword by Rob Thomas. About the technology Analyzing enterprise data starts by reading, filtering, and merging files and streams from many sources. The Spark data processing engine handles this varied volume like a champ, delivering speeds 100 times faster than Hadoop systems. Thanks to SQL support, an intuitive interface, and a straightforward multilanguage API, you can use Spark without learning a complex new ecosystem. About the book Spark in Action, Second Edition, teaches you to create end-to-end analytics applications. In this entirely new book, you’ll learn from interesting Java-based examples, including a complete data pipeline for processing NASA satellite data. And you’ll discover Java, Python, and Scala code samples hosted on GitHub that you can explore and adapt, plus appendixes that give you a cheat sheet for installing tools and understanding Spark-specific terms. What's inside Writing Spark applications in Java Spark application architecture Ingestion through files, databases, streaming, and Elasticsearch Querying distributed datasets with Spark SQL About the reader This book does not assume previous experience with Spark, Scala, or Hadoop. About the author Jean-Georges Perrin is an experienced data and software architect. He is France’s first IBM Champion and has been honored for 12 consecutive years. Table of Contents PART 1 - THE THEORY CRIPPLED BY AWESOME EXAMPLES 1 So, what is Spark, anyway? 2 Architecture and flow 3 The majestic role of the dataframe 4 Fundamentally lazy 5 Building a simple app for deployment 6 Deploying your simple app PART 2 - INGESTION 7 Ingestion from files 8 Ingestion from databases 9 Advanced ingestion: finding data sources and building your own 10 Ingestion through structured streaming PART 3 - TRANSFORMING YOUR DATA 11 Working with SQL 12 Transforming your data 13 Transforming entire documents 14 Extending transformations with user-defined functions 15 Aggregating your data PART 4 - GOING FURTHER 16 Cache and checkpoint: Enhancing Spark’s performances 17 Exporting data and building full data pipelines 18 Exploring deployment