Apache Spark Implementation On Ibm Z Os Book in PDF, ePub and Kindle version is available to download in english. Read online anytime anywhere directly from your device. Click on the download button below to get a free pdf file of Apache Spark Implementation On Ibm Z Os book. This book definitely worth reading, it is an incredibly well-written.
Apache Spark Implementation on IBM z/OS by Lydia Parziale,Joe Bostian,Ravi Kumar,Ulrich Seelbach,Zhong Yu Ye,IBM Redbooks Pdf
The term big data refers to extremely large sets of data that are analyzed to reveal insights, such as patterns, trends, and associations. The algorithms that analyze this data to provide these insights must extract value from a wide range of data sources, including business data and live, streaming, social media data. However, the real value of these insights comes from their timeliness. Rapid delivery of insights enables anyone (not only data scientists) to make effective decisions, applying deep intelligence to every enterprise application. Apache Spark is an integrated analytics framework and runtime to accelerate and simplify algorithm development, depoyment, and realization of business insight from analytics. Apache Spark on IBM® z/OS® puts the open source engine, augmented with unique differentiated features, built specifically for data science, where big data resides. This IBM Redbooks® publication describes the installation and configuration of IBM z/OS Platform for Apache Spark for field teams and clients. Additionally, it includes examples of business analytics scenarios.
Apache Spark for the Enterprise: Setting the Business Free by Oliver Draese,Eberhard Hechler,Hong Min,Catherine Moxey,Pallavi Priyadarshini,Mark Simmonds,Mythili Venkatakrishnan,George Wang,IBM Redbooks Pdf
Analytics is increasingly an integral part of day-to-day operations at today's leading businesses, and transformation is also occurring through huge growth in mobile and digital channels. Enterprise organizations are attempting to leverage analytics in new ways and transition existing analytics capabilities to respond with more flexibility while making the most efficient use of highly valuable data science skills. The recent growth and adoption of Apache Spark as an analytics framework and platform is very timely and helps meet these challenging demands. The Apache Spark environment on IBM z/OS® and Linux on IBM z SystemsTM platforms allows this analytics framework to run on the same enterprise platform as the originating sources of data and transactions that feed it. If most of the data that will be used for Apache Spark analytics, or the most sensitive or quickly changing data is originating on z/OS, then an Apache Spark z/OS based environment will be the optimal choice for performance, security, and governance. This IBM® RedpaperTM publication explores the enterprise analytics market, use of Apache Spark on IBM z SystemsTM platforms, integration between Apache Spark and other enterprise data sources, and case studies and examples of what can be achieved with Apache Spark in enterprise environments. It is of interest to data scientists, data engineers, enterprise architects, or anybody looking to better understand how to combine an analytics framework and platform on enterprise systems.
Installing and Configuring IBM Db2 AI for IBM z/OS v1.4.0 by Tim Hogan,Lydia Parziale,Guanjun Cai,Janet Figone,IBM Redbooks Pdf
Artificial intelligence (AI) enables computers and machines to mimic the perception, learning, problem-solving, and decision-making capabilities of the human mind. AI development is made possible by the availability of large amounts of data and the corresponding development and wide availability of computer systems that can process all that data faster and more accurately than humans can. What happens if you infuse AI with a world-class database management system, such as IBM Db2®? IBM® has done just that with Db2 AI for z/OS (Db2ZAI). Db2ZAI is built to infuse AI and data science to assist businesses in the use of AI to develop applications more easily. With Db2ZAI, the following benefits are realized: Data science functionality Better built applications Improved database performance (and DBA's time and efforts are saved) through simplification and automation of error reporting and routine tasks Machine learning (ML) optimizer to improve query access paths and reduce the need for manual tuning and query optimization Integrated data access that makes data available from various vendors including private cloud providers. This IBM Redpaper® publication helps to simplify your installation by tailoring and configuration of Db2 AI for z/OS®. It was written for system programmers, system administrators, and database administrators.
Securely Leverage Open-Source Software with Python AI Toolkit for IBM z/OS by Joe Bostian,Evan Rivera,IBM Redbooks Pdf
Open-source software (OSS) is widely available and serves as an essential component for enterprises in the artificial intelligence (AI) and machine learning (ML) industry. Specifically, the open-source programming language Python is one of the most versatile and popular programming languages that are used in the world at the time of writing. This situation is especially true in the data science community, where Python provides many libraries and tools that enable essential AI and ML functions, and where it is supported by a large community of developers that actively contribute to its development. Understanding and managing vulnerabilities within OSS can be complex because of the many components, dependencies, and contributors that are involved. Although the nature of OSS helps balance access to programming and technology, it also results in fast-paced changes to software, which emphasizes the importance of software currency to minimize security concerns. Enterprises understand the critical need to have access to and leverage reputable open-source projects with proper maintenance, updates, transparency, reliable support, and a sense of control to form a secure foundation for implementing AI solutions. Python AI Toolkit for IBM® z/OS® is a powerful set of tools and libraries that is used to establish a secure foundation for AI development and deployment on z/OS so that enterprises can leverage their existing infrastructure for these mission-critical applications. The OSS that is provided within Python AI Toolkit for IBM z/OS is scanned and vetted for security vulnerabilities so that users can make informed decisions when leveraging these Python packages. Packages can be installed and managed by using the Package Installer for Python (pip), which is a common Python package manager, enabling a familiar, flexible, and agile delivery experience while empowering developers to build AI solutions.
IBM z15 (8562) Technical Guide by Octavian Lascu,IBM Redbooks Pdf
This IBM® Redbooks® publication describes the features and functions the latest member of the IBM Z® platform, the IBM z15TM Model T02 (machine type 8562). It includes information about the IBM z15 processor design, I/O innovations, security features, and supported operating systems. The z15 is a state-of-the-art data and transaction system that delivers advanced capabilities, which are vital to any digital transformation. The z15 is designed for enhanced modularity, which is in an industry standard footprint. This system excels at the following tasks: Making use of multicloud integration services Securing data with pervasive encryption Accelerating digital transformation with agile service delivery Transforming a transactional platform into a data powerhouse Getting more out of the platform with IT Operational Analytics Accelerating digital transformation with agile service delivery Revolutionizing business processes Blending open source and Z technologies This book explains how this system uses new innovations and traditional Z strengths to satisfy growing demand for cloud, analytics, and open source technologies. With the z15 as the base, applications can run in a trusted, reliable, and secure environment that improves operations and lessens business risk.
IBM z15 (8561) Technical Guide by Octavian Lascu,John Troy,Jannie Houlbjerg,Frank Packheiser,Paul Schouten,Kazuhiro Nakajima,Anna Shugol,Hervey Kamga,Bo XU,IBM Redbooks Pdf
This IBM® Redbooks® publication describes the features and functions the latest member of the IBM Z® platform, the IBM z15TM (machine type 8561). It includes information about the IBM z15 processor design, I/O innovations, security features, and supported operating systems. The z15 is a state-of-the-art data and transaction system that delivers advanced capabilities, which are vital to any digital transformation. The z15 is designed for enhanced modularity, which is in an industry standard footprint. This system excels at the following tasks: Making use of multicloud integration services Securing data with pervasive encryption Accelerating digital transformation with agile service delivery Transforming a transactional platform into a data powerhouse Getting more out of the platform with IT Operational Analytics Accelerating digital transformation with agile service delivery Revolutionizing business processes Blending open source and Z technologies This book explains how this system uses new innovations and traditional Z strengths to satisfy growing demand for cloud, analytics, and open source technologies. With the z15 as the base, applications can run in a trusted, reliable, and secure environment that improves operations and lessens business risk.
DB2 12 for z Optimizer by Terry Purcell,IBM Redbooks Pdf
There has been a considerable focus on performance improvements as one of the main themes in recent IBM DB2® releases, and DB2 12 for IBM z/OS® is certainly no exception. With the high-value data retained on DB2 for z/OS and the z Systems platform, customers are increasingly attempting to extract value from that data for competitive advantage. Although customers have historically moved data off platform to gain insight, the landscape has changed significantly and allowed z Systems to again converge operational systems with analytics for real-time insight. Business-critical analytics is now requiring the same levels of service as expected for operational systems, and real-time or near real-time currency of data is expected. Hence the resurgence of z Systems. As a precursor to this shift, IDAA brought the data warehouse back to DB2 for z/OS and, with its tight integration with DB2, significantly reduces data latency as compared to the ETL processing that is involved with moving data to a stand-alone data warehouse environment. That change has opened up new opportunities for operational systems to extend the breadth of analytics processing without affecting the mission-critical system and integrating near real-time analytics within that system, all while maintaining the same z Systems qualities of service. Apache Spark on z/OS and Linux for System z also allow analytics in-place, in real-time or near real-time. Enabling Spark natively on z Systems reduces the security risk of multiple copies of the Enterprise data, while providing an application developer-friendly platform for faster insight in a simplified and more secure analytics framework. How is all of this relevant to DB2 for z/OS? Given that z Systems is proving again to be the core Enterprise Hybrid Transactional/Analytical Processing (HTAP) system, it is critical that DB2 for z/OS can handle its traditional transactional applications and address the requirements for analytics processing that might not be candidates for these rapidly evolving targeted analytics systems. And not only are there opportunities for DB2 for z/OS to play an increasing role in analytics, the complexity of the transactional systems is increasing. Analytics is being integrated within the scope of those transactions. DB2 12 for z/OS has targeted performance to increase the success of new application deployments and integration of analytics to ensure that we keep pace with the rapid evolution of IDAA and Spark as equal partners in HTAP systems. This paper describes the enhancements delivered specifically by the query processing engine of DB2. This engine is generally called the optimizer or the Relational Data Services (RDS) components, which encompasses the query transformation, access path selection, run time, and parallelism. DB2 12 for z/OS also delivers improvements targeted at OLTP applications, which are the realm of the Data Manager, Index Manager, and Buffer Manager components (to name a few), and are not identified here. Although the performance measurement focus is based on reducing CPU, improvement in elapsed time is likely to be similarly achieved as CPU is reduced and performance constraints alleviated. However, elapsed time improvements can be achieved with parallelism, and DB2 12 does increase the percentage offload for parallel child tasks, which can further reduce chargeable CPU for analytics workloads.
Optimized Inferencing and Integration with AI on IBM zSystems: Introduction, Methodology, and Use Cases by Makenzie Manna,Erhan Mengusoglu,Artem Minin,Krishna Teja Rekapalli,Thomas Rüter,Pia Velazco,Markus Wolff,IBM Redbooks Pdf
In today's fast-paced, ever-growing digital world, you face various new and complex business problems. To help resolve these problems, enterprises are embedding artificial intelligence (AI) into their mission-critical business processes and applications to help improve operations, optimize performance, personalize the user experience, and differentiate themselves from the competition. Furthermore, the use of AI on the IBM® zSystems platform, where your mission-critical transactions, data, and applications are installed, is a key aspect of modernizing business-critical applications while maintaining strict service-level agreements (SLAs) and security requirements. This colocation of data and AI empowers your enterprise to optimally and easily deploy and infuse AI capabilities into your enterprise workloads with the most recent and relevant data available in real time, which enables a more transparent, accurate, and dependable AI experience. This IBM Redpaper publication introduces and explains AI technologies and hardware optimizations, and demonstrates how to leverage certain capabilities and components to enable AI solutions in business-critical use cases, such as fraud detection and credit risk scoring, on the platform. Real-time inferencing with AI models, a capability that is critical to certain industries and use cases, now can be implemented with optimized performance thanks to innovations like IBM zSystems Integrated Accelerator for AI embedded in the Telum chip within IBM z16TM. This publication describes and demonstrates the implementation and integration of the two end-to-end solutions (fraud detection and credit risk), from developing and training the AI models to deploying the models in an IBM z/OS® V2R5 environment on IBM z16 hardware, and integrating AI functions into an application, for example an IBM z/OS Customer Information Control System (IBM CICS®) application. We describe performance optimization recommendations and considerations when leveraging AI technology on the IBM zSystems platform, including optimizations for micro-batching in IBM Watson® Machine Learning for z/OS. The benefits that are derived from the solutions also are described in detail, including how the open-source AI framework portability of the IBM zSystems platform enables model development and training to be done anywhere, including on IBM zSystems, and enables easy integration to deploy on IBM zSystems for optimal inferencing. Thus, allowing enterprises to uncover insights at the transaction-level while taking advantage of the speed, depth, and securability of the platform. This publication is intended for technical specialists, site reliability engineers, architects, system programmers, and systems engineers. Technologies that are covered include TensorFlow Serving, WMLz, IBM Cloud Pak® for Data (CP4D), IBM z/OS Container Extensions (zCX), IBM CICS, Open Neural Network Exchange (ONNX), and IBM Deep Learning Compiler (zDLC).
IBM Data Engine for Hadoop and Spark by Dino Quintero,Luis Bolinches,Aditya Gandakusuma Sutandyo,Nicolas Joly,Reinaldo Tetsuo Katahira,IBM Redbooks Pdf
This IBM® Redbooks® publication provides topics to help the technical community take advantage of the resilience, scalability, and performance of the IBM Power SystemsTM platform to implement or integrate an IBM Data Engine for Hadoop and Spark solution for analytics solutions to access, manage, and analyze data sets to improve business outcomes. This book documents topics to demonstrate and take advantage of the analytics strengths of the IBM POWER8® platform, the IBM analytics software portfolio, and selected third-party tools to help solve customer's data analytic workload requirements. This book describes how to plan, prepare, install, integrate, manage, and show how to use the IBM Data Engine for Hadoop and Spark solution to run analytic workloads on IBM POWER8. In addition, this publication delivers documentation to complement available IBM analytics solutions to help your data analytic needs. This publication strengthens the position of IBM analytics and big data solutions with a well-defined and documented deployment model within an IBM POWER8 virtualized environment so that customers have a planned foundation for security, scaling, capacity, resilience, and optimization for analytics workloads. This book is targeted at technical professionals (analytics consultants, technical support staff, IT Architects, and IT Specialists) that are responsible for delivering analytics solutions and support on IBM Power Systems.
Turning Data into Insight with IBM Machine Learning for z/OS by Samantha Buhler,Guanjun Cai,John Goodyear,Edrian Irizarry,Nora Kissari,Zhuo Ling,Nicholas Marion,Aleksandr Petrov,Junfei Shen,Wanting Wang,He Sheng Yang,Dai Yi,Xavier Yuen,Hao Zhang,IBM Redbooks Pdf
The exponential growth in data over the last decade coupled with a drastic drop in cost of storage has enabled organizations to amass a large amount of data. This vast data becomes the new natural resource that these organizations must tap in to innovate and stay ahead of the competition, and they must do so in a secure environment that protects the data throughout its lifecyle and data access in real time at any time. When it comes to security, nothing can rival IBM® Z, the multi-workload transactional platform that powers the core business processes of the majority of the Fortune 500 enterprises with unmatched security, availability, reliability, and scalability. With core transactions and data originating on IBM Z, it simply makes sense for analytics to exist and run on the same platform. For years, some businesses chose to move their sensitive data off IBM Z to platforms that include data lakes, Hadoop, and warehouses for analytics processing. However, the massive growth of digital data, the punishing cost of security exposures as well as the unprecedented demand for instant actionable intelligence from data in real time have convinced them to rethink that decision and, instead, embrace the strategy of data gravity for analytics. At the core of data gravity is the conviction that analytics must exist and run where the data resides. An IBM client eloquently compares this change in analytics strategy to a shift from "moving the ocean to the boat to moving the boat to the ocean," where the boat is the analytics and the ocean is the data. IBM respects and invests heavily on data gravity because it recognizes the tremendous benefits that data gravity can deliver to you, including reduced cost and minimized security risks. IBM Machine Learning for z/OS® is one of the offerings that decidedly move analytics to Z where your mission-critical data resides. In the inherently secure Z environment, your machine learning scoring services can co-exist with your transactional applications and data, supporting high throughput and minimizing response time while delivering consistent service level agreements (SLAs). This book introduces Machine Learning for z/OS version 1.1.0 and describes its unique value proposition. It provides step-by-step guidance for you to get started with the program, including best practices for capacity planning, installation and configuration, administration and operation. Through a retail example, the book shows how you can use the versatile and intuitive web user interface to quickly train, build, evaluate, and deploy a model. Most importantly, it examines use cases across industries to illustrate how you can easily turn your massive data into valuable insights with Machine Learning for z/OS.
Getting Started: Journey to Modernization with IBM Z by Makenzie Manna,Ravinder Akula,Matthew Cousens,Pabitra Mukhopadhyay,Anand Shukla,IBM Redbooks Pdf
Modernization of enterprise IT applications and infrastructure is key to the survival of organizations. It is no longer a matter of choice. The cost of missing out on business opportunities in an intensely competitive market can be enormous. To aid in their success, organizations are facing increased encouragement to embrace change. They are pushed to think of new and innovative ways to counter, or offer, a response to threats that are posed by competitors who are equally as aggressive in adopting newer methods and technologies. The term modernization often varies in meaning based on perspective. This IBM® Redbooks® publication focuses on the technological advancements that unlock computing environments that are hosted on IBM Z® to enable secure processing at the core of hybrid. This publication is intended for IT executives, IT managers, IT architects, System Programmers, and Application Developer professionals.
Enabling Real-time Analytics on IBM z Systems Platform by Lydia Parziale,Oliver Benke,Willie Favero,Ravi Kumar,Steven LaFalce,Cedrine Madera,Sebastian Muszytowski,IBM Redbooks Pdf
Regarding online transaction processing (OLTP) workloads, IBM® z SystemsTM platform, with IBM DB2®, data sharing, Workload Manager (WLM), geoplex, and other high-end features, is the widely acknowledged leader. Most customers now integrate business analytics with OLTP by running, for example, scoring functions from transactional context for real-time analytics or by applying machine-learning algorithms on enterprise data that is kept on the mainframe. As a result, IBM adds investment so clients can keep the complete lifecycle for data analysis, modeling, and scoring on z Systems control in a cost-efficient way, keeping the qualities of services in availability, security, reliability that z Systems solutions offer. Because of the changed architecture and tighter integration, IBM has shown, in a customer proof-of-concept, that a particular client was able to achieve an orders-of-magnitude improvement in performance, allowing that client's data scientist to investigate the data in a more interactive process. Open technologies, such as Predictive Model Markup Language (PMML) can help customers update single components instead of being forced to replace everything at once. As a result, you have the possibility to combine your preferred tool for model generation (such as SAS Enterprise Miner or IBM SPSS® Modeler) with a different technology for model scoring (such as Zementis, a company focused on PMML scoring). IBM SPSS Modeler is a leading data mining workbench that can apply various algorithms in data preparation, cleansing, statistics, visualization, machine learning, and predictive analytics. It has over 20 years of experience and continued development, and is integrated with z Systems. With IBM DB2 Analytics Accelerator 5.1 and SPSS Modeler 17.1, the possibility exists to do the complete predictive model creation including data transformation within DB2 Analytics Accelerator. So, instead of moving the data to a distributed environment, algorithms can be pushed to the data, using cost-efficient DB2 Accelerator for the required resource-intensive operations. This IBM Redbooks® publication explains the overall z Systems architecture, how the components can be installed and customized, how the new IBM DB2 Analytics Accelerator loader can help efficient data loading for z Systems data and external data, how in-database transformation, in-database modeling, and in-transactional real-time scoring can be used, and what other related technologies are available. This book is intended for technical specialists and architects, and data scientists who want to use the technology on the z Systems platform. Most of the technologies described in this book require IBM DB2 for z/OS®. For acceleration of the data investigation, data transformation, and data modeling process, DB2 Analytics Accelerator is required. Most value can be achieved if most of the data already resides on z Systems platforms, although adding external data (like from social sources) poses no problem at all.
Four Ways to Transform Your Mainframe for a Hybrid Cloud World by Guillaume Arnould,Guillaume Hoareau,Herve Sabrie,Sebastien Llaurency,Yann Kindelberger,IBM Redbooks Pdf
The IBM® mainframe remains a widely used enterprise computing workhorse, hosting essential IT for the majority of the world's top banks, airlines, insurers and more. As the mainframe continues to evolve, the newest IBM Z® servers offer solutions for AI and analytics, blockchain, cloud, DevOps, security and resiliency, with the aim of making the client experience similar to that of using cloud services. Many organizations today face challenges with their core IT infrastructure: Complexity and stability An environment might have years of history and be seen as too complex to maintain or update. Problems with system stability can impact operations and be considered a high risk for the business. Workforce challenges Many data center teams are anticipating a skills shortage within the next 5 years due to a retiring and declining workforce specialized in the mainframe, not to mention the difficulty of attracting new talent. Total cost of ownership Some infrastructure solutions are seen as too expensive, and it's not always easy to balance up-front costs with the life expectancy and benefits of a given platform. Lack of speed and agility Older applications can be seen as too slow and monolithic as organizations face an increasing need for faster turnaround and release cycles. Some software vendors suggest addressing these challenges with the "big bang" approach of moving your entire environment to a public cloud. But public cloud isn't the best option for every workload, and a hybrid multicloud approach can offer the best of both worlds. IBM Z is constantly being developed to address the real challenges businesses face today, and every day we're helping clients modernize their IT environments. There are 4 strategic elements to consider when modernizing your mainframe environment: Infrastructure Applications Data access DevOps chain This paper focuses on these four modernization dimensions.
Oracle on IBM Z by Susan Adamovich,Sam Amsavelu,Armelle Chevé,Helene Grosch,Moshe Reder,David J Simpson,Narjisse Zaki,IBM Redbooks Pdf
Oracle Database 12c Release 1 running on Linux is available for deployment on the IBM ZTM family of servers. The enterprise-grade Linux on IBM Z solution is designed to add value to Oracle Database solutions, including the new functions that are introduced in Oracle Database 12c. In this IBM Redbooks® publication, we explore the IBM and Oracle Alliance and describe how Oracle Database benefits from the IBM Z platform. We then explain how to set up Linux guests to install Oracle Database 12c. We also describe how to use the Oracle Enterprise Manager Cloud Control Agent to manage Oracle Database 12c Release 1. We also describe a successful consolidation project from sizing to migration, performance management topics, and high availability. Finally, we end with a chapter about surrounding Oracle with Open Source software. The audience for this publication includes database consultants, installers, administrators, and system programmers. This publication is not meant to replace Oracle documentation, but to supplement it with our experiences while installing and using Oracle products.
