I Heart Logs. Event Data, Stream Processing, and Data Integration (e-book)

Why a book about logs? That...s easy: the humble log is an abstraction that lies at the heart of many systems, from NoSQL databases to cryptocurrencies. Even though most engineers don...t think much about them, this short book shows you why logs are worthy of your attention.Based on his popular blog posts, LinkedIn principal engineer Jay Kreps shows you how logs work in distributed systems, and then delivers practical applications of these concepts in a variety of common uses-data integration, enterprise architecture, real-time stream processing, data system design, and abstract computing models.Go ahead and take the plunge with logs; you...re going love them.Learn how logs are used for programmatic access in databases and distributed systemsDiscover solutions to the huge data integration problem when more data of more varieties meet more systemsUnderstand why logs are at the heart of real-time stream processingLearn the role of a log in the internals of online data systemsExplore how Jay Kreps applies these ideas to his own work on data infrastructure systems at LinkedIn Spis treści:
Preface
Conventions Used in This Book
Safari Books Online
How to Contact Us
1. Introduction
What Is a Log?
Logs in Databases
Logs in Distributed Systems
Variety of Log-Centric Designs
An Example
Logs and Consensus
Changelog 101: Tables and Events Are Dual
Whats Next
2. Data Integration
Data Integration: Two Complications
Data is More Diverse
The Explosion of Specialized Data Systems
Log-Structured Data Flow
My Experience at LinkedIn
Relationship to ETL and the Data Warehouse
ETL and Organizational Scalability
Where Should We Put the Data Transformations?
Decoupling Systems
Scaling a Log
3. Logs and Real-Time Stream Processing
Data Flow Graphs
Logs and Stream Processing
Reprocessing Data: The Lambda Architecture and an Alternative
What is a Lambda Architecture and How Do I Become One?
Whats Good About This?
And the Bad...
An Alternative
Stateful Real-Time Processing
Log Compaction
4. Building Data Systems with Logs
Unbundling?
The Place of the Log in System Architecture
Conclusion
A. References
Academic Papers, Systems, Talks, and Blogs
Enterprise Software
Open Source