012 Lambda Architecture – The New Big Data Architecture
1. Objective
1.目标
In this blog, we will discuss Lambda Architecture big data. Also, Lambda Architecture applications, advantages as well as disadvantages of Lambda Architecture in Big Data. Moreover, we will discuss Lambda Architecture in detail, how it works.
在这篇文章中，我们将讨论 Lambda 架构. 此外，Lambda 架构应用程序，Lambda 架构的优点和缺点 大数据.，同时我们也将详细讨论 Lambda 架构及其工作原理.
Lambda Architecture – The New Big Data Architecture
2. What is Lambda Architecture
2. 什么是 Lambda 架构
This is the new big data architecture. Also, this was designed to ingest and process. Also, to query both fresh and historical (batch) data in a single data architecture.
这是新的大数据架构，这个设计了数据摄取和数据加工. 此外，在单个数据体系结构中查询新数据和历史 (批处理) 数据.
We use this architecture is to solve the problem of computing arbitrary functions. Also, the problems contain three layers:
我们使用这种架构是为了解决任意函数的计算问题.此外，问题包含三层:
Batch layer,
Serving layer, and
Speed layer
批量层
服务层
加速层
Lambda Architecture – Layers
Basically, we used to call the batch layer a “data lake” system like Hadoop. Also, use this historical archive to hold all of the data ever collected. Moreover, this layer helps into supports batch query. Also, we use batch processing to generate analytics or ad hoc.
基本上,我们曾经将批处理层称为 “数据湖” 系统，如 Hadoop.此外，使用这个历史档案保存 所有的收集到的数据此外，该层还支持批量查询.此外，我们使用批处理来生成分析或临时分析.
Secondly, we used to call the speed layer a combination of queuing, streaming. Also, the speed layer is like the batch layer in that it computes similar analytics. It except that it computes that analytics in real-time on only the most recent data. The analytics the batch layer calculates.
其次,我们过去把加速层称为排队、流的组合.此外，加速层就像它计算类似分析的批处理层.除了它只对最近的数据进行实时分析之外.批处理层计算的分析.
For example
it may be based on data one hour old. It is the speed layer’s responsibility to calculate real-time analytics. That is based on fast-moving data that is zero to one hour old.
它可能 基于一个小时前的数据计算实时分析是加速层的责任.那个基于在 0 到 1 小时的快速移动数据上.
The third layer – we used to call the serving layer handles serving up results. Also, combined with both the speed and batch layer.
第三层 -- 我们曾经调用服务层来处理服务结果.此外，结合加速层和批处理层.
Lambda Architecture – Working
a. As all data enters in the system it will be dispatched to both the batch layer and the speed layer for processing.
a.当所有数据进入系统时，它会被派遣分发用于处理的批处理层和加速层.
b. The batch layer has the two most important functions:
b. 批处理层有两个最重要的功能:
(i) managing the master dataset
(i) 管理主数据集
(ii) to pre-compute the batch views.
(ii) 批处理视图与计算
c. Also, we use serving layer to indexes the batch views. Thus, they can be queried in low-latency, ad-hoc way.
c. 此外，我们使用服务层对批处理视图进行索引.因此，他们可以在低延迟，自组织的方式进行数据查询
d. The speed layer compensates for the high latency of updates to the serving layer. Also deals with recent data only.
d. 加速层补偿了服务层更新的高延迟.也只处理最近的数据.
e. We can answer any incoming query by merging results from batch views and real-time views.
e. 我们可以通过合并批处理视图和实时视图的结果来响应 任何上层传入的查询.
3. Typical Lambda Applications
3. 典型的 Lambda 应用
As we know it is an emerging paradigm in Big Data computing. However, log ingestion and accompanying analytics are use cases of Lambda-based applications.
众所周知，这是大数据计算的一个新兴模式. 然而,日志获取和附带的分析是基于 Lambda 的应用程序的用例.
Moreover, log messages often are created at a high velocity. Also, they are immutable. Also, we can call it as the “fast data”. The ingestion of each log message does not require a response to the entity that delivered the data. It is a one-way data pipeline.
此外，日志消息经常 被以很高的速度创建，此外，它们是不可变的. 另外，我们可以称之为 “快速数据”.每个日志消息的接收不需要对传递数据的实体的响应.这是一个单向的数据管道.
For example
We can say that the analytics for website click logs could be counting page hits and page popularity.
我们可以说，网站点击日志的分析可以计算页面点击量和页面受欢迎程度.
4. Advantages of Lambda Architectures
4. Lambda 架构
As a result, emphasizes retaining the input data unchanged. Also, the discipline of modeling data transformation. Moreover, this is one of the things that makes large MapReduceworkflows tractable. As it enables you to debug each stage independently.
因此，强调保持输入数据不变. 还有数据转换的建模规范. 此外，这是使大 MapReduce可跟踪的工作流.因为它使您能够调试每个阶段独立.
This highlights the problem of reprocessing data. As the reprocessing process is one of the key challenges of stream processing. Also, by this process, input data over again to re-derive output. This is a completely obvious but often ignored requirement. Also, a code will always change.
这凸显了数据再处理的问题.由于后处理过程是流处理的关键挑战之一.此外，通过这个过程，再次输入数据以重新导出输出.这是一个非常明显但经常被忽视的问题要求.同时代码总是会改变.
5. Disadvantages of Lambda Architectures
5. Lambda 结构的优缺点
There is a problem with the Lambda Architecture. That is to maintain the code. Also, that needs to produce the same result in two complex distributed systems. That is exactly as painful as it seems like it would be. To do programming in frameworks like Storm and Hadoop is complex. Also, the code ends up being towards the framework it runs on.
Lambda 架构的确存在一些问题. 那就是维护代码. 此外，这需要在两个复杂的分布式系统中产生相同的结果.这和看起来的一样痛苦.做规划框架，如 Storm, Hadoop很复杂. 代码最终指向它运行的框架.
Why can’t the stream processing system be improved to handle the full problem set in its target domain?
流处理系统为什么不能 得到改进要处理其目标域中的完整问题集？
To fix this we have only one approach that is we need to have a language or either framework. Moreover, that abstracts over both the real-time and batch framework. You can easily write your code using this higher level framework. Then it “compiles down” to stream processing or MapReduce under the covers. “Summingbird” is an only framework that can easily do this. Furthermore, this will definitely make things a little better, but I don’t think it solves the problem.
为了解决这个问题，我们只有一种方法，那就是我们需要有一种语言或框架.此外，这在实时和批处理框架中都是抽象的.你可以轻松使用这个更高级别的框架编写代码.然后它 “向下编译” 到流处理或 MapReduce 在封面下 “Summingbird” 是唯一一个可以轻松这样做.此外，这肯定会让事情变得更好，但我不认为这样能够很好的解决问题.
6. Conclusion
6. 结论
As a result, we have studied What is Lambda Architecture. Also, Lambda Architecture working and applications, Lambda Architectures limitations, and benefits of Lambda Architectures. I hope this New Big Architecture will clear your concept about its working too. Furthermore, if you have any query, feel free to ask in a comment section.
我们研究了什么是 Lambda 架构. Lambda 体系结构的工作和应用、 Lambda 体系结构的限制以及 Lambda 体系结构的优势.我希望这个新的大架构也会让你对它的工作有一个清晰的概念.此外，如果你有任何疑问，可以在评论部分提出.
