This series of blog posts present a collection of low-latency techniques in Flink. In part one, we discussed the types of latency in Flink and the way we measure end-to-end latency and presented a few techniques that optimize latency directly. In this post, we will continue with a few more direct latency optimization techniques. Just like in part one, for each optimization technique, we will clarify what it is, when to use it, and what to keep in mind when using it. We will also show experimenta

228 技术 lddgo 分享于 2022-09-13

Apache Flink is a stream processing framework well known for its low latency processing capabilities. It is generic and suitable for a wide range of use cases. As a Flink application developer or a cluster administrator, you need to find the right gear that is best for your application. In other words, you don’t want to be driving a luxury sports car while only using the first gear.

81 技术 lddgo 分享于 2022-09-13

One of the most important characteristics of stream processing systems is end-to-end latency, i.e. the time it takes for the results of processing an input record to reach the outputs. In the case of Flink, end-to-end latency mostly depends on the checkpointing mechanism, because processing results should only become visible after the state of the stream is persisted to non-volatile storage (this is assuming exactly-once mode; in other modes, results can be published immediately).

72 技术 lddgo 分享于 2022-09-13

Deciding proper parallelisms of operators is not an easy work for many users. For batch jobs, a small parallelism may result in long execution time and big failover regression. While an unnecessary large parallelism may result in resource waste and more overhead cost in task deployment and network shuffling.

75 技术 lddgo 分享于 2022-09-13

对于非渲染引擎相关工作的开发者来说，可能认为即使构建最简单的3D程序也非常困难，但事实上并非如此，本篇文章将通过简单的200多行的纯 Java代码，去实践正交投影、简单三角形光栅化、z缓冲（深度缓冲区）和平面着色等基本的3D渲染技术，然后在下一片文章中，将着重介绍光线追踪的知识。

123 技术 lddgo 分享于 2022-09-13

容器本质是一项隔离技术，很好的解决了他的前任 - 虚拟化未解决的问题：运行环境启动速度慢、资源利用率低，而容器技术的两个核心概念，Namespace 和 Cgroup，恰到好处的解决了这两个难题。Namespace 作为看起来是隔离的技术，替代了 Hypervise 和 GuestOS，在原本在两个 OS 上的运行环境演进成一个，运行环境更加轻量化、启动快，Cgroup 则被作为用起来是隔离的技术，限制了一个进程只能消耗整台机器的部分 CPU 和内存。

113 技术 lddgo 分享于 2022-09-12

Dubbo 作为一个微服务框架，Dubbo SDK 与应用服务绑定在同一个进程内，它跟随着应用服务被部署在分布式集群各个位置，为了在分布式环境下实现各个应用服务间的协作， Dubbo 定义了一些中心化组件

115 技术 lddgo 分享于 2022-09-12

微服务给大家带来了敏捷开发的特性，基于敏捷开发带来的便利，让我们可以在同一个时间内多个迭代/feature 并行开发。但微服务架构本身也给开发环境带来了一定的复杂性：每个 feature 的修改点都可能会被分散在多个应用中，需要多个应用互相配合才能完成整体的逻辑。这些应用既需要互相配合好，又不能让他们互相影响，故敏捷开发有时候也不是那么容易。

116 技术 lddgo 分享于 2022-09-12

KubeVela 是一个简单易用且高度可扩展的应用交付和管理平台，基于 Kubernetes 与 OAM 技术构建。其核心功能是让开发人员方便快捷地在 Kubernetes 上定义与交付现代微服务应用，而无需了解任何 Kubernetes 本身相关的细节。

100 技术 lddgo 分享于 2022-09-12

目前，业界已经总结出了几种常见的服务发布策略来解决版本升级过程中带来的流量有损问题。本文首先会对这些普遍的发布策略进行简单的原理解析，最后结合阿里云的云原生网关对这些发布策略进行实践。

106 技术 lddgo 分享于 2022-09-12