Apache Flink is a very versatile tool for all kinds of data processing workloads. It can process incoming data within a few milliseconds or crunch through petabytes of bounded datasets (also known as batch processing).
To best understand state and state backends in Flink, it’s important to distinguish between in-flight state and state snapshots. In-flight state, also known as working state, is the state a Flink job is working on. It is always stored locally in memory (with the possibility to spill to disk) and can be lost when jobs fail without impacting job recoverability. State snapshots, i.e., checkpoints and savepoints, are stored in a remote durable storage, and are used to restore the local state
Flink has supported resource management systems like YARN and Mesos since the early days; however, these were not designed for the fast-moving cloud-native architectures that are increasingly gaining popularity these days, or the growing need to support complex, mixed workloads (e.g. batch, streaming, deep learning, web services). For these reasons, more and more users are using Kubernetes to automate the deployment, scaling and management of their Flink applications.
This new release brings remote functions to the front and center of StateFun, making the disaggregated setup that separates the application logic from the StateFun cluster the default. It is now easier, more efficient, and more ergonomic to write applications that live in their own processes or containers. With the new Java SDK this is now also possible for all JVM languages, in addition to Python.
Streaming jobs which run for several days or longer usually experience variations in workload during their lifetime. These variations can originate from seasonal spikes, such as day vs. night, weekdays vs. weekend or holidays vs. non-holidays, sudden events or simply the growing popularity of your product.
市面上已有lottie，svga，pag等优秀的矢量动画库，但是在B站业务场景下，并不能完美的解决自身业务痛点。B站拥有着一套自研的渲染引擎 Chronos，目前已支撑了移动端视频播放界面弹幕等视频特效业务，部分视频中的弹幕特效，以及大部分视频中的三连组件。Chronos 拥有对GPU接口的抽象和渲染管线，我们可以在Chronos基础上，完成一个矢量动画绘制模块，解决B站部分场景下特效动画的痛点。
对互联网公司来说，数据安全一直是极为重视和敏感的话题。涉及客户安全数据或者一些商业性敏感数据，如身份证号、手机号、卡号、客户号等个人信息如果被泄露出去，就会引发严重的数据安全风险。 在真实业务场景中，相关业务开发团队往往需要针对公司安全部门需求，自行实行并维护一套加解密系统，自行维护的加解密系统往往又面临着重构或修改风险。因此希望实现一个通用的敏感数据处理框架，如何在不修改业务逻辑、业务SQL的情况下，透明化、安全低风险地实现无缝进行数据加解密改造。
