티스토리 뷰

공부

[flink] basic

승가비 2022. 11. 3. 17:06
728x90

https://nightlies.apache.org/flink/flink-docs-master/docs/concepts/flink-architecture/

 

Flink Architecture

Flink Architecture # Flink is a distributed system and requires effective allocation and management of compute resources in order to execute streaming applications. It integrates with all common cluster resource managers such as Hadoop YARN and Kubernetes,

nightlies.apache.org

https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/resource-providers/standalone/docker/#flink-with-docker-compose

 

Docker

Docker Setup # Getting Started # This Getting Started section guides you through the local setup (on one machine, but in separate containers) of a Flink cluster using Docker containers. Introduction # Docker is a popular container runtime. There are offici

nightlies.apache.org

https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/

 

Kafka

Apache Kafka Connector # Flink provides an Apache Kafka connector for reading data from and writing data to Kafka topics with exactly-once guarantees. Dependency # Apache Flink ships with a universal Kafka connector which attempts to track the latest versi

nightlies.apache.org

https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/connectors/datastream/kafka/#consumer-offset-committing

 

Kafka

Apache Kafka Connector # Flink provides an Apache Kafka connector for reading data from and writing data to Kafka topics with exactly-once guarantees. Dependency # Apache Flink ships with a universal Kafka connector which attempts to track the latest versi

nightlies.apache.org

https://riptutorial.com/apache-flink

 

apache-flink Tutorial => Getting started with apache-flink

Learn apache-flink - This section provides an overview of what apache-flink is, and why a developer might want to use it.It should also mention any large...

riptutorial.com

https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/overview/

 

Overview

Deployment # Flink is a versatile framework, supporting many different deployment scenarios in a mix and match fashion. Below, we briefly explain the building blocks of a Flink cluster, their purpose and available implementations. If you just want to start

nightlies.apache.org

https://gyrfalcon.tistory.com/entry/Flink-1-%EC%86%8C%EA%B0%9C-Basic-Concept

 

Flink 시작하기 #1 소개 (Basic Concept)

#Flink란? stream processing framework for distributed, high-performing, always-available, and accurate data streaming applications. 라고 소개되고 있다. 간단히 말해 최근 많이 사용되는 apache Storm, Spark Streaming과 같은 스트리

gyrfalcon.tistory.com

https://stackoverflow.com/questions/55134009/flink-how-to-set-parallelism-properly-when-using-multiple-kafka-source

 

Flink, how to set parallelism properly when using multiple Kafka source?

I still cannot get a clear idea of parallelism, let's say we have a flink cluster which has enough slots. In our flink job, we consume 3 kafka topics from 3 different kafka clusters, each topic has...

stackoverflow.com

https://stackoverflow.com/questions/70538578/flink-1-13-2-noresourceavailableexception

 

Flink 1.13.2: NoResourceAvailableException

This is with Flink 1.13.2 running in Amazon's Kinesis Data Analytics Flink environment. This application is running on Kafka topics. When the topics had smaller traffic volumes, this application ra...

stackoverflow.com

 

https://programmer.group/flink-kafka-source-god-operated-flink-kafka-connector.html

 

Flink Kafka source God operated Flink Kafka connector

Keywords: Java Spring Back-end introduction Flink provides a special Kafka connector to read or write data to Kafka topic. Flink Kafka Consumer integrates Flink's Checkpoint mechanism to provide exactly once processing semantics. For this reason, Flink doe

programmer.group

https://gyrfalcon.tistory.com/entry/Flink-6-Windows

 

Flink 시작하기 #6 Windows

스트리밍 데이터는 unbounded 데이터로 처음과 끝의 개념이 없다. element의 데이터를 개별적으로 처리하는 연산만 사용한다면 큰 문제는 없지만, 집계연산을 사용한다면 문제가 생긴다. 만약 평균

gyrfalcon.tistory.com

https://velog.io/@zxshinxz/Flink%EB%A1%9C-%EC%8B%9C%EC%9E%91%ED%95%98%EB%8A%94-Stream-processing-5-Window-Operator-pt1

 

Flink로 시작하는 Stream processing 5 - Window Operator pt1

오늘은 개인적으로 Stream processing의 꽃이라고 생각하는 window operator에 대해 적어 보려고 합니다. Window operator는 간단하게 설명하자면, stream으로 흘들어 오는 이벤트를 묶어서 처리 하는 것입니다.

velog.io

https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/operators/windows/

 

Windows

Windows # Windows are at the heart of processing infinite streams. Windows split the stream into “buckets” of finite size, over which we can apply computations. This document focuses on how windowing is performed in Flink and how the programmer can ben

nightlies.apache.org

 

728x90
저작자표시 비영리

'공부' 카테고리의 다른 글

[AWS] graviton  (0) 2022.11.07
[AWS] SES (email)  (0) 2022.11.07
[kafka] docker-compose  (0) 2022.11.03
[jq] jq -r ".values[].name" && cURL cookie  (0) 2022.11.03
[kotlin] toObject, toJson  (0) 2022.11.02
댓글