[airflow] AIRFLOW__KUBERNETES_EXECUTOR__DELETE_WORKER_PODS_ON_FAILURE=true

https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#delete-worker-pods-on-failure Configuration Reference — Airflow Documentation airflow.apache.org https://github.com/apache/airflow/issues/28751 KubernetesExecutor leaves failed pods due to deepcopy issue with Google providers · Issue #28751 · apache/airflow Apache Airflow version Other Airflow 2 version (please specify..
공부 2023. 7. 16. 13:00
[airflow] configurations

config: AIRFLOW__CORE__DEFAULT_TIMEZONE: Asia/Seoul AIRFLOW__CORE__DEFAULT_TASK_EXECUTION_TIMEOUT: 10800 AIRFLOW__CORE__LOAD_EXAMPLES: false AIRFLOW__CORE__PARALLELISM: 128 AIRFLOW__CORE__MAX_ACTIVE_RUNS_PER_DAG: 64 AIRFLOW__LOGGING__REMOTE_LOGGING: true AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER: s3://asdf AIRFLOW__LOGGING__REMOTE_LOG_CONN_ID: aws_default AIRFLOW__WEBSERVER__EXPOSE_CONFIG: false ..
공부 2023. 7. 16. 12:46
[빠니보틀] 틀려도 무조건 해야 돼

좋은 에너지를 주는 것들 2023. 7. 16. 12:42
[github] actions not working

RequestError [HttpError]: Resource not accessible by integration at /home/runner/work/_actions/seunggabi/auto-close-milestone/v1.0.0/dist/index.js:[4](https://github.com/seunggabi/copy-money/actions/runs/5564921370/jobs/10164850823#step:3:5)23[5](https://github.com/seunggabi/copy-money/actions/runs/5564921370/jobs/10164850823#step:3:6):21
공부 2023. 7. 16. 12:31
[aws] emr machine usage (mem & cpu)

r5a.8xlargeConfiguration optionDefault value mapreduce.map.java.opts -Xmx6349m mapreduce.reduce.java.opts -Xmx12698m mapreduce.map.memory.mb 7936 mapreduce.reduce.memory.mb 15872 yarn.app.mapreduce.am.resource.mb 15872 yarn.scheduler.minimum-allocation-mb 32 yarn.scheduler.maximum-allocation-mb 253952 yarn.nodemanager.resource.memory-mb 253952 https://docs.aws.amazon.com/emr/latest/ReleaseGuide/..
공부 2023. 7. 16. 11:01
1
728x90
728x90