배우기를 멈추는 사람은 20세건 80세건 늙은 것이다.

import boto3 from botocore.config import Config _MAX_ATTEMPTS = 15 def boto3_client(type): config = Config( retries={ "max_attempts": _MAX_ATTEMPTS, "mode": "standard" } ) return boto3.client(type, config=config)

d=db t=table p=partition ALTER TABLE ${d}.${t} SET TBLPROPERTIES('EXTERNAL'='TRUE'); ALTER TABLE ${d}.${t} DROP PARTITION (${p} ''); MSCK REPAIR TABLE ${d}.${t}; https://118k.tistory.com/349 [하이브] 매니지드 테이블과 익스터널 테이블 변경하기 하이브의 테이블은 매니지드(MANAGED) 테이블과 익스터널(EXTERNAL) 테이블 타입이 존재한다. 매니지드 테이블은 테이블을 drop 하면 관리하는 파일도 삭제가 되고, 익스터널 테이블은 파일은 보관된 118k.tistory.com https://stackoverflow.com/questions/46307667..

https://stackoverflow.com/questions/19750653/how-to-append-text-files-using-batch-files How to append text files using batch files How can I append file1 to file2, from a batch file? Text files and only using what is "standard" on windows. stackoverflow.com

#!/bin/bash input=$1 output=$2 rm $output n=0 while read line; do comma="${line//[^,]}" cnt="${#comma}" echo "Line No. ${n} : ${cnt}" >> $output n=$((n+1)) done < ${input} var="text,text,text,text" res="${var//[^,]}" echo "$res" echo "${#res}" ,,, 3 https://stackoverflow.com/questions/16679369/count-occurrences-of-a-char-in-a-string-using-bash Count occurrences of a char in a string using Bash I..

sudo systemctl list-units

https://stackoverflow.com/questions/2061439/string-concatenation-in-jinja String concatenation in Jinja I just want to loop through an existing list and make a comma delimited string out of it. Something like this: my_string = 'stuff, stuff, stuff, stuff' I already know about loop.last, I just need to stackoverflow.com

https://stackoverflow.com/questions/69048706/airflow-how-to-get-an-airflow-variable-inside-the-bash-command-in-bash-operato

https://stackoverflow.com/questions/33742895/how-to-show-full-column-content-in-a-spark-dataframe

yarn.nodemanager.resource.memory-mb yarn.nodemanager.resource.cpu-vcores yarn.scheduler.minimum-allocation-mb yarn.scheduler.maximum-allocation-mb yarn.scheduler.minimum-allocation-vcores yarn.scheduler.maximum-allocation-vcores https://wooono.tistory.com/145 [Spark] java.lang.IllegalArgumentException: Required executor memory (13312), overhead (2496 MB), and PySpark memory (0 MB) is a 우선 YARN R..

https://jaemunbro.medium.com/zeppelin-%EB%8B%A4%EC%A4%91-interpreter-binding%EA%B3%BC-interpreter-timeout-ce7ad4c3312c [Zeppelin] 다중 Interpreter binding과 Interpreter Timeout 설정하기 EMR의 Spark Zeppelin을 운영하고 있는데 여러 사용자가 들어와서 Job을 수행하는 경우가 잦다. 이러한 Multi Tenant Zepplin을 운영하는데 조금더 필요한 설정들이 무엇이 있을까? jaemunbro.medium.com https://aws.amazon.com/ko/premiumsupport/knowledge-center/yarn-uses-resources-after..

https://stackoverflow.com/questions/37254681/spark-throwing-filenotfoundexception-when-overwriting-dataframe-on-s3 Spark throwing FileNotFoundException when overwriting dataframe on S3 I have partitioned parquet files stored on two locations on S3 in the same bucket: path1: s3n://bucket/a/ path2: s3n://bucket/b/ The data has the same structure. I want to read the files from the... stackoverflow...

https://devpouch.tistory.com/127 [bash] 리눅스 쉘 스크립트에서 sleep 함수 사용법 리눅스 bash 쉘스크립트의 sleep 명령어를 통해 프로그램 실행을 일시적으로 정지할 수 있다. sleep 명령어는 아래와 같이 사용할 수 있다. sleep 명령어 사용법 sleep 1 # 1초 일시 정지 sleep 1s # 1초 일시 devpouch.tistory.com

pip3 install jq parse() { key=$1 python3 -c " import sys import jq import json input = json.load(sys.stdin) output = jq.compile('$key').input(input).all() if(isinstance(output, list)): output = ' '.join(output) print(output) " } name=$(aws emr describe-cluster --cluster-id $id | parse ".Cluster.Name") echo $name https://stackoverflow.com/questions/1955505/parsing-json-with-unix-tools?page=2&tab=..

fun id(): String { return make() .sparkContext() .applicationId() } https://knight76.tistory.com/entry/YARN%EC%97%90-%EB%B0%B0%ED%8F%AC%EB%90%9C-Spark-%EC%95%A0%ED%94%8C%EB%A6%AC%EC%BC%80%EC%9D%B4%EC%85%98%EC%9D%98-Application-ID-%EC%96%BB%EA%B8%B0 YARN에 배포된 Spark 애플리케이션의 Application ID 얻기 How to get applicationId of Spark application deployed to YARN in ... https://spark.apache.org/docs/2.3.0/a..

ALTER TABLE EMP_DTLS MODIFY COLUMN EMP_ID INT(10) FIRST ALTER TABLE EMP_DTLS MODIFY COLUMN EMP_ID INT(10) AFTER id https://stackoverflow.com/questions/20179801/place-an-existing-column-at-first-position-in-mysql place an existing column at first position in mysql please tell me how to place an existing column(contained values) at first position in mysql. Suppose i have a table EMP_DTLS and there..

val numbers = emptyList() val sumFromTen = numbers.fold(10) { total, num -> total + num } println("folded: $sumFromTen") // folded: 10 val sum = numbers.reduce { total, num -> total + num } println("reduced: $sum") folded: 10 Empty collection can't be reduced. java.lang.UnsupportedOperationException: Empty collection can't be reduced. at kr.leocat.test.FoldTest.test(FoldTest.kt:35) ... https://b..

@Test fun toNull() { // given data class Person( val name: String?, val job: String?, val age: Int ) val spark = SparkUtil.make() val data = spark.createDataFrame( mutableListOf( Person("null", "a", 25), Person("Bob", "null", 30), Person("null", "null", 35) ), Person::class.java ).toDF() val expected = spark.createDataFrame( mutableListOf( Person(null, "a", 25), Person("Bob", null, 30), Person(n..

https://json8.tistory.com/177 [안드로이드] uses-sdk:minSdkVersion declared in library 에러 해결 방법 원인 : Android SDK 11 버전에서 지원하지 않은 library 사용 수정 방법 : build.gradle minSdkVersion 변경 (appcompat-v7:26.1.0 경우 min SDK 14로 변경 필요) build.gradle 기존 설정 상태 minSdkVersion 11 에러 로그 Manifest mer json8.tistory.com https://progdev.tistory.com/50 flutter.minSdkVersion, flutter.targetSdkVersion가 선언된 위치 defaultConfig { // T..

https://stackoverflow.com/questions/59958294/how-do-i-execute-terraform-actions-without-the-interactive-prompt How do I Execute Terraform Actions Without the Interactive Prompt? How am I able to execute the following command: terraform apply #=> . . . Do you want to perform these actions? Terraform will perform the actions described above. Only 'yes' will be acce... stackoverflow.com