배우기를 멈추는 사람은 20세건 80세건 늙은 것이다.

fun drop(table: String) { val spark = SparkUtil.instance(CommonUtil.methodName()) spark.catalog().dropTempView(table) } object SparkUtil { fun instance(name: String? = ""): SparkSession { return make(name!!) } private fun make(name: String): SparkSession { return config( SparkSession.builder().appName(name) ).enableHiveSupport().orCreate } private fun config(builder: SparkSession.Builder): Spark..

You need to iterate over steps.items(), because an iteration over dict only returns its keys. >>> x = sorted(steps.items()) >>> x [(1, 'value1'), (2, 'value3'), (5, 'value2')] Iterate over sorted keys: >>> for key in sorted(steps): ... # use steps[keys] to get the value https://stackoverflow.com/questions/16710112/python-iterate-over-dictionary-sorted-by-key python: iterate over dictionary sorte..

https://www.youtube.com/watch?v=5ZGsrbQbcbE

https://www.youtube.com/watch?v=RBDoN0ehZXE

https://qkqhxla1.tistory.com/1136 parquet vs orc vs avro (big data file format ) 1. www.datanami.com/2018/05/16/big-data-file-formats-demystified/ 공통점. 3개 타입은 전부 하둡에 저장하는데에 최적화되어있다. orc, parquet, avro 3개 전부 기계가 읽을수 있는 바이너리 포맷이다. orc, p.. qkqhxla1.tistory.com https://www.quora.com/Why-is-parquet-best-for-Spark-and-not-ORC-although-both-are-columnar-based-file-formats Why is parquet best for Spark..

# emr.sh #!/bin/sh TAG=$1 ENV=$2 CLASS=$3 ARGS=$4 SRC=s3://src/${ENV}/jar/batch/batch.jar LOG=s3://log/${ENV}/batch/ SUBNET_ID=subnet-0e3653577617c98a3 echo $( \ aws emr create-cluster \ \ --auto-scaling-role EMR_AutoScaling_DefaultRole \ --instance-groups file://./batch/static/json/instance.json \ \ --name ${CLASS} \ --release-label emr-6.7.0 \ --auto-terminate \ --applications Name=Spark \ --u..

ImportError: cannot import name 'json' from 'itsdangerous' (/Users/seunggab.kim/seunggabi/workspace/ads/ads-bot/venv/lib/python3.10/site-packages/itsdangerous/__init__.py) https://velog.io/@___pepper/Flask-ImportError-cannot-import-name-json-from-itsdangerous [Flask] ImportError: cannot import name 'json' from 'itsdangerous' itsdangerous import error in flask velog.io ImportError: cannot import ..

Example private fun token(email: String, password: String): String { val url = "https://asdf.com/tokens" val headers = mapOf( "Accept" to "application/json", "Content-Type" to "application/json", "Authorization" to "null" ) val json = mapOf( "auth_type" to "CREDENTIAL", "credential_type_payload" to mapOf( "email" to email, "password" to password, ), ).toJson() return CrawlUtil.post(url, headers,..