[yarn] cost tool `migration-hadoop-to-emr-tco-simulator`

https://github.com/awslabs/migration-hadoop-to-emr-tco-simulator GitHub - awslabs/migration-hadoop-to-emr-tco-simulator Contribute to awslabs/migration-hadoop-to-emr-tco-simulator development by creating an account on GitHub. github.com
공부 2023. 2. 16. 01:00
[glue] crawler (load s3)

https://inpa.tistory.com/entry/AWS-%F0%9F%93%9A-Glue-Crawler%EB%A1%9C-%ED%85%8C%EC%9D%B4%EB%B8%94-%EB%A7%8C%EB%93%A4%EA%B3%A0-Athena%EB%A1%9C-%EC%A1%B0%ED%9A%8C%ED%95%98%EA%B8%B0 [AWS] 📚 Glue Crawler로 테이블 만들고 Athena로 조회하기 Glue Crawler로 S3 스키마 생성 지난 포스팅에서는 csv파일을 S3에 업로드하고 Athena에서 직접 테이블 쿼리문을 실행하여 수동으로 만들어 조회하는 시간을 가져보았다. 이번에는 AWS Glue 서비스가 제 inpa.tistory.com https://docs.aws.amazon.com/ko_kr/gl..
공부 2023. 2. 16. 00:59
[yarn] yarn.resourcemanager.am.max-attempts=1

https://stackoverflow.com/questions/38709280/how-to-limit-the-number-of-retries-on-spark-job-failure How to limit the number of retries on Spark job failure? We are running a Spark job via spark-submit, and I can see that the job will be re-submitted in the case of failure. How can I stop it from having attempt #2 in case of yarn container failure or stackoverflow.com
공부 2023. 2. 16. 00:58
[sh] while loop (sub shell) not assign global variables -> for in loop

https://unix.stackexchange.com/questions/402750/modify-global-variable-in-while-loop Modify global variable in while loop I have a script that process a folder, and count the files in the mean time. i=1 find tmp -type f | while read x do i=$(($i + 1)) echo $i done echo $i However, $i is always 1, how do I res... unix.stackexchange.com https://skylit.tistory.com/321 Bash shell에서 for 루프 사용하기 OS: U..
공부 2023. 2. 16. 00:54
1
728x90
728x90