[HDFS] check exists & zero file; check directory size

path=hdfs://a/b/c hdfs dfs -test -e ${path} if [ $? -eq 0 ]; then echo "exists !!!" fi hdfs dfs -test -z ${path} if [ $? -eq 0 ]; then echo "zero !!!" fi size=`hdfs dfs -count ${path} | awk '{print $3}'` echo $size https://stackoverflow.com/questions/43900364/find-whether-file-exists-or-not-in-hdfs-using-shell-script/43903810 Find whether file exists or not in HDFS using shell script I have a sh..
공부 2021. 1. 8. 23:51
[Pyspark] show all results

df.show(df.count(), false) https://stackoverflow.com/questions/33742895/how-to-show-full-column-content-in-a-spark-dataframe
공부 2021. 1. 8. 23:37
[Regexp] ignore case sensitivity

# include ... log, Log, LOg, LOG, loG, lOG, ... (?i).*log.* https://stackoverflow.com/questions/9655164/regex-ignore-case-sensitivity Regex: ignore case sensitivity How can I make the following regex ignore case sensitivity? It should match all the correct characters but ignore whether they are lower or uppercase. G[a-b].* stackoverflow.com
공부 2021. 1. 8. 23:35
[Java] remove line break `(\n|\r|\u0001)`

private static Column fixText(Column column) { return regexp_replace( regexp_replace( regexp_replace( column, "\\n", ""), "\\r", ""), "\\u0001", ""); } text .replace("\\n", "") .replace("\\r", "") .replace("\\u0001", "") https://stackoverflow.com/questions/2163045/how-to-remove-line-breaks-from-a-file-in-java How to remove line breaks from a file in Java? How can I replace all line breaks from a..
공부 2021. 1. 8. 23:33
[Spark][SQL] GET_JSON_OBJECT

SELECT GET_JSON_OBJECT(ext, '$.id') track_id, COUNT(1) cnt FROM {db}.{table} WHERE 1=1 AND day = 20210107 AND GET_JSON_OBJECT(ext, '$.tp') = 'TRACK' GROUP BY GET_JSON_OBJECT(ext, '$.id') LIMIT 1000
공부 2021. 1. 8. 07:11
1
728x90
728x90