碎片文件过多
sqly语句最后加上
distribute by ss_sold_date_sk, cast(rand() * 5 as int);
解析json文件并处理换行符
regexp_replace(get_json_object(json_content, '$.body'), '[\\n\\r]', '') as content
避免mapreduce
set hive.exce.mode.local.auto = true;
避免float错误
0.2对于float 类型是0.2000001，而对于double类型是0.200000000001
>cast(0.2 as float)
扫描一次数据
FROM history
INSERT OVERWRITE sales SELECT * WHERE action='purchased'
INSERT OVERWRITE credits SELECT * WHERE action='returned';
presto与hive的行列互转
concat_ws(',', collect_set(order_id))
array_join(array_distinct(array_agg(order_id)), ',')
lateral view explode(split(order_ids, ',')) b as order_id
cross join unnest(split(order_ids, ',')) as b(order_id)
插入总行数
sum(1) over()
presto时间加减
date '$$begindate' - interval '2' day
将map类型合并
mapmerge(map(),paramcontext, factorcontext)
