用python写hive的UDF函数?

如梦似幻梦幻泡影 发布于 2017/02/27 17:34
阅读 447
收藏 0

我用python写UDF函数时总报错:

python:

import sys

def main():
for line in sys.stdin:
        line = line.strip()
        fields = line.split('\t')
        if fields[2] is None or fields[2]=""
            fields[2] = "o"
        print '%s\t%s\t%s' % (fields[0],fields[1],fields[2])
        
if __name__ == "__main__":
    main()

hql语句:

select transform(device_id,device_type,gender) using 'python gender.py' as(device_id,device_type,gender) from dm_device_gender where year=2017 and month=02 and day=23 limit 20;

报错:

Error: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row {"device_id":"pom7vptz-wnu7-tvxj-xeqm-7anh1ga5vo9x","device_type":"gaid","package_names":"com.k12.equivalenceTiles","source":"m:dsp","gender":"f","ratio":"0.5002738483938325","tag":"calc","year":"2017","month":"02","day":"23"}
        at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:195)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:455)

 

请问大神这是什么原因?

加载中
0
b
birdgogo
请问你解决了么?我也遇到了这个问题?是import包的问题?你怎么解决的?
如梦似幻梦幻泡影
如梦似幻梦幻泡影
中文 异常
返回顶部
顶部