Wordcount pyspark

text_file = sc.textFile("hdfs://...")
counts = text_file.flatMap(lambda line: line.split(" ")) \
             .map(lambda word: (word, 1)) \
             .reduceByKey(lambda a, b: a + b)
counts.saveAsTextFile("hdfs://...")

Source: spark.apache.org

Similar codes

19 votes305 views2 answers
Renaming headers pandasd
Python
0 vote322 views1 answer
Python sys environment
Python
0 vote355 views1 answer
Fetch json array from mysql django
Python
0 vote349 views1 answer
Set a random seed
Python
0 vote270 views1 answer
Python find cursor position
Python
0 vote305 views1 answer
Gevent with flask
Python
1 vote323 views1 answer
Python windows get file modified date
Python
4 votes325 views6 answers
Rgb to hex python
Python
0 vote343 views1 answer
Weather api python
Python
7 votes296 views4 answers
Print multiple lines python
Python