Wordcount pyspark

text_file = sc.textFile("hdfs://...")
counts = text_file.flatMap(lambda line: line.split(" ")) \
             .map(lambda word: (word, 1)) \
             .reduceByKey(lambda a, b: a + b)
counts.saveAsTextFile("hdfs://...")

Source: spark.apache.org

Similar codes

0 vote225 views1 answer
Remove multiple strings from list python
Python
45 votes221 views4 answers
Append to front of list python
Python
0 vote298 views1 answer
Striding in python
Python
0 vote250 views1 answer
Non venomous snakes
Python
2 votes236 views3 answers
Python rotate list
Python
0 vote203 views1 answer
How to make only one space in python
Python
1 vote261 views1 answer
How to use query_params in get_object djangorestframework
Python Django
4 votes246 views3 answers
Python division
Python
0 vote224 views1 answer
Set lable of field django
Python
2 votes237 views1 answer
Flask if statement
Python