Wordcount pyspark

text_file = sc.textFile("hdfs://...")
counts = text_file.flatMap(lambda line: line.split(" ")) \
             .map(lambda word: (word, 1)) \
             .reduceByKey(lambda a, b: a + b)
counts.saveAsTextFile("hdfs://...")

Source: spark.apache.org

Similar codes

1 vote165 views1 answer
Ipykernel pip
Python
1 vote187 views1 answer
Python how to remove the title of the index from dataframe
Python
0 vote179 views1 answer
Django create object with default today date
Python
4 votes194 views3 answers
ArrayField in Django
Python Django
4 votes177 views1 answer
Pandas replace empty strings with NaN
Python
12 votes182 views1 answer
Link shortener
Python
0 vote193 views1 answer
Sys.path[0]
Python
0 vote221 views1 answer
Django on delete set default
Python Django
0 vote184 views1 answer
Simple Kivy pong game
Python
1 vote159 views1 answer
Hide cmd in python
Python