Wordcount pyspark

text_file = sc.textFile("hdfs://...")
counts = text_file.flatMap(lambda line: line.split(" ")) \
             .map(lambda word: (word, 1)) \
             .reduceByKey(lambda a, b: a + b)
counts.saveAsTextFile("hdfs://...")

Source: spark.apache.org

Similar codes

0 vote373 views1 answer
Check pip installed packages inside virtualenv
Python
10 votes335 views1 answer
Pandas sample seed
Python
2 votes328 views2 answers
Or in django query
Python
0 vote296 views1 answer
Check if list elememnt in dataframe column
Python
14 votes308 views6 answers
How to print hello world in python
Python
0 vote308 views1 answer
Python seaborn violin stack overflow
Python
2 votes334 views1 answer
Reverse order np array
Python
2 votes318 views1 answer
Range python iterate by 2
Python
0 vote327 views1 answer
Create tiff stack in python
Python
3 votes326 views2 answers
Module 'cv2' has no 'imread' member
Python