Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

0 vote355 views1 answer
Django pull from Google Sheets
Python
13 votes349 views5 answers
How to check if an input is a number in python
Python
2 votes328 views1 answer
Python how to get number of strings in a list
Python
0 vote389 views1 answer
Youtube-dl python download to specific folder
Python
11 votes358 views6 answers
BeautifulSoup
Python
0 vote375 views1 answer
Isidentifier method in python
Python
9 votes343 views2 answers
Install python linux ubuntu
Python
0 vote347 views1 answer
Flatmap in python
Python
0 vote338 views1 answer
Python digit string
Python
0 vote364 views1 answer
Omr sheet python stackoverflow
Python