Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

5 votes316 views3 answers
Converting string array to int array python
Python
2 votes374 views2 answers
Python fibonacci while loop
Python
-1 vote288 views1 answer
Filter groupby pandas
Python
0 vote304 views1 answer
Selenium.common.exceptions.TimeoutException: Message:
Python
0 vote289 views1 answer
Python selenium disable JavaScript Detection
Python
0 vote309 views1 answer
Os.name == 'nt'
Python
0 vote279 views1 answer
Make a white image numpy
Python
0 vote386 views1 answer
Set header in dataframe 2nd line
Python
0 vote280 views1 answer
How to sort the order in multiple index pandas
Python
0 vote307 views1 answer
Django phone number
Python