Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

0 vote308 views1 answer
Pandas show head and tail
Python
21 votes318 views1 answer
How to plot a graph using matplotlib
Python
-1 vote338 views1 answer
Using comma as the thousand separator
Python
0 vote330 views1 answer
Promote a row in panda dataframe to header
Python
0 vote314 views1 answer
Static Assets in Django
Python
0 vote332 views1 answer
How to use self.list.setCurrentRow() in pyqt5
Python
0 vote367 views1 answer
Production mode flask
Python Flask
-2 votes330 views1 answer
Django e docker
Python Django
1 vote387 views1 answer
Requirements.txt flask
Python Flask
0 vote304 views1 answer
How to get input from pyqt line edit
Python