Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

0 vote292 views1 answer
Set a random seed
Python
5 votes262 views4 answers
Get user django
Python
-3 votes254 views2 answers
Multiplication table python
Python
0 vote278 views1 answer
Python selenium set attribute of element
Python
0 vote236 views1 answer
Subplots whitespace
Python
0 vote245 views1 answer
Unicodedata no accent
Python
0 vote269 views1 answer
Python tar a directory
Python
1 vote247 views1 answer
Apostrophe in python
Python
1 vote247 views1 answer
Convert a tuple into string python
Python
1 vote242 views1 answer
How to convert int in python
Python