Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

0 vote282 views1 answer
Ladnha;
Python
3 votes272 views2 answers
Python decouple
Python
0 vote266 views2 answers
Drip bucket limiter python
Python
0 vote295 views1 answer
Access key through value python
Python
2 votes298 views1 answer
How to reomve certain row from dataframe pandas
Python
0 vote269 views2 answers
Count repeated characters in a string python
Python
0 vote263 views1 answer
Pandas check is field is null or empty
Python
-1 vote337 views3 answers
How to get the code of a website in python
Python
2 votes275 views2 answers
Python greatest common divisor
Python
0 vote269 views1 answer
Find no of 1's in a binary number
Python