Python site-packages pyspark

import os

def myfun(x):`
        os.system("pip install shapely")
        return x
rdd = sc.parallelize([1,2,3,4]) ## assuming 4 worker nodes
rdd.map(lambda x: myfun(x)).collect() 
## call each cluster to run the code to import the library

Source: stackoverflow.com

Similar codes

0 vote273 views2 answers
Numpy array into tuple
Python
20 votes285 views3 answers
Download image python
Python
0 vote275 views1 answer
How to calculate the variance of all columns in python
Python
1 vote261 views1 answer
Create spark dataframe in python
Python
0 vote326 views1 answer
Flask put request example
Python Flask
8 votes263 views1 answer
Python program to remove duplicate characters of a given string.
Python
0 vote297 views1 answer
Parquet pyspark
Python
7 votes276 views1 answer
Python html to pdf
Python
1 vote266 views1 answer
Palindrome rearranging python
Python
0 vote268 views1 answer
Django creat app return _bootstrap._gcd_import
Python