/Python/Remove outliers python dataframe

Remove outliers python dataframe

Remove outliers python pandas

#------------------------------------------------------------------------------
# accept a dataframe, remove outliers, return cleaned data in a new dataframe
# see http://www.itl.nist.gov/div898/handbook/prc/section1/prc16.htm
#------------------------------------------------------------------------------
def remove_outlier(df_in, col_name):
    q1 = df_in[col_name].quantile(0.25)
    q3 = df_in[col_name].quantile(0.75)
    iqr = q3-q1 #Interquartile range
    fence_low  = q1-1.5*iqr
    fence_high = q3+1.5*iqr
    df_out = df_in.loc[(df_in[col_name] > fence_low) & (df_in[col_name] < fence_high)]
    return df_out

Source: stackoverflow.com

Remove outliers python dataframe

cols = ['col_1', 'col_2'] # one or more

Q1 = df[cols].quantile(0.25)
Q3 = df[cols].quantile(0.75)
IQR = Q3 - Q1

df = df[~((df[cols] < (Q1 - 1.5 * IQR)) |(df[cols] > (Q3 + 1.5 * IQR))).any(axis=1)]

Source: stackoverflow.com

Outliers removal pandas

df = pd.DataFrame(np.random.randn(100, 3))

from scipy import stats
df[(np.abs(stats.zscore(df)) < 3).all(axis=1)]

Source: stackoverflow.com

Pandas removing outliers from dataframe

df[(df["col"] >= x ) & (df["col"] <= y )]

but it's more readable to use:

df[df["col"].between(x,y)]

Source: app.dataquest.io

Similar codes

2 votes305 views1 answer
Matplotlib 3D plots reduce margins
Python
1 vote298 views1 answer
Length of a matrix in python
Python
0 vote267 views1 answer
Python advanced programming exercises
Python
0 vote286 views1 answer
Python copy dictionary keep original same
Python
8 votes272 views1 answer
Remove too short strings from a list python
Python
1 vote301 views1 answer
Save video cv2
Python
1 vote336 views1 answer
List to tuple
Python
0 vote292 views1 answer
Higlight words in python
Python
32 votes310 views6 answers
Get count of unique values in column pandas
Python
18 votes281 views6 answers
Remove punctuation from string python
Python