Перейти к содержимому

Pandas как заменить nan на среднее

  • автор:

Pandas: как заполнить значения NaN средним (3 примера)

Вы можете использовать функцию fillna() для замены значений NaN в кадре данных pandas.

Вот три распространенных способа использования этой функции:

Метод 1: заполнить значения NaN в одном столбце средним значением

Метод 2: заполнить значения NaN в нескольких столбцах средним значением

Метод 3: заполнить значения NaN во всех столбцах средним значением

В следующих примерах показано, как использовать каждый метод на практике со следующими пандами DataFrame:

Пример 1. Заполнение значений NaN в одном столбце средним значением

Следующий код показывает, как заполнить значения NaN в столбце рейтинга средним значением столбца рейтинга :

Среднее значение в столбце рейтинга было 85,125 , поэтому каждое из значений NaN в столбце рейтинга было заполнено этим значением.

Пример 2. Заполнение значений NaN в нескольких столбцах средним значением

В следующем коде показано, как заполнить значения NaN в столбцах рейтинга и очков соответствующими значениями столбцов:

Значения NaN как в столбцах рейтингов , так и в столбцах очков были заполнены соответствующими средними значениями столбцов.

Пример 3. Заполнение значений NaN во всех столбцах средним значением

В следующем коде показано, как заполнить значения NaN в каждом столбце значениями столбца:

Обратите внимание, что значения NaN в каждом столбце были заполнены средним значением столбца.

Полную онлайн-документацию по функции fillna() можно найти здесь .

Дополнительные ресурсы

В следующих руководствах объясняется, как выполнять другие распространенные операции в pandas:

Pandas: Replace NaN with mean or average in Dataframe using fillna()

In this article we will discuss how to replace the NaN values with mean of values in columns or rows using fillna() and mean() methods.

In data analytics we sometimes must fill the missing values using the column mean or row mean to conduct our analysis. Python provides users with built-in methods to rectify the issue of missing values or ‘NaN’ values and clean the data set. These functions are,

Dataframe.fillna()

The fillna() method is used to replace the ‘NaN’ in the dataframe. We have discussed the arguments of fillna() in detail in another article.

The mean() method:

Parameters:

  • axis :
    • Axis for the function to be applied on.
    • Exclude NA/null values when computing the result.
    • If the axis is a MultiIndex (hierarchical), count along a particular level, collapsing into a Series.
    • If None, will attempt to use everything, then use only numeric data. Not implemented for Series.

    We will be using the default values of the arguments of the mean() method in this article.

    Returns:

    Frequently Asked:

    • It returns the average or mean of the values.

    Now let’s look at some examples of fillna() along with mean(),

    Pandas: Replace NaN with column mean

    We can replace the NaN values in a complete dataframe or a particular column with a mean of values in a specific column.

    Suppose we have a dataframe that contains the information about 4 students S1 to S4 with marks in different subjects

    This is the DataFrame that we have created,

    If we calculate the mean of values in ‘S2’ column, then a single value of float type is returned

    Output:

    Replace NaN values in a column with mean of column values

    Now let’s replace the NaN values in column S2 with mean of values in the same column i.e. S2,

    Output:

    Since the mean() method is called by the ‘S2’ column, therefore value argument had the mean of the ‘S2’ column values. Then ‘NaN’ values in the ‘S2’ column got replaced with the value we got in the ‘value’ argument i.e. the mean of the ‘S2’ column.

    Replace all NaN values in a Dataframe with mean of column values

    Now if we want to change all the NaN values in the DataFrame with the mean of ‘S2’ we can simply call the fillna() function with the entire dataframe instead of a particular column name. Let me show you what I mean with the example,

    Output:

    Notice that all the values are replaced with the mean on ‘S2’ column values. In the above examples values we used the ‘inplace=True’ to make permanent changes in the dataframe.

    We can even use the update() function to make the necessary updates.

    The above line will replace the NaNs in column S2 with the mean of values in column S2.

    Pandas: Replace NANs with mean of multiple columns

    Let’s reinitialize our dataframe with NaN values,

    Output

    Now if we want to work on multiple columns together, we can just specify the list of columns while calling mean() function

    Output:

    It returned a series containing 2 values i.e. mean of values in column S2 & S3.

    Now let’s replace the NaN values in the columns ‘S2’ and ‘S3’ by the mean of values in ‘S2’ and ‘S3’ as returned by the mean() method. The ‘value’ attribute has a series of 2 mean values that fill the NaN values respectively in ‘S2’ and ‘S3’ columns. Here ‘value’ is of type ‘Series’,

    Output:

    Pandas: Replace NANs with row mean

    We can fill the NaN values with row mean as well. Here the NaN value in ‘Finance’ row will be replaced with the mean of values in ‘Finance’ row. For this we need to use .loc(‘index name’) to access a row and then use fillna() and mean() methods. Here ‘value’ argument contains only 1 value i.e. mean of values in ‘History’ row value and is of type ‘float’

    Conclusion:

    So, these were different ways to replace NaN values in a column, row or complete dataframe with mean or average values.

    How to fill NAN values with mean in Pandas?

    It is a quite compulsory process to modify the data we have as the computer will show you an error of invalid input as it is quite impossible to process the data having ‘NaN’ with it and it is not quite practically possible to manually change the ‘NaN’ to its mean. Therefore, to resolve this problem we process the data and use various functions by which the ‘NaN’ is removed from our data and is replaced with the particular mean and ready be get process by the system.

    How to Fill NaNs in a Pandas DataFrame

    Missing values are common and occur either due to human error, instrument error, processing from another team, or otherwise just a lack of data for a certain observation.

    In this Byte, we'll take a look at how to fill NaNs in a DataFrame , if you choose to handle NaNs by filling them.

    First off, let's create a mock DataFrame with some random values dropped out:

    Let's plot, say, the third column:

    When filled with various techniques — this NaN-filled graph can be replaced with:

    fillna() — Mean, Median, Mode

    You can fill these values into a new column and assign it to the column you wish to fill, or in-place using the inplace argument. Here, we'll be extracting the filled values in a new column for ease of inspection:

    The median, mean and mode of the column are -0.187669 , -0.110873 and 0.000000 and these values will be used for each NaN respectively. This is effectively filled with constant values, where the value being input depends on the entirety of the column.

    First, filling with median values results in:

    With mean values:

    With mode values:

    fillna() — Constant Value

    You can also fill with a constant value instead:

    Free eBook: Git Essentials

    Check out our hands-on, practical guide to learning Git, with best-practices, industry-accepted standards, and included cheat sheet. Stop Googling Git commands and actually learn it!

    This results in a constant value (0) being put instead of each NaN. 0 is close to our median and mean and equal to the mode, so the filled values will resemble that method closely for our mock dataset:

    fillna() — Forward and Backward Fill

    On each row — you can do a forward or backward fill, taking the value either from the row before or after:

    With forward-filling, since we're missing from row 2 — the value from row 1 is taken to fill the second one. The values propagate forward:

    With backward-filling, the opposite happens. Row 2 is filled with the value from row 3:

    Though, if there's more than one NaN in a sequence — these won't do well and can cascade NaNs further down, skewing the data and removing actually recorded values.

    interpolate()

    The interpolate() method delegates the interpolation of values to SciPy's suite of methods for interpolating values. It accepts a wide variety of arguments, including, nearest , zero , slinear , quadratic , cubic , spline , barycentric , polynomial , krogh , piecewise_polynomial , spline , pchip , akima , cubicspline , etc.

    Interpolation is much more flexible and "smart" than just filling values with constants or half-variables such as previous methods.

    Interpolation can properly fill a sequence in a way that no other methods can, such as:

    The default interpolation is linear, and assuming that 1. 5 is likely a 1, 2, 3, 4, 5 sequence isn't far-fetched (but isn't guaranteed). Both constant filling and forward or backward-filling fail miserably here. Generally speaking — interpolation is usually going to be a good friend when it comes to filling NaNs in noisy signals, or corrupt datasets.

    Experimenting with types of interpolation may yield better results.

    Here are two interpolation methods ( splice and polynomial require an order argument):

Добавить комментарий

Ваш адрес email не будет опубликован. Обязательные поля помечены *