13

This line returns the first 4 rows in the dataframe combined for feature_a

combined.iloc[0:4]["feature_a"]

As expected, this next line returns the 2nd, 4th, and 16th rows in the dataframe for column feature_a:

combined.iloc[[1,3,15]]["feature_a"]

This line sets the first 4 rows in the dataframe for feature_a to 77.

combined.iloc[0:4]["feature_a"] = 77

This line does something. Some sort of computations are happening since it takes longer when applied to a longer list.

combined.iloc[[1,3,15]]["feature_a"] = 88

The 2nd, 4th, and 16th rows are not set to 88 when checked with this:

combined.iloc[[1,3,15]]["feature_a"]

How can I set an arbitrary list of rows of a column of a dataframe to a value without taking a massive coding detour?

This scenario seems like it should be pretty straightforward and common.

Stephen Rauch
  • 406
  • 1
  • 7
  • 15
Doug7
  • 313
  • 1
  • 3
  • 7
  • This is a question about programming only (no statistics) and thus belongs on [Stack Overflow](https://stackoverflow.com/) – Jake Westfall Jun 04 '17 at 23:49
  • Without a minimal reproducible example this kind of question would be off topic on stackoverflow as well – Glen_b Jun 05 '17 at 00:31

1 Answers1

26

If you reverse the selectors, and select by column first, it will work fine:

Code:

df.feature_a.iloc[[1, 3, 15]] = 88

Why?

When you did the first (non-working way) you are selecting a non-contiguous section of the data frame. You should have received the warning:

A value is trying to be set on a copy of a slice from a DataFrame. Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-> docs/stable/indexing.html#indexing-view-versus-copy

This is because there are two independent operations taking place.

  1. combined.iloc[[1,3,15]] creates a new dataframe of only three rows, and the frame is necessarily copied. then...
  2. select one column via ["feature_a"] but it is selected against the copy.

So the assignment goes to the copy. There are various ways to fix this, but in this case, it is easier (and cheaper) to select the column first, then select parts of the columns for assignment.

Test Code:

df = pd.DataFrame(np.zeros((20, 3)), columns=['feature_a', 'b', 'c'])
df.feature_a.iloc[[1, 3, 15]] = 88
print(df)

Results:

    feature_a    b    c
0         0.0  0.0  0.0
1        88.0  0.0  0.0
2         0.0  0.0  0.0
3        88.0  0.0  0.0
4         0.0  0.0  0.0
5         0.0  0.0  0.0
6         0.0  0.0  0.0
7         0.0  0.0  0.0
8         0.0  0.0  0.0
9         0.0  0.0  0.0
10        0.0  0.0  0.0
11        0.0  0.0  0.0
12        0.0  0.0  0.0
13        0.0  0.0  0.0
14        0.0  0.0  0.0
15       88.0  0.0  0.0
16        0.0  0.0  0.0
17        0.0  0.0  0.0
18        0.0  0.0  0.0
19        0.0  0.0  0.0
Stephen Rauch
  • 406
  • 1
  • 7
  • 15