In the context of machine learning, consider the iris dataset, what exactly the sample space, attribute space, feature space is?

Question

Sample space, attribute space, feature space are used everywhere.

It seems that a lot posts talk about these concepts without a really specific example.

This post uses a conceptual example a little bit away from real life.

1. there is no real data in there.
2. does't talk about the relation between feature space and sample space
3. setting features like $\text{distance travelled} \in R$, $\text{time duration} \in R$ does not make sense to me (more precisely, I cannot understand), why would time duration be negative value in real life?

Columbia's glossary gives

Feature Space is The set X of feature vectors x that can be used as input to a classifier.

and Feature Vector is A vector of features, denoted by x. In general, a classification function is a function defined on feature vectors and taking values in a set of class labels. set Y.

wiki gives

The vector space associated with these vectors is often called the feature space

Let's talk about these based on the iris dataset, where there are 150 instances and 4 attributes.

Attribute Information:

1. sepal length in cm 
2. sepal width in cm 
3. petal length in cm 
4. petal width in cm 
5. class: 
-- Iris Setosa 
-- Iris Versicolour 
-- Iris Virginica

I assume attribute space and feature space are the same thing, concerned about the 4 attributes.

Based on all above, what exactly the sample space, attribute space/feature space is?

Answer 1

Let's say that we've measured subjects on variables $X_1$, $X_2$, and $X_3$. However, for our regression or classification equation, we want to use $X_1$, $X_3$, $X_1X_3$, $X_1^2$, and $cos(X_1)$ as the predictors ($X_2$ omitted intentionally).

Attribute space: $X_1$, $X_2$, and $X_3$

Feature space: $X_1$, $X_3$, $X_1X_3$, $X_1^2$, and $cos(X_1)$

The attributes are what you measure. The features are what you put into the regression. You may elect to exclude attributes. You may elect to transform attributes, such as interaction terms, squaring (cubing, etc), trig functions, or anything else you find interesting. That's the feature extraction part that your linked answer mentions.

If you know what PCA means, I propose the following exercise.

Exercise: You measure subjects on 10,000 attributes. Then you use PCA and decide to include 15 PCs in your regression equation. What is your feature space?

Edit

You wanted to discuss iris in particular. There are four attributes:

$X_1$: length of pedals

$X_2$: width of pedals

$X_3$: length of sepals

$X_4$: width of sepals

Let's say that you're convinced that some explicit measure of pedal area should make it into your regression. Sure, you don't think pedals are rectangular, but $X_1X_2$ ought to give some idea of pedal area. Then define $X_5 = X_1X_2$. Now run your regression.

$$Y = \beta_0 + \beta_1X_1 + \beta_2X_2 + \beta_3X_3 + \beta_4X_4 + \beta_5X_5$$