Questions tagged [basis-function]

A basis function is an element of a set of functions that span a function space.

Basis functions are spanning sets of functions. These include sums of sine waves to match any waveform, radial basis functions used for artificial neural networks, basis vectors of a linear algebra, and $\Sigma_0^{n-1} t^i$ to span a polynomial of degree $n-1$. For the latter, $\Sigma t^i$ is called a cardinal or standard basis because the coefficients of $t^i$ are all 1's.

42 questions
18
votes
1 answer

Understanding Gaussian Process Regression via infinite dimensional basis function view

It is often said that gaussian process regression corresponds (GPR) to bayesian linear regression with a (possibly) infinite amount of basis functions. I am currently trying to understand this in detail to get an intuition for what kind of models I…
Julian Karch
  • 1,433
  • 1
  • 13
  • 26
14
votes
3 answers

Understanding Gaussian Basis function parameters to be used in linear regression

I'd like to apply the Gaussian basis function into a linear regression implementation. Unfortunately I'm having a hard time understanding a couple parameters in the basis function. Specifically $\mu$ and $\sigma$. My dataset is a 10,000 x 31…
user2743
  • 241
  • 1
  • 2
  • 4
11
votes
2 answers

Splines - basis functions - clarification

I have been reading the very helpful introduction on splines on http://freakonometrics.hypotheses.org/9184 and on http://www.stats.uwo.ca/faculty/braun/ss3859/chapters/splines/splines.pdf, as well as the examples so helpfully given by whuber on this…
Wouter
  • 2,102
  • 3
  • 17
  • 26
7
votes
1 answer

Is spline basis orthogonal?

When we talk about the basis, we have the concept like orthogonal, unit length etc. for vectors. I think the same concept also exist in Fourier basis and Polynomial basis. But how about spline (Say cubic B-spline) basis?
Haitao Du
  • 32,885
  • 17
  • 118
  • 213
7
votes
1 answer

What are periodic version of splines?

In this What's wrong to fit periodic data with polynomials? post, I tried to use Fourier basis expansion and Polynomial basis expansion to fit a toy periodic data (daily temperature data set). I got excellent answer from @Cliff AB and @Aksakal on…
Haitao Du
  • 32,885
  • 17
  • 118
  • 213
5
votes
1 answer

Spline – basis functions

I am trying to wrap my head around splines and the concept of basis functions using the Elements of Statistical Learning. I understand that the goal is to find polynomials that are continuous at first and second derivatives. However, following the…
user1885116
  • 2,128
  • 3
  • 23
  • 26
5
votes
2 answers

What is Dimension of basis in splines

From mgcv package, gam(for generalized additive model fit) function uses the parameter 'k' which is dimension of basis. Can anyone explain to me what does dimension of basis function means in spline in layman terms?
5
votes
1 answer

In using a basis model to represent a cubic spline, what is the meaning of the truncated power basis function and its math in lay terms?

I am going over “Introduction to Statistical Learning” (James, et al). While I understand the concept of a spline (piecewise polynomial with continuous 1st and 2nd derivative at each knot), I’ve been having a lot of trouble understanding this “power…
4
votes
1 answer

Are $h_i(x)=x^{-\alpha_i}$ okay basis functions for fitting?

I have some pairs of data ${(x_1,y_1),..., (x_n,y_n)}$ genereated by some process and would like to fit it with a function so that $y_i \approx \hat{f}(x_i)$. By plotting the $(X,Y)$ on a 2D plot, and eyeballing, we find the relationship of data is…
Tom Bennett
  • 677
  • 4
  • 15
4
votes
1 answer

Please correct my assumption on how regression trees work

I'm trying to understand how regression trees work, I've been experimenting with catboost and xgboost in python, and I'm getting results which I don't expect, can someone please clarify (and apologies in advance if this is a coding error) I've…
David Waterworth
  • 516
  • 3
  • 11
3
votes
1 answer

Difference between s() and ti() terms in mgcv package when applied to one variable

I am using the mgcv package in R to fit logistic GAMs to survey data. In one of my models I use an interaction between two covariates. I am currently trying to fit this model using RStan and so am constructing the basis functions myself using…
3
votes
0 answers

Adaptive knot selection for B-spline fitting

When fitting a B-spline for regression purposes I've seen a lot of cases where knots are fixed uniformly ,but in some situations this could lead to poor estimations because the behaviour of the curve is not uniform. Knots should be denser when…
3
votes
0 answers

How to calculate the output of ns() function in R by hand

ns() function in R can generate the natural cubic spline basis matrix. I checked the reference but still do not know how to calculate such matrix by hand. For example, I have x = (-2.377, 0.029, 0.106, 2.184). Using ns function with df =3, I can…
cow3918
  • 31
  • 4
3
votes
1 answer

bad fit - nomenclature for breeds

Question: What is it called when one uses a basis, like the pure line instead of the sigmoid/logistic, in a manner that grossly departs from the "physics" of the problem? There should be a word for that. Humorous introduction: Lots of physics and…
EngrStudent
  • 8,232
  • 2
  • 29
  • 82
3
votes
0 answers

How are radial basis functions (RBFs) networks extended to use multiple layers?

I am trying to understand the interpretation of radial basis functions (RBFs) as networks and then trying to understand the relationship it has to "normal" neural networks and how to extend them to multiple layers. For this I was watching the…
1
2 3