DSP

Least Squares in Gaussian Noise – Maximum Likelihood

From the previous posts on Linear Regression (using Batch Gradient descent, Stochastic Gradient Descent, Closed form solution), we discussed couple of different ways to estimate the  parameter vector in the least square error sense for the given training set. However, how does the least square error criterion work when the training set is corrupted by [...]

0 comments Read the full article →

Newton’s method to find square root, inverse

Some of us would have used Newton’s method (also known as Newton-Raphson method) in some form or other. The method has quite a bit of history,  starting with the Babylonian way of finding the square root and later over centuries reaching the present recursive way of finding the solution. In this post, we will describe [...]

0 comments Read the full article →

Closed form solution for linear regression

In the previous post on Batch Gradient Descent and Stochastic Gradient Descent, we looked at two iterative methods for finding the parameter vector  which minimizes the square of the error between the predicted value  and the actual output  for all  values in the training set. A closed form solution for finding the parameter vector  is possible, and in this post [...]

8 comments Read the full article →

Stochastic Gradient Descent

For curve fitting using linear regression, there exists a minor variant of Batch Gradient Descent algorithm, called Stochastic Gradient Descent. In the Batch Gradient Descent, the parameter vector  is updated as, . (loop over all elements of training set in one iteration) For Stochastic Gradient Descent, the vector gets updated as, at each iteration the [...]

2 comments Read the full article →

Batch Gradient Descent

I happened to stumble on Prof. Andrew Ng’s Machine Learning classes which are available online as part of Stanford Center for Professional Development. The first lecture in the series discuss the topic of fitting parameters for a given data set using linear regression.  For understanding this concept, I chose to take data from the top [...]

7 comments Read the full article →

Approximate Vector Magnitude Computation

In this post, let us discuss a simple implementation friendly scheme for computing the absolute value of a complex number . The technique called (alpha Max + beta Min) algorithm is discussed in Chapter 13.2 of Understanding Digital Signal Processing, Richard Lyons and is also available online at Digital Signal Processing Tricks – High-speed vector [...]

7 comments Read the full article →

Using CORDIC for phase and magnitude computation

In a previous post (here), we looked at using CORDIC (Co-ordinate Rotation by DIgital Computer) for understanding how a complex number can be rotated by an angle without using actual multipliers. Let us know try to understand how we can use CORDIC for finding the phase and magnitude of a complex number. Basics The CORDIC [...]

0 comments Read the full article →

CORDIC for phase rotation

My understanding of the CORDIC (Co-ordinate Rotation by DIgital Computer) thanks to the nice article in [DSPGURU-CORDIC].

2 comments Read the full article →

Straight line fit using least squares estimate

Two points suffice for drawing a straight line. However we may be presented with a set of data points (more than two?) presumably forming a straight line. How can one use the available set of data points to draw a straight line? A probable approach is to draw a straight line which hopefully minimizes the [...]

6 comments Read the full article →

Interpreting the output of fft() operation in Matlab

It might be interesting to interpret the output of the fft() function in Matlab. Consider the following simple examples. fsMHz = 20; % sampling frequency fcMHz = 1.5625; % signal frequency N = 128; % fft size % generating the time domain signal x1T = exp(j*2*pi*fcMHz*[0:N-1]/fsMHz); x1F = fft(x1T,N); % 128 pt FFT figure; plot([-N/2:N/2-1]*fsMHz/N,fftshift(abs(x1F))) [...]

19 comments Read the full article →