- DSP log - http://www.dsplog.com -

Newton’s method to find square root, inverse

Posted By Krishna Sankar On December 25, 2011 @ 7:36 pm In DSP | 1 Comment

Some of us would have used Newton’s method [1] (also known as Newton-Raphson method) in some form or other. The method has quite a bit of history,  starting with the Babylonian way of finding the square root [2] and later over centuries reaching the present recursive way of finding the solution. In this post, we will describe Newton’s method and apply it to find the square root and the inverse of a number.

Geometrical interpretation

We know that the derivative of a function $f(x)$ at $x_0$ is the slope of the tangent (red line) at $x_0$ i.e.,

$f^'(x_0) = \frac{f(x_0)}{x_0-x_1}$.

Rearranging, the intercept of the tangent at x-axis is,

$x_1 = x_0 - \frac{f(x_0)}{f^'(x_0)}$.

From the figure above, can see that the tangent (red line) intercepts the x-axis at $x_1$ which is closer to  the $f(x)=0$ where compared to $x_0$. Keep on doing this operation recursively, and it converges to the zero of the function OR in another words the root of the function.

In general for $n^{th}$iteration, the equation is :

$\Huge x_{n+1} = x_{n} - \frac{f(x_n)}{f^'(x_n)}$.

Finding square root

Let us, for example try to use this method for finding the square root of D=100. The function to zero out in the Newton’s method frame work  is,

$f(x) = x^2 - D$, where $D=100$.

The first derivative is

$f^'(x) = 2x$.

The recursive equation is,

$x_{n+1} = x_{n} - \frac{x_n^2-D}{2x_n}$.

Matlab code snippet

clear ; close all;
D = 100; % number to find the square root
x = 1; % initial value
for ii = 1:10
fx = x.^2 - D;
f1x = 2*x;
x = x - fx/f1x;
x_v(ii) = x;
end
x_v' =
50.5000000000000
26.2400990099010
15.0255301199868
10.8404346730269
10.0325785109606
10.0000528956427
10.0000000001399
10.0000000000000
10.0000000000000
10.0000000000000

We can see that the it converges within around 8 iterations. Further, playing around with the initial value,

a) if we start with initial value of x = -1, then we will converge to -10.

b) if we start with initial value of x = 0, then we will not converge

and so on…

Finding inverse (division)

Newton’s method can be used to find the inverse of a variable D. One way to write the function to zero out is$f(x) = xD - 1$, but we soon realize that this does not work as we need know $\frac{1}{D}$ in the first place.

Alternatively the function to zero out can be written as,

$f(x) = \frac{1}{x} - D$.

The first derivative is,

$f^'(x) = -\frac{1}{x^2}$.

The equation in the recursive form is,

$\begin{array}{lll}x_{n+1} &= & x_{n} - $$\frac{\frac{1}{x}-D}{-\frac{1}{x^2}}$$\\ & = & x_n\(2-x_n)\end{array}$.

Matlab code snippet

clear ; close all;
D = .1; % number to find the square root
x = [.1:.2:1]; % initial value
for ii = 1:10
fx = (1./x) - D;
f1x = -1./x.^2;
x = x - fx./f1x;
x_v(:,ii) = x;
end
plot(x_v');
legend('0.1', '0.3', '0.5', '0.7', '0.9');
grid on; xlabel('number of iterations'); ylabel('inverse');
title('finding inverse newton''s method');

The following plot shows the convergence of inverse computation to the right value for different values of$x_0$ for this example matlab code snippet.

Figure : convergence of inverse computation

Finding the minima of a function

To find the minima of a function, we to find where the derivative of the function becomes zero i.e.  $f^'(x) = 0$.

Using Newton’s method, the recursive equation becomes :

$x_{n+1} = x_n - \frac{f^'(x)}{f^{''}(x)} = 0$.

Thoughts

We have  briefly gone through the Newton’s method and its applications to find the roots of a function, inverse, minima etc. However, there are quite a few aspects which we did not go over, like :

a) Impact of the initial value on the convergence of the function

b) Rate of the convergence

c) Error bounds of the converged result

d) Conditions where the convergence does not happen

and so on…

Hoping to discuss those in another post…