Moore's Law and Computational Complexity

Notation:

Moore's Law has come to connote the proposition that, in terms of basic operations. , the processing power of a single processor will double approximately every two years. It can be useful to refer to Moore's law in thinking about computational complexity and parallel processing. To this end we will use the term Moore cycle to refer to:

The engineering time cycle required to develop a processor capable of doing twice the number of basic operations per "clock tic" that can be done with today's processors. Read $\QTR{Large}{2n}$ years for $\QTR{Large}{n}$ Moore cycles.

or

The additional number of processors (measured in powers of $\QTR{Large}{2\ }$ ) required to do twice the number of "basic operations" per "clock tic" that can be done with a cluster of a given size. Read processors for $\QTR{Large}{n}$ Moore cycles, where $\QTR{Large}{p}$ is the number of processors presently in the cluster.

Informally, suppose I have a cluster with $\QTR{Large}{p}$ processors and I want $\QTR{Large}{4}$ times the performance. I need to spend $\QTR{Large}{2}$ Moore cycles. Either wait $\QTR{Large}{4}$ years and buy $\QTR{Large}{p\ }$ new processors, or wait $\QTR{Large}{2}$ years and buy $\QTR{Large}{2p}$ new processors, or buy $\QTR{Large}{3p}$ of today's processors. (The extent that, in practice, the promise of this calculation can be realized will be subjected to a good deal scrutiny a bit later in the semester.)

Finally, again informally, it suffices to have table-lookup or a comparison in mind when thinking about the term "basic operation."

______________________________________________________________________

There are two related situations involving complexity that we want to consider:

We apply an algorithm to a small dataset. We would like to apply it to a larger dataset, but, at present, lack the computing resources to do it in a timely way. We want to know how many Moore cycles will be required to do the computation.
We do not want others to be able to apply an algorithm to a given data set (e.g. in cryptography). We need to estimate how many Moore cycles will be required before they will have the necessary compute-power.

____________________________________________________________

Two Examples:

Addition:

Consider the following template for addition

In the worst case, with carries, it requires approximately $\QTR{Large}{2n}$ addition table lookups to add two $\QTR{Large}{n}$ digit numbers. That is, if I have the resources to add two $\QTR{Large}{n}$ digit numbers today, one Moore cycle will be required to add two $\QTR{Large}{2n}$ digit numbers. Note that whether it is $\QTR{Large}{2n}$ addition table lookups to add two $\QTR{Large}{n}$ digit numbers or $\QTR{Large}{kn}$ addition table lookups to add two $\QTR{Large}{n}$ digit numbers, $\QTR{Large}{k}$ some other fixed constant, the conclusion that one Moore cycle will be required to add two $\QTR{Large}{2n}$ digit numbers, is the same .

Multiplication:

Consider the following template for multiplication

Taking into account some arithmetic considerations, worst case, there is a constant $\QTR{Large}{\ k}$ such that multiplication table lookups suffices to multiply to $\QTR{Large}{n}$ digit numbers. If I have those resources today, two Moore cycles will be required before it will be practical to multiply two $\QTR{Large}{2n}$ digit numbers.

_____________________________________________________________

A More General Worst Case Calculation.

Suppose we have analyzed an algorithm $\QTR{Large}{A(d)}$ and determined that for any $\QTR{Large}{n}$ and any $\QTR{Large}{n\ }$ digit number $\QTR{Large}{d}$ , $\QTR{Large}{A(d)}$ can be computed in no more than

basic operations, .

An outline of the calculation:

Using a little algebra, we can show that there is some such that for ,

Hence, for any , and $\QTR{Large}{d}$ any $\QTR{Large}{n\ }$ digit number, $\QTR{Large}{A(d)}$ can be computed in no more than basic operations.
Fixing $\QTR{Large}{n}$ suppose we have the resources to compute $\QTR{Large}{A(d)}$ where $\QTR{Large}{d}$ any $\QTR{Large}{n\ }$ digit number. In particular, we have the resources to do basic operations in an acceptable amount of time. To compute $\QTR{Large}{A(d)}$ where $\QTR{Large}{d}$ any $\QTR{Large}{2n\ }$ digit number would, worst case, require
Restated, for any algorithm with such a polynomial bound, we can choose a fixed $\QTR{Large}{m}$ such that if we can compute $\QTR{Large}{A(d)}$ where $\QTR{Large}{d}$ is any $\QTR{Large}{n\ }$ digit number today, in $\QTR{Large}{m}$ Moore cycles we would have the resources to compute $\QTR{Large}{A(d)}$ for $\QTR{Large}{d}$ any $\QTR{Large}{2n\ }$ digit number.

______________________________________________________________

A More "Complex" Calculation:

Fixing the base in which integers are represented as $\QTR{Large}{10}$ ,Suppose we wanted to check if a given integer $\QTR{Large}{d}$ is a prime. We could simply divide $\QTR{Large}{d}$ by every integer $\QTR{Large}{a<d.}$ If any of these numbers divide $\QTR{Large}{d}$ evenly then $\QTR{Large}{d}$ is not a prime, and if none divide evenly then $\QTR{Large}{d}$ is a prime. Using this brute force method if $\QTR{Large}{d}$ is any $\QTR{Large}{n\ }$ digit number we might have to do approximately divisions. Even if a division was a basic operation, doubling the number of digits would require divisions.

Thus, as opposed to the calculations above, $\QTR{Large}{n}$ increases, the number of Moore cycles required to achieve acceptable performance on $\QTR{Large}{2n}$ digit numbers also increases.

In fact one can do a lot better. Again, a little algebras show that it suffices to try all So rather than divisions we need only do (floor function) $\QTR{Large}{.}$ However, the observation that as $\QTR{Large}{n}$ increases, the number of Moore cycles required to achieve acceptable performance on $\QTR{Large}{2n}$ digit numbers also increases, still holds.