8/3/2019 ENEE 660 HW Sol #1

1/8

1

Electrical and Computer Engineering Department

University of Maryland College Park

ENEE 660

System Theory

Fall 2008

Professor John S. Baras

Solutions to Homework Set #1

8/3/2019 ENEE 660 HW Sol #1

2/8

the binary representation. Let us suppose that #U/EXB is finite and equals n. Then let r be aFibonacci (in binary) number with binary string length larger than n. We can find such a Fibonacci

number no matter what n is because the recursion shows that Fibonacci numbers can be arbitrarilylarge. Then we can write r as r = wvz where w,v,z are binary numbers. If vi is concatenation itimes, then wviz will also be in the Fibonacci sequence for all i, because the states have to cyclethrough a loop since the input string length is longer than n.Consider r = wvz and r = wviz. Let nw, nv, nz be the value of w, v and z treated as binarynumbers. The numerical value of r is

nw2|z|+|v| + nv2

|z| + nz

and that of r is

nw2|z|+i|v| + nv2|z| (1 + 2|v| + + 2(i1)|v|) + nz

Then both r, r are Fibonacci. And r is Fibonacci for all i. Clearly given the recursion thatFibonacci numbers satisfy this cannot be true. Given any n, I can find i for which the recursionwill not be satisfied.However, given the recursion it will become clear in a couple of lectures in the course that I canindeed realize this sequence using a two dimensional linear system. But the two dimensional linearsystem uses infinite number of states as an automaton.

(b) We want to examine if the sequence of prime numbers

P = {2, 3, 5, 7, 11, 13, }

is realizable. We formulate the problem as follows. First we represent each prime number in binaryrepresentation. This gives us the set of strings of 0s and 1s, beginning with a 1, whose value isinterpreted as the binary representation of a prime. So let U = {0, 1}, consider U

R = {u U, first element of u is 1 , u is the binary representation of a prime}

2 is the string 103 is the string 115 is the string 1017 is the string 111

11 is the string 101113 is the string 1101

etc.etc.

This set of strings from

U form R

Take as output set Y = {0, 1}. We can now state the problem as:Has the I/O map:

2

8/3/2019 ENEE 660 HW Sol #1

3/8

XR : U Y

a finite set of Nerode equivalent classes? The answer is no. We prove this below.To prove the result we need two results from number theory. First, it is a fact that the number ofprimes is infinite and therefore there are arbitrarily large primes. Second, it is also a fact (provenby Fermat) that 2p1 1 is divisible by p for any prime p > 2. Or, what is the same, 2p1 1 mod

p.We proceed by contradiction. Suppose #U/EXR = n. Let z be the binary representation of aprime p such that p > 2n. Such a prime exists since there are infinitely many primes. Then we canwrite z = uvw, where the length of v is 1 and uviw (where vi is the concatenation of v , i times)is the binary representation of a prime for all i. This is true because the length of z is longer thann (the number of states) and so there must be a piece of z that creates a loop in the state set.

Let nu, nv, nw be the value of u,v, and w treated as binary numbers. Ifu or w are , the nu or nw,is 0. Let i = p. Then uvpw is the binary representation of a prime q. The numerical value of q is

nu2|w|+p|v| + nv2

|w|(1 + 2|v| + + 2(p1)|v|) + nw ,

where | | is the binary strings length.By Fermats theorem 2(p1) 1 mod p. We raise both sides to the power |v| and get 2(p1)|v| 1mod p. Then 2p|v| = 2(p1)|v|2|v| 2|v| mod p.Let s = 1 + 2|v| + + 2(p1)|v|. Then (2|v| 1)s = 2p|v| 1 = 2|v| 1 mod p. So (2|v| 1)(s 1) isdivisible by p. But 1 |v| n and so 2 2|v| 2n < p. So p cannot divide 2|v| 1, and thereforeit must divide s 1. Or s 1 mod p. But

q = nu2|w|+p|v| + nv2

|w|s + nw

and therefore

q nu2|w|+|v| + nv2

|w| + nw mod p ,

where the r.h.s. is the numerical value ofp. Therefore q p mod p; or p divides q. Since q > p > 1, qcannot be prime. But we started from a q that was prime. So contradiction.

Problem 3

(a) Since U/ f F(U/Ef) (i.e., U

/ f is a subset of the set of functions of U/Ef into itself)

we have

n #F(U/Ef) = nn

Since x f y implies xEfy we also have #U/ f #U

/Ef or n n.

3

8/3/2019 ENEE 660 HW Sol #1

4/8

(b) Here U = {u1, u2, , un1} = Y. All inputs are reset inputs i.e. f(zuk) = uk for all z U

f() = u1

We know in general u f u implies uEfu. Now suppose uEfu. Then f(vuz) = u =last element of the string z from U = f(vuz).

So u f u also.

Therefore #U/ f= #U/Ef.

(c) Here U = {u1, u2, , unn

}, Y = {y1. , yn}. Let {g1, g2, gnn

} be the set of functions ofY into itself. We can re-label them by the inputs, since their numbers are the same. So we

let {gu1

, gu2

, , gunn

} be these functions.

The I/O map is

f() = y1Y

f(zuj) = guj

(f(z))

Now uEfu = f(uz) = f(uz), z U

= guz

(f(u)) = guz

(f(u))where uz is the element of Uuniquely defined by the string z

= f(u) = f(u)

Here uz is an element ofU such that guz

= gzl(gz

l1(. . . (gz

1

(.)))) as functions, where the stringz has the elements z = z1z2 . . . zl. So the elements of U/Ef are in one to one correspondencewith the distinct images of elements of U, which are obviously n, since Y has n elements.The elements of U/ f are therefore the maps of Y into itself induced by elements of U

.

By definition of f these include all the functions guk

. Thus n = nn.

Problem 4

The input-output description is

U = {0, 1}, Y = {0, 1, 2, 3}

U = U

and

f(u) = f(ui; i = 1, n) = (n

i=1

ui) mod 4

Nerode equivalence classes

4

8/3/2019 ENEE 660 HW Sol #1

5/8

x0 = [0] = input strings with 4 1s

x1 = [1] = input strings with 4+ 1 1sx2 = [11] = input strings with 4+ 2 1

sx3 = [111] = input strings with 4+ 3 1

s

where is an arbitrary integer. Now the Nerode equivalence relation off is actually a congruence.Indeed, if u1Efu

2 we have

n1

i=1

u1i

mod 4 =

n2k=1

u2k

mod 4.

Let v, w be strings of length nv, nw respectively. Then

f(vu1w) =

nv

k=1

vk +n1i=1

u1i +nw=1

w

mod 4

=

nv

k=1

vk +n2i=1

u2i +nw=1

w

mod 4

= f(vu2w) .

Therefore, the Nerode and Myhill equivalence classes coincide! So, the Nerode and Myhill realiza-tions coincide as well! The state transition map is

a([x], u) = [xu] = [x] + [u]

where addition is mod 4. Recall from above that [u] = # of 1s in the string u. The read-out mapis

c([x], u) = [x] + [u] .

The number of output values is equal to the number of states. Diagramatically, we have

0,2

0,11,1

1,21,0

0,0

0,3 1,3

x

x

x

x0 1

23

The construction for counting 1s mod 5 is identical. We now have 5 states instead of 4. Yes, theMyhill semigroup is a group. Namely, the additive group Z5. Similarly, for any integer p, countingones mod p leads to a machine with Myhill semigroup Zp (additive).

5

8/3/2019 ENEE 660 HW Sol #1

6/8

Problem 5

The input-output map of the system is

fI/O = XL

with U = {a, b} and Y = {0, 1}. Here, L is the set {anbn : n = 1, 2, }. The criterion forrealizability of this map by a finite state machine is that the set of Nerode equivalence classes mustbe finite. We want to show that there are infinite number of Nerode equivalence classes. Indeed,we claim that ak and an are in different Nerode equivalence classes for k = n. Here, by ak weunderstand a a

k

. Now, recall that

akEfan f(akw) = f(anw)

for all wU.Let w = bk. Then, f(akbk) = 1, while f(anbk) = 0. So ak is not Nerode equivalent to an for k = n.Therefore, there are an infinite number of Nerode equivalence classes.

Problem 6

We let U = R, Y = R, X = R.The parameter set is {0, 1, 2, 3, }, i.e. discrete time. The next state transition map is

(t + 1; t,x,) = x + pt+1 ,

and the readout map is

(t,x,) = t,n+1x(t)

where i,j is the Kronecker delta symbol. To represent the Horner rule we initialize the machine atx(0) = 0.Solving for the state sequence in Horners rule gives

x(t) =t1i=0

ipti

which is a convolution! So in matrix notation

x(1)x(2)

...x(n + 1)

=

1 0 0

1 0. . . 0

... 1. . .

...n n1 1

p1p2

...pn+1

The optimality of Horners rule is discussed in A. Borodins Horners rule is uniquely optimal,in Theory of Machines and Computations (1971), A. Kohavi and A. Paz (editors). From the last

6

8/3/2019 ENEE 660 HW Sol #1

7/8

computation, we deduce that Horners rule is the optimal first order recursion to compute theproduct of a Toeplitz matrix of the above form with any vector!

The input-output map of the system here is the map which associates to a given value(s) of sthe corresponding value(s) of the polynomial. Clearly, if we compute the value of the polynomial

naively (non-recursively) we will need (n + (n 1) + + 1) multiplications

= n(n+1)2

and n

additions. The use of recursions reduces the number of multiplications. Note that when we considerthe simultaneous evaluation of the value of the polynomial for several values of , the reductionobtained by the Horner rule is even more dramatic. Recursions are equivalent to introducingmemory in the computational algorithm. Horners rule requires just a scalar as the state. Inthis sense, the recursion utilizes the minimum memory, i.e. one state variable only. Because ofthe structure of the dynamics this leads to optimality with respect to number of additions andmultiplications, but this is tricky to prove.

Problem 7

You can find more details about Strassens algorithm in V. Strassen, Gaussian Elimination is notOptimal, Numerische Math, Vol. 13, pp. 354-356 (1969).Let

A =

a11 a12a21 a22

, B =

b11 b12b21 b22

, C = AB =

c11 c12c21 c22

Naive multiplication requires 8 multiplications. Strassens algorithm is:

Let

z1 = (a11 + a22)(b11 + b22) 1 multiplicationz2 = (a21 + a22)b11 1 multiplicationz3 = a11(b12 b22) 1 multiplicationz4 = (a11 + a12)b22 1 multiplicationz5 = a22(b11 + b21) 1 multiplicationz6 = (a11 + a21)(b11 + b12) 1 multiplicationz7 = (a12 a22)(b21 + b22) 1 multiplication

Now verify that

c11 = z1 + z5 z4 + z7c12 = z3 + z4c21 = z2 + z5c22 = z1 z2 + z3 + z6

You have used only 7 multiplications. In return, you have used many more additions. Sincemultiplications are more costly i.e. take more time, people are interested in reducing the numberof multiplications. You can also verify that the algorithm works without assuming commutativity

7

8/3/2019 ENEE 660 HW Sol #1

8/8

in the scalar multiplication! As a consequence of this obervation, the algorithm remains valid evenif aij, bij are matrices.

We can, therefore, extend it to higher dimensional matrices with dramatic savings in the numberof multiplications. For example, for 4 4 matrices A, B, naive multiplication C = AB, requires 64multiplications. Now use Strassens algorithm. Partition to block 2 2 matrices

A =

A11 A12A21 A22

, B =

B11 B12B21 B22

.

The product C requires 7 2 2 matrix multiplications. Each 2 2 matrix multiplication requires7 multiplications. So, total required is 7 7 = 49. Think of the savings for 100 100 matrices!Naive multiplication requires 1003 = 106 = 1, 000, 000 multiplications. Strassens algorithmrequires 7log2100 = 77 = 823, 543. indicates the smallest integer larger than.

The input-output map of the system here accepts as inputs the values of the elements of thematrices A, B, and produces as output the values of the elements of the product matrix C = AB.This map is multilinear (in the elements of A, B). To reduce multiplications, we must introduceintermediate computations and proceed recursively. Strassens algorithm computes 7 intermediatevariables, using 7 multiplications. Applications of state realization theory to the evaluation ofmultilinear functions have been considered by researchers. In this particular problem, we do nothave a complete minimality theory; only theories producing lower and upper bounds on the numberof multiplications are needed.

8