Semidefinite programming, binary codes and a graph coloring problem

The largest size of a binary codeLP and SDP bound

Coloring problem and Dual SDP

Semidefinite programming, binary codes and agraph coloring problem

Chao Li

Advised by Prof. Martin

[email protected]

May 12, 2015

Chao Li SDP, codes and a graph coloring problem


Coloring problem and Dual SDP

Outline I

The largest size of a binary codeBinary codesProblem introduction

LP and SDP boundDelsarte’s Linear Programming boundSchrijver’s Semidefinite Programming (SDP) Bound

Coloring problem and Dual SDPThe graph coloring problemThe dual semidefinite programming (D-SDP) formulation



Coloring problem and Dual SDPBinary codesProblem introduction

Binary codes

I Let F = 0, 1 denote the binary field.

I Then F n is the set of all binary strings with n bits.I Hamming distance: ∂(x , y) = k means xi 6= yi for exactly k

values of i . e.g. ∂(000111, 001001) = 3.I A binary error-correcting code C of length n and distance d is

a collection of x ∈ F n where ∂(x , y) ≥ d , ∀x 6= y ∈ C .I For example a code C = 0100, 1000, 0111 is a code of

length 4 and minimum Hamming distance 2.I Is C the largest such code?I A(4, 2) = 8 because we can choose

C ′ = 0000, 0011, 0101, 0110, 1001, 1010, 1100, 1111.




Binary codes

I Let F = 0, 1 denote the binary field.I Then F n is the set of all binary strings with n bits.

I Hamming distance: ∂(x , y) = k means xi 6= yi for exactly kvalues of i . e.g. ∂(000111, 001001) = 3.

I A binary error-correcting code C of length n and distance d isa collection of x ∈ F n where ∂(x , y) ≥ d , ∀x 6= y ∈ C .

I For example a code C = 0100, 1000, 0111 is a code oflength 4 and minimum Hamming distance 2.

I Is C the largest such code?I A(4, 2) = 8 because we can choose

C ′ = 0000, 0011, 0101, 0110, 1001, 1010, 1100, 1111.




Binary codes

I Let F = 0, 1 denote the binary field.I Then F n is the set of all binary strings with n bits.I Hamming distance: ∂(x , y) = k means xi 6= yi for exactly k

values of i . e.g. ∂(000111, 001001) = 3.

I A binary error-correcting code C of length n and distance d isa collection of x ∈ F n where ∂(x , y) ≥ d , ∀x 6= y ∈ C .



C ′ = 0000, 0011, 0101, 0110, 1001, 1010, 1100, 1111.




Binary codes



a collection of x ∈ F n where ∂(x , y) ≥ d , ∀x 6= y ∈ C .



C ′ = 0000, 0011, 0101, 0110, 1001, 1010, 1100, 1111.




Binary codes




length 4 and minimum Hamming distance 2.


C ′ = 0000, 0011, 0101, 0110, 1001, 1010, 1100, 1111.




Binary codes




length 4 and minimum Hamming distance 2.I Is C the largest such code?

I A(4, 2) = 8 because we can chooseC ′ = 0000, 0011, 0101, 0110, 1001, 1010, 1100, 1111.




Binary codes




length 4 and minimum Hamming distance 2.I Is C the largest such code?I A(4, 2) = 8 because we can choose

C ′ = 0000, 0011, 0101, 0110, 1001, 1010, 1100, 1111.




Our problem

I Find the maximum size A(n, d) of a binary error-correctingcode C of length n and minimum distance d : ∂(x , y) ≥ d forall x 6= y in C .

I We require large codes with large minimum distance tomaintain high information rate while still allowing recoveryfrom channel noise.

I From 1950s forward, we have seen many constructions andmany upper bounds. But for most values of n and d , A(n, d)is not known.

I We will focus on a recently discovered upper bound.



Coloring problem and Dual SDPDelsarte’s Linear Programming boundSchrijver’s Semidefinite Programming (SDP) Bound

Association scheme

An association scheme (X ,R) with n classes consists of a finite setX , of size v say, together with n + 1 relations Ri on X which,viewed as n + 1 v × v adjacency matrices Di with entries 0 and 1,satisfy

Di = Dᵀi

n∑i=0

Di = J all ones matrix

D0 = I

DiDj =n∑

k=0pk

ij Dk




An example of an association scheme

1 2 3 4 5 61 0 1 1 2 3 32 1 0 1 3 2 33 1 1 0 3 3 24 2 3 3 0 1 15 3 2 3 1 0 16 3 3 2 1 1 0




Bose-Mesner algebra

Given an association scheme (X ,R), the vector space

A = n∑

t=0αtDt | αt ∈ C

is closed under matrix multiplication and constitutes a subalgebraof Cv×v whose dimension is n + 1. This is called the Bose-Mesneralgebra of the association scheme (X ,R).Note that A is also closed under entry-wise multiplication andcontains the identities, I and J , for both of these multiplications.




An example

00 01

10 11

R0

00 01

10 11

R1

00 01

10 11

R2

Figure: Hamming scheme for n = 2




An example

D0 =

1 0 0 00 1 0 00 0 1 00 0 0 1

D1 =

0 1 1 01 0 0 11 0 0 10 1 1 0

D2 =

0 0 0 10 0 1 00 1 0 01 0 0 0

D0,D1,D2 is the usual basis of the Bose-Mesner algebra for theprevious Hamming scheme.




Eigenmatrix Q

The Bose-Mesner algebra admits a basis of positive semidefinitematrices E0,E1, . . . ,Ed and the change of basis matrix Q from Aito Ei is known as the second eigenmatrix and is given byQij = Kj(i) where Kj(x) is the Krawtchouk polynomial

Kj(x) :=j∑

h=0(−1)h(q − 1)j−h

(xh

)(n − xj − h

)

where q = 2 in the binary case.




Characteristic vector

I The characteristic vector x of a code C has one entry for eachc ∈ F n, xc = 1 if c ∈ C ; xc = 0 otherwise.

I For example, for F 2 = 00, 01, 10, 11, a code C = 00, 01will have the characteristic vector [1, 1, 0, 0].

I Defineai = 1

|C |xᵀAix

I Later we call this xC instead of x .




Linear Programming constraints

I xᵀCAixC counts pairs of codewords at distance i .

I ai := 1|C |x

ᵀCAixC is the average number of codewords of

distance i from c ∈ C .I Now define bj := 2n

|C |xᵀCEjxC

I Clearly:

ai ≥ 0a0 = 1a1 = · · · = ad−1 = 0

n∑i=0

ai = |C |






I ai := 1|C |x


distance i from c ∈ C .

I Now define bj := 2n

|C |xᵀCEjxC

I Clearly:

ai ≥ 0a0 = 1a1 = · · · = ad−1 = 0

n∑i=0

ai = |C |






I ai := 1|C |x


distance i from c ∈ C .I Now define bj := 2n

|C |xᵀCEjxC

I Clearly:

ai ≥ 0a0 = 1a1 = · · · = ad−1 = 0

n∑i=0

ai = |C |





I Since Ej 0, xᵀEjx ≥ 0. So bj ≥ 0.

I But we know

Ej = 12n

n∑i=0

Kj(i)Ai

xᵀEjx = 12n

n∑i=0

Kj(i)xᵀAix

xᵀEjx = |C |2n

n∑i=0

Qijai ≥ 0

bj =n∑

i=0aiQij ≥ 0





I Since Ej 0, xᵀEjx ≥ 0. So bj ≥ 0.I But we know

Ej = 12n

n∑i=0

Kj(i)Ai

xᵀEjx = 12n

n∑i=0

Kj(i)xᵀAix

xᵀEjx = |C |2n

n∑i=0

Qijai ≥ 0

bj =n∑

i=0aiQij ≥ 0




Linear Programming formulation for code size

Then we obtain our LP formulation for code size:

max∑n

i=0ais.t. aQ≥ 0

a ≥ 0a0 = 1a1 = · · · = ad−1 = 0

Since every code C gives a feasible solution, with objective value|C |, the summation of ai gives an upper bound on the maximumsize of any code C with length n and minimum distance d .




Implementation of the Linear Programming bound

I We use C++ linked with CPLEX to formulate and solve thelinear programming model.

I If we insist on exact solutions, our program can only solvelinear programming formulations up to n < 32, since whenn = 32 and d = 2 there will be overflow.




Analytic results

I The results indicate A(n, d) = 2n−1 when d = 2,A(n, d) ≈ 2n

n+1 when d = 3 and A(n, d) = 2 when d > 2n/3,which can be proved as well.

I So we can do linear programming without the computer.I Some bounds

I Plotkin boundI Levenshtein boundI MRRW bound




MRRW v.s. Gilbert-Varshamov

MCELIECE-RODEMICH-RUMSEY-WELCH

UPPER BOUND

GILBERT-VARSHAMOV

LOWER BOUND

0.1 0.2 0.3 0.4 0.5

d

n

0.2

0.4

0.6

0.8

1.0

R

Figure: Asymptotic bounds on the best binary codes




Semidefinite Programming (SDP)

I Semidefinite programming is a new branch of conicprogramming, developed since 1990s.

I It searches for solutions on a section of a positive semidefinitecone. Since the semidefinite cone is convex, it’s a convexoptimization problem.

I Inner product 〈C ,X 〉 = tr(CᵀX ).I We call Hermitian matrix X positive semidefinite if vᵀXv ≥ 0

for all v .I X 0 means X is positive semidefinite.







I Inner product 〈C ,X 〉 = tr(CᵀX ).

I We call Hermitian matrix X positive semidefinite if vᵀXv ≥ 0for all v .

I X 0 means X is positive semidefinite.








for all v .

I X 0 means X is positive semidefinite.








for all v .I X 0 means X is positive semidefinite.




Generic form of SDP formulation

The generic form of a SDP formulation is as follows:

max 〈C , χ〉

s.t. 〈Ai , χ〉 = bi (1 ≤ i ≤ m)

〈Bj , χ〉 ≤ dj (1 ≤ j ≤ k)

χ 0 ,

in which χ represents the semidefinite variable, a Hermitian v × vmatrix. Ai , Bj and C are constant matrices. bi and dj are constantnumbers.




Basis for Terwilliger algebraI Let α = (α0, α1, α2, α3), and let α ` n indicate that

n = α0 + α1 + α2 + α3 and all αi ≥ 0.

I Let α2 + α3 = i α1 + α3 = j α1 + α2 = kI Define wt(x) be the Hamming distance between vector x and

the 0 vector, say ∂(0, x).I Define

(Lα)x ,y =

1 if wt(x) = i ,wt(y) = j , ∂(x , y) = k0 o.w .

be a F n × F n matrix. Where a F n × F n matrix means therows and columns of this matrix are both indexed by elementsof F n.

I We call the set Lα : ∀α ` n the usual basis for Terwilligeralgebra.





n = α0 + α1 + α2 + α3 and all αi ≥ 0.I Let α2 + α3 = i α1 + α3 = j α1 + α2 = k

I Define wt(x) be the Hamming distance between vector x andthe 0 vector, say ∂(0, x).

I Define

(Lα)x ,y =








n = α0 + α1 + α2 + α3 and all αi ≥ 0.I Let α2 + α3 = i α1 + α3 = j α1 + α2 = kI Define wt(x) be the Hamming distance between vector x and

the 0 vector, say ∂(0, x).

I Define

(Lα)x ,y =








n = α0 + α1 + α2 + α3 and all αi ≥ 0.I Let α2 + α3 = i α1 + α3 = j α1 + α2 = kI Define wt(x) be the Hamming distance between vector x and

the 0 vector, say ∂(0, x).I Define

(Lα)x ,y =







Example for the basis of Terwilliger algebra

00 01

10 11

R0

00 01

10 11

R1

00 01

10 11

R2

00 01

10 11

R3

00 01

10 11

R4

00 01

10 11

R5

00 01

10 11

R6




00 01

10 11

R7

00 01

10 11

R8

00 01

10 11

R9




Block diagonalization

Schrijver observes that Tn = ∑α`n xαLα | ∀xα ∈ C is a

C*-algebra hence there exists a unitary matrix U and positiveintegers p0, q0, . . . , pm, qm such that U∗TnU is equal to thecollection of all block diagonal matrices. For M ∈ Tn,M =

∑α`n xαLα:

U∗MU =

C0 0 · · · 00 C1 · · · 0...

... . . . ...0 0 · · · Cm




in which Cj is a block diagonal matrix with qj repeated identicalblocks of order pk :

Ck =

Bk 0 · · · 00 Bk · · · 0...

... . . . ...0 0 · · · Bk




By deleting all repeated blocks, we obtain an algebra isomorphism:

ϕ(Tn) 7→

B0 0 · · · 00 B1 · · · 0...

... . . . ...0 0 · · · Bm

Schrijver finds block Bk has size pk = n + 1− 2k and is repeatedqk =

(nk)−( n

k−1)

times.




The exact entries of Bk

Let

βkα =

n∑u=0

(−1)u−α3

(uα3

)(n − 2ku − k

)(n − k − uα2 + α3 − u

)(n − k − uα1 + α3 − u

)

then we will get the kth block matrix Bk as a(n − 2k + 1)× (n − 2k + 1) matrix:∑α`n

(n − 2k

α2 + α3 − k

)− 12(

n − 2kα1 + α3 − k

)− 12

βkαxα

n−k

α2+α3=k,α1+α3=k




Some notations

Define

α(α) = (α0 + α2, 0, α1 + α3, 0)α(α) = (α0 + α1, 0, α2 + α3, 0)α(α) = (α0 + α3, 0, α1 + α2, 0)




Semidefinite matrices

Let C ⊆ F n be a code, and Π be the set of all automorphisms π ofF n. Let x be the characteristic vector of code C . Define

Π0 = π ∈ Π : 0 ∈ π(C)Π1 = π ∈ Π : 0 /∈ π(C)

R = 1|Π0|

∑π∈Π0

xπ(C)xᵀπ(C)

R ′ = 1|Π1|

∑π∈Π1

xπ(C)xᵀπ(C)




Let λα be the number of triples (x , y , z) ∈ C3, where

∂(x , y) = α2 + α3 ∂(x , z) = α1 + α3 ∂(y , z) = α1 + α2

Definexα = 1

|C |( nα0,α1,α2,α3

)λα (1)

Then we can show that

R =∑α`n

xαLα R ′ = |C |2n − |C |

∑α`n

(xα − xα)Lα

are both positive semidefinite matrices.




Linear constraints

Along with the semidefinite conditions, Schrijver’s formulation hasthe following linear constraints:

I xα(0) = 1II 0 ≤ xα ≤ xα, ∀α ∈ T and xα + xα ≤ 1 + xα,∀α ∈ T

III xα = xα′ if (α1, α2, α3) is a permutation of (α′1, α′2, α′3)IV xα = 0 if α1 + α2, α2 + α3, α1 + α3 ∩ 1, . . . , d − 1 6= ∅




Objective functionThe objective function is to maximize |C | =

∑α`n

( nα2+α3

)xα.

Since

λα =

∣∣∣∣∣∣∣(x , y , z) ∈ C3 :

∂(x , y) = α2 + α3∂(x , z) = 0∂(y , z) = α2 + α3

∣∣∣∣∣∣∣

Hence ∑α

λα = |C |2

By substituting (1) into this, we obtain that

|C | =∑α`n

(n

α2 + α3

)xα




Implementation of Schrijver’s SDP formulation

I We use Matlab and CVX to model the previous SDPformulation.

I CVX is a convex optimization package used with MatlabI CVX has the capability modeling and solving semidefinite

programming models.

I We build the linear constraints III & IV implicitly whenvalidating the αs. Since x only depends on α, we eliminatesome αs when we enumerate all

(n+33)

of them.I For constraint III, we assign them the same name if α′ is a

permutation of α in α1, α2 and α3.I Constraint IV allows us to elimintate many variables.







programming models.I We build the linear constraints III & IV implicitly when

validating the αs. Since x only depends on α, we eliminatesome αs when we enumerate all

(n+33)

of them.

I For constraint III, we assign them the same name if α′ is apermutation of α in α1, α2 and α3.

I Constraint IV allows us to elimintate many variables.









(n+33)


permutation of α in α1, α2 and α3.

I Constraint IV allows us to elimintate many variables.









(n+33)


permutation of α in α1, α2 and α3.I Constraint IV allows us to elimintate many variables.




Orthogonality graph

I Observe that (±)1-vectors, u, v in Rn are orthogonal iff thecorresponding binary vectors are at Hamming distance n/2.

I The graph with all 01-tuples as vertices is called theorthogonality graph if x ∼ y iff ∂(x , y) = n

2 . We denote thisby Ω(n). We note that Ω(n) is k-regular for k =

( nn/2).




A quantum information game

I The orthogonality graph coloring problem is inspired from aquantum information game, expanding to classical bits.

I Two players A and B are asked questions xA and xB, coded asn-bit rings satisfying ∂(xA, xB) ∈ 0, n/2. A and B win thegame if they give answers yA and yB, coded as binary string oflength r such that yA = yB ⇔ xA = xB. Galliard et al.pointed out that whether or not the game can always be wonis equivalent to the question

χ(Ω(n)) ≤ r?




The graph coloring problem

We want to find the minimum number of colors required to colorΩ(n) a priori, so that the two questions xA and xB are viewed astwo vertices of Ω(n) and A and B answer their respective questionsby giving the colors of the vertices xA and xB respectively, coded asbinary string of length log2(n) = r . If the two vertices have thesame color, then they win.




The SDP formulation

De Klerk and Pasechnik give a very similar formulation based onSchrijver’s code size upper bound formulation. The only differenceis in linear constraint IV:

xα = 0 if α1 + α2, α1 + α3, α2 + α3 ∩ n/2 = ∅




Implementation of the SDP formulation for graph coloringproblem

Just with moderate modification, we obtained our SDPformulation for the graph coloring problem, and produce the sameresults as de Klerk and Pasechnik did.




Duality form of semidefinite programming

max∑α uαxα

s.t.∑α xαBα C

xα ≥ 0

min 〈C , χ〉

〈Bα, χ〉 ≤ −uαχ 0




Can we generate the dual SDP automatically

I Based on our implementation we are able to transfer theprimal SDP to its dual problem.

I We explicitly assign 1 to x0 and 0 to x1.I We expand the block diagonal matrix by appending the linear

constraints to the end.I We split the positive semidefinite diagonal matrix into the

sum of constant matrices, which are the coefficients of xαs.I Finally we use the constant matrices to formulate the dual

SDP problem.






I We explicitly assign 1 to x0 and 0 to x1.

I We expand the block diagonal matrix by appending the linearconstraints to the end.

I We split the positive semidefinite diagonal matrix into thesum of constant matrices, which are the coefficients of xαs.

I Finally we use the constant matrices to formulate the dualSDP problem.







constraints to the end.

I We split the positive semidefinite diagonal matrix into thesum of constant matrices, which are the coefficients of xαs.









sum of constant matrices, which are the coefficients of xαs.









sum of constant matrices, which are the coefficients of xαs.I Finally we use the constant matrices to formulate the dual

SDP problem.




Implementation

I Use the Matlab code to output the constraints and thesemidefinite cone to text files.

I Use C++ program to parse the text files.I After parsing the text file, the C++ program obtains the

coefficient matrices and output them as text files.I Then we use another Matlab code to parse the text files of

the coefficient matrices and build the dual SDP problem.




Contribution and future work

I Now we have obtained the dual SDP problem and can obtainthe optimal solution of it.

I Our program could easily build and solve the dual SDPproblems for small values of n.

I By comparing the optimal solution of the primal SDP and theoptimal solution of the dual SDP for different values of n, wewant to find the pattern behind them.

I Once the pattern is clear, we can try to prove it on paper forarbitrarily large n.


Semidefinite programming, binary codes and a graph coloring problem

Science

Transcript of Semidefinite programming, binary codes and a graph coloring problem