Kobe University Repository : ThesisPhysics" beyond the SM at the TeV scale. Typical examples of...

141
Kobe University Repository : Thesis 学位論文題目 Title Flavor Physics and Anomalous Interaction in Gauge-Higgs Unification(ゲージ・ヒッグス統一理論におけるフレーバー物理および 異常な相互作用) 氏名 Author Kurahashi, Nobuaki 専攻分野 Degree 博士(理学) 学位授与の日付 Date of Degree 2012-03-25 資源タイプ Resource Type Thesis or Dissertation / 学位論文 報告番号 Report Number 5576 権利 Rights JaLCDOI URL http://www.lib.kobe-u.ac.jp/handle_kernel/D1005576 ※当コンテンツは神戸大学の学術成果です。無断複製・不正使用等を禁じます。著作権法で認められている範囲内で、適切にご利用ください。 PDF issue: 2021-06-02

Transcript of Kobe University Repository : ThesisPhysics" beyond the SM at the TeV scale. Typical examples of...

  • Kobe University Repository : Thesis

    学位論文題目Tit le

    Flavor Physics and Anomalous Interact ion in Gauge-HiggsUnificat ion(ゲージ・ヒッグス統一理論におけるフレーバー物理および異常な相互作用)

    氏名Author Kurahashi, Nobuaki

    専攻分野Degree 博士(理学)

    学位授与の日付Date of Degree 2012-03-25

    資源タイプResource Type Thesis or Dissertat ion / 学位論文

    報告番号Report Number 甲5576

    権利Rights

    JaLCDOI

    URL http://www.lib.kobe-u.ac.jp/handle_kernel/D1005576※当コンテンツは神戸大学の学術成果です。無断複製・不正使用等を禁じます。著作権法で認められている範囲内で、適切にご利用ください。

    PDF issue: 2021-06-02

  • Doctoral Dissertation

    abbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbcdddddddddd

    Flavor Physics and Anomalous Interaction

    in Gauge-Higgs Unification

    ( )

    eeeeeeeeeefgggggggggggggggggggggggggggggggggggggggggh

    January 2012

    Graduate School of Science, Kobe University

    Nobuaki Kurahashi

  • Doctoral Dissertation

    Flavor Physics and Anomalous Interaction

    in Gauge -Higgs Unification

    January 2012January 2012

    Graduate School of Science, Kobe UniversityGraduate School of Science, Kobe University

    Nobuaki KurahashiNobuaki Kurahashi

    http://www.sci.kobe-u.ac.jp/http://www.kobe-u.ac.jp/http://www.sci.kobe-u.ac.jp/http://www.kobe-u.ac.jp/

  • Abstract

    We discuss flavor physics and anomalous Higgs interactions in the scenario of gauge-Higgs unifi-

    cation (GHU), which is an attractive candidate of physics beyond the Standard Model (SM).

    We firstly discuss flavor mixing and resultant flavor changing neutral current (FCNC) processes

    in the 5 dimensional (5D) SU(3)color ⊗ SU(3) GHU model on the orbifold S1/Z2. As the FCNCprocess we calculate the rate of K0 – K̄0 mixing and D0 – D̄0 mixing due to the exchange of non-

    zero Kaluza-Klein (KK) gluons at the tree level. To achieve flavor violation is a challenging issue

    in the scenario, since the Yukawa couplings are originally higher dimensional gauge interactions.

    We argue that the presence of Z2-odd bulk masses of fermions plays a crucial role as the new

    source of flavor violation. Flavor mixing is argued to be realized by the fact that the bulk mass

    term and brane localized mass term are not diagonalized simultaneously unless bulk masses are

    degenerate. Thus the FCNC process disappears for degenerate bulk masses and as the consequence

    we find “GIM-like” suppression mechanism is operative for the FCNC processes of light quarks.

    We therefore obtain a lower bound on the compactification scale of order O(10TeV) from K0 –K̄0 mixing and of order O(1TeV) from D0 – D̄0 mixing by comparing our prediction on the massdifference of neutral K meson or neutral D meson with recent experimental data, which is much

    milder than what we naively expect assuming only the decoupling of non-zero KK mode gluons.

    We also argue another typical FCNC processes, B0d – B̄0d mixing and B

    0s – B̄

    0s mixing, in the

    more realistic 5D SU(3)color ⊗SU(3)⊗U ′(1) GHU model on the orbifold S1/Z2. In this model, thefermion of 3rd generation has no bulk mass in order to realize the observed top quark mass. Thus

    the “GIM-like” suppression mechanism mentioned above does not work so strongly for the 3rd

    generation containing top and bottom quarks and apparently the constraint from such B0 – B̄0

    mixing is expected to be dangerously large. However, it turns out that the rate of the FCNC

    processes are suppressed by small mixing between the 3rd generation and lighter generations, and

    we obtain lower bounds on the compactification scale of order O(1TeV), which is much milderthan what we naively expect.

    Furthermore, we study CP violation due to the flavor mixing in the above scenarios. To achieve

    CP violation is also a challenging issue in the GHU scenario. Although the flavor mixing is due

    to the “interplay” between brane localized interaction and bulk mass as was mentioned above, it

    generally has complex components. So we consider the general n generation model and point out

    that CP-violating phase appears due to the non-zero KK gluons even in the 2 generation model,

    while at least 3 generation is needed to break the CP symmetry in the Standard Model. We

    also discuss the K0 – K̄0 system as a representative CP-violating FCNC process, and estimate the

    constraint from ∆S = 2 process on the compactification scale by comparing the mass difference

    ∆mK and particularly the parameter εK in the minimal 2 generation model with experimental

    results.

    iii

  • iv Abstract

    Secondly, we argue another interesting topic of GHU, anomalous interaction of Higgs. In the

    scenario, Higgs originates from higher dimensional gauge field and has a physical meaning as

    Aharonov-Bohm phase or Wilson-loop. As its inevitable consequence, physical observables are

    expected to be periodic in the Higgs field. In particular, the Yukawa coupling is expected to

    show some periodic and non-linear behavior as the function of the Higgs vacuum expectation

    value (VEV), while it is just a constant in the SM. For a specific choice of the VEV, the Yukawa

    coupling of KK zero-mode fermion even vanishes. On the other hand, the Yukawa coupling is

    originally provided by higher dimensional gauge interaction, which is clearly linear in the Higgs

    field.

    We discuss how such 2 apparent contradiction about the non-linearity of the Yukawa coupling

    can be reconciled and at the same time how these 2 “pictures” give different predictions in the

    simplest framework of the scenario: SU(3) electroweak model in 5D flat space-time with orbifold

    extra space. The deviation of the Yukawa coupling from the SM prediction is also calculated for

    arbitrary VEV. Furthermore, we study the property of “H-parity” symmetry, which guarantees

    the stability of the Higgs field for a specific choice of the VEV.

    Also discussed is the Higgs interaction with W± and Z0. It turns out that in our framework

    of flat space-time the interaction does not show deviation from the SM, except for the specific

    case of the VEV.

  • Contents

    Abstract iii

    Contents v

    INTRODUCTION 1

    1 Introduction 3

    1.1 Gauge-Higgs unification model as New Physics . . . . . . . . . . . . . . . . . . . . 3

    1.2 Challenging issues of GHU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

    1.3 Flavor physics in GHU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

    1.3.1 Flavor mixing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

    1.3.2 FCNC process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

    1.3.3 CP violation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

    1.4 Anomalous interactions in GHU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

    1.5 Outline of the dissertation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

    1.5.1 Outline of part I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

    1.5.2 Outline of part II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

    ——————————————————–

    I FLAVOR PHYSICS 17

    2 Flavor mixing and FCNC process 19

    2.1 The model : SU(3)color ⊗ SU(3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.1.1 Lagrangian and matter contents . . . . . . . . . . . . . . . . . . . . . . . . 19

    2.1.2 The mass eigenvalues and mode functions of fermion . . . . . . . . . . . . . 21

    2.1.3 Brane localized mass term . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

    2.1.4 Some comments on this model . . . . . . . . . . . . . . . . . . . . . . . . . 22

    2.2 Flavor mixing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

    2.2.1 Identification of the SM quark doublet . . . . . . . . . . . . . . . . . . . . . 24

    2.2.2 Yukawa coupling and the diagonalization . . . . . . . . . . . . . . . . . . . 24

    2.2.3 2 generations model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

    2.3 FCNC process : K0 – K̄0 mixing and D0 – D̄0 mixing . . . . . . . . . . . . . . . . . 28

    v

  • vi Contents

    2.3.1 Natural flavor conservation . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

    2.3.2 Strong interaction and Feynman diagrams . . . . . . . . . . . . . . . . . . . 29

    2.3.3 K0 – K̄0 mixing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

    2.3.3.1 KL –KS mass difference . . . . . . . . . . . . . . . . . . . . . . . . 32

    2.3.3.2 The lower bound on the compactification scale . . . . . . . . . . . 34

    2.3.4 D0 – D̄0 mixing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

    2.3.4.1 The lower bound on the compactification scale . . . . . . . . . . . 37

    2.4 GIM-like suppression mechanism of FCNC . . . . . . . . . . . . . . . . . . . . . . . 41

    2.5 More realistic model : SU(3)color ⊗ SU(3)⊗ U ′(1) . . . . . . . . . . . . . . . . . . 432.5.1 Some issues of introducing 3rd generation . . . . . . . . . . . . . . . . . . . 43

    2.5.2 3 generations model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

    2.5.3 Flavor mixing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

    2.6 FCNC process : B0 – B̄0 mixing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

    2.6.1 Natural flavor conservation . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

    2.6.2 Strong interaction and Feynman diagrams . . . . . . . . . . . . . . . . . . . 50

    2.6.3 The lower bound on the compactification scale . . . . . . . . . . . . . . . . 52

    3 CP violation due to flavor mixing 57

    3.1 The model containing CP-violating phase . . . . . . . . . . . . . . . . . . . . . . . 57

    3.1.1 General counting argument of CP violating phases . . . . . . . . . . . . . . 57

    3.1.2 2 generation case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

    3.1.3 CP violation due to the strong interaction . . . . . . . . . . . . . . . . . . . 59

    3.2 ∆S = 2 process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

    3.2.1 K0 – K̄0 system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

    3.2.2 Constraint on the lower bound for 1/R from ∆S = 2 process . . . . . . . . . 61

    Appendices on part I 67

    A KK mode summation SKK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

    B The behavior of parameters for 2 generation model . . . . . . . . . . . . . . . . . . 70

    C Values of the experimental data used in our analysis . . . . . . . . . . . . . . . . . 73

    ——————————————————–

    II ANOMALOUS INTERACTION 77

    4 Anomalous Higgs interaction 79

    4.1 The model : SU(3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

    4.1.1 The equations of motion for fermion . . . . . . . . . . . . . . . . . . . . . . 80

    4.2 Mode functions and mass eigenvalues for fermion . . . . . . . . . . . . . . . . . . . 82

    4.2.1 The boundary condition at y = 0 . . . . . . . . . . . . . . . . . . . . . . . . 83

    4.2.2 The boundary condition at |y| = πR . . . . . . . . . . . . . . . . . . . . . . 844.2.3 Mass spectra for fermion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

    4.2.4 The relation between ψ̂(n)2 (x) and ψ̂

    (n)3 (x) . . . . . . . . . . . . . . . . . . . 86

    4.2.5 The normalization of mode functions . . . . . . . . . . . . . . . . . . . . . . 88

  • Contents vii

    4.3 Anomalous Higgs interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

    4.3.1 The diagonal Yukawa coupling . . . . . . . . . . . . . . . . . . . . . . . . . 91

    4.3.2 On the difference of 2 pictures . . . . . . . . . . . . . . . . . . . . . . . . . 92

    4.3.3 The deviation of Yukawa coupling from the SM prediction . . . . . . . . . . 93

    4.3.4 Numerical results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

    4.4 H-parity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

    4.4.1 H-parity for fermion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

    4.4.2 H-parity for gauge boson . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

    4.5 Higgs interactions with massive gauge bosons . . . . . . . . . . . . . . . . . . . . . 98

    Appendix on part II 99

    D Fermion mass as a function of x and M̄ . . . . . . . . . . . . . . . . . . . . . . . . 101

    ——————————————————–

    SUMMARY 103

    5 Summary and outlook 105

    5.1 Flavor physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

    5.1.1 Flavor mixing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

    5.1.2 CP violation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

    5.2 Anomalous interactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

    Acknowledgments 113

    List of Figures 115

    List of Tables 118

    Bibliography & References 119

  • INTRODUCTIONINTRODUCTION

  • · Chapter 1

    Introduction

    1.1 Gauge-Higgs unification model as New Physics

    The Standard Model (SM) has been very successful in describing observed phenomena. However,

    the mechanism of spontaneous gauge symmetry breaking is still not conclusive. Higgs particle,

    responsible for the spontaneous breaking seems to have various theoretical problems. One of the

    biggest problems is the stability of the electroweak scale against quadratically divergent radiative

    corrections to the Higgs mass. Such divergences imply that the parameter of low energy scale is

    sensitive to contributions of heavy fields with masses lying at the cut-off scale, which in principle

    can reach the Planck scale. It drives the Higgs mass to unacceptably large values unless the

    tree-level bare mass parameter is finely tuned to cancel the large quantum corrections. This

    problem is called hierarchy problem (or fine-tuning problem) and suggests the presence of “New

    Physics” beyond the SM at the TeV scale. Typical examples of physics beyond the SM (PBSM)

    are supersymmetry (SUSY) [64, 81, 160, 171, 172], where such divergences can be removed by a

    symmetry relating boson and fermion [133] and the Randall-Sundrum (RS) model where the

    hierarchy can be understood via a warp factor associated with the bulk dimension [154, 155].1

    Little Higgs (LH) model where Higgs is regarded as pseudo-Nambu-Goldstone boson, inspired

    by dimensional deconstruction [21, 22, 94, 95, 114], also offers promising solution to the problem

    [23,24,41–45,80,115,116,130,151,161–164,168].

    In this thesis, we discuss gauge-Higgs unification (GHU) theory [16, 55, 56, 61, 62, 66, 97–99,

    131, 156] as a scenario of PBSM. During the last several decades much attention has been paid

    to gauge theories in higher dimensions, and GHU in particular is one of the attractive scenarios

    solving the hierarchy problem without invoking SUSY, so many interesting works have been done

    from various points of view [10–12, 46, 52, 59, 74–78, 82, 83, 85, 90, 102, 104–106, 125, 126, 137–139,

    145–149, 166, 167, 170]. In this scenario, Higgs doublet in the SM is identified with the extra

    spatial components of the higher dimensional gauge fields. This is based on the fascinating idea

    that gauge and Higgs fields can be unified in higher dimensions. Remarkable feature is that the

    quantum correction to Higgs mass is insensitive to the cut-off scale of the theory and calculable

    regardless of the non-renormalizability of higher dimensional gauge theory, which is guaranteed

    by the higher dimensional gauge invariance. The radiatively induced finite Higgs mass should

    be understood as to be described by the Wilson-line phase, that is a non-local operator and free

    from UV-divergence. This fact has opened up a new avenue to the solution of the hierarchy

    problem [91]. The finiteness of the Higgs mass has been studied and verified in various models

    1Higher dimensional SUSY model has been also studied [18,57], for example.

    3/128

  • 4/128 1.1 Gauge-Higgs unification model as New Physics

    and types of compactification at 1-loop level [17,51,70,127]2 and even at the 2-loop level [103,140].

    Interestingly, GHU is closely related to other attractive scenarios aimed to solve the hierarchy

    problem, such as dimensional deconstruction and LH model mentioned above [30]. This is not

    surprising, since the theory of dimensional deconstruction can be regarded as a latticized GHU

    where the role of Wilson-line is played by the link-variable. Also, the relation of super-string

    theory to GHU should be emphasized. Namely, the point particle limit of open string sector is

    10-dimensional E8 supersymmetric (pure) Yang-Mills theory, which can be regarded as a sort of

    GHU.

    The SM has another serious problem, i.e. the flavor problem. The Higgs sector of the SM

    has many arbitrary parameters whose values cannot be theoretically predicted, while the gauge

    sector is theoretically uniquely determined by gauge principle. Especially we have no definite idea

    about the origin of quark and lepton masses and generation or flavor mixings, i.e. the origin of

    the Yukawa couplings of Higgs field. Flavor physics is therefore very important clue, not only to

    the confirmation of the SM but also to the search for “New Physics” beyond the SM. In GHU

    model, the fact that the Higgs is a part of gauge fields indicates that the Higgs interactions are

    basically governed by gauge principle. Thus, the scenario may also shed some light on the long

    standing arbitrariness problem in the Higgs interactions.

    2For the case of gravity-GHU, see [88,89].

  • Introduction 5/128

    1.2 Challenging issues of GHU

    To see whether the scenario is viable, it will be of crucial importance to address the following

    questions.

    a . Does the scenario have characteristic and generic predictions on the observables subjectto the precision test ?

    b . Though there is a hope that the problem of the arbitrariness of Higgs interactions may besolved, how are the variety of fermion masses and flavor mixing realized ?

    c . In view of the fact that Higgs interactions are basically gauge interactions with real gaugecoupling constants, how is the CP violation realized ?

    Let us note that the problem b and c are also shared by super-string theories, since the low

    energy effective theory of the open string sector is regarded as a sort of (10-dimensional) GHU,

    as was mentioned earlier.

    Concerning the issue a , it will be desirable to find out finite (UV-insensitive) and calculable

    observables subject to the precision tests, although the theory is non-renormalizable and observ-

    ables are very UV-sensitive in general. Works from such viewpoint already have been done in the

    literature [7, 8, 31,38–40,124,135,136,141].

    In this thesis, especially in part I, we focus on the remaining issues b and c concerning

    flavor physics in the GHU scenario, the argument being based on our recent works [2–4] and [5]

    respectively. Note that the issue c has been also addressed in recent papers [9] and [128]. The

    former claims that CP symmetry is broken “spontaneously” by the vacuum expectation value

    (VEV) of the Higgs field since the Higgs boson behaves as CP-odd scalar filed. In the latter,

    CP violation is achieved by the geometry of the compactified extra space which has a complex

    structure. Although both of them are the CP-violation mechanisms specific to higher dimensional

    gauge theory, CP-violation due to the flavor mixing such as Kobayashi-Maskawa (KM) theory [121]

    is not argued precisely. Then in this thesis we attempt to address the issue of CP violation due

    to the flavor mixing in the GHU scenario.

    It is highly non-trivial problem to account for the variety of fermion masses, flavor mixings and

    the resultant CP violations in this scenario, since the gauge interactions are real and universal for

    all generations of matter fields, while (global) symmetry among generations, say flavor symmetry,

    has to be broken (flavor violation) eventually in order to distinguish each flavor and to get their

    mixings with CP-violating phases.

  • 6/128 1.3 Flavor physics in GHU

    1.3 Flavor physics in GHU

    As we have seen in section 1.1, flavor physics, particularly Flavor Changing Neutral Current

    (FCNC) processes are of special interest. In the SM, FCNC processes do not exist at tree (classical)

    level and induced only at loop (quantum) level. The observation of such FCNC processes provide

    us with valuable information on the all particles as the intermediate states. Therefore, they are

    very suitable to search for possible heavy particle effects of New Physics. The rates of such FCNC

    processes are known to be suppressed by the higher order of the perturbation theory and the small

    mass difference from Glashow-Iliopoulos-Maiani (GIM) mechanism [72] or small flavor mixings.

    Thus the FCNC processes are frequently called “rare processes”.

    Actually, historically the rare processes have played crucial role in the foundation of the

    particle physics. Some of the most important events, for instance, are listed below:

    The charm quark c was introduced in order to naturally suppress the rare processes ofneutral kaon such as K0 – K̄0 mixing (Glashow-Iliopoulos-Maiani [72]).

    The mass of such predicted c-quark was predicted from the calculation of KL –KS massdifference before the discovery of c-quark (Gaillard-Lee [67]).

    The 3rd generation was introduced to implement the observed CP violationin the neutralkaon system (Kobayashi-Maskawa [121]).

    The lower bound on the mass of the top (or truth) quark was imposed by the data onB0 – B̄0 mixing before the direct discovery of t-quark.

    The processes of CP violation are also known to happen rarely in neutral kaon system. In

    fact, the possibility of the observation of CP violation in the kaon decay process was suggested

    in [123] and CP violation had been found via the rare decay of neutral kaon since then [47]. In

    such sense, the physics of CP violation is also expected to provide us with valuable information

    of heavy unknown particles of New Physics. CP violation is also needed to explain an imbalance

    between matter and anti-matter [159]. It is interesting to note that CP violation in the KM model

    of 3 generations [121] is known to be too small to generate the imbalance, although the KM model

    was originally devised as the theory to accommodate CP violation. Thus to give a explanation of

    the observed asymmetry between matter and anti-matter in the universe is one of the unsolved

    theoretical problem, which the SM cannot account for. This also suggests the presence of some

    new mechanism of CP violation implemented by New Physics.

    1.3.1 Flavor mixing

    The gauge group of models for GHU theory must be larger than the SM gauge group in order to

    obtain Higgs fields which transform according to the fundamental representation of SU(2)L. The

    original gauge symmetry can be reduced to that of the SM by compactifying the extra dimensions

    on an orbifold. Orbifolding [69, 118–120, 152, 153] is a technique used to break a gauge group

    without the use of Higgs fields. It has many applications not only in GHU models but also in

    other higher dimensional models [20, 25, 92, 93, 144]. A genuine feature of the higher dimensional

    theories with orbifold compactification is that the gauge invariant bulk mass terms for fermions,

  • Introduction 7/128

    generically written as

    Mϵ(y)ψ̄ψ with ϵ(y) =

    {+1 (y > 0)

    −1 (y < 0)

    where y is extra space coordinate, are allowed. The bulk masses may be different depending on

    each generation and can be an important new source of the violation of flavor symmetry. The

    presence of the mass terms causes the localization of Weyl fermions in 2 different fixed points of the

    orbifold depending on their chiralities and the Yukawa coupling obtained by the overlap integral

    over the extra coordinate y of the mode functions of Weyl fermions with different chiralities is

    suppressed by a factor

    ∼ 2M̄e−M̄(M̄ ≡ π M

    R−1

    )where R is the size of the extra space, which is otherwise just gauge coupling g and universal for all

    flavors. Thus in GHU scenario, fermion masses are all equal and of weak scale MW to start with

    and the observed hierarchical small fermion masses can be achieved without fine tuning thanks

    to the exponential suppression factor e−M̄ .

    One may expect that the bulk masses need not to be diagonal in the base of generation and

    lead to the flavor mixing. Unfortunately, it is not the case: for each representation R of gaugegroup, ψ(R), the bulk mass terms are generically written as

    Mij(R)ϵ(y)ψ̄i(R)ψj(R)

    (Mij(R) : a hermitian matrixi, j : generation indices

    ),

    which can be diagonalized by a suitable unitary transformation keeping the kinetic terms un-

    touched, and therefore we are always able to start with the base where the bulk mass terms are

    diagonalized. So the fermion mass eigenstates are essentially equal to gauge (weak) eigenstates

    and the flavor mixing may not occur in the bulk space.

    We are thus led to introduce a brane localized interaction to achieve the flavor mixings as was

    proposed in [35]. Even in the base where the bulk mass terms are diagonalized, the brane localized

    mass (BLM) terms still have off-diagonal elements in the flavor base in general. It is interesting

    to note that though the brane localized interaction contains theoretically unfixed parameters

    behaving as the source of flavor mixing in this base, the bulk masses still play important roles.

    At first thought, one might think that only the BLM terms are enough to generate the flavor

    mixings since they can be put by hand. However, it is not the case. As we will see below, we

    show that the flavor mixings exactly disappear in the limit of universal bulk masses where the

    hierarchy of fermion masses is absent [2]. The reason is in this limit the bulk mass terms remain

    flavor-diagonal for arbitrary unitary transformation of each representation of bulk fermions, i.e.

    there is no way to distinguish each generation of bulk fermions. By use of this degree of freedom

    the Yukawa couplings are readily made diagonal. Thus, the “interplay” between brane localized

    interaction and bulk mass leads to physical flavor mixing, and the fact that in general 2 types

    of fermion mass terms cannot be diagonalized simultaneously is essential. This is a remarkable

    feature of the GHU scenario, not shared by, e.g. the scenario of Universal Extra Dimension (UED)

    model where flavor mixing may be caused by Yukawa couplings just as in the SM, irrespectively

    of bulk masses.

  • 8/128 1.3 Flavor physics in GHU

    d s

    Ga(n)µ

    s d

    (a) KK gluon exchange

    d s

    γ(n)µ or Z(n)µ

    s d

    (b) KK photon or Z boson exchange

    Figure 1.3.1: Typical example of FCNC process: (non-zero KK mode) gauge bosonexchange diagram at the tree level for K0 – K̄0 mixing

    1.3.2 FCNC process

    Once the flavor mixings are realized it will be important to discuss the resultant FCNC processes,

    which have been playing a crucial role for checking the viability of various New Physics model,

    as is well-known in the case of supersymmetric models. In supersymmetric models the SUSY

    breaking masses of squarks and sleptons can be new source of flavor violation. Thus the condition

    to suppress FCNC processes severely constrain the mechanism of SUSY breaking. This issue was

    first discussed in [58] in the context of extra dimensions. A central issue is whether “natural flavor

    conservation” is realized, i.e. whether FCNC processes at tree level are “naturally” forbidden in

    the GHU scenario. In ordinary 4-dimensional (4D) framework, there exists a useful criteria

    discussed by Glashow and Weinberg (GW) [73] and Paschos [150] to ensure the natural flavor

    conservation:

    Fermions with the same electric charge and the same chirality should possess the same quantumnumbers, such as the 3rd component of weak isospin I3.

    Since our model is expected to reduce to the SM at low energies, it is expected that there is

    no FCNC processes at the tree level with respect to the zero-mode fields. However, as a new

    feature of higher dimensional model, in the low energy processes of zero-mode fermions due to

    the exchange of non-zero Kaluza-Klein (KK) modes of gauge bosons the FCNC processes are

    known to be possible already at the tree level, even though the amplitudes are suppressed by the

    compactification scale Mc = 1/R due to the decoupling of heavy gauge bosons.

    As was mentioned above, the bulk masses of each fermion is a new source of flavor violation.

    This means that the condition of GW in 4D space-time is not enough to ensure natural flavor

    conservation. Namely, the gauge couplings of non-zero KK modes of gauge boson, whose mode

    functions are y-dependent, are no longer universal even for Weyl fermions with definite chirality

    and the same quantum numbers, since the overlap integral of mode function of fermion and KK

    gauge boson depends on the balk mass M . Thus once we move to the base of mass-eigenstates

    FCNC appears at the tree level (See figure 1.3.1).

    As a typical concrete example of FCNC process, we firstly calculate the K0 – K̄0 mixing

    amplitude at the tree level via non-zero KK gluon exchange and obtain the lower bounds for the

    compactification scale 1/R as the predictions of our 5-dimensional (5D) GHU model [2]. What we

    calculate is the dominant contribution to the process, the tree diagram with the exchange of non-

    zero KK gluons. Comparing the obtained finite contribution to the mixing with the experimental

  • Introduction 9/128

    data, we put the lower bound on the compactification scale. Interestingly, the obtained lower

    bound of O(10)TeV is much milder than we naively expect assuming that the amplitude issimply suppressed by the inverse (square) powers of the compactification scale, say O

    (103)TeV.

    We point out the presence of “GIM-like” suppression mechanism of the FCNC process, operative

    for light fermions in the GHU model. As was mentioned above, fermion masses much smaller

    than MW are realized by the localizations of fermions. Larger the bulk mass M , the steeper

    localization of fermion and therefore for the fermions the mode functions of KK gluons seem to

    be almost constant. Thus for light fermions the gauge couplings of KK gluons become almost

    universal, just as in the case of the zero-mode sector.

    Secondly, we turn to the D0 – D̄0 mixing, which is caused by the mixing between up and charm

    quarks [3]. Our mechanisms of the flavor mixing and the suppression of FCNC should be also

    applicable to the up-type quark sector. The D0 – D̄0 mixing is not only the typical FCNC process

    in up-type quark sector, but also plays special role in exploring PBSM. Namely, in the SM the

    ∆C = 2 FCNC process is realized through “box diagram” where internal quarks are of down-

    type, though in addition to such “short distance” (SD) contribution poorly known “long distance”

    contribution due to non-perturbative quantum chromodynamics (QCD) effects are claimed to be

    important. The mass-squared differences of down-type quarks are much smaller than those of

    up-type quarks. Thus the expected SD contribution to the mass difference of neutral D meson

    ∆MD(SD) due to D0 – D̄0 mixing is expected to be small in the SM:

    xD(SD) =∆MD(SM)

    ΓD≲ 10−3 ,

    where ΓD is the decay width of neutral D meson. Hence if the D0 – D̄0 mixing and/or associated

    CP violating observable with relatively large rates are found it suggests the presence of some New

    Physics. As the matter of fact, recently impressive progress has been made by BABAR and Belle

    in the measurement [177]

    xD(Exp) = (1.00± 0.25)× 10−2 .

    We will calculate the dominant contribution to the process at the tree level by the exchange

    of non-zero KK gluons. Comparing the obtained finite contribution to the mixing with the

    allowed range for the New Physics contribution derived from the experimental data, we put the

    lower bound on the compactification scale 1/R. It will be also discussed how the extent of the

    suppression of FCNC process is different depending on the type of contributing effective 4-Fermi

    operators, i.e. the operators made by the product of currents with the same chirality (LL and

    RR type) and different chiralities (LR type).

    Similarly, in addition to these 2 FCNC processes we will also consider the B0 – B̄0 mixing,

    which is caused by the mixing between the down-type quark of 3rd generation and those of first 2

    generations, and estimate the lower bounds on 1/R by comparing the obtained finite contribution to

    the mixing with the allowed range for the New Physics contribution derived from the experimental

    data [4]. However, a serious issue is how to implement the t-quark mass since the bulk mass is

    effective only for light quarks, i.e. the upper bound of fermion mass mf is MW ≃ 80.4GeV whilemt ≃ 173GeV. Therefore it is necessary to modify our model to get the real top mass. Fortunately,it is known that considering the higher rank representation of gauge group the upper bound is

    accordingly increased [37, 134], and we can construct the realistic 3 generation model. Then,

    for the 3rd generation containing top and bottom quarks, the suppression mechanism mentioned

    above is expected not to work so strongly by the absence of bulk masses necessary for realizing the

    http://www-public.slac.stanford.edu/babar/http://belle.kek.jp/

  • 10/128 1.3 Flavor physics in GHU

    observed top mass. So it is expected that the dangerous large FCNC containing the 3rd generation

    such as B0d – B̄0d or B

    0s – B̄

    0s mixing arises and more stringent constraints will be obtained. Thus

    it would be more desirable to discuss the FCNC process in the 3 generation scheme.

    1.3.3 CP violation

    As was argued above, we introduced 2 types of mass terms, i.e. the bulk mass term and the BLM

    term, and the “interplay” between these mass terms is crucial to get flavor mixing. Then, since

    the BLM term can be arbitrary which is put by hand, it generally has some complex phases (in the

    base where the bulk mass terms are diagonalized) and they are expected to induce CP violations.

    Not all complex phases have physical meaning, however, as some of them can be removed by

    “re-phasing” (redefinition of quark fields). In fact, as we will discuss in detail in section 3.1.1, it

    turns out that the (maximal) number of physical complex phases are

    (n− 1)2 for n generation.

    A remarkable feature is that the non-trivial CP-violating phase appears in the interactions between

    the zero-mode fermion and non-zero KK gluons even in 2 generation scheme in our model since

    FCNC vertices exist in the strong interaction [5, 6], while at least 3 generations are needed to

    break the CP symmetry in the SM.

    For the illustrative purpose to confirm the mechanism of CP violation due to the flavor mixing,

    we will see how the realistic quark masses and mixing are reproduced, and calculate the Wilson-

    coefficient caused by the ∆S = 2 process, i.e. K0 – K̄0 mixing, via non-zero KK gluon exchange at

    the tree level in order to compare the mass difference of 2 neutral K mesons ∆mK and especially

    the parameter εK as the typical CP violating observable in the minimal 2 generation model with

    experimental result. We also estimate the lower bound for the compactification scale 1/R by

    comparing the obtained result with the experimental data.

  • Introduction 11/128

    1.4 Anomalous interactions in GHU

    While GHU relying on gauge principle may shed some lights on the long-standing problems

    of Higgs interactions, it is of crucial importance whether the scenario makes its characteristic

    predictions which are not shared by the SM as the inevitable consequence of the fact that Higgs

    is a gauge boson.

    From such point of view, we secondly discuss anomalous Higgs interaction in GHU in part II

    in this thesis. Namely, we argue that in contrast to the case of the SM, Yukawa coupling is

    non-diagonal, in general, even in the base of mass eigenstates of quarks and when focused on the

    KK zero-mode sector, the Yukawa coupling deviates from that of the SM and even vanishes in an

    extreme case. This argument is based on our recent works [86].

    Such anomalous Higgs interactions are known to be inevitable consequence of the Higgs as a

    gauge field. To see this, let us begin with the fact that in gauge theories with spontaneous gauge

    symmetry breaking the fermion mass term is generically written as

    m(v)ψ̄ψ (1.1)

    for a given mass eigenstate of fermion ψ, where m(v) is a function of the VEV v of Higgs field.

    Physical Higgs field h is a shift of the Higgs field from the VEV and therefore the interaction of

    h with ψ is naturally anticipated to be obtained by replacing v by v + h. This procedure works

    perfectly well for the SM. Namely, in the case of the SM m(v) = fv, where f is a Yukawa coupling

    constant, and the replacement v → v + h correctly gives the Yukawa interaction of h with ψ :m(v + h)ψ̄ψ = f(v + h)ψ̄ψ. We also note the Yukawa coupling is given as the 1st derivative of

    the function:

    f =d

    dvm(v) . (1.2)

    So far everything seems to be just trivial.

    We, however, realize that in GHU the situation is not trivial. In GHU, our Higgs field is

    the zero-mode of extra space component of gauge field A(0)y (assuming 5D space-time). Thus

    the VEV v is a constant gauge field, which having vanishing field strength is usually regarded as

    unphysical, i.e. pure gauge. However, in the case where the extra space is a circle S1, non-simply-

    connected space, the zero mode A(0)y has a physical meaning as a Aharonov-Bohm (AB) [15] phase

    or Wilson-loop:

    W = P exp

    {ig

    2

    ∮dy Ay

    }= eig4πRA

    (0)y

    (g, g4 : 5D & 4D gauge coupling

    ).

    where the line integral is along S1 and R is the radius of S1. The contour integral may be regarded

    as a magnetic flux Φ penetrating inside the circle (see figure 1.4.1),

    g4A(0)y = g

    Φ

    2πR,

    and therefore is physical and cannot be gauged away.

    It is interesting to note that Wilson-loop W is a periodic function of A(0)y . In other words,

    Higgs field appears in the form of “non-linear realization” in GHU. Such periodicity in the Higgs

    field never appears in the SM and therefore is expected to lead to quite characteristic prediction of

  • 12/128 1.4 Anomalous interactions in GHU

    extra dimension

    4D

    radius R

    Figure 1.4.1: The contour integral behaves just like a magnetic flux.

    GHU scenario. Namely, as the characteristic feature of GHU, we expect that physical observables

    have periodicity in the Higgs field:

    v −→ v + 2g4R

    . (1.3)

    A similar thing happens in the quantization condition of magnetic flux in super-conductor: Φ =2πn/e (n : integer), where the unit of the quantization 2π/e corresponds to the period in (1.3).

    The effective potential as the function of the Higgs (VEV) is a typical example of the observables

    showing such periodicity:

    V (v) ∝ 34π2

    1

    (2πR)4

    ∞∑n=1

    cos(ng4πRv)

    n5,

    which is the simplified formula for the contributions of the fields with vanishing bulk masses.

    We expect that the mass eigenvalue m(v) in (1.1) also has the periodicity. In fact, we will

    show that the mass eigenvalues for light zero-mode quarks with “Z2-odd” bulk masses are well

    approximated by

    m(v) ∝ sin(g42πRv

    ),

    which leads to a Higgs interactions with quarks, behaving as trigonometric function of h and thus

    non-linear interactions ! Namely,

    m(v + h) ∝ sin{g42πR(v + h)

    }(1.4)

    and the Yukawa coupling, i.e. the coupling of the linear interaction of Higgs hψ̄ψ, is given as

    f =d

    dvm(v) ∝ cosx

    (x ≡ g4

    2πRv

    ). (1.5)

    We now realize that the Yukawa coupling even vanishes for an extreme case of x = π/2.

    This kind of “anomalous” Higgs interaction has been first pointed out in curved RS 5D space-

    time and for the gauge group SO(5) × U(1) model [100, 101, 107–109, 158]. Even the possibilitythat the Higgs, being rather stable, plays the role of dark matter has been pointed out [101].

    We, however, know that the Yukawa interaction given in the original Lagrangian does not

    have such non-linearity and is linear in the physical Higgs field h, just as in the SM:

    ψ̄

    {i/∂ − γ5∂y + iγ5g4

    λ62(v + h)−Mϵ(y)

    }ψ , (1.6)

    • > > >

  • Introduction 13/128

    mnR

    vR

    (a) KK mass eigenvalues of fermion

    mnR

    vR

    (b) The eigenvalues after chiral transformation

    Figure 1.4.2: Mass spectra of KK mode fermion mass in flat space-time

    mnR

    vR

    (a) “Level crossing” (M = 0)

    mnR

    vR

    (b) Mixing among KK modes (M ̸= 0)

    Figure 1.4.3: The level crossing is avoided by the shift of degenerate mass eigenvaluesof O(M).

    which is the relevant part in the SU(3) model we discuss later and λ6 is a Gell-Mann matrix. In

    fact, the KK mass eigenvalues for a specific case of vanishing bulk mass M are known to be linear

    in v:

    mn =n

    R+g42v

    (n : integers

    ).

    In this specific case, although the eigenvalues themselves are linear in v, the mass spectrum as the

    whole is known to be periodic as is seen in figure 1.4.2(a). We note that in this case the Yukawa

    coupling given by (1.2) is just a constant as in the SM, except the specific situation x = π/2. In

    figure 1.4.2(b), which is obtained from figure 1.4.2(a) by chiral transformations for negative KK

    modes n < 0, there appears a level crossing at x = π/2 and derivative cannot be defined. Though

    we expect that the level crossing is lifted once the mixing among the crossing 2 KK modes is

    taken into account, the mixing seems not to be allowed for vanishing bulk mass, because of the

    conservation of extra space component of momentum. We will see later that by introducing the

    bulk mass M the level crossing is avoided as shown in figure 1.4.3(b). This may be understood

    as the result of the violation of translational invariance in the extra space due to the introduction

    of the bulk mass.

    At the first glance, these 2 viewpoints or “pictures”, i.e. the one which claims non-liner Higgs

  • 14/128 1.4 Anomalous interactions in GHU

    interactions as is shown in (1.4) and the other one which claims linear Yukawa interaction of h as

    is shown in (1.6), seem to be contradictory with each another. Both pictures, however, are based

    on some reliable arguments and there should be a way to reconcile these two.

    Hence, the main purpose is to study the interesting properties of anomalous interactions,

    in particular to clearly understand how these 2 pictures are reconciled with each another, in

    the simplest framework of GHU, i.e. SU(3) electro-weak gauge model in 5D space-time with an

    orbifold S1/Z2 as its extra space [122, 165]. As the matter field, we introduce a SU(3) triplet

    fermion. We are also interested in the issue whether these 2 pictures make different predictions

    in some range of supposed energies.

    It will be shown that the Higgs interaction with fermion is linear in h as is seen in (1.6) and

    can be written in the form of matrix in the base of fermion’s 4D mass eigenstates, i.e. KK modes.

    In contrast to the case of the SM, the “Yukawa coupling matrix” is generally non-diagonal. For

    instance in the specific case x = π/2, all diagonal elements are known to disappear and the matrix

    becomes completely off-diagonal. The mass function m(v + h) such as (1.4) is nothing but the

    eigenvalue of the 4D mass operator for the zero-mode fermion, where h is regarded as a constant

    on an equal footing with the VEV v. Namely, it is an eigenvalue of the matrix in the base of all

    KK modes, obtained from the y-integral (y is an extra space coordinate) of the free Lagrangian

    (1.6) with the 4D kinetic term being ignored:∫ πR−πRdy ψ̄

    {γ5∂y − iγ5g4

    λ62(v + h) +Mϵ(y)

    }ψ . (1.7)

    As long as the Yukawa coupling matrix, which is the part linear in h in (1.7) has off-diagonal

    elements, the eigenvalues of the matrix obtained from (1.7) can be non-linear in h. Thus the 2

    pictures are not contradictory with each another. On the other hand, we will point out that the

    predictions for the quadratic Higgs interactions in 2 pictures show some difference when Higgs

    mass and/or Higgs 4-momentum cannot be ignored, which reasonably may be the case in the

    situation of Large Hadron Collider (LHC) experiment or future linear collider.

    In addition, the “H-parity” proposed in [101, 109] to implement the stability of the Higgs

    at x = π/2 is investigated from our own viewpoint in our model. Also discussed is the Higgs

    interaction with massive zero-mode gauge bosons W± and Z0.

  • Introduction 15/128

    1.5 Outline of the dissertation

    This dissertation is organized in 2 parts. Part I, containing 2 chapters (chapter 2 & chapter 3, is

    about the flavor physics, and part II, containing 1 chapter (chapter 4), is about the anomalous

    interactions. After we discuss these topics, we devote in chapter 5 for the summary.

    1.5.1 Outline of part I

    In chapter 2, after introducing 5D SU(3)color ⊗ SU(3) GHU model in section 2.1, we perform ageneral analysis how the flavor mixing is realized in the context of the GHU scenario in section 2.2.

    In section 2.3, as an application of the flavor mixing discussed in section 2.2, we calculate the

    mass difference of neutral K-mesons (D-mesons) caused by the K0 – K̄0 mixing(D0 – D̄0 mixing

    )via non-zero KK gluon exchange at the tree-level. We also obtain the lower bounds for the

    compactification scale from these FCNC processes by comparing the obtained result with the

    experimental data. The origin of the “GIM-like” suppression mechanism of FCNC process is

    discussed in section 2.4, emphasizing the importance of the localization of quark fields and the

    fact that FCNC is controlled by the non-degeneracy of quark masses, which is specific to the

    GHU. Also discussed is the origin of the different extent of the suppression depending on the

    chirality of the relevant 4-Fermi operator.

    Furthermore, we also introduce a more realistic 5D SU(3)color ⊗ SU(3) ⊗ U ′(1) GHU modeland briefly summarize how the flavor mixing is realized in section 2.5, which is clarified and

    described in detail in section 2.2. In section 2.6, we calculate the mass difference of neutral B

    mesons caused by the B0d – B̄0d mixing and B

    0s – B̄

    0s mixing via non-zero KK gluon exchange at the

    tree-level, similarly to section 2.3. We also estimate the constraints on the compactification scale

    from the FCNC processes.

    In chapter 3, in order to analyze the CP violation due to the flavor mixing, we construct the

    2 generation model as the simplest example of CP-violating model and argue the CP violation

    due to the strong interaction in section 3.1. In section 3.2, we discuss the K0 – K̄0 system as

    the typical CP-violating FCNC process, and also estimate the constraint on the compactification

    scale from ∆S = 2 process.

    1.5.2 Outline of part II

    In section 4.1, our model is briefly described and in section 4.2 quark mass eigenvalues together

    with corresponding mode functions are derived. In section 4.3, anomalous Higgs interaction with

    quarks is discussed. First, by use of the wisdom of quantum mechanics, we argue that 2 pictures

    can be reconciled with each another. By use of such wisdom we point out that Yukawa coupling

    of the Higgs with the zero-mode d quark can be calculated in 2 different ways and we confirm

    by explicit calculations that these 2 methods provide exactly the same result. At the same time

    we point out that 2 pictures make different predictions on the quadratic Higgs interaction with

    the quark under some circumstance. The formula to give the deviation of the anomalous Yukawa

    coupling from the SM prediction for an arbitrary Higgs VEV is obtained and an approximated

    formula for light quarks is shown to be in good agreement with exact result. In section 4.4,

    H-parity symmetry is discussed and we show that only in the specific case of x = π/2 the parity

    symmetry is not broken spontaneously, and therefore meaningful. In section 4.5, we address the

    issue of Higgs interaction with massive gauge bosons W± and Z0. We show that except for the

    specific case x = π/2 the Higgs interaction is always linear and there is no deviation from the SM

    prediction, in contrast to the result in refs. [107,108].

  • Part I

    FLAVOR PHYSICSFLAVOR PHYSICS

  • · Chapter 2

    Flavor mixing and FCNC process

    In this chapter, we firstly discuss flavor mixing in the 5D SU(3)color ⊗ SU(3) GHU model com-pactified on an orbifold S1/Z2 and resulting FCNC processes, K0 – K̄0 mixing and D0 – D̄0 mixing

    in the 2 generation model. Also we secondly discuss flavor mixing and resulting FCNC processes,

    B0d – B̄0d and B

    0s – B̄

    0s mixing in more realistic 3 generation model in the SU(3)color⊗SU(3)⊗U ′(1)

    GHU scenario.

    This argument is mainly based on [2] and [3] for the 2 generation model(K0 – K̄0 mixing and

    D0 – D̄0 mixing)and [4] for the 3 generation model

    (B0 – B̄0 mixing

    ).

    2.1 The model : SU(3)color ⊗ SU(3)

    We consider a 5D SU(3)color ⊗SU(3) GHU model compactified on an orbifold S1/Z2 with a radiusR of S1. The SU(3) unifies the electro-weak interactions SU(2) ⊗ U(1). As matter fields, weintroduce n generations of bulk fermion in the fundamental representation and the (complex

    conjugate of) 2nd-rank symmetric tensor representation of SU(3) gauge group,

    ψi(3) = Qi3 ⊕ di

    ψi(6̄) = Σi ⊕Qi6 ⊕ ui(i = 1, 2, · · · , n

    ),

    which contain ordinary quarks of the SM in the zero-mode sector, i.e. a pair of SU(2) doublet

    Qi3 and Qi6, and SU(2) singlets d

    i and ui. ψi(6̄) also contain SU(2) triplet exotic states Σi [35].

    2.1.1 Lagrangian and matter contents

    The bulk Lagrangian is given by

    L =− 12Tr(FMNF

    MN)− 1

    2Tr(GMNG

    MN)

    + ψ̄i(3){i /D3 −Miϵ(y)

    }ψi(3) +

    1

    2Tr[ψ̄i(6̄)

    {i /D6 −Miϵ(y)

    }ψi(6̄)

    ],

    where

    FMN = ∂MAN − ∂NAM − ig[AM , AN

    ],

    GMN = ∂MGN − ∂NGM − igs[GM , GN

    ],

    19/128

  • 20/128 2.1 The model : SU (3)color ⊗ SU (3)

    /D3ψi(3) = ΓM (∂M − igAM − igsGM )ψi(3) ,

    /D6ψi(6̄) = ΓM

    [∂Mψ

    i(6̄)+ ig

    {A∗Mψ

    i(6̄) + ψi(6̄)A†M

    }− igsGMψi(6̄)

    ].

    The gauge fields AM and GM are written in a matrix form, e.g. AM = AaM

    λa

    2 in terms of

    Gell-Mann matrices λa. It should be understood that AM in the covariant derivative DM =

    ∂M − igAM − igsGM acts properly depending on the representations of the fermions and GM actson the color indices. M,N = 0, 1, 2, 3, 5 and the 5D gamma matrices are

    ΓM =(γµ , iγ5

    ) (µ = 0, 1, 2, 3

    ). (2.1)

    g and gs are 5D gauge coupling constants of SU(3) and SU(3)color, respectively. Mi are Z2-odd

    generation dependent bulk mass parameters of the fermions with the sign function

    ϵ(y) =

    {+1 (y > 0)

    −1 (y < 0). (2.2)

    As was discussed in the introduction, here we take the base where the bulk mass term is flavor-

    diagonal.

    The periodic boundary condition is imposed along S1 and Z2 parity assignments are taken for

    gauge fields as

    Aµ =

    (+,+) (+,+) (−,−)(+,+) (+,+) (−,−)(−,−) (−,−) (+,+)

    , Ay = (−,−) (−,−) (+,+)(−,−) (−,−) (+,+)

    (+,+) (+,+) (−,−)

    , (2.3a)Gµ =

    (+,+) (+,+) (+,+)(+,+) (+,+) (+,+)(+,+) (+,+) (+,+)

    , Gy = (−,−) (−,−) (−,−)(−,−) (−,−) (−,−)

    (−,−) (−,−) (−,−)

    , (2.3b)where (+,+) etc. stand for Z2 parities at fixed points y = 0 and y = πR, respectively. We can see

    that the gauge symmetry SU(3) is explicitly broken to SU(2)×U(1) by the boundary conditions.The gauge fields with Z2 parities (+,+) and (−,−) are mode-expanded by use of mode functions,which are just trigonometric functions, i.e.

    Sn(y) ≡1√πR

    sinn

    Ry , Cn(y) ≡

    1√2πR

    (n = 0)

    1√πR

    cosn

    Ry (n ̸= 0)

    . (2.4)

    The fermions are assigned the following Z2 parities with all colors having the same parity:

    Ψi(3) ={Qi3L(+,+) +Q

    i3R(−,−)

    }⊕{diL(−,−) + diR(+,+)

    },

    Ψi(6̄) ={ΣiL(−,−) + ΣiR(+,+)

    }⊕{Qi6L(+,+) +Q

    i6R(−,−)

    }⊕{uiL(−,−) + uiR(+,+)

    },

    where Qi3 and Qi6 are SU(2) doublets and d

    i and ui are SU(2) singlets. ψi(6̄) also contain SU(2)

    triplet exotic states Σi written in a form of 2 × 2 symmetric matrix [35]. In this way a chiraltheory is realized in the zero-mode sector by Z2 orbifolding.

  • Flavor mixing and FCNC process 21/128

    2.1.2 The mass eigenvalues and mode functions of fermion

    Let us derive fermion mass eigenvalues and mode functions necessary for the argument of flavor

    mixing.

    The fundamental representation ψi(3) is expanded by an ortho-normal set of mode functions

    as follows:

    ψi(3) =

    Qi3Lf

    iL(y) +

    ∞∑n=1

    {Q

    i(n)3L f

    i(n)L (y) +Q

    i(n)3R Sn(y)

    }diRf

    iR(y) +

    ∞∑n=1

    {di(n)R f

    i(n)R (y) + d

    i(n)L Sn(y)

    } (n ≥ 1) . (2.5)

    The mode functions are given in [9]:

    f iL(y) =

    √Mi

    1− e−2πRMie−Mi|y| , f iR(y) =

    √Mi

    e2πRMi − 1eMi|y| , (2.6a)

    fi(n)L (y) =

    1√πR

    {n/R

    mincos

    n

    Ry − Mi

    minϵ(y) sin

    n

    Ry

    }, (2.6b)

    fi(n)R (y) =

    1√πR

    {n/R

    mincos

    n

    Ry +

    Mimin

    ϵ(y) sinn

    Ry

    }(2.6c)

    with

    min ≡√M2i +

    ( nR

    )2.

    The mode functions f iL and fiR are those for the zero modes, and f

    i(n)L and f

    i(n)R are for non-zero

    KK modes. We can see that before the spontaneous electroweak symmetry breaking the fermion

    mass terms are diagonalized by use of these mode functions:∫ πR−πRdy ψ̄i(3)

    {iΓ y∂y −Miϵ(y)

    }ψi(3) =

    ∞∑n=1

    min

    (Q̄

    i(n)3 Q

    i(n)3 − d̄

    i(n)di(n))

    −→−∞∑n=1

    min

    (Q̄

    i(n)3 Q

    i(n)3 + d̄

    i(n)di(n)). (2.7)

    In the 2nd line, a chiral rotation Qi(n)3 → ei

    π2γ5Q

    i(n)3 is performed.

    The 2nd-rank symmetric tensor representation 6̄ in a matrix form can be decomposed into 3

    different SU(2)× U(1) representations as follows:

    ψi(6̄)=

    iσ2Σi

    (iσ2)T 1√

    2iσ2Qi6

    1√2Qi†6(iσ2)T

    ui

    , (2.8)where iσ2 denotes an SU(2) invariant anti-symmetric tensor

    (iσ2)αβ

    = ϵαβ. Each component is

    expanded by the same mode functions (2.6) as in the fundamental representation:

    Σi = ΣiRfiR(y) +

    ∞∑n=1

    {Σi(n)R f

    i(n)R (y) + Σ

    i(n)L Sn(y)

    },

  • 22/128 2.1 The model : SU (3)color ⊗ SU (3)

    Qi6 = Qi6Lf

    iL(y) +

    ∞∑n=1

    {Q

    i(n)6L f

    i(n)L (y) +Q

    i(n)6R Sn(y)

    },

    ui = uiRfiR(y) +

    ∞∑n=1

    {ui(n)R f

    i(n)R (y) + u

    i(n)L Sn(y)

    }.

    The mass terms of ψ(6̄) are also diagonalized, ignoring the VEV of Ay:

    Tr ψ̄i(6̄){iΓ y∂y −Miϵ(y)

    }ψi(6̄) =−

    ∞∑n=1

    min

    (Tr Σ̄i(n)Σi(n) − Q̄i(n)6 Q

    i(n)6 + ū

    i(n)ui(n))

    −→−∞∑n=1

    min

    (Tr Σ̄i(n)Σi(n) + Q̄

    i(n)6 Q

    i(n)6 + ū

    i(n)ui(n)),

    where a chiral rotation Qi(n)6 → ei

    π2γ5Q

    i(n)6 is performed.

    2.1.3 Brane localized mass term

    We notice that there are 2 left-handed quark doublets Q3L and Q6L per generation in the zero-

    mode sector in this model, which are massless before electro-weak symmetry breaking. In a

    simplified 1 generation case, for instance, one of 2 independent linear combinations of these

    doublets should correspond to the ordinary quark doublet of the SM, but the other one is an

    exotic state. Moreover, we have an exotic fermion ΣR. We therefore introduce brane localized 4D

    Weyl spinors to form SU(2)×U(1) invariant brane localized Dirac mass terms in order to removethese exotic massless fermions from the low-energy effective theory [13,35].

    LBLM =∫ πR−πRdy

    √2πR δ(y)Q̄iR(x)

    {ηijBLMQ

    j3L(x, y) + λ

    ijBLMQ

    j6L(x, y)

    }+

    ∫ πR−πRdy

    √2πRmBLMδ(y − πR)Tr

    {Σ̄iR(x, y)Σ

    iL(x)

    }+ h.c. , (2.9)

    where QR and ΣL are the brane localized Weyl fermions of the doublet and the triplet of SU(2)

    respectively. The n×n matrices ηijBLM, λijBLM and mBLM are mass parameters. These BLM terms

    are introduced at opposite fixed points such that QR (ΣL) couples to Q3,6L (ΣR) localized on the

    brane at y = 0 (y = πR). Let us note that the matrices ηijBLM, λijBLM can be non-diagonal, which

    causes the flavor mixing [35].

    2.1.4 Some comments on this model

    Some comments on this model are in order. The predicted Weinberg angle of this model is not

    realistic, sin2θW = 3/4. Possible modification is to introduce an extra U(1)1 or the brane localized

    gauge kinetic term [165]. However, the wrong Weinberg angle is irrelevant to our argument, since

    our interest is in the flavor mixing and resultant K0 – K̄0 mixing and D0 – D̄0 mixing (and also

    B0 – B̄0 mixing) via KK gluon exchange in the QCD sector, whose amplitude is independent of

    the Weinberg angle.

    Second, in our model the bulk masses of fermions are generation-dependent, but are taken

    as common for both ψi(3) and ψi(6̄). In general, the bulk masses of each representation are

    1An extra U ′(1) is introduced when we construct the realistic 3 generations model in section 2.5.

  • Flavor mixing and FCNC process 23/128

    mutually independent and there is no physical reason to take such a choice. It would be justified

    if we have some Grand Unified Theory (GUT) where the 3 and 6̄ representations are embedded

    into a single representation of the GUT gauge group. For instance, if we consider the following

    gauge symmetry breaking pattern

    Sp(8) −→ Sp(6)× SU(2) −→ SU(3)× U(1)× SU(2) ,

    then we find that 3 and 6̄ of SU(3) can be embedded into the adjoint representation 36 of

    Sp(8) [169]. This is because the adjoint representation is decomposed as follows;

    36 −→ (1,3)⊕ (21,1)⊕ (6,2) −→ (1,3)⊕ (1⊕ 6⊕ 6̄⊕ 8,1)⊕ (3⊕ 3̄,2) .

  • 24/128 2.2 Flavor mixing

    2.2 Flavor mixing

    In the previous section we worked in the base where fermion bulk mass terms are written in a

    diagonal matrix in the generation space. Then the Lagrangian for fermions, which includes Yukawa

    couplings as the gauge interaction of Ay is completely diagonalized in the generation space. Thus

    flavor mixing does not occur in the bulk and the BLM terms for the doubled doublets Q3L and

    Q6L is expected to lead to the flavor mixing. We now confirm the expectation and discuss how

    the flavor mixing is realized in this model.

    2.2.1 Identification of the SM quark doublet

    Let us focus on the sector of quark doublets and singlets, which contain fermion zero modes.

    First, we identify the SM quark doublet by diagonalizing the relevant BLM term,∫ πR−πRdy

    √2πR δ(y)Q̄R(x)

    [ηBLM λBLM

    ][ Q3L(x, y)Q6L(x, y)

    ]

    ⊃√2πR Q̄R(x)

    [ηBLMfL(0) λBLMfL(0)

    ][ Q3L(x)Q6L(x)

    ]

    =√2πR Q̄′R(x)

    [mdiag 0n×n

    ][ QHL(x)QSML(x)

    ], (2.10)

    where [U1 U3

    U2 U4

    ][QHL(x)

    QSML(x)

    ]=

    [Q3L(x)

    Q6L(x)

    ], U Q̄QR(x) = Q

    ′R(x) , (2.11a)

    U Q̄[ηBLMfL(0) λBLMfL(0)

    ][ U1 U3U2 U4

    ]=[mdiag 0n×n

    ]. (2.11b)

    In eq. (2.10), ηBLMfL(0) is an abbreviation of a n × n matrix whose (i, j) element is given byηijBLMf

    jL(0), for instance. U3, U4 are n× n matrices which indicate how the quark doublets of the

    SM are contained in each of Q3L(x) and Q6L(x) and compose a 2n× 2n unitary matrix togetherwith U1, U2, which diagonalizing the BLM matrix. The eigenstate QH becomes massive and

    decouples from the low energy processes, while QSM remains massless at this stage and therefore

    is identified with the SM quark doublet. U1, · · · , U4 satisfy the following unitarity condition:

    U †1U1 + U†2U2 = 1ln×n , (2.12a)

    U †3U3 + U†4U4 = 1ln×n , (2.12b)

    U †1U3 + U†2U4 = 0n×n . (2.12c)

    2.2.2 Yukawa coupling and the diagonalization

    After this identification of the SM doublet, Yukawa couplings are read off from the higher dimen-

    sional gauge interaction of Ay, whose zero mode is the Higgs field H(x):∫ πR−πRdy[−g2ψ̄i(3)Aayλ

    aΓ yψi(3) + gTr{ψ̄i(6̄)Aay(λ

    a)∗Γ yψi(6̄)}]

  • Flavor mixing and FCNC process 25/128

    ⊃−∫ πR−πRdy{g2Q̄i3L(x, y)H(x, y)d

    iR(x, y) +

    g

    2

    √2Q̄i6L(x, y)iσ

    2H∗(x, y)uiR(x, y) + h.c.}

    ⊃− g42

    {⟨H†⟩d̄iR(x)I

    i(00)RL U

    ij3 Q

    jSML(x) +

    √2⟨HT⟩iσ2ūiR(x)I

    i(00)RL U

    ij4 Q

    jSML(x)

    }+ h.c.

    where g4 ≡ g/√

    2πR and the overlap integral of mode function Ii(00)RL is given as

    Ii(00)RL ≡

    ∫ πR−πRdy f iLf

    iR =

    M̄isinh M̄i

    (M̄i ≡ π

    MiR−1

    ), (2.13)

    which behaves as

    Ii(00)RL ∼ 2M̄ie

    −M̄i for M̄i ≫ 1 ,

    thus realizing the hierarchical small quark masses without fine tuning of Mi. We thus know that

    the matrices of Yukawa coupling constant g4Yu/2 and g4Yd/2 are given as

    g42Yd =

    g42I

    (00)RL U3 ,

    g42Yu =

    g42

    √2I

    (00)RL U4 , (2.14)

    where the matrix I(00)RL has elements

    (I

    (00)RL

    )ij= δijI

    i(00)RL . These matrices are diagonalized by bi-

    unitary transformations as in the SM and Cabibbo-Kobayashi-Maskawa (CKM) matrix is defined

    in a usual way [36,121].{Ŷd = diag(m̂d, m̂s, · · · ) = V †dRYdVdLŶu = diag(m̂u, m̂c, · · · ) = V †uRYuVuL

    , VCKM = V†dLVuL , (2.15)

    where all the quark masses are normalized by the W -boson mass as m̂f = mf/MW . A remarkable

    point is that the Yukawa couplings g4Yu/2 and g4Yd/2 are mutually related by the unitarity condition

    eq. (2.12b), on the contrary those are completely independent in the SM. Thus if we set bulk

    masses of fermion to be universal among generations, i.e. M1 = M2 = M3 = · · · = Mn, thenI

    (00)RL is proportional to the unit matrix. In such a case, Y

    †uYu ∝ U †4U4 and Y

    †d Yd ∝ U

    †3U3 can be

    simultaneously diagonalized because of the unitarity condition eq. (2.12b). This means that the

    flavor mixing disappears in the limit of universal bulk masses, as was expected in the introduction.

    In reality, off course the bulk masses should be different to explain the variety of quark masses

    and therefore the flavor mixing does not vanish.

    2.2.3 2 generations model

    For an illustrative purpose to confirm the mechanism of flavor mixing, let us consider the 2

    generations. We will see how the realistic quark masses and mixing are reproduced. The argument

    here will be useful also for the calculation in the next section. For simplicity, we ignore CP

    violation and assume that U3 and U4 are real for the moment. By noting that an arbitrary

    2 × 2 matrix can be written in a form O1MdiagO2 in terms of 2 orthogonal matrices O1,2 and adiagonal matrix Mdiag and by use of unitarity condition (2.12b), 2× 2 matrices U3 and U4 can beparametrized without loss of generality as

    U3 =

    [cos θ −sin θsin θ cos θ

    ][ca1 0

    0 ca2

    ], U4 =

    [cos θ′ −sin θ′

    sin θ′ cos θ′

    ][sa1 0

    0 sa2

    ], (2.16)

  • 26/128 2.2 Flavor mixing

    where sai ≡ sin ai, cai ≡ cos ai. Actually the most general forms of U3 and U4 have a commonorthogonal matrix multiplied from the right, being consistent with (2.12b). The matrix, however,

    can be eliminated by suitable unitary transformation among the members of degenerate doublets

    QSML(x) and has no physical meaning. In another word, the common orthogonal matrix can be

    absorbed into VdL, VuL without changing VCKM. Thus, without loss of generality we can adopt

    the parametrization (2.16).

    Let us note that if we wish, instead of the base where bulk mass term is diagonalized, we

    can move to another base where θ = θ′ = 0 by suitable unitary transformations of Q3 and Q6.

    Then in this base the bulk mass term is no longer diagonal in the generation space unless bulk

    masses are degenerate, and the off-diagonal elements lead to flavor mixing. In the specific case of

    degenerate bulk masses, the bulk mass term is still diagonal and flavor mixing disappears. This

    is another proof of why flavor mixing disappears for degenerate bulk masses.

    The overlap integral (2.13) is parametrized as follows.

    I(00)RL =

    [b1 0

    0 b2

    ]where bi ≡

    M̄isinh M̄i

    . (2.17)

    Now physical observables m̂u, m̂c, m̂d, m̂s and the Cabibbo angle θc are written in terms of a1,

    a2, b1, b2 and θ, θ′. Namely trivial relations{

    det(Ŷ †d Ŷd

    )= m̂2dm̂

    2s

    det(Ŷ †u Ŷu

    )= m̂2um̂

    2c

    ,

    {Tr(Ŷ †d Ŷd

    )= m̂2d + m̂

    2s

    Tr(Ŷ †u Ŷu

    )= m̂2u + m̂

    2c

    (2.18)

    provide through eqs. (2.14), (2.15), (2.16), (2.17) with2

    m̂2dm̂2s = c

    2a1c

    2a2b

    21b

    22 , (2.19a)

    m̂2d + m̂2s =

    1

    2

    {(c2a1 + c

    2a2

    )(b21 + b

    22

    )+(c2a1 − c

    2a2

    )(b21 − b22

    )cos 2θ

    }, (2.19b)

    m̂2um̂2c = 4s

    2a1s

    2a2b

    21b

    22 , (2.19c)

    m̂2u + m̂2c =

    (s2a1 + s

    2a2

    )(b21 + b

    22

    )+(s2a1 − s

    2a2

    )(b21 − b22

    )cos 2θ′ . (2.19d)

    We also note that the θc is given as

    tan 2θc = tan 2(θuL − θdL) , (2.20a)

    tan 2θuL =2sa1sa2

    (b22 − b21

    )sin 2θ′(

    s2a1 − s2a2)(b21 + b

    22

    )−(s2a1 + s

    2a2

    )(b22 − b21

    )cos 2θ′

    , (2.20b)

    tan 2θdL =2ca1ca2

    (b22 − b21

    )sin 2θ(

    c2a1 − c2a2)(b21 + b

    22

    )−(c2a1 + c

    2a2

    )(b22 − b21

    )cos 2θ

    , (2.20c)

    where angles θdL, θuL are angles parameterizing VdL, VuL, respectively. Note that 5 physical

    observables are written in terms of 6 parameters, a1, a2, b1, b2 and θ, θ′.3 So our theory has

    1 degree of freedom, which cannot be determined by the observables. We choose θ′ as a free

    parameter. Then once we choose the value of θ′, other 5 parameters can be completely fixed by

    the observables, by solving eqs. (2.19) and (2.20) numerically for a1, a2, b1, b2 and θ. The result

    is shown in table 2.2.1. (See also figure B.1 in appendix B.)4

    2The conditions(Ŷ †d Ŷd

    )22

    >(Ŷ †d Ŷd

    )11

    and(Ŷ †u Ŷu

    )22

    >(Ŷ †u Ŷu

    )11

    are also necessary, strictly speaking.3For the case of n generations, there are n(n+ 1) parameters in our model.4Note that sin θ′ has the upper and lower limits. As | sin θ′| goes to 1, the bulk mass of the 2nd generation M2

    goes to 0 (i.e. b2 = 1). When sin θ′ takes a value beyond these limits, a solution doesn’t exist.

  • Flavor mixing and FCNC process 27/128

    Table 2.2.1: Numerical result for the relevant parameters fixed by quark masses andCabibbo angle

    sin θ′ s2a1 s2a2

    b21 b22 sin θ

    0.9999 0.000015 0.999998 3.77×10−9 1 0.000150.8 0.0444 0.9954 3.90×10−9 3.33×10−4 0.003290.6704 0.0650 0.9931 3.97×10−9 2.24×10−4 00.6 0.0740 0.9921 4.01×10−9 1.95×10−4 -0.002330.4 0.0923 0.9899 4.13×10−9 1.52×10−4 -0.01000.2 0.101 0.9888 4.24×10−9 1.35×10−4 -0.01820 0.102 0.9887 4.36×10−9 1.30×10−4 -0.0259-0.2 0.0960 0.9895 4.47×10−9 1.35×10−4 -0.0322-0.4 0.0828 0.9910 4.59×10−9 1.52×10−4 -0.0363-0.6 0.0626 0.9934 4.72×10−9 1.96×10−4 -0.0368-0.8 0.0352 0.9964 4.85×10−9 3.38×10−4 -0.0313-0.9999 0.000011 0.999999 4.99×10−9 1 -0.00063

    Thus we have confirmed that observed quark masses and flavor mixing angle can be reproduced

    in our model of GHU. Let us note that in eq. (2.20a) Cabibbo angle θc disappears in the limit of

    universal bulk mass, i.e. M1 =M2 and therefore b1 = b2, as is expected.

    Some comments are in order. One might think that the above analysis of the diagonalization

    of fermion mass matrices restricting only to the zero-mode sector is not complete, since it ignores

    possible mixings between zero-mode and massive exotic states and the zero mode and non-zero KK

    modes given in section 2.1 may mix with each other to form mass eigenstates once the VEV ⟨Ay⟩is switched on. (Let us recall that the SM quark doublets do not mix with brane localized fermion

    by construction.) Such mixings, however, are easily known not to exist in the limit of vanishing

    VEV, ⟨Ay⟩ = 0. Hence, even in the presence of the VEV such mixings will be suppressed by thesmall ratios of the VEV to the compactification scale or large BLMs. Therefore, our analysis is a

    good approximation at the leading order.

    Introducing the source of flavor mixing in the BLMs has already been considered in [53], for

    instance. The difference between their model and ours is that in our model the interplay with

    the bulk masses and the Yukawa couplings in the bulk is crucial, while the flavor mixing is put

    by hand in ref. [53], since Yukawa coupling is not allowed in the bulk in the model.

  • 28/128 2.3 FCNC process : K0 – K̄0 mixing and D0 – D̄0 mixing

    2.3 FCNC process : K0 – K̄0 mixing and D0 – D̄0 mixing

    In this section, we firstly apply the results of the previous section to a representative FCNC process

    due to the flavor mixing i.e. K0 – K̄0 mixing responsible for the mass difference of 2 neutral K

    mesons ∆mK . Then, we also apply to another representative FCNC process due to the flavor

    mixing, D0 – D̄0 mixing similarly.

    2.3.1 Natural flavor conservation

    As we have discussed in the introduction, in our model natural flavor conservation is not realized,

    i.e. FCNC processes are possible, already at the tree level.

    To begin with, let us consider the processes where zero-mode gauge bosons are exchanged.

    We firstly restrict ourselves to the FCNC processes of zero-mode down-type quarks due to gauge

    boson exchange at the tree level. If such type of diagrams exist with a sizable amplitudes, it will

    easily spoil the viability of the model.

    Concerning the zero-mode gauge boson, especially the Z-boson, it is in principle possible

    to cause the tree-level FCNC. The exchanges of zero-mode photon and gluon trivially do not

    possess FCNC, since the mode function of the zero-mode gauge boson is y-independent, and the

    overlap integral of mode functions is universal, i.e. generation independent, just as the kinetic

    terms of fermions are. Thus the gauge coupling of zero-mode gauge boson depends on only the

    relevant quantum numbers such as the 3rd component of weak isospin I3. Therefore the condition

    proposed by GW [73] and Paschos [150] to guarantee natural flavor conservation for the theories

    of 4D space-time is relevant. At the first glance, the GW condition seems to be not satisfied in

    our model, since there are right-handed down-type quarks belonging to different representations,

    i.e. quarks belonging to ψ(3) and ψ(6̄) of SU(3). Then the Z boson exchange seems to yield

    FCNC. Fortunately, however, the down-type quarks belonging to ψ(6̄) (more precisely the triplet

    Σi) is known to have the same quantum number I3 as that of di belonging to ψ(3). Therefore,

    for the down-type quark sector, this 2 generation model satisfies the GW condition and FCNC

    does not arise even after moving to the mass eigenstates.

    Secondly, we focus on the FCNC processes of zero-mode up-type quarks due to the zero-mode

    gauge boson exchange at the tree level. Similarly to the down-type quark sector, the exchanges

    of zero-mode photon and gluon trivially do not possess FCNC in this case, too. Unfortunately,

    however, it turns out that the GW condition is not satisfied for the up-type quark sector. Note

    that we have 2 right-handed up-type quarks belonging to ψ(6̄), SU(2) singlet uiR and a member

    of SU(2) triplet ΣiR in (2.8), and they have different isospin I3, i.e. 0 and 1, while they have

    the same electric charge and chirality. Thus FCNC process due to the exchange of the zero-mode

    Z-boson arises at tree-level. However, the triplet ΣiR is an exotic fermion and acquires large SU(2)

    invariant brane mass. Thus the mixing between uiR and ΣiR is inversely suppressed by the power

    of mBLM in (2.9) and the FCNC vertex of Z-boson can be safely neglected. We may say that

    the GW condition is satisfied in a good approximation in the processes via the zero-mode gauge

    boson exchange. Furthermore, the contribution by the weak gauge boson exchange is expected to

    be small compared with that by the gluon exchange.

    Therefore the remaining possibility is the process via the exchange of non-zero KK mode gauge

    bosons. In this case, the mode functions of non-zero KK mode gauge bosons are y-dependent and

    their couplings to fermions are no longer universal in both down- and up-type quark sectors even if

    the GW condition is met. Namely, non-degenerate bulk masses of fermions for each generation is

  • Flavor mixing and FCNC process 29/128

    a new source of flavor violation and the coupling constants in the effective 4D Lagrangian become

    generation-dependent, thus leading to FCNC after moving to the mass eigenstates.

    Along this line of argument, we study K0 – K̄0 mixing in the down-type quark sector and

    D0 – D̄0 mixing in the up-type quark sector caused by the non-zero KK mode gluon exchange at

    the tree level as the dominant contribution to these FCNC processes.

    2.3.2 Strong interaction and Feynman diagrams

    For such purpose, we derive the strong interaction vertices: restricting to the zero-mode sector of

    down-type quarks and integrating over the 5th dimensional coordinate y, we obtain the relevant

    4D interactions:

    Ls ⊃gs

    2√2πR

    Gaµ

    (d̄iRγ

    µλadiR + Q̄i3Lλ

    aγµQi3L + Q̄i6Lλ

    aγµQi6L

    )+

    ∞∑n=1

    gs2Ga(n)µ

    {d̄iRλ

    aγµdiRIi(0n0)RR +

    (Q̄i3Lλ

    aγµQi3L + Q̄i6Lλ

    aγµQi6L)Ii(0n0)LL

    }⊃ gs

    2√2πR

    Gaµ

    (¯̃diRγ

    µλad̃iR +¯̃diLλ

    aγµd̃iL

    )+

    ∞∑n=1

    gs2Ga(n)µ

    ¯̃diRλ

    aγµd̃jR

    (V †dRI

    (0n0)RR VdR

    )ij

    +

    ∞∑n=1

    gs2Ga(n)µ

    ¯̃diLλ

    aγµd̃jL(−1)n{V †dL

    (U †3I

    (0n0)RR U3 + U

    †4I

    (0n0)RR U4

    )VdL

    }ij. (2.21a)

    Similarly, for up-type quark sector,

    Ls ⊃gs

    2√2πR

    Gaµ(¯̃uiRγ

    µλaũiR + ¯̃uiLλ

    aγµũiL)+

    ∞∑n=1

    gs2Ga(n)µ ¯̃u

    iRλ

    aγµũjR

    (V †uRI

    (0n0)RR VuR

    )ij

    +

    ∞∑n=1

    gs2Ga(n)µ ¯̃u

    iLλ

    aγµũjL(−1)n{V †uL

    (U †3I

    (0n0)RR U3 + U

    †4I

    (0n0)RR U4

    )VuL

    }ij. (2.21b)

    I(0n0)RR is a overlap integral relevant for gauge interaction

    Ii(0n0)RR =

    1√πR

    ∫ πR−πRdy(f iR)2

    cosn

    Ry =

    1√πR

    (2M̄i)2

    (2M̄i)2 + (nπ)2(−1)ne2M̄i − 1

    e2M̄i − 1(2.22)

    where M̄i = πMiR−1 and the mode expansion of gluon

    Gaµ(x, y) =1√2πR

    Gaµ +

    ∞∑n=1

    1√πR

    Ga(n)µ cosn

    Ry

    has been substituted. Let us note that the overlap integrals for left-handed quarks Ii(0n0)LL is

    related to Ii(0n0)RR as

    Ii(0n0)LL = I

    i(0n0)RR

    ∣∣∣M̄i→−M̄i

    = (−1)nIi(0n0)RR , (2.23)

    since the chirality exchange corresponds to the exchange of 2 fixed points. In eq. (2.21), d̃ and ũ

    denote mass eigenstates,(d̃1, d̃2

    )=(d, s)and

    (ũ1, ũ2

    )=(u, c). The derivation of the last line of

    the equation (2.21a) and (2.21b) is easily understood, since ignoring QHL{Q3L ∼ U3QSMLQ6L ∼ U4QSML

    , QiSML =

    [uiL

    diL

    ]

  • 30/128 2.3 FCNC process : K0 – K̄0 mixing and D0 – D̄0 mixing

    and [d1L

    d2L

    ]= VdL

    [d̃1L

    d̃2L

    ],

    [d1R

    d2R

    ]= VdR

    [d̃1R

    d̃2R

    ],

    [u1L

    u2L

    ]= VuL

    [ũ1L

    ũ2L

    ],

    [u1R

    u2R

    ]= VuR

    [ũ1R

    ũ2R

    ].

    We can see from (2.21) that the FCNC appears in the couplings of non-zero KK gluons due

    to the fact that I(0n0)RR is not proportional to the unit matrix (the breaking of universality), while

    the coupling of the zero-mode gluon is flavor conserving, as we expected.

    The Feynman rules necessary for the calculation of K0 – K̄0 mixing can be read off from

    (2.21a).

    dR sR

    Ga(n)µ

    =gs2

    (V †dRI

    (0n0)RR VdR

    )21λaγµR , (2.24a)

    dL sL

    Ga(n)µ

    =gs2(−1)n

    {V †dL

    (U †3I

    (0n0)RR U3 + U

    †4I

    (0n0)RR U4

    )VdL

    }21λaγµL (2.24b)

    and for the calculation of D0 – D̄0 mixing, from (2.21b),

    uR cR

    Ga(n)µ

    =gs2

    (V †uRI

    (0n0)RR VuR

    )21λaγµR , (2.25a)

    uL cL

    Ga(n)µ

    =gs2(−1)n

    {V †uL

    (U †3I

    (0n0)RR U3 + U

    †4I

    (0n0)RR U4

    )VuL

    }21λaγµL (2.25b)

    and the propagator is common for both processes,

    Ga(n)µ G

    b(n′)ν = δnn′δab

    ηµν

    k2 − (n/R)2(’t Hooft-Feynman gauge

    ). (2.26)

    L and R are chiral projection operators. We can verify utilizing the unitarity condition that the

    coefficient vanishes in the limit of universal bulk masses M1 = M2 = · · · = Mm and thereforewhen I

    (0n0)RR is proportional to the unit matrix, as we expect since in this limit flavor mixing just

    disappears;

    V †dRI(0n0)RR VdR

    M1=M2 = ···−−−−−−−−→ V †dRVdRI(0n0)RR ∝ 1lm×m ,

    V †dL

    (U †3I

    (0n0)RR U3 + U

    †4I

    (0n0)RR U4

    )VdL

    M1=M2 = ···−−−−−−−−→ V †dL(U †3U3 + U

    †4U4

    )VdLI

    (0n0)RR ∝ 1lm×m

    and so on. The non-zero KKmode gluon exchange diagrams, which give the dominant contribution

    to the processes ofK0 – K̄0 mixing andD0 – D̄0 mixing, are depicted in figure 2.3.1 and figure 2.3.2,

    respectively.

  • Flavor mixing and FCNC process 31/128

    dL sL

    Ga(n)µ

    sL dL

    (a) LL type

    dR sR

    Ga(n)µ

    sR dR

    (b) RR type

    dL sL

    Ga(n)µ

    sR dR

    (c) LR type

    Figure 2.3.1: The diagrams of K0 – K̄0 mixing via KK gluon exchange

    cL uL

    Ga(n)µ

    uL cL

    (a) LL type

    cR uR

    Ga(n)µ

    uR cR

    (b) RR type

    cL uL

    Ga(n)µ

    uR cR

    (c) LR type

    Figure 2.3.2: The diagrams of D0 – D̄0 mixing via KK gluon exchange

    Before starting the concrete calculation, some comments are in order. At first glance, the

    contribution of LR-type processes in figure 2.3.1(c) seems to be dominant, because the diagram

    yields effective 4-Fermi operator, which is product of left-handed and right-handed currents. Let

    us note that in the SM the 1-loop box diagram yields 4-Fermi operator, which is product of pure

    left-handed currents. Thus in the case of the SM to form a pseudo-scalar state, the neutral K

    meson, from d and s̄, chirality flip is needed and the amplitude is suppressed by small current

    quark masses. On the other hand, the 4-Fermi operator of our interest has both left-handed and

    right-handed quarks and the amplitude is not suppressed by small quark masses. This means the

    amplitude is relatively enhanced compared to the case of the SM with an “enhancement” factor(mK

    md +ms

    )2∼ 23 . (2.27)

    In addition, the effective 4-Fermi operator from the LR-type diagram is known to have the

    strongest Renormalization Group (RG) enhancement from QCD correction [48, 181]. As we will

    see later, however, the KK mode summation for LR-type processes turns out to be much less than

    those for LL and RR type. Therefore, the contribution of LL- and RR-type processes is not less

    important, and even give dominant contribution.

    Second, one may wonder whether the exchange of extra space component of gluon, Ga(n)y , also

    gives similar contribution with the enhancement factor, since scalar-type coupling causes chirality

    flip. We, however, find the contribution is relatively suppressed by small masses of external quarks

    mq (mq = md, ms). Let us note that the zero mode Ga(0)y is “modded out” by orbifolding and

    non-zero KK modes of Ga(n)y (n ̸= 0) are absorbed as the longitudinal components of massive

  • 32/128 2.3 FCNC process : K0 – K̄0 mixing and D0 – D̄0 mixing

    gluons Ga(n)µ through Higgs-like mechanism. In the unitarity gauge, the contribution of such

    longitudinal components are taken into account by adding to the propagator eq. (2.26) a piece

    proportional to kµkν/(n/R)2, where kµ is the momentum transfer. By use of equations of motion for

    external quarks, its contribution to the amplitude is relatively suppressed by a factor

    m2q(n/R)2

    = O(m2qR

    2)

    and we can safely neglect the contribution of Ga(n)y exchange.

    2.3.3 K0 – K̄0 mixing

    First, we discuss the K0 – K̄0 mixing.5 By noting the fact k2 ≪ (n/R)2 for n ̸= 0, the contributionof diagram of figure 2.3.1 is written in the form of effective 4-Fermi Lagrangian obtained by use

    of Feynman rules listed in (2.24) and (2.26),

    dL sL

    Ga(n)µ

    sL dL

    ∼− g2s

    4

    1

    (1/R)2

    ∞∑n=1

    (s̄Lλ

    aγµdL)(s̄Lλ

    aγµdL)

    × 1n2

    {V †dL

    (U †3I

    (0n0)LL U3 + U

    †4I

    (0n0)LL U4

    )VdL

    }221, (2.28a)

    dR sR

    Ga(n)µ

    sR dR

    ∼− g2s

    4

    1

    (1/R)2

    ∞∑n=1

    (s̄Rλ

    aγµdR)(s̄Rλ

    aγµdR)(V †dRI

    (0n0)RR VdR

    )221, (2.28b)

    dL sL

    Ga(n)µ

    sR dR

    ∼− g2s

    4

    1

    (1/R)2

    ∞∑n=1

    (s̄Lλ

    aγµdL)(s̄Rλ

    aγµdR)

    × 1n2

    {V †dL

    (U †3I

    (0n0)LL U3 + U

    †4I

    (0n0)LL U4

    )VdL

    }21

    (V †dRI

    (0n0)RR VdR

    )221.

    (2.28c)

    The sum over the integer n is convergent and the coefficient of the effective Lagrangian (2.28) is

    suppressed by 1/M2c , where Mc ≡ 1/R: the decoupling effects of non-zero KK gluons.6

    2.3.3.1 KL –KS mass difference

    The relevant hadronic matrix elements are written by use of the “bag parameters” Bi (i = 1 ∼ 5)which denote the deviation from the approximation of vacuum saturation and whose numerical

    results are obtained by lattice calculations [32] :⟨K̄0∣∣∣s̄αLγµdαL · s̄βLγµdβL∣∣∣K0⟩ = ⟨K̄0 ∣∣∣s̄αRγµdαR · s̄βRγµdβR∣∣∣K0⟩ ≈ 13f2KmKB1 ,

    5For the studies of K0 – K̄0 mixing in other New Physics models, see for instance [48,54,117].6gsI

    (0n0)LL and gsI

    (0n0)RR are dimensionless quantities.

  • Flavor mixing and FCNC process 33/128

    ⟨K̄0∣∣∣s̄αLγµdβL · s̄βLγµdαL∣∣∣K0⟩ = ⟨K̄0 ∣∣∣s̄αRγµdβR · s̄βRγµdαR∣∣∣K0⟩ ≈ 13f2KmKB1 ,⟨

    K̄0∣∣∣s̄αLγµdβL · s̄βRγµdαR∣∣∣K0⟩ ≈

    {1

    12+

    1

    2

    (mK

    md +ms

    )2}f2KmKB4 ,

    ⟨K̄0∣∣∣s̄αLγµdαL · s̄βRγµdβR∣∣∣K0⟩ ≈

    {1

    4+

    1

    6

    (mK

    md +ms

    )2}f2KmKB5 ,

    where α, β are color indices and fK is the kaon decay constant. mK , md, ms denote the kaon

    mass and the current quark masses of down and strange quarks. Note that the color indices are

    contracted in different ways in these matrix elements. Using these results and a following relation

    about Gell-Mann matrices λa;

    8∑a=1

    (λa)αβ(λa)γδ = 2δαδδβγ −

    2

    3δαβδγδ

    (Tr(λaλb

    )= 2δab

    ), (2.29)

    the hadronic matrix element of the effective 4-Fermi operator is obtained as⟨K̄0∣∣∣s̄LλaγµdL