saas.ecloud.10086.cn · Web viewPaaS（Platform as a...
Transcript of saas.ecloud.10086.cn · Web viewPaaS（Platform as a...
DevOps
V0.1
01
2018/11/06
1
58
215163
http://cmsoft.10086.cn
4001-10086-5
DEVOPS
012018-11-06
- 2 -
- 2 -
- 2 -
- 2 -
- 2 -
- 3 -
1 - 5 -
1.1 - 5 -
1.2 - 7 -
1.3 - 10 -
1.4 - 11 -
1.5 - 14 -
2 - 15 -
2.1 - 15 -
2.2 - 15 -
2.2.1 - 15 -
2.2.2 - 17 -
2.2.3 - 19 -
2.2.4 - 25 -
2.3 - 27 -
2.3.1 - 27 -
2.4 - 28 -
2.4.1 - 29 -
2.4.2 - 30 -
3 - 32 -
3.1 - 32 -
3.1.1 - 32 -
3.1.2 - 37 -
3.1.3 - 42 -
3.1.4 - 45 -
3.2 - 59 -
3.2.1 - 59 -
3.3 - 62 -
3.3.1 - 62 -
3.3.2 - 64 -
3.3.3 - 68 -
3.3.4 - 84 -
3.3.5 - 86 -
3.4 - 100 -
3.4.1 - 101 -
3.4.2 - 102 -
3.4.3 - 115 -
3.4.4 - 116 -
3.4.5 - 119 -
3.4.6 - 128 -
3.4.7 - 139 -
3.4.8 - 143 -
3.4.9 - 149 -
4 - 152 -
4.1 - 153 -
4.2 - 155 -
4.2.1 - 155 -
4.2.2 - 160 -
4.2.3 - 161 -
4.2.4 - 163 -
4.2.5 - 164 -
4.2.6 - 171 -
4.2.7 - 172 -
4.2.8 - 178 -
4.2.9 - 182 -
4.3 - 184 -
4.4 - 185 -
4.5 - 186 -
5 - 189 -
DEVOPSPaaS
DEVOPSPaaS 11
11 PaaS
DEVOPS->->->->
DEVOPS 12
12
PaaS
PaaSPlatform as a ServiceIaaSInfrastructure-as-a-ServiceSaaSSoftware-as-a-ServiceIaaSSaaSPaaSPaaS
UbuntuUbuntuApacheApache
PaaSPaaS
Element
PaaS
DevOps
DevOpsDevelopmentOperations/QA->->->->DevOps 14
13 DevOps
14
14
Pipeline 15
15
DEVOPS 11
11
admin
admin
admin
admin
admin
CICD
DEVOPS 16
16 BDPaaS
admin 17 18
17 admin
18
12
12
CICD
19 DEVOPS
1. >
MPP 31
31
32
32
33
33
> > 34
34
3.1.3
1. >
1. MPP 35
35
36
36
37
37
//
1. > 38
38
39
39
310
310
1. > 311
311
1. CPU 312
312
1. Oracle/MySQLDB 313
313 Oracle
314
314
MPPBC-RDB HybridBC-RDB HybridDBURL 315
315 BC-RDB Hybrid
Comment by wuyan:
1. >
1. 316
316
317
317
Oracle
1. > Oracle
1. 3
DBCPUsession 318
318 Oracle
Oracle 319
319 Oracle
1. >
320
320
321
321
on sliderYarnon sliderYarnYarn
HiveHdfsHdfsHiveHdfs
SparkYarnYarn0
Yarn
a. YarnIO 322
322 Yarn
MR
SparkSpark StreamingYarn
323
323
FAIR
FIFO
DRFvCore
PRIORITY
324
324
> 325
325
MPP
BC-RDB Hybrid
> BC-RDB Hybrid
DBDB 326
326 BC-RDB Hybrid
BC-RDB Hybrid 327
327 MPP
/
1. >
///
328
328
MySQL
1. >
329
329
DEVOPSDEVOPS
WordPress
1. >
330
330
!
331
332
332
tar
333
333
DEVOPS
1. >
1. 334
334
1.
1.
0~9164
1.
335
335
1. >
1.
1.
1.
0~9164
1.
YAML 336
336
1. > 337
337
1. 338
338
339
339
340
340
341
341
1. >
1.
1. 342
342
DEVOPSDEVOPSDEVOPSDEVOPS
1. >
1.
1. 343
343
344
344
registryaddress/library/reponame
31
31
- 32
32 -
DeploymentvolumepodReplication Controllerpodpodpod
StatefulSet
-&- 33
33 -
0~92~60
keyvalue
keyvalue
0~2001
0~20020
0
podpods
BDPaaS4.15Macvlanvolume00%1Deployment
100%BDPaaS4.94volume100%1
102310+210-3
00%
00BDPaaS1
-Service 34
34 -
Service
0~91~32
IPClusterIPIP
IPHeadlessNodePort
30000~32000ClusterIPHeadless
1~65535HTTP80HTTPS443TCP10000~11000
HTTPURLTCPTCPHTTP
TCPHTTP
- 35
35 -
0~91~32
Docker
docker pull
registryaddress/library/reponame
latest
CPU
-- 36
36 -
PVCPVCPVCPV
ConfigMapConfigMapPod
37
&
ConfigMapConfigMapConfigMap
key
ConfigMap
ConfigMapConfigMaps
/var/log/
/var/log/aaa
37
HTTP
300
1~12060
HTTP1~30030
0
05
HTTPHTTPS
/login.html
headernamevalue
TCP
300
1~12060
HTTP1~30030
0
05
EXEC
300
1~12060
HTTP1~30030
0
05
docker execdocker exec
pod-
Service 38
38 Service
Service
0~91~32
IPClusterIPIP
IPHeadlessNodePort
30000~32000ClusterIPHeadless
1~65535HTTP80HTTPS443TCP10000~11000
HTTPURLTCPTCPHTTP
TCPHTTP
YAMLYAML
YAML
YAML 345
345
1. >
1.
1. 346
346
0~91~32
YAML 347
347 YAML
YANLYAML
YAML
apiVersion: apps/v1beta1kind: Deploymentmetadata: name: redis209 # namespace: defaultspec: replicas: 2 # strategy: # type: RollingUpdate template: metadata: labels: key1: value1 spec: containers: - image: nginx:1.9.10 # imagePullPolicy: Always # Always name: nginx - command: # Entrypoint - /bin/sh - args: # Command - -c - while true; do echo hello; sleep 10;done - env: # - name: ACCESS_KEY_ID - name: ACCESS_KEY_SECRET envFrom: # - configMapRef: name: test-info volumeMounts: - mountPath: /tmp/hbase2 # name: ebs-volume # Kubernetes volume volumes - mountPath: /tmp/hbase4 # name: rbdpd-volume # Kubernetes volume volumes livenessProbe: # failureThreshold: 5 httpGet: path: /metrics port: 10054 scheme: HTTP initialDelaySeconds: 60 periodSeconds: 10 successThreshold: 1 timeoutSeconds: 5 readinessProbe: # exec: command: - cat - /tmp/healthy initialDelaySeconds: 5 periodSeconds: 5 resources: # limits: # cpu: 34m memory: 204Mi requests: # cpu: 34m memory: 204Mi hostNetwork: True # Host IP nodeSelector: # ip: 172.22.22.4 restartPolicy: Always # Always volumes: - hostPath: path: /alauda/hbase # - name: ebs-volume
1. >
1. 38
39
1.
348
348
310
310
CPU
CPU
YAML
YAMLYAML
Comment by wuyan:
BDPaaS
YAMLService Binding
1. >
1.
1.
1. 349
349
311
311
IPIP
CPU2
YAML
YAML
Kubernetes Service
Service Binding
Service InstanceService InstanceInstance
ConfigMap
DeploymentStatefulSet
1. >
1.
1.
1.
1.
12 350
350
DeploymentStatefulSet
1. >
1.
1.
1.
1.
b.
c.
1. >
1.
1.
1.
1. >
351
351
1. >
1.
1.
1.
1. >
312
312
1~65535HTTP80HTTPS443TCP10000~11000
HTTPURLTCPTCPHTTP
TCPHTTP
352
352
1. >
1.
1.
1.
1. >
37
CPU
1. >
1.
1.
1.
1. >
CPU 353
353
ConfigMap
ConfigMap
1. >
1. ConfigMap
1.
1.
1. > ConfigMap
ConfigMapConfigMap 354
354 ConfigMap
1. >
1.
1.
1.
1. >
355
355
ConfigMapConfigMap
key
docker exec
docker exec
1. >
1.
1.
1.
1. > EXEC
EXEC 313
313 EXEC
pod
SSHShell/bin/sh/bin/bash
EXEC
EXEC
SSH
Console 356
356 EXEC
1. >
1.
1.
1.
1. >
357
357
1. >
1.
1.
1.
1. >
358
358
1. >
1.
1.
1.
1.
359
359
360 CICD
git >
> > CI
cicd_groupcicdnewusertest
1. test000 361
361
> 362
362
> 363
363
>
+ 364
364
.gitlab-ci.yml 365
#
stages:
- test
- build
- upload
- deploy
#
test:
stage: test
script:
#
- echo "run unit tests"
#
- mvn test
#
build:
stage: build
script:
#
- echo "build docker image"
# -u-p
- docker login 10.139.17.66:5000 -u alauda -p Alauda
# test000
- docker build -t test000 .
#
upload:
stage: upload
script:
#
- echo "push docker image"
#
- docker login 10.139.17.66:5000 -u alauda -p Alauda
# test000
- pipline-push -g 3 -r test000:latest test000:latest
#
deploy:
stage: deploy
script:
#
- echo "deploy app"
#
- pipline-deploy -g 3 deploy-ci.json
365
Dockerfile
#
FROM tomcat:latest
#
MAINTAINER chenjiong "[email protected]"
#
RUN mkdir /usr/local/tomcat/webapps/webtest
#
ADD src/main/resources/webapp/index.html /usr/local/tomcat/webapps/webtest
deploy-ci.json
{
"resource": {
"name": "test000",
"create_method": "UI"
},
"namespace": {
"name": "space3",
"uuid": "362d6ea5-d801-11e8-b798-fa163ed82a90"
},
"cluster": {
"uuid": "6c15de90-1010-4c1d-b49b-dd60da6fb614",
"name": "global"
},
"kubernetes": [{
"apiVersion": "extensions/v1beta1",
"kind": "Deployment",
"metadata": {
"namespace": "",
"name": "test000"
},
"spec": {
"template": {
"metadata": {
"labels": {
"service.alauda.io/name": "test000"
}
},
"spec": {
"affinity": {},
"containers": [{
"name": "test000",
"image": "10.139.17.66:5000/pro3/test000:latest",
"imagePullPolicy": "Always",
"resources": {
"requests": {
"memory": "1G",
"cpu": 1
}
}
}],
"hostNetwork": false
}
},
"replicas": 1,
"strategy": {
"type": "RollingUpdate"
}
}
}, {
"apiVersion": "autoscaling/v1",
"kind": "HorizontalPodAutoscaler",
"metadata": {
"name": "test000",
"namespace": ""
},
"spec": {
"scaleTargetRef": {
"apiVersion": "extensions/v1beta1",
"kind": "Deployment",
"name": "test000"
},
"minReplicas": 1,
"maxReplicas": 20
}
}, {
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "test000",
"namespace": "",
"annotations": {
"loadbalancer.alauda.io/bind": "[]"
}
},
"spec": {
"selector": {
"service.alauda.io/name": "test000"
},
"type": "NodePort",
"ports": [{
"name": "http-0",
"protocol": "TCP",
"port": 8080,
"targetPort": 8080,
"nodePort": 60100
}]
}
}]
}
366
366
1. CloneCloneURL 367
git clone [email protected]:newuser/test000.git
367 URL
git commit -am "modify"
# originmaster
git push origin master
> > 368
368
369
369
1. > 370
370
> / 371
371
372
372
1. CICD
373
373
374
374
64
200
URL
375
375
passedblockedretestfailedskippeduntested906030147 376
376
> >
377
377
BDPaaS
/
issue 378
378
1.
379
379
authorassigneemilestonelabel 380
380
authorassigneemilestonelabel 381
381
382
382
() 383
383
1.
384
384
385
385
386
386
> > 387
387
388
388
requestfeatureuser storydevelop tasktest taskbugto do/not starteddoing/in progressresovled 389
389
>
390
390
391
391
392
392
CI/CD
393
393
1. 393
SHA 394
394
1. 393
395
395
1. 393
396
396
1. ziptar.gztar.bz2tar 397
397
/
1. // 398
398 /
1. 393
399
399
3100
3100
3101
3101
1.
3102
3102
3103
3103
3104
3104
3105
3105
CI/CD
.gitlab-ci.yml.gitlab-ci.yml
stages:
-
-
-
-
-
-
Compile:
stage:
script:
- echo ""
- echo ""
FindBugs:
stage:
script:
- echo "FindBugs"
- echo "FindBugs"
PMD:
stage:
script:
- echo "PMD"
- echo "PMD"
CheckStyle:
stage:
script:
- echo "CheckStyle"
- echo "CheckStyle"
JUnit:
stage:
script:
- echo ""
- echo ""
Clover:
stage:
script:
- echo ""
- echo ""
Library:
stage:
script:
- echo ""
- echo ""
Deploy Test Env:
stage:
script:
- echo ""
- echo ""
API Automation Test:
stage:
script:
- echo "API"
- echo "API"
UI Automation Test:
stage:
script:
- echo "UI"
- echo "UI"
Performance Automation Test:
stage:
script:
- echo ""
- echo ""
Security Automation Test:
stage:
script:
- echo ""
- echo ""
Release:
stage:
script:
- echo ""
- echo ""
CI/CD 3106
3106 CI/CD
jarwartgzzip 3107
3107
1.
3108
3108
Docker 3109
3109
1.
3110
3110
tar
3111
3111
1.
3112
3112
OpenCompleted 3113
3113
Completed 3114
3114 Completed
Open 3115
3115
Open 3116
3116 Open
3117
3117
1.
3118
3118
3119
3119
1.
3120
3120
3121
3121
3122
3122
3123
3123
/ 3124
3124
1.
3125
3125
HTTP
51 BDPaaS
ID
400000001
1.
400000001
400000003
beanjson
beanjson
bean
400000004
400000005
400000006
core-site.xml
core-site.xml
400000007
core-site.xmlBC-HControl
BC-HControl
BC-HControl
400000008
UpdateCoreSitejson
UpdateCoreSite
400000009
400000010
systemId = {systemId}
systemId
400000011
systemId={systemId}parentId={parentId}name={name}
400000012
400000013
attr:{}
400000021
1.
400000022
1.
400000023
1.
400000024
1.
400000025
1
2
400000026
400000027
logmanager session id
logmanager session id
400000031
1.
400000032
1.
400000040
1.
400000041
1.
400000042
1.
400000043
1.
400000044
1.
400000050
1.
401000001
1.
401000010
402000001
402000002
1.
403000001
403000002
1.
400
500
404000001
{id}
Id
roleid
404000002
accessKeyId
accessKeyId
404000003
messageid={}
messageid={}
message
404000010
systemId={systemId} and userId={userId}
systemIduserId
404000011
404000012
module id
404000013
bc-streaming
bc-streaming
streaming
409000001
409000002
409000003
409000004
409000005
{}
409000006
key:{key}
409000007
key:{key}
409000008
name ={}
name ={}
409000014
key:{key}
409000015
key:{key}
409000016
key:{key}
410000001
url{url}
410000002
{type}
containermirrorgbasehybridoraclecmh-hadoopmysql
410000003
CPU
CPU
CPU
410000004
410000005
410000006
410000007
410000008
410000009
can't get response
Oracle
410010001
IP
IP
IP
410110002
IP
IP
IP
410110003
410110004
- 1 -
- 2 -
YesNo
YesNo
