OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ......
-
Upload
trankhuong -
Category
Documents
-
view
217 -
download
1
Transcript of OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ......
![Page 1: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/1.jpg)
Cisco Public © 2010 Cisco and/or its affiliates. All rights reserved. 1
OpenStack Overview
Shannon McFarland – CCIE #5245 Principal Engineer – Cloud and Virtualization Group @eyepv6
![Page 2: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/2.jpg)
© 2013 Cisco and/or its affiliates. All rights reserved. Cisco Public 2
Agenda•
9:00–9:30 a.m. - Registration and Tech Expo 9:30–10:30 a.m- Introduction to SDN 10:30–11:30 a.m.- SDN Protocols and Controllers Part One 11:30 a.m.–12:00- p.m.- Lunch and Tech Expo 12:00–1:00 p.m.- SDN Protocols and Controllers Part Two 1:00–1:45 p.m.- SDN and Splunk 1:45–2:45 p.m.- What's New with OpenStack 2:45–3:00 p.m.- Wrap Up
![Page 3: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/3.jpg)
Cisco Public 3 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ What is OpenStack? § Common Use Cases § OpenStack Participation § Deployment Summary § OpenStack + SDN
![Page 4: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/4.jpg)
Cisco Public 4 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
What is OpenStack?
![Page 5: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/5.jpg)
Cisco Public 5 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
“OpenStack is a collection of open source technologies delivering a massively scalable
cloud operating system” - openstack.org
5
![Page 6: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/6.jpg)
Cisco Public 6 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Austin – Oct 2010
Bexar– Feb 2011
Diablo – September 2011
Essex– April 2012
Catus – April 2011
Folsom –Sept 2012
Grizzly– April 2013
Havana – October 2013
2011 2012 2013 2014
Icehouse– April 2014
2015
Kilo – April 2015
![Page 7: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/7.jpg)
Cisco Public 7 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Core Projects Shown
7
New!
Compute “Nova”
- Houses VMs - API driven - Support for multi-hypervisors
Storage Image, Object, Block
“Glance, Swift, Cinder” - Instance/VM image storage - Cloud object storage - Persistent block level storage
Dashboard “Horizon”
- Web app for controlling OpenStack resources - Self-service portal
Identity “Keystone”
- Centralized policies - Tenant mgmt. - RBAC - Ext. integration (LDAP)
Networking “Neutron”
- Networking as a service - Multiple models - IP address mgmt. - Plugins to external HW
Telemetry “Ceilometer”
- Central collection point - Metering and monitoring
Orchestration “Heat”
- Template-based orchestration engine - More rapid deployment of applications
Database “Trove”
-DBaaS -Single-tenant DB within instance
Data Processing “Sahara”
- Fast provisioning of Hadoop clusters
![Page 8: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/8.jpg)
Cisco Public 8 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ PTL Juno Summary: https://www.youtube.com/playlist?list=PLKqaoAnDyfgqpX5f3PCuOgsDm-_UJu2aU
§ Nova § NFV § Improvements in live upgrades (introduced in Icehouse) § http://blog.russellbryant.net/2014/07/07/juno-preview-for-openstack-compute-nova/
§ Heat § Rollback § non-Admin resource creation § http://www.zerobanana.com/archive/2014/07/10#heat-juno-update
§ Neutron § Distributed Virtual Router
§ L3 HA
§ New LBaaS API 8
Reference
![Page 9: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/9.jpg)
Cisco Public 9 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Common Use Cases
![Page 10: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/10.jpg)
Cisco Public 10 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ OpenStack, at least today, is targeted at hosting modern day distributed applications written for the cloud – This isn’t your grandpa’s server virtualization platform built for individual VM HA/Mobility
§ Proof-of-Concept environment to work out CI/CD § A place to research, learn and test CI/CD processes
§ PoC web applications along with ‘practicing’ the new DevOps methodology
§ A place to learn the whole cloud deployment framework, document, train, move to production
§ Host traditional development environments § Using the lessons learned in the sandbox phase: ‒ Build Dev, QA and production environments
‒ Apply CI/CD processes
‒ Slow-role Web application deployment either on ‘standard’ OpenStack or in conjunction with a PaaS deployment
§ Data Processing environments – Big Data clusters, etc..
§ Training systems – Cheap and fast to build and tear down for each class
§ Revenue generating applications – Vertical applications (only if the app doesn’t care much about VM-level HA)
10
![Page 11: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/11.jpg)
Cisco Public 11 © 2013-2014 Cisco and/or its affiliates. All rights reserved. 11
![Page 12: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/12.jpg)
Cisco Public 12 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Boot the Instance
Config Management
App is Deployed
Rinse & Repeat
12
- Cloud-init for Puppet/Chef/etc.. - Image already has agent/script
http://docs.openstack.org/user-guide/content/user-data.html
# Nodes for web server instances
node 'sales-web-01' {
include lamp
}
root@build-server:~# tree /etc/puppet/modules/lamp/
/etc/puppet/modules/lamp/
├── files
│ ├── apache2.conf
│ ├── index.php
│ └── php5.conf
└── manifests
└── init.pp
nova boot --user-data ./cloud-config-puppet.txt --image precise-x86_64 --flavor m1.tiny --key_name ctrl-key --nic net-id=42823c88-bb86-4e9a-9f7b-ef1c0631ee5e sales-web-01!
![Page 13: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/13.jpg)
Cisco Public 13 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ Heat provides you a way to deploy resources (network, router, load-balancer, VM and applications) all from a ‘template’
§ Today, Heat orchestrates resources inside a tenant space § https://wiki.openstack.org/wiki/Heat § http://docwiki.cisco.com/wiki/
OpenShift_Origin_Heat_Deployment_Guide § http://blog.scottlowe.org/2014/05/01/an-introduction-to-openstack-heat/
§ https://github.com/shmcfarl/my-heat-templates
13
![Page 14: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/14.jpg)
Cisco Public 14 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
OpenStack Participation
![Page 15: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/15.jpg)
Cisco Public 15 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ Choice § There is no one-size fits all option for cloud computing – Amazon or VMware are cool but not the
be all/end all solution § There is no single vendor who can fill all needs of a cloud stack – You will likely engage with
multiple partners
§ Community § Open Source § Community driven – Individual, organizational § Better time-to-market and faster feature velocity
§ Commercialization § Start with the ‘baseline’ OpenStack components § Vendor opportunities for value-add integration on top of OpenStack baseline ‒ Design, deployment, automation, operation, high-availability, applications, etc…
15
![Page 16: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/16.jpg)
Cisco Public 16 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
• Cisco Designs on specific releases in ‘beachhead’ accounts
• Start simple, build from there – Focus on automation and HA
• Evangelization of what Cisco is doing - Thought Leadership – Help customers know What, When, Where & How
Engineering
Customers
Community • Cisco Product Integration • Nexus Plugins – Neutron • UCS • CSR/ASR • Co-developed solutions (Red
Hat, Canonical, SUSE)
• Neutron – Network Service • Horizon – Dashboard • Keystone – Identity • Swift – Object Storage • Ceph/Cinder – Block Storage • Automation – PuppetLabs • HA Design
![Page 17: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/17.jpg)
Cisco Public 17 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ Cisco.com OpenStack: http://www.cisco.com/web/solutions/openstack/index.html § Red Hat:
§ UCSO: http://www.cisco.com/c/en/us/td/docs/solutions/Enterprise/Data_Center/OpenStack/UCSO/Starter/1-0/UCSO.pdf
§ http://www.cisco.com/c/en/us/td/docs/solutions/Enterprise/Data_Center/OpenStack/RHEL-UCS/Red-Hat-Openstack-Platform-UCS.pdf
§ http://www.cisco.com/c/dam/en/us/td/docs/unified_computing/ucs/UCS_CVDs/ucs_rhos.pdf § http://www.cisco.com/c/dam/en/us/products/collateral/switches/nexus-7000-series-switches/
wp_openstack.pdf § http://www.cisco.com/c/dam/en/us/solutions/collateral/data-center-virtualization/unified-fabric/solution-brief-
c22-729865.pdf
§ Ubuntu: http://www.cisco.com/c/dam/en/us/td/docs/unified_computing/ucs/UCS_CVDs/ucs_ubuntu.pdf
17
Reference
![Page 18: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/18.jpg)
Cisco Public 18 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ Red Hat OpenStack (RHOS/RDO) – PackStack and Foreman/Staypuft: http://www.redhat.com/openstack/ http://openstack.redhat.com/Main_Page Spinal Stack: http://spinal-stack.readthedocs.org/en/latest/index.html
§ Canonical/Ubuntu – MAAS and JuJu: http://www.ubuntu.com/cloud § SUSE: https://www.suse.com/products/suse-cloud/features/ § Mirantis Fuel: http://software.mirantis.com/main/
§ Piston Cloud: http://www.pistoncloud.com/ § Others …
18
Reference
![Page 19: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/19.jpg)
Cisco Public 19 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ Nexus 1000v: http://www.cisco.com/c/en/us/support/switches/nexus-1000v-kvm/products-installation-guides-list.html
§ Nexus 3000 and Higher: http://www.cisco.com/en/US/prod/collateral/switches/ps9441/ps11541/data_sheet_c78-727737.html
§ Cisco Nexus + OpenStack Deployment: http://docwiki.cisco.com/wiki/OpenStack/ML2NexusMechanismDriver
§ Cisco CSR 1000v: http://www.cisco.com/c/en/us/td/docs/routers/csr1000/software/configuration/csr1000Vswcfg/installkvm.html
§ Cisco ACI with OpenStack: http://www.cisco.com/c/dam/en/us/solutions/collateral/data-center-virtualization/unified-fabric/solution-brief-c22-729865.pdf
§ Cisco APIC driver for OpenStack Neutron ML2: http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/guide-c07-732454.html
§ Group-Based Policy (GBP) for OpenStack: http://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-733126.html
19
Reference
![Page 20: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/20.jpg)
Cisco Public 20 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Deployment Summary
![Page 21: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/21.jpg)
Cisco Public 21 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ OpenStack components live South of the Top-of-Rack switch
§ Your existing DC, Internet Edge and BN architecture stays the same
§ It’s about the compute, storage and orchestration/management tiers
§ Your apps go largely unchanged
Ser
vice
s
Access Layer
Agg Layer
Core Layer
UC
S C
-Ser
ies
UC
S B
-Series
Enterprise/ Internet
![Page 22: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/22.jpg)
Cisco Public 22 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
AIO Controller/
Compute/Storage
AIO Controller: - MySQL, MariaDB, etc - RabbitMQ, Qpid, etc.. - API Endpoints:
- Keystone - Glance - Nova - Neutron - Cinder - Heat - Swift
AIO Controller
Compute/Storage
Compute/Storage
Compute
Compute
Storage
Storage
Storage Compute
AIO Controller
![Page 23: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/23.jpg)
Cisco Public 23 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Data Center Infrastructure
OOB
Compute
Network Node(s)
AIO Controller
Compute
Network Node(s)
AIO Controller
Compute
Network Node(s)
AIO Controller
Spine/Agg Layer
TOR(s) TOR(s) TOR(s)
Spine/Agg Layer
Block Storage
Block Storage
Block Storage
AIO Controllers: - Galera/MySQL - RabbitMQ - API Endpoints:
- Keystone - Glance - Nova - Neutron - Cinder - Heat - Swift
OOB OOB SLB
Infrastructure Services
Build/PXE
Automation
DNS
DHCP
NTP
Logging
Object Storage
Object Storage
Object Storage
![Page 24: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/24.jpg)
Cisco Public 24 © 2013-2014 Cisco and/or its affiliates. All rights reserved. Cisco Public © 2013-2014 Cisco and/or its affiliates. All rights reserved.
Data Center Infrastructure
OOB
Spine/Agg Layer
TOR(s) TOR(s) TOR(s)
Spine/Agg Layer
OOB OOB
Object Storage
Object Storage
Swift Proxies
TOR(s)
Object Storage
OOB OOB
RabbitMQ
API Endpoints
Galera
TOR(s) TOR(s)
Compute
OOB
Block Storage
Object Storage
RabbitMQ
API Endpoints
Galera
Compute
Block Storage
Object Storage
RabbitMQ
API Endpoints
Galera
Compute
Block Storage
Object Storage
Compute
Network Node(s)
Compute
Compute
Compute
Compute
Network Node(s)
Compute
Compute
Compute
Block Storage
Block Storage
Compute Compute
Service Cloud Tenant Cloud
Reference
![Page 25: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/25.jpg)
Cisco Public 25 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ It’s the ‘underlay’ Cloud
§ Used as a hosting platform for tenant cloud services – usually in a large cloud (1000s of instances with 100-1000s of tenants)
§ It is an OpenStack deployment that will host (virtually) the OpenStack control functions used by each tenant
Service Cloud
AIO Controller
AIO Controller
AIO Controller Tenant 1
AIO Controller
AIO Controller
AIO Controller Tenant 2
Compute
Compute
Reference
![Page 26: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/26.jpg)
Cisco Public 26 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
OpenStack + SDN
![Page 27: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/27.jpg)
Cisco Public 27 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
1. Physical Resource Layer • Networking, Storage and Compute resources • Hardware-based networking services
2. Network Resource and SDN Layer • Resource Virtualization Provisioning, and Management NX-OS API SDN API Other
Network Mgmt/Cntl API
Compute Service
Virtual Servers
Storage Service
Virtual Disks
Network Service
Virtual Networks
Plug-In Architecture
3. OpenStack Cloud Platform Layer • Presents compute and networking virtualization interfaces to application developers in a multi-tenant environment
Cloud App Developer API
App
OS
VM
DataBase
OS
VM
App
OS
VM
Virtual Appliance
4. User Application Layer • Self-provision resources through APIs • Only see virtualized resources
Virtual Appliance
![Page 28: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/28.jpg)
Cisco Public 28 © 2013-2014 Cisco and/or its affiliates. All rights reserved.
§ Trends include the movement to or back to private clouds with the design option to
link workloads between various clouds (Public<>Private) § Don’t go down the path of OpenStack if your goal is a ‘free’ alternative to another
server virtualization system – Use OpenStack for what OpenStack was built for § OpenStack is for real, but immature in some aspects, especially networking and
upgrades – Gaps are closing quickly
§ Real value-add is not in installation and the basic OpenStack components but rather on overall system design, DevOps, optimization and scale-out above what a baseline OpenStack system provides
![Page 29: OpenStack Overview - · PDF fileOpenStack Overview Shannon McFarland – CCIE #5245 ... There is no single vendor who can fill all needs of a cloud stack – You will likely engage](https://reader031.fdocuments.net/reader031/viewer/2022030403/5a78fabd7f8b9a5a148e9868/html5/thumbnails/29.jpg)
Thank you.