Ambari Views - Overview

AMBARI VIEWS

Ambari UX So Far…

Mostly for addressing “Operator” Concerns

Host went down!

Disk is full!

Need to tweak configs

The DataNodes are down!

Need to add hosts

Need to secure cluster

Need NameNod

e HA

Ambari UX So Far…

Some for addressing “Data Worker” Concerns

Hmm…why is my query slow?

Across Hadoop…a whole bunch of other UIs out there!

You may have used…• Native UIs for various Hadoop ecosystem components:

MapReduce Job History, NameNode, ResourceManager, HBase, Storm, Oozie, Falcon, etc.

• Hue• Ambrose (Twitter)• White Elephant (LinkedIn)• Lipstick (Netflix)• …and so on

!@#$

Ambari: Common UX for Hadoop

Provide a common, secure and pluggable approach for UX across:• Operators, System Admin• Data Workers• Application Developers• …and others

Yay!

Yay!

Ambari Views: Goals

Single point of entry✔ Common URL for common user communities✔ “Views” embedded in Ambari UI

Pluggable UI Framework✔ “Views” contributed and shared as plugins✔ No code changes to the core✔ Browse published Views and install

Ambari Views: Goals

Authorization✔ Control who can access which views and which aspects of views✔ Deployment model supports connecting to different LDAP/ADs by user community

Runs on Ambari Server✔ No extra daemons needed

Runs Ambari “standalone”✔ No need to deploy cluster via Ambari to use Views

Example Views

Operators• Capacity Scheduler

Queue Manager• YARN Resource Utilization• Heatmaps• HDFS / Hive Mirroring

Data Workers• Pig Query Editor • Hive Query Editor• Workflow Design• HDFS File Browser• Hive/Tez

Visualization

Application Developers• Job Visualization• Streaming Topology Visualization

Views and the Framework

Views Framework

Views

Core to Ambari

Plugins to Ambari

Components of a View

VIEWClient-side

assets(.js, html)

AMBARI WEB

VIEWServer-side resources

(java)

AMBARI SERVER

{rest}Hadoop

and other systems

View Packaging• View descriptor : view.xml• Resource / Service classes : JAX-RS annotated• UI classes : html, Servlets deployed as web app (WEB-INF/web.xml).• Application logic : Supporting classes• Dependencies : 3rd party jars or classes

├── WEB-INF│ └── web.xml├── org│ └── apache│ └── ambari│ └── view│ └── filebrowser│ ├── DownloadService.class│ ├── FileBrowserService.class│ └── FileOperationService.class└── view.xml

View Versions + Instances

• Multiple versions of a View• Multiple View Instances of each version

View NameView

InstancesView

Versions

ViewVersions

View InstancesView

Instances

View Instances

• View Context– View and user information

• Instance Data– Lightweight name/value (for prefs)

• Events– Framework and Custom events

Framework Services

Client-Side Server-Side ViewContext.getUsername()ViewContext.getInstanceData()ViewController.fireEvent()

View Events

• Server-side Framework and Custom eventsonDeploy(), onCreate(), onDestroy()

Develop Deploy Create instancesPackage

onDeploy() onCreate()

onDestroy()

Authentication

VIEW

AMBARI SERVER

Ambari DB

LDAP

{rest}

<html>

User AuthN Source

User-Permission Mapping

Authenticate

Provide principal via ViewContext

Views Deployment

• Deploy Views as part of an operational Ambari Server• Or deploy standalone “Ambari Views Server” for data workers

AmbariServer

HADOOPStore & Process

AmbariViews Server

Operators manage the cluster, may have Views deployed

Data Workers use the cluster and use the Ambari Views Server for Views (no agents)

Learn More

• Frameworkhttps://github.com/apache/ambari/tree/trunk/ambari-viewshttps://github.com/apache/ambari/blob/trunk/ambari-views/docs/index.md

• Framework Exampleshttps://github.com/apache/ambari/tree/trunk/ambari-views/examples

• View Contribshttps://github.com/apache/ambari/tree/trunk/contrib/views

https://github.com/apache/ambari/tree/trunk/ambari-views

https://github.com/apache/ambari/tree/trunk/ambari-views

https://github.com/apache/ambari/blob/trunk/ambari-views/docs/index.md



https://github.com/apache/ambari/tree/trunk/ambari-views/examples

https://github.com/apache/ambari/tree/trunk/ambari-views/examples

https://github.com/apache/ambari/tree/trunk/contrib/views

https://github.com/apache/ambari/tree/trunk/contrib/views

EXTRAS

© Hortonworks Inc. 2014

The Deployed View

• Views are deployed by placing the view package in the Ambari view folder.

• Once deployed, views and view instances are available through the Ambari REST API.

GET http://c6401.ambari.apache.org:8080/api/v1/views/WEATHER/ { "href" : "http://c6401.ambari.apache.org:8080/api/v1/views/", "items" : [ { "href" : "http://c6401.ambari.apache.org:8080/api/v1/views/WEATHER", "ViewInfo" : { "view_name" : "WEATHER" } } ]}


View Descriptor : view.xml

• name – the internal name of the view (must be unique)• label – the public display name of the view• version – the version of the view• parameter – metadata about view instance properties• resource – the names of the classes required to support a

view sub-resource• instance – optional property sets which define view instances.<view> <name>MYVIEW</name> <label>My View</label> <version>1.0.0</version> <parameter> <name>scope</name> <description>The scope of the instance.</description> <required>true</required> </parameter></view>


View Descriptor : parameter

• Values for view parameters are given as properties when a view instance is defined.

– name – the name of the parameter– description – a brief description of the parameter– required – indicates whether the parameter is required for instance

definition<view> … <parameter> <name>scope</name> <description>The scope of the instance.</description> <required>true</required> </parameter> <instance> <name>GLOBAL_INSTANCE</name> <property> <key>scope</key> <value>global</value> </instance></view>


View Descriptor : instance

• Values for view parameters are given as properties when a view instance is defined.

• Multiple instance may be defined for a view.– name – the name of the instance– property– key / value pair. Name should match a view parameter.

<view> … <parameter> <name>scope</name> <description>The scope of the instance.</description> <required>true</required> </parameter> <instance> <name>GLOBAL_INSTANCE</name> <property> <key>scope</key> <value>global</value> </instance></view>


View Descriptor : resource

• The defined resources of a view plug into the Ambari REST API.

– name – the name of the resource– plural-name – the plural name as represented in the API– id-property – the identifying property of the resource– resource-class – the JavaBean resource class– provider-class – the ResourceProvider implementation– service-class – the JAX-RS annotated resource service class

<view> … <resource> <name>city</name> <plural-name>cities</plural-name> <id-property>id</id-property> <resource-class>org.apache.ambari.view.weather.CityResource</resource-class> <provider-class>org.apache.ambari.view.weather.CityResourceProvider</provider-class> <service-class>org.apache.ambari.view.weather.CityService</service-class> </resource></view>



A resource class can be any JavaBean. The resource instances will be provided by an implementation of a ResourceProvider, which is written by the view developer.

public class CityResource {

private String id; private Map<String, Object> weather; private String units;

public String getId() { return id; }

public void setId(String id) { this.id = id; } …}



A resource provider class should implement ResourceProvider. Note the injected ViewContext in the following example…

public class CityResourceProvider implements ResourceProvider<CityResource> {

@Inject ViewContext viewContext;

@Override public CityResource getResource(String resourceId, Set<String> propertyIds) throws SystemException, NoSuchResourceException, UnsupportedPropertyException {

Map<String, String> properties = viewContext.getProperties();

String units = properties.get("units");

try { return getResource(resourceId, units, propertyIds); } catch (IOException e) { throw new SystemException("Can't get city resource " + resourceId + ".", e); } } …}



A resource service class should be annotated with JAX-RS annotations to handle service requests. Note the injected ViewResourceHandler in the following example…

public class CityService { @Inject ViewResourceHandler resourceHandler;

@GET @Path("{cityName}") @Produces({"text/plain", "application/json"}) public Response getCity(@Context HttpHeaders headers, @Context UriInfo ui, @PathParam("cityName") String cityName) { return resourceHandler.handleRequest(headers, ui, cityName); } …}



The defined resources of a view plug into the Ambari REST API and are accessed through the given JAX-RS annotated ResourceProvider implementation.GET http://c6401.ambari.apache.org:8080/api/v1/views/WEATHER/versions/0.1.0/instances/US_WEST/ { "ViewInstanceInfo" : { "instance_name" : "US_WEST", "view_name" : "WEATHER", "properties" : { "cities" : "Palo Alto, US;Los Angeles, US;Portland, US;Seattle, US", "units" : "imperial" } }, "cities" : [ { "href" : "http://….org:8080/api/v1/views/WEATHER/instances/US_WEST/cities/Los Angeles, US", "id" : "Los Angeles, US", "instance_name" : "US_WEST” }, { "href" : "http://…:8080/api/v1/views/WEATHER/instances/US_WEST/cities/Palo Alto, US", "id" : "Palo Alto, US", "instance_name" : "US_WEST” },…

http://c6401.ambari.apache.org:8080/api/v1/views/WEATHER/instances/US_WEST/





View Interfaces: ViewContext

• Available to the view components through injection.• Provides access to the view and instance attributes.• Provides access to Ambari configuration.• Provides access to run time information about the current execution context.

public interface ViewContext { public String getUsername(); public String getViewName(); public String getInstanceName(); public Map<String, String> getProperties(); public String getAmbariProperty(String key); public ResourceProvider<?> getResourceProvider(String type); public URLStreamProvider getURLStreamProvider();}


View Interfaces : ResourceProvider

• Optional SPI. • Only required if the view defines a resource that needs to be

plugged into the Ambari API framework.• Used to monitor and manage view sub-resources through

CRUD operations.• The view developer may choose not to support all

operations.

public interface ResourceProvider<T> { public T getResource(String resourceId, Set<String> properties); public Set<T> getResources(ReadRequest request; public void createResource( String resourceId, Map<String, Object> properties; public boolean updateResource( String resourceId, Map<String, Object> properties; public boolean deleteResource(String resourceId);}


View UI

• Optional. • A view package may include a WEB-INF/web.xml so that the view may be deployed as a web app.

<web-app xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd" version="2.4">

<display-name>Weather Application</display-name> <description> This is the weather view application. </description> <servlet> <servlet-name>WeatherServlet</servlet-name> <servlet-class>org.apache.ambari.view.weather.WeatherServlet</servlet-class> </servlet> <servlet-mapping> <servlet-name>WeatherServlet</servlet-name> <url-pattern>/ui</url-pattern> </servlet-mapping></web-app>


View UI

• Servlet specified in web.xml. • Note the use of the ViewContext in the following example …

public class WeatherServlet extends HttpServlet {

private ViewContext viewContext;

@Override public void init(ServletConfig config) throws ServletException { super.init(config);

ServletContext context = config.getServletContext(); viewContext = (ViewContext) context.getAttribute(ViewContext.CONTEXT_ATTRIBUTE); }

@Override protected void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException { … PrintWriter writer = response.getWriter(); writer.println("<h1>" + viewContext.getInstanceName() + " Weather</h1>");


View UI

• Access View UI. { "href" : "http://c6401.ambari.apache.org:8080/api/v1/views/WEATHER/versions/0.1.0/instances/US_WEST/", "ViewInstanceInfo" : { "context_path" : "/views/WEATHER/US_WEST", "instance_name" : "US_WEST", "view_name" : "WEATHER", "properties" : { "cities" : "Palo Alto, US;Los Angeles, US;Portland, US;Seattle, US", "units" : "imperial" } },

Ambari Views - Overview

Technology

Transcript of Ambari Views - Overview