Post on 26-Jan-2015
description
AMBARI VIEWS
Ambari UX So Far…
Mostly for addressing “Operator” Concerns
Host went down!
Disk is full!
Need to tweak configs
The DataNodes are down!
Need to add hosts
Need to secure cluster
Need NameNod
e HA
Ambari UX So Far…
Some for addressing “Data Worker” Concerns
Hmm…why is my query slow?
Across Hadoop…a whole bunch of other UIs out there!
You may have used…• Native UIs for various Hadoop ecosystem components:
MapReduce Job History, NameNode, ResourceManager, HBase, Storm, Oozie, Falcon, etc.
• Hue• Ambrose (Twitter)• White Elephant (LinkedIn)• Lipstick (Netflix)• …and so on
!@#$
Ambari: Common UX for Hadoop
Provide a common, secure and pluggable approach for UX across:• Operators, System Admin• Data Workers• Application Developers• …and others
Yay!
Yay!
Ambari Views: Goals
Single point of entry✔ Common URL for common user communities✔ “Views” embedded in Ambari UI
Pluggable UI Framework✔ “Views” contributed and shared as plugins✔ No code changes to the core✔ Browse published Views and install
Ambari Views: Goals
Authorization✔ Control who can access which views and which aspects of views✔ Deployment model supports connecting to different LDAP/ADs by user community
Runs on Ambari Server✔ No extra daemons needed
Runs Ambari “standalone”✔ No need to deploy cluster via Ambari to use Views
Example Views
Operators• Capacity Scheduler
Queue Manager• YARN Resource Utilization• Heatmaps• HDFS / Hive Mirroring
Data Workers• Pig Query Editor • Hive Query Editor• Workflow Design• HDFS File Browser• Hive/Tez
Visualization
Application Developers• Job Visualization• Streaming Topology Visualization
Views and the Framework
Views Framework
Views
Core to Ambari
Plugins to Ambari
Components of a View
VIEWClient-side
assets(.js, html)
AMBARI WEB
VIEWServer-side resources
(java)
AMBARI SERVER
{rest}Hadoop
and other systems
View Packaging• View descriptor : view.xml• Resource / Service classes : JAX-RS annotated• UI classes : html, Servlets deployed as web app (WEB-INF/web.xml).• Application logic : Supporting classes• Dependencies : 3rd party jars or classes
├── WEB-INF│ └── web.xml├── org│ └── apache│ └── ambari│ └── view│ └── filebrowser│ ├── DownloadService.class│ ├── FileBrowserService.class│ └── FileOperationService.class└── view.xml
Page 11
View Versions + Instances
• Multiple versions of a View• Multiple View Instances of each version
View NameView
InstancesView
Versions
ViewVersions
View InstancesView
Instances
View Instances
• View Context– View and user information
• Instance Data– Lightweight name/value (for prefs)
• Events– Framework and Custom events
Framework Services
Client-Side Server-Side ViewContext.getUsername()ViewContext.getInstanceData()ViewController.fireEvent()
View Events
• Server-side Framework and Custom eventsonDeploy(), onCreate(), onDestroy()
Develop Deploy Create instancesPackage
onDeploy() onCreate()
onDestroy()
Authentication
VIEW
AMBARI SERVER
Ambari DB
LDAP
{rest}
<html>
User AuthN Source
User-Permission Mapping
Authenticate
Provide principal via ViewContext
Views Deployment
• Deploy Views as part of an operational Ambari Server• Or deploy standalone “Ambari Views Server” for data workers
Page 16
AmbariServer
HADOOPStore & Process
AmbariViews Server
Operators manage the cluster, may have Views deployed
Data Workers use the cluster and use the Ambari Views Server for Views (no agents)
Learn More
• Frameworkhttps://github.com/apache/ambari/tree/trunk/ambari-viewshttps://github.com/apache/ambari/blob/trunk/ambari-views/docs/index.md
• Framework Exampleshttps://github.com/apache/ambari/tree/trunk/ambari-views/examples
• View Contribshttps://github.com/apache/ambari/tree/trunk/contrib/views
EXTRAS
Page 18
© Hortonworks Inc. 2014
The Deployed View
Page 19
• Views are deployed by placing the view package in the Ambari view folder.
• Once deployed, views and view instances are available through the Ambari REST API.
GET http://c6401.ambari.apache.org:8080/api/v1/views/WEATHER/ { "href" : "http://c6401.ambari.apache.org:8080/api/v1/views/", "items" : [ { "href" : "http://c6401.ambari.apache.org:8080/api/v1/views/WEATHER", "ViewInfo" : { "view_name" : "WEATHER" } } ]}
© Hortonworks Inc. 2014
View Descriptor : view.xml
Page 20
• name – the internal name of the view (must be unique)• label – the public display name of the view• version – the version of the view• parameter – metadata about view instance properties• resource – the names of the classes required to support a
view sub-resource• instance – optional property sets which define view instances.<view> <name>MYVIEW</name> <label>My View</label> <version>1.0.0</version> <parameter> <name>scope</name> <description>The scope of the instance.</description> <required>true</required> </parameter></view>
© Hortonworks Inc. 2014
View Descriptor : parameter
Page 21
• Values for view parameters are given as properties when a view instance is defined.
– name – the name of the parameter– description – a brief description of the parameter– required – indicates whether the parameter is required for instance
definition<view> … <parameter> <name>scope</name> <description>The scope of the instance.</description> <required>true</required> </parameter> <instance> <name>GLOBAL_INSTANCE</name> <property> <key>scope</key> <value>global</value> </instance></view>
© Hortonworks Inc. 2014
View Descriptor : instance
Page 22
• Values for view parameters are given as properties when a view instance is defined.
• Multiple instance may be defined for a view.– name – the name of the instance– property– key / value pair. Name should match a view parameter.
<view> … <parameter> <name>scope</name> <description>The scope of the instance.</description> <required>true</required> </parameter> <instance> <name>GLOBAL_INSTANCE</name> <property> <key>scope</key> <value>global</value> </instance></view>
© Hortonworks Inc. 2014
View Descriptor : resource
Page 23
• The defined resources of a view plug into the Ambari REST API.
– name – the name of the resource– plural-name – the plural name as represented in the API– id-property – the identifying property of the resource– resource-class – the JavaBean resource class– provider-class – the ResourceProvider implementation– service-class – the JAX-RS annotated resource service class
<view> … <resource> <name>city</name> <plural-name>cities</plural-name> <id-property>id</id-property> <resource-class>org.apache.ambari.view.weather.CityResource</resource-class> <provider-class>org.apache.ambari.view.weather.CityResourceProvider</provider-class> <service-class>org.apache.ambari.view.weather.CityService</service-class> </resource></view>
© Hortonworks Inc. 2014
View Descriptor : resource
Page 24
A resource class can be any JavaBean. The resource instances will be provided by an implementation of a ResourceProvider, which is written by the view developer.
public class CityResource {
private String id; private Map<String, Object> weather; private String units;
public String getId() { return id; }
public void setId(String id) { this.id = id; } …}
© Hortonworks Inc. 2014
View Descriptor : resource
Page 25
A resource provider class should implement ResourceProvider. Note the injected ViewContext in the following example…
public class CityResourceProvider implements ResourceProvider<CityResource> {
@Inject ViewContext viewContext;
@Override public CityResource getResource(String resourceId, Set<String> propertyIds) throws SystemException, NoSuchResourceException, UnsupportedPropertyException {
Map<String, String> properties = viewContext.getProperties();
String units = properties.get("units");
try { return getResource(resourceId, units, propertyIds); } catch (IOException e) { throw new SystemException("Can't get city resource " + resourceId + ".", e); } } …}
© Hortonworks Inc. 2014
View Descriptor : resource
Page 26
A resource service class should be annotated with JAX-RS annotations to handle service requests. Note the injected ViewResourceHandler in the following example…
public class CityService { @Inject ViewResourceHandler resourceHandler;
@GET @Path("{cityName}") @Produces({"text/plain", "application/json"}) public Response getCity(@Context HttpHeaders headers, @Context UriInfo ui, @PathParam("cityName") String cityName) { return resourceHandler.handleRequest(headers, ui, cityName); } …}
© Hortonworks Inc. 2014
View Descriptor : resource
Page 27
The defined resources of a view plug into the Ambari REST API and are accessed through the given JAX-RS annotated ResourceProvider implementation.GET http://c6401.ambari.apache.org:8080/api/v1/views/WEATHER/versions/0.1.0/instances/US_WEST/ { "ViewInstanceInfo" : { "instance_name" : "US_WEST", "view_name" : "WEATHER", "properties" : { "cities" : "Palo Alto, US;Los Angeles, US;Portland, US;Seattle, US", "units" : "imperial" } }, "cities" : [ { "href" : "http://….org:8080/api/v1/views/WEATHER/instances/US_WEST/cities/Los Angeles, US", "id" : "Los Angeles, US", "instance_name" : "US_WEST” }, { "href" : "http://…:8080/api/v1/views/WEATHER/instances/US_WEST/cities/Palo Alto, US", "id" : "Palo Alto, US", "instance_name" : "US_WEST” },…
© Hortonworks Inc. 2014
View Interfaces: ViewContext
Page 28
• Available to the view components through injection.• Provides access to the view and instance attributes.• Provides access to Ambari configuration.• Provides access to run time information about the current execution context.
public interface ViewContext { public String getUsername(); public String getViewName(); public String getInstanceName(); public Map<String, String> getProperties(); public String getAmbariProperty(String key); public ResourceProvider<?> getResourceProvider(String type); public URLStreamProvider getURLStreamProvider();}
© Hortonworks Inc. 2014
View Interfaces : ResourceProvider
Page 29
• Optional SPI. • Only required if the view defines a resource that needs to be
plugged into the Ambari API framework.• Used to monitor and manage view sub-resources through
CRUD operations.• The view developer may choose not to support all
operations.
public interface ResourceProvider<T> { public T getResource(String resourceId, Set<String> properties); public Set<T> getResources(ReadRequest request; public void createResource( String resourceId, Map<String, Object> properties; public boolean updateResource( String resourceId, Map<String, Object> properties; public boolean deleteResource(String resourceId);}
© Hortonworks Inc. 2014
View UI
Page 30
• Optional. • A view package may include a WEB-INF/web.xml so that the view may be deployed as a web app.
<web-app xmlns="http://java.sun.com/xml/ns/j2ee" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/j2ee http://java.sun.com/xml/ns/j2ee/web-app_2_4.xsd" version="2.4">
<display-name>Weather Application</display-name> <description> This is the weather view application. </description> <servlet> <servlet-name>WeatherServlet</servlet-name> <servlet-class>org.apache.ambari.view.weather.WeatherServlet</servlet-class> </servlet> <servlet-mapping> <servlet-name>WeatherServlet</servlet-name> <url-pattern>/ui</url-pattern> </servlet-mapping></web-app>
© Hortonworks Inc. 2014
View UI
Page 31
• Servlet specified in web.xml. • Note the use of the ViewContext in the following example …
public class WeatherServlet extends HttpServlet {
private ViewContext viewContext;
@Override public void init(ServletConfig config) throws ServletException { super.init(config);
ServletContext context = config.getServletContext(); viewContext = (ViewContext) context.getAttribute(ViewContext.CONTEXT_ATTRIBUTE); }
@Override protected void doGet(HttpServletRequest request, HttpServletResponse response) throws IOException { … PrintWriter writer = response.getWriter(); writer.println("<h1>" + viewContext.getInstanceName() + " Weather</h1>");
© Hortonworks Inc. 2014
View UI
Page 32
• Access View UI. { "href" : "http://c6401.ambari.apache.org:8080/api/v1/views/WEATHER/versions/0.1.0/instances/US_WEST/", "ViewInstanceInfo" : { "context_path" : "/views/WEATHER/US_WEST", "instance_name" : "US_WEST", "view_name" : "WEATHER", "properties" : { "cities" : "Palo Alto, US;Los Angeles, US;Portland, US;Seattle, US", "units" : "imperial" } },