Download - Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

Transcript
Page 1: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page1of15

QuickDeploymentStep-by-stepinstructionstodeployOracleBigDataLiteVirtualMachine

Version4.11

LastUpdated:1/10/2018

Pleasenote:Thisapplianceisfortestingandeducationalpurposesonly;itisunsupportedandnottobeusedinproduction.

CONTENTS

Introduction........................................................................................................................................................1

InstallationSteps.................................................................................................................................................3

TroubleshootingTips...........................................................................................................................................9

UsefulTips.........................................................................................................................................................11

INTRODUCTION

OracleBigDataLiteVirtualMachineprovidesanintegratedenvironmenttohelpyougetstartedwiththeOra-cleBigDataplatform.ManyOracleBigDataplatformcomponentshavebeeninstalledandconfigured-allow-ingyoutobeginusingthesystemrightaway.SeetheBigDataLitelandingpage(http://www.oracle.com/technetwork/database/bigdata-appliance/oracle-bigdatalite-2104726.html)onOTNfortutorials,videos,whitepapersandmore.

ThefollowingcomponentsareincludedonOracleBigDataLite:

• OracleEnterpriseLinux6.9

Page 2: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page2of15

• OracleDatabase12cRelease1EnterpriseEdition(12.1.0.2)-includingOracleBigDataSQL-enabledex-ternaltables,OracleMultitenant,OracleAdvancedAnalytics,OracleOLAP,OraclePartitioning,OracleSpatialandGraph,andmore.

• ClouderaDistributionincludingApacheHadoop(CDH5.13.1)• ClouderaManager(5.13.1)• OracleBigDataSpatialandGraph2.4• OracleBigDataConnectors4.11

o OracleSQLConnectorforHDFS3.8.1o OracleLoaderforHadoop3.9.1o OracleDataIntegrator12c(12.2.1.3.0)o OracleRAdvancedAnalyticsforHadoop2.7.1o OracleXQueryforHadoop4.9.1o OracleDataSourceforApacheHadoop1.2.1o OracleShellforHadoopLoaders1.3.1

• OracleNoSQLDatabaseEnterpriseEdition12cR1(4.5.12)• OracleJDeveloper12c(12.2.1.2.0)• OracleSQLDeveloperandDataModeler17.3.1withOracleRESTDataServices3.0.7• OracleDataIntegrator12cR1(12.2.1.3.0)• OracleGoldenGate12c(12.3.0.1.2)• OracleRDistribution3.3.0• OraclePerfectBalance2.10.0

BigDataLiteincludessoftwareproductsthatareoptionalontheOracleBigDataAppliance(BDA),includingOracleNoSQLDatabaseEnterpriseEdition,OracleBigDataSQL,OracleBigDataSpatialandGraphandOracleBigDataConnectors.

Togetstarted,youshouldperformthefollowinghighleveltasks:

• Ensureyourhostcomputermeetstherequirementsforrunningthevirtualmachine• DownloadandinstallVirtualBoxandVirtualBoxExtensionPack–whichisacross-platformvirtualiza-

tionapplication• DownloadthezipfilesfromOTNanduse7-ziptoextracttheappliancefile• CreatetheVMbyimportingtheBigDataLite-xxx.ovafileintoVirtualBox• Startthemachine!• LogintoBigDataLiteusing:

Page 3: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page3of15

userid:oraclepassword:welcome1

• OpentheStartHeredocumentonthedesktopfordetailsabouttheenvironment.

INSTALLATIONSTEPS

1. Prepareyourhostsystem.

• Minimum8GBofrealmemory;moreisbetter.5GmemorywillbededicatedtotheVM.Somedetails:o BigDataLitew/oClouderaManager(CM)requires5GBmemorydedicatedtotheVMo BigDataLitew/CMrunningrequires10GBmemory

• TurnonVirtualAssistfeaturesintheBIOS(usuallydonebydefault).RefertoTroubleshootingtipsinStep12formoredetails.

• ~73GBdiskspaceneededtodownloadandinstall:o 11zipfiles:~24GB(Canberemovedafterextraction)o .ovafile:~24GB(Resultofunzip.Itcanberemovedafterimporting–butnicetokeeparound)o importedimage:~49GB.

• Downloadandinstall7Zip.• Downloadandinstallmd5sum(optional)

2.DownloadandinstallOracleVirtualBoxandVirtualBoxExtensionPack(version4.3.26andaboveisrec-ommend).

ItishighlyrecommendedthatyoualsoinstalltheVirtualBoxGuestAdditions–whichprovidesen-hancedmousesupport,clipboardsupportandsharingoffilesbetweenthevirtualmachineanditshost.

3. DownloadBigDataLitefilesfromOracleTechnologyNetwork

• GototheBigDataLitelandingpageat:http://www.oracle.com/technetwork/database/bigdata-appliance/oracle-bigdatalite-2104726.html

• ClicktoaccepttheOTNlicense.• Downloadallofthefilestoasingledirectory.

4. Afterallzipfilesaredownloaded,extractthemwith7zip.

InWindows:• Right-clickonfileBigDataLite-xxx.7z.001fileandselect7-zip->ExtractHere

Page 4: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page4of15

InLinux:

• Runthefollowingcommandtoextractthefile:

7za e BigDataLite-xxx.7z.001

Result:TheextractionwillcreatetheBigDataLite-xxx.ovaappliancefile.ThisfilewillbeusedtocreateanewmachineinVirtualBox.Thissinglefilecontainstheentiremachinedefinition,includingthephysicaldisksaswellasdefaultsforthemachineconfiguration(e.g.network,CPUs,memory,etc.).

5. StartOracleVirtualBoxManagerandImporttheAppliance

AfterstartingVirtualBox,createthenewvirtualmachineusingtheimportwizard:• ClickFile->ImportAppliancetolaunchtheimportwizard• IntheImportVirtualAppliancepage,clickOpenappliance…• LocatetheBigDataLite-xxx.ovafileandclickOpen.ClickNext.• Appliancesettingsgiveyouanoverviewoftheconfiguration.Inthisstep,youmayneedto

updatethelocationoftheVirtualDiskImages.Thesefileswillinitiallyuseapproximately50GBofdisk–andcanexpandasyouusethemachine.ChangethedefaultlocationoftheVir-tualDiskImagesifthecurrentlocationdoesnothaveenoughcapacity.Note,theVMuses4separatedisks.ThesedisksshouldnotbemovedafterimportingtheVM:

Page 5: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page5of15

ClickImport

Result:AnewBigDataLite-xxxVirtualMachineisavailableintheVirtualBoxManager:

Page 6: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page6of15

6.VirtualMachineSettings

BigDataLiteisconfiguredwiththefollowingparameters,whichisidealforan8GBhostsystem:• 5GBRAM• 2processors• NATNetworkAdapter

Asmentionedearlier,ifyouwouldliketorunClouderaManager,youwillneedtoincreasethememoryallocatedtotheVM.YoucanupdatethesedefaultsettingsbyselectingtheBigDataLiteVMinVirtualBoxManagerandclickingSettings:

YoucanupdatetheamountofmemoryandCPUsallocatedtothemachinefromtheSystemtab.PleasereviewtheVirtualBoxdocumentationtolearndetailsaboutupdatingtheconfiguration.

7.StartyourBigDataLite!

Double-clickonBigDataLite-xxxtostarttheVM.Logonastheoracleusertogetstarted:id: oraclepassword: welcome1

ThiswillbringyoutoaLinuxdesktop.TheStartHeredocumentonthedesktopwillprovideyoude-tailsabouttheinstalledsoftware,idsandpasswords,andmore.

Page 7: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page7of15

BigDataLiteDesktop

8. Start/StopServices

UsetheStart/StopServicesscripttostart/stopyourservices(youcanalsotypeservicesatthelinuxprompt).Servicesonlyappeariftheminimummemoryrequirementismet.Ensurethatyoustartallservicesrequiredforanapplication(e.g.HiverequiresHDFS).

Page 8: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page8of15

Starting/Stoppingservices

9. BigDataLiteSamples

BigDataLitesamplesarenowavailableongithub(https://github.com/oracle/BigDataLite).ThismakesiteasytoupdateyourVMwiththelatestcollateralavailable.ThereisabranchforeachBigDataLiterelease–startingwithBigDataLite4.3.0.Double-clicktheRefreshSamplesiconandfollowthein-structionstorefreshyourlocalcopyofthesampleswiththehostedrepository:

Page 9: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page9of15

RefreshSamplesThesamplesaredownloadedtothe/home/oracle/srcdirectory.Note:thecontentsinthisdirectorywillbedeletedaspartoftherefreshprocess–soremembertosaveanycustomizationsyoumadeinthisdirectory.Also,directory/home/oracle/movieisabindmountthatreferstodirectory/home/oracle/src/movie.Thismeansthatthesetwofoldersareequivalent;anychangestothecontentsinonefolderisreflectedintheother.

TROUBLESHOOTINGTIPS

1.Failedtoimportappliance<path>/BigDataLite-xxx.ova.CouldnotopenthecurrentfileintheOVApack-age(VERR_TAR_CHKSUM_MISMATCH).

Thiserroroccurswhenattemptingtoimportanimage.Tosolvetheproblem,simplyupdateyourver-sionofVirtualBox.

2.ERROR:“Failedtoopenasessionforthevirtualmachine”whenattemptingtostartthemachine

TheVMrequiresthathardwarevirtualizationisenabledforthehostcomputer.Ifitisnotenabled,youwillreceivethefollowingerrorwhenattemptingtostartthemachine:FailedtoopenasessionforthevirtualmachineBigDataLite-xxx.VT-xisdisabledintheBIOS(VERR_VMX_MSR_VMXON_DISABLED)YoucanupdatetheBIOSatboottimeforthehost.Ensurethatboth"Intel(R)VirtualizationTechnolo-gy"and"Intel(R)VT-dFeature"areenabled.PleasereviewtheinstructionsprovidedbyyourhardwaremanufacturerthatdescribeupdatingBIOSsettings.

Page 10: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page10of15

3.ERROR:"Thiskernelrequiresanx86-64CPU,butonlydetectedani686CPU.Unabletoboot-pleaseuseakernelappropriateforyourCPU"

EnsurethatyourmachineisrunningOracle(64-bit)Linux.IntheVMsettingsselect"General"andsettheVersiontoOracle(64-bit).

4.ERROR:“ImplementationoftheUSB2.0controllernotfound!”

InstalltheappropriateVirtualBoxExtensionPackforyourversionofVirtualBox.

5.Whiletheimageisstartingup,youinitiallyseeafewmessagesandthenablankscreen

Ifthishappens,thenstoptheimage.GobacktoVirtualBoxManagerandclickontheimage’sSet-tings.GotoDisplayandincreasetheVideomemorytoatleast6MB.Starttheimageandyoushouldnolongerseetheblankscreen.

6.Unabletocopy/pastebetweenthehostandBigDataLite

First,ensurethatyouhaveinstalledtheVirtualBoxGuestAdditions.IfyouhaveinstalledGuestAddi-tionsandcopy/pastestilldoesn’twork–gototheDevicesmenuandensurethatSharedClipboardisenabled.

7.IncorrectJDKusedbyHadoop.Note:thiswillnotoccurunlessyouaddJDK1.7totheVM.

TherearenumerouserrorsthatcanoccurbecauseCDHisusingJDK1.7insteadofJDK1.8(seebelow).TousethecorrectJDK:

• Openaterminalwindowbysingle-clickingtheTerminaliconinthetaskbar.• Enterthefollowingcommand:

o sudovi/etc/default/bigtop-utilso Addthefollowinglineattheendofthefile:

exportJAVA_HOME=/usr/java/latest• RestarttheVMtomakesurethattheupdatedsettingsareused.

ManyoftheseerrorsoccurwhenODIisattemptingtorunjobsontheVM.Forexample:

• com.sunopsis.core.SnpsForeignKeyReferencedObjectMissingException:ODI-10244:CannotimportDataTypeConversion:949-62-184(withglobalid:7663ea6c-f7c3-4d0b-bd80-

Page 11: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page11of15

3117c1e06868andinternalid:949|62|184)becausereferencedTechnologywithglobalid7ffd3917-2b27-35a3-96a7-3bde26098531doesnotexist.

• Error:QueryResult:Unsupportedmajor.minorversion52.0

USEFULTIPS

1.SharingfoldersbetweenBigDataLiteanditshost

VirtualBoxGuestAdditionsprovidesausefulfeaturethatenablesyoutoshareahost’sfolderwithinthevirtualmachine.Toconfigureasharedfolder:

• SelectBigDataLite-xxxfromthelistofmachinesandclickSettings

Page 12: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page12of15

• GotoSharedFoldersandclickAddFolder

• SpecifythefolderyouwouldliketoshareandselectAuto-mount.ThenclickOK.Below,the

H:\driveisbeingshared.

Youcanrepeatthisprocessformultiplesharedfolders.

Page 13: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page13of15

• ToaccessthesharedfolderfromBigDataLite,navigateto/media/sf_<folder_name>

2.UsingtheVMasaHadoopClient

YoucanuseBigDataLiteasaHadoopclient–allowingyoutouseitasadevelopmentenvironmentfortheOracleBigDataAppliance.ToconnecttheBigDataLitetoBDA,logintotheClouderaManagerin-stancethatismanagingyourHadoopclusterandselectActions->ViewClientConfigurationURLs:

ThiswilltakeyoutotheClientConfigurationURLspage.SelecttheappropriateClientConfiguration(e.g.MapReduce):

Page 14: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page14of15

Thiswilldownloadazipfilecontainingallthefilesrequiredtoconnecttothecluster.Afterdownload-ingthefile,executethefollowingtwosteps:

• Unzipthefileinto/home/oracle.• SettheappropriateenvironmentvariablethatwillpointBigDataLitetotheremotecluster.

Forexample,ifyoudownloadedbothhiveandmapreduceconfigurations:export HADOOP_CONF_DIR=/home/oracle/hadoop-conf export HIVE_CONF_DIR=/home/oracle/hive-conf

3.ConnectingexternalclientstoBigDataLite

YoumaywanttoconnecttoBigDataLitefromyourhostcomputer.VirtualBoxportforwardinghasbeenimplementedallowingyoutoconnecttheguestusingabrowserorSSH.Youwillconnecttoserver“localhost:<port#>”andVirtualBoxwillforwardtherequesttotheguest.ListedbelowaretheportmappingsforservicesrunninginBigDataLite:

Service PortNameNode 50070SecondaryNameNode 50090ResourceManager 8088NodeManager 8042JobHistoryServer 19888Hue 8888OozieServer 11000HistoryServer 18088

Page 15: Quick Deployment Step-by-step instructions to deploy ... · Oracle Big Data Lite 4.11 Quick Deployment Page 1 of 15 Quick Deployment Step-by-step instructions to deploy Oracle Big

OracleBigDataLite4.11QuickDeployment

Page15of15

CloudeaManager 7180Moviedemo 7001SolrAdmin 8983Oracle 1521Apex 8080SSH 2222NoSQLDBAdmin 5001BigDataSpatial&Graph 8045