Hue Installation
Hue - Hadoop User Experience
What is Hue?
●
Hue is a Web interface for analyzing data
with Apache Hadoop.
●
Query and visualize data directly from
your Browser.
Installation
Prerequisites:
1.
Centos 6/7
2.
Hdp 2.3 above
3.
Apache ambari 1.7 above.
Preparing
Environment and downloading Hue:
●
Sudo yum groupinstall "Development
Tools"
●
Sudo yum install ant
●
Sudo yum install gcc g++
●
Sudo yum install libkrb5-dev
libmysqlclient-dev
●
Sudo yum install libssl-dev libsasl2-dev
libsasl2-modules-gssapi-mit
●
Sudo yum install libsqlite3-dev
●
Sudo yum install libtidy-0.99-0
libxml2-dev libxslt-dev
●
Apache
maven install:
$ cd /opt
$ sudo tar xzf
apache-maven-3.3.9-bin.tar.gz
$ sudo ln -s
apache-maven-3.3.9 maven
export
M2_HOME=/opt/maven
export
PATH=${M2_HOME}/bin:${PATH}
source
/etc/profile.d/maven.sh
●
Sudo yum install libldap2-dev
●
Sudo yum instal python-dev
python-simplejson python-setuptools
Make
sure Java 1.7 or above installed and configured correctly:
$ sudo yum install
openjdk-7-jre openjdk-7-jdk
$
sudo echo “JAVA_HOME=\”/usr/lib/jvm/java-7-openjdk-amd64/jre\”"
>> /etc/environment
Hue
Repository:
Make Install:
$ cd hue-3.8.1
$ make apps
It
will generate hue,hue.ini,desktop and configuration files. It will be installed
on /root/hue-3.8.1/ and file ownership
to root file permission.
Let's fix that run
correctly without root permission access.
$ sudo chown -R varun:varun
/root/hue-3.8.1
Let's fix
readonly/writeonly permission for desktop.db
$ sudo chmod 777
/root/hue-3.8.1/desktop
$ sudo
chmod 777 /root/hue-3.8.1/desktop/desktop.db
Configuring
Hue and Hadoop:
Step
1: You need to stop all the services in Ambari to modify the
configuration file. Use the following steps to stop the services in Ambari.
(i) login as admin
Stop
all service
Make
sure all service stopped.
Step
2: Configure HDP:
(a) Modify hadoop hdfs-site xml
$ sudo
vi /etc/Hadoop/conf/hdfs-site.xml
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.support.append</name>
<value>true</value>
</property>
(b)
Modify the hadoop core-site.xml file -
$ sudo
vi /etc/Hadoop/conf/core-site.xml
<property>
<name>hadoop.proxyuser.hue.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hue.groups</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hcat.groups</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.hcat.hosts</name>
<value>*</value>
</property>
(c) Disable the hdfs safe mode option:
$ hdfs dfsadmin
-safemode leave
(d)
Modify the oozie site xml:
$ vi
/etc/oozie/conf/oozie-site.xml
<property>
<name>oozie.service.ProxyUserService.proxyuser.hue.hosts</name>
<value>*</value>
</property>
<property>
<name>oozie.service.ProxyUserService.proxyuser.hue.groups</name>
<value>*</value>
</property>
Step
3: Restart all service via Ambari login from admin. go to Actions->start
all, it will start all the services again.
Step
4: Confirm WebHDFS running:
Step
5: HUE Configuration:
$ cd
/root/hue-3.8.1/desktop/conf
$
vi hue.ini
(A) Configure Web Server - Insert those value as
specified below configuration
[desktop]
# Set this to a random string, the longer
the better.
# This is used for secure hashing in the
session store.
secret_key=jFE93j;2[290-eiw.KEiwN2s3['d;/.q[eIW^y#e=+Iei*@Mn<qW5o
#
Webserver listens on this address and port
http_host=0.0.0.0
http_port=8000
(B) Configure HDFS Configuration:
[hadoop]
#
Configuration for HDFS NameNode
#
------------------------------------------------------------------------
[[hdfs_clusters]]
# HA support by using HttpFs
[[[default]]]
# Enter the filesystem uri
fs_defaultfs=hdfs://sandbox.hortonworks.com:8020
# NameNode logical name.
##
logical_name=
# Use
WebHdfs/HttpFs as the communication mechanism.
# Domain should be the NameNode or HttpFs
host.
#
Default port is 14000 for HttpFs.
webhdfs_url=http://sandbox.hortonworks.com:50070/webhdfs/v1
# Change
this if your HDFS cluster is Kerberos-secured
(C)
Configure YARN cluster Resource Manager, Hive, Oozie, etc:
(i) [[yarn_clusters]]
[[[default]]]
#
Enter the host on which you are running the ResourceManager
resourcemanager_host=localhost
#
The port where the ResourceManager IPC listens on
##
resourcemanager_port=8032
#
Whether to submit jobs to this cluster
submit_to=True
#
Resource Manager logical name (required for HA)
##
logical_name=
# Change
this if your YARN cluster is Kerberos-secured
##
security_enabled=false
# URL of
the ResourceManager API
resourcemanager_api_url=http://localhost:8088
# URL of
the ProxyServer API
# URL of the
HistoryServer API
history_server_api_url=http://localhost:19888
(ii)
[liboozie]
# The URL where the Oozie service runs
on. This is required in order for
# users to submit jobs. Empty value
disables the config check.
#
Requires FQDN in oozie_url if enabled
## security_enabled=false
(iii)
[beeswax]
#
Host where HiveServer2 is running.
# If Kerberos security is enabled, use
fully-qualified domain name (FQDN).
hive_server_host=localhost
#
Port where HiveServer2 Thrift server runs on.
## hive_server_port=10000
(iv)
[hbase]
#
Comma-separated list of HBase Thrift servers for clusters in the format of
'(name|host:port)'.
#
Use full hostname with security.
#
If using Kerberos we assume GSSAPI SASL, not PLAIN.
hbase_clusters=(Cluster|localhost:9090)
#
HBase configuration directory, where hbase-site.xml is located.
## hbase_conf_dir=/etc/hbase/conf
(V) Find app_blacklist disable security
under [desktop]
#
Comma separated list of apps to not load at server startup.
# e.g.: pig,zookeeper
app_blacklist=impala,security
# The directory where to store the auditing
logs. Auditing is disable if the value is empty.
# e.g. /var/log/hue/audit.log
## audit_event_log_dir=
(vi) Solr Search configuration
[search]
#
URL of the Solr Server
solr_url=http://localhost:8983/solr/
#
Requires FQDN in solr_url if enabled
## security_enabled=false
## Query sent when no term is entered
## empty_query=*:*
Step
6: Start Hue Supervisor
$ cd /root/hue-3.8.1/build/env/bin
$ ./supervisor
Step
7: Open new terminal:
$
cd /root/hue-3.8.1/build/env/bin
$ ./hue livy_server
It
Works!!!
File
Browser
Manage
Users:
Query
Editors:
References:
Comments
Post a Comment