This chapter will cover access to Apache HBase either through non-Java languages and through custom protocols. For information on using the native HBase APIs, refer to User API Reference and the HBase APIs chapter.

97. REST

Representational State Transfer (REST) was introduced in 2000 in the doctoral dissertation of Roy Fielding, one of the principal authors of the HTTP specification.

REST itself is out of the scope of this documentation, but in general, REST allows client-server interactions via an API that is tied to the URL itself. This section discusses how to configure and run the REST server included with HBase, which exposes HBase tables, rows, cells, and metadata as URL specified resources. There is also a nice series of blogs on How-to: Use the Apache HBase REST Interface by Jesse Anderson.

97.1. Starting and Stopping the REST Server

The included REST server can run as a daemon which starts an embedded Jetty servlet container and deploys the servlet into it. Use one of the following commands to start the REST server in the foreground or background. The port is optional, and defaults to 8080.

  1. # Foreground
  2. $ bin/hbase rest start -p <port>
  4. # Background, logging to a file in $HBASE_LOGS_DIR
  5. $ bin/hbase-daemon.sh start rest -p <port>

To stop the REST server, use Ctrl-C if you were running it in the foreground, or the following command if you were running it in the background.

  1. $ bin/hbase-daemon.sh stop rest

97.2. Configuring the REST Server and Client

For information about configuring the REST server and client for SSL, as well as doAs impersonation for the REST server, see Configure the Thrift Gateway to Authenticate on Behalf of the Client and other portions of the Securing Apache HBase chapter.

97.3. Using REST Endpoints

The following examples use the placeholder server http://example.com:8000, and the following commands can all be run using curl or wget commands. You can request plain text (the default), XML , or JSON output by adding no header for plain text, or the header “Accept: text/xml” for XML, “Accept: application/json” for JSON, or “Accept: application/x-protobuf” to for protocol buffers.

Unless specified, use GET requests for queries, PUT or POST requests for creation or mutation, and DELETE for deletion.

Endpoint HTTP Verb Description Example
/version/cluster GET Version of HBase running on this cluster 
  1. curl -vi -X GET \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/version/cluster"

|
| /status/cluster | GET | Cluster status |

  1. curl -vi -X GET \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/status/cluster"

|
| / | GET | List of all non-system tables |

  1. curl -vi -X GET \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/"

|

Endpoint HTTP Verb Description Example
/namespaces GET List all namespaces 
  1. curl -vi -X GET \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/namespaces/"

|
| /namespaces/_namespace_ | GET | Describe a specific namespace |

  1. curl -vi -X GET \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/namespaces/special_ns"

|
| /namespaces/_namespace_ | POST | Create a new namespace |

  1. curl -vi -X POST \
  2.   -H "Accept: text/xml" \
  3.   "example.com:8000/namespaces/special_ns"

|
| /namespaces/_namespace_/tables | GET | List all tables in a specific namespace |

  1. curl -vi -X GET \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/namespaces/special_ns/tables"

|
| /namespaces/_namespace_ | PUT | Alter an existing namespace. Currently not used. |

  1. curl -vi -X PUT \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/namespaces/special_ns

|
| /namespaces/_namespace_ | DELETE | Delete a namespace. The namespace must be empty. |

  1. curl -vi -X DELETE \
  2.   -H "Accept: text/xml" \
  3.   "example.com:8000/namespaces/special_ns"

|

Endpoint HTTP Verb Description Example
/_table_/schema GET Describe the schema of the specified table. 
  1. curl -vi -X GET \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/users/schema"

|
| /_table_/schema | POST | Update an existing table with the provided schema fragment |

  1. curl -vi -X POST \
  2.   -H "Accept: text/xml" \
  3.   -H "Content-Type: text/xml" \
  4.   -d '<?xml version="1.0" encoding="UTF-8"?><TableSchema name="users"><ColumnSchema name="cf" KEEP_DELETED_CELLS="true" /></TableSchema>' \
  5.   "http://example.com:8000/users/schema"

|
| /_table_/schema | PUT | Create a new table, or replace an existing table’s schema |

  1. curl -vi -X PUT \
  2.   -H "Accept: text/xml" \
  3.   -H "Content-Type: text/xml" \
  4.   -d '<?xml version="1.0" encoding="UTF-8"?><TableSchema name="users"><ColumnSchema name="cf" /></TableSchema>' \
  5.   "http://example.com:8000/users/schema"

|
| /_table_/schema | DELETE | Delete the table. You must use the /_table_/schema endpoint, not just /_table_/. |

  1. curl -vi -X DELETE \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/users/schema"

|
| /_table_/regions | GET | List the table regions |

  1. curl -vi -X GET \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/users/regions

|

Endpoint HTTP Verb Description Example
/_table_/_row_ GET Get all columns of a single row. Values are Base-64 encoded. This requires the “Accept” request header with a type that can hold multiple columns (like xml, json or protobuf). 
  1. curl -vi -X GET \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/users/row1"

|
| /_table_/_row_/_column:qualifier_/_timestamp_ | GET | Get the value of a single column. Values are Base-64 encoded. |

  1. curl -vi -X GET \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/users/row1/cf:a/1458586888395"

|
| /_table_/_row_/_column:qualifier_ | GET | Get the value of a single column. Values are Base-64 encoded. |

  1. curl -vi -X GET \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/users/row1/cf:a"
  5. curl -vi -X GET \
  6.   -H "Accept: text/xml" \
  7.    "http://example.com:8000/users/row1/cf:a/"

|
| /_table_/_row_/_column:qualifier_/?v=_number_of_versions_ | GET | Multi-Get a specified number of versions of a given cell. Values are Base-64 encoded. |

  1. curl -vi -X GET \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/users/row1/cf:a?v=2"

|

Endpoint HTTP Verb Description Example
/_table_/scanner/ PUT Get a Scanner object. Required by all other Scan operations. Adjust the batch parameter to the number of rows the scan should return in a batch. See the next example for adding filters to your scanner. The scanner endpoint URL is returned as the Location in the HTTP response. The other examples in this table assume that the scanner endpoint is http://example.com:8000/users/scanner/145869072824375522207. 
  1. curl -vi -X PUT \
  2.   -H "Accept: text/xml" \
  3.   -H "Content-Type: text/xml" \
  4.   -d '<Scanner batch="1"/>' \
  5.   "http://example.com:8000/users/scanner/"

|
| /_table_/scanner/ | PUT | To supply filters to the Scanner object or configure the Scanner in any other way, you can create a text file and add your filter to the file. For example, to return only rows for which keys start with u123 and use a batch size of 100, the filter file would look like this:[source,xml] —— { “type”: “PrefixFilter”, “value”: “u123” } ——Pass the file to the -d argument of the curl request. |

  1. curl -vi -X PUT \
  2.   -H "Accept: text/xml" \
  3.   -H "Content-Type:text/xml" \
  4.   -d @filter.txt \
  5.   "http://example.com:8000/users/scanner/"

|
| /_table_/scanner/_scanner-id_ | GET | Get the next batch from the scanner. Cell values are byte-encoded. If the scanner has been exhausted, HTTP status 204 is returned. |

  1. curl -vi -X GET \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/users/scanner/145869072824375522207"

|
| _table_/scanner/_scanner-id_ | DELETE | Deletes the scanner and frees the resources it used. |

  1. curl -vi -X DELETE \
  2.   -H "Accept: text/xml" \
  3.   "http://example.com:8000/users/scanner/145869072824375522207"

|

Endpoint HTTP Verb Description Example
/_table_/_row_key_ PUT Write a row to a table. The row, column qualifier, and value must each be Base-64 encoded. To encode a string, use the base64 command-line utility. To decode the string, use base64 -d. The payload is in the --data argument, and the /users/fakerow value is a placeholder. Insert multiple rows by adding them to the <CellSet> element. You can also save the data to be inserted to a file and pass it to the -d parameter with syntax like -d @filename.txt. 
  1. curl -vi -X PUT \
  2.   -H "Accept: text/xml" \
  3.   -H "Content-Type: text/xml" \
  4.   -d '<?xml version="1.0" encoding="UTF-8" standalone="yes"?><CellSet><Row key="cm93NQo="><Cell column="Y2Y6ZQo=">dmFsdWU1Cg==</Cell></Row></CellSet>' \
  5.   "http://example.com:8000/users/fakerow"
  7. curl -vi -X PUT \
  8.   -H "Accept: text/json" \
  9.   -H "Content-Type: text/json" \
  10.   -d '{"Row":[{"key":"cm93NQo=", "Cell": [{"column":"Y2Y6ZQo=", "$":"dmFsdWU1Cg=="}]}]}'' \
  11.   "example.com:8000/users/fakerow"

|

97.4. REST XML Schema

  1. <schema xmlns="http://www.w3.org/2001/XMLSchema" xmlns:tns="RESTSchema">
  3.   <element name="Version" type="tns:Version"></element>
  5.   <complexType name="Version">
  6.     <attribute name="REST" type="string"></attribute>
  7.     <attribute name="JVM" type="string"></attribute>
  8.     <attribute name="OS" type="string"></attribute>
  9.     <attribute name="Server" type="string"></attribute>
  10.     <attribute name="Jersey" type="string"></attribute>
  11.   </complexType>
  13.   <element name="TableList" type="tns:TableList"></element>
  15.   <complexType name="TableList">
  16.     <sequence>
  17.       <element name="table" type="tns:Table" maxOccurs="unbounded" minOccurs="1"></element>
  18.     </sequence>
  19.   </complexType>
  21.   <complexType name="Table">
  22.     <sequence>
  23.       <element name="name" type="string"></element>
  24.     </sequence>
  25.   </complexType>
  27.   <element name="TableInfo" type="tns:TableInfo"></element>
  29.   <complexType name="TableInfo">
  30.     <sequence>
  31.       <element name="region" type="tns:TableRegion" maxOccurs="unbounded" minOccurs="1"></element>
  32.     </sequence>
  33.     <attribute name="name" type="string"></attribute>
  34.   </complexType>
  36.   <complexType name="TableRegion">
  37.     <attribute name="name" type="string"></attribute>
  38.     <attribute name="id" type="int"></attribute>
  39.     <attribute name="startKey" type="base64Binary"></attribute>
  40.     <attribute name="endKey" type="base64Binary"></attribute>
  41.     <attribute name="location" type="string"></attribute>
  42.   </complexType>
  44.   <element name="TableSchema" type="tns:TableSchema"></element>
  46.   <complexType name="TableSchema">
  47.     <sequence>
  48.       <element name="column" type="tns:ColumnSchema" maxOccurs="unbounded" minOccurs="1"></element>
  49.     </sequence>
  50.     <attribute name="name" type="string"></attribute>
  51.     <anyAttribute></anyAttribute>
  52.   </complexType>
  54.   <complexType name="ColumnSchema">
  55.     <attribute name="name" type="string"></attribute>
  56.     <anyAttribute></anyAttribute>
  57.   </complexType>
  59.   <element name="CellSet" type="tns:CellSet"></element>
  61.   <complexType name="CellSet">
  62.     <sequence>
  63.       <element name="row" type="tns:Row" maxOccurs="unbounded" minOccurs="1"></element>
  64.     </sequence>
  65.   </complexType>
  67.   <element name="Row" type="tns:Row"></element>
  69.   <complexType name="Row">
  70.     <sequence>
  71.       <element name="key" type="base64Binary"></element>
  72.       <element name="cell" type="tns:Cell" maxOccurs="unbounded" minOccurs="1"></element>
  73.     </sequence>
  74.   </complexType>
  76.   <element name="Cell" type="tns:Cell"></element>
  78.   <complexType name="Cell">
  79.     <sequence>
  80.       <element name="value" maxOccurs="1" minOccurs="1">
  81.         <simpleType><restriction base="base64Binary">
  82.         </simpleType>
  83.       </element>
  84.     </sequence>
  85.     <attribute name="column" type="base64Binary" />
  86.     <attribute name="timestamp" type="int" />
  87.   </complexType>
  89.   <element name="Scanner" type="tns:Scanner"></element>
  91.   <complexType name="Scanner">
  92.     <sequence>
  93.       <element name="column" type="base64Binary" minOccurs="0" maxOccurs="unbounded"></element>
  94.     </sequence>
  95.     <sequence>
  96.       <element name="filter" type="string" minOccurs="0" maxOccurs="1"></element>
  97.     </sequence>
  98.     <attribute name="startRow" type="base64Binary"></attribute>
  99.     <attribute name="endRow" type="base64Binary"></attribute>
  100.     <attribute name="batch" type="int"></attribute>
  101.     <attribute name="startTime" type="int"></attribute>
  102.     <attribute name="endTime" type="int"></attribute>
  103.   </complexType>
  105.   <element name="StorageClusterVersion" type="tns:StorageClusterVersion" />
  107.   <complexType name="StorageClusterVersion">
  108.     <attribute name="version" type="string"></attribute>
  109.   </complexType>
  111.   <element name="StorageClusterStatus"
  112.     type="tns:StorageClusterStatus">
  113.   </element>
  115.   <complexType name="StorageClusterStatus">
  116.     <sequence>
  117.       <element name="liveNode" type="tns:Node"
  118.         maxOccurs="unbounded" minOccurs="0">
  119.       </element>
  120.       <element name="deadNode" type="string" maxOccurs="unbounded"
  121.         minOccurs="0">
  122.       </element>
  123.     </sequence>
  124.     <attribute name="regions" type="int"></attribute>
  125.     <attribute name="requests" type="int"></attribute>
  126.     <attribute name="averageLoad" type="float"></attribute>
  127.   </complexType>
  129.   <complexType name="Node">
  130.     <sequence>
  131.       <element name="region" type="tns:Region"
  132.    maxOccurs="unbounded" minOccurs="0">
  133.       </element>
  134.     </sequence>
  135.     <attribute name="name" type="string"></attribute>
  136.     <attribute name="startCode" type="int"></attribute>
  137.     <attribute name="requests" type="int"></attribute>
  138.     <attribute name="heapSizeMB" type="int"></attribute>
  139.     <attribute name="maxHeapSizeMB" type="int"></attribute>
  140.   </complexType>
  142.   <complexType name="Region">
  143.     <attribute name="name" type="base64Binary"></attribute>
  144.     <attribute name="stores" type="int"></attribute>
  145.     <attribute name="storefiles" type="int"></attribute>
  146.     <attribute name="storefileSizeMB" type="int"></attribute>
  147.     <attribute name="memstoreSizeMB" type="int"></attribute>
  148.     <attribute name="storefileIndexSizeMB" type="int"></attribute>
  149.   </complexType>
  151. </schema>

97.5. REST Protobufs Schema

  1. message Version {
  2.   optional string restVersion = 1;
  3.   optional string jvmVersion = 2;
  4.   optional string osVersion = 3;
  5.   optional string serverVersion = 4;
  6.   optional string jerseyVersion = 5;
  7. }
  9. message StorageClusterStatus {
  10.   message Region {
  11.     required bytes name = 1;
  12.     optional int32 stores = 2;
  13.     optional int32 storefiles = 3;
  14.     optional int32 storefileSizeMB = 4;
  15.     optional int32 memstoreSizeMB = 5;
  16.     optional int32 storefileIndexSizeMB = 6;
  17.   }
  18.   message Node {
  19.     required string name = 1;    // name:port
  20.     optional int64 startCode = 2;
  21.     optional int32 requests = 3;
  22.     optional int32 heapSizeMB = 4;
  23.     optional int32 maxHeapSizeMB = 5;
  24.     repeated Region regions = 6;
  25.   }
  26.   // node status
  27.   repeated Node liveNodes = 1;
  28.   repeated string deadNodes = 2;
  29.   // summary statistics
  30.   optional int32 regions = 3;
  31.   optional int32 requests = 4;
  32.   optional double averageLoad = 5;
  33. }
  35. message TableList {
  36.   repeated string name = 1;
  37. }
  39. message TableInfo {
  40.   required string name = 1;
  41.   message Region {
  42.     required string name = 1;
  43.     optional bytes startKey = 2;
  44.     optional bytes endKey = 3;
  45.     optional int64 id = 4;
  46.     optional string location = 5;
  47.   }
  48.   repeated Region regions = 2;
  49. }
  51. message TableSchema {
  52.   optional string name = 1;
  53.   message Attribute {
  54.     required string name = 1;
  55.     required string value = 2;
  56.   }
  57.   repeated Attribute attrs = 2;
  58.   repeated ColumnSchema columns = 3;
  59.   // optional helpful encodings of commonly used attributes
  60.   optional bool inMemory = 4;
  61.   optional bool readOnly = 5;
  62. }
  64. message ColumnSchema {
  65.   optional string name = 1;
  66.   message Attribute {
  67.     required string name = 1;
  68.     required string value = 2;
  69.   }
  70.   repeated Attribute attrs = 2;
  71.   // optional helpful encodings of commonly used attributes
  72.   optional int32 ttl = 3;
  73.   optional int32 maxVersions = 4;
  74.   optional string compression = 5;
  75. }
  77. message Cell {
  78.   optional bytes row = 1;       // unused if Cell is in a CellSet
  79.   optional bytes column = 2;
  80.   optional int64 timestamp = 3;
  81.   optional bytes data = 4;
  82. }
  84. message CellSet {
  85.   message Row {
  86.     required bytes key = 1;
  87.     repeated Cell values = 2;
  88.   }
  89.   repeated Row rows = 1;
  90. }
  92. message Scanner {
  93.   optional bytes startRow = 1;
  94.   optional bytes endRow = 2;
  95.   repeated bytes columns = 3;
  96.   optional int32 batch = 4;
  97.   optional int64 startTime = 5;
  98.   optional int64 endTime = 6;
  99. }

98. Thrift

Documentation about Thrift has moved to Thrift API and Filter Language.

99. C/C++ Apache HBase Client

FB’s Chip Turner wrote a pure C/C++ client. Check it out.

C++ client implementation. To see HBASE-14850.

100. Using Java Data Objects (JDO) with HBase

Java Data Objects (JDO) is a standard way to access persistent data in databases, using plain old Java objects (POJO) to represent persistent data.

Dependencies

This code example has the following dependencies:

  1. HBase 0.90.x or newer

  2. commons-beanutils.jar (https://commons.apache.org/)

  3. commons-pool-1.5.5.jar (https://commons.apache.org/)

  4. transactional-tableindexed for HBase 0.90 (https://github.com/hbase-trx/hbase-transactional-tableindexed)

Download hbase-jdo

Download the code from http://code.google.com/p/hbase-jdo/.

Example 26. JDO Example

This example uses JDO to create a table and an index, insert a row into a table, get a row, get a column value, perform a query, and do some additional HBase operations.

  1. package com.apache.hadoop.hbase.client.jdo.examples;
  3. import java.io.File;
  4. import java.io.FileInputStream;
  5. import java.io.InputStream;
  6. import java.util.Hashtable;
  8. import org.apache.hadoop.fs.Path;
  9. import org.apache.hadoop.hbase.client.tableindexed.IndexedTable;
  11. import com.apache.hadoop.hbase.client.jdo.AbstractHBaseDBO;
  12. import com.apache.hadoop.hbase.client.jdo.HBaseBigFile;
  13. import com.apache.hadoop.hbase.client.jdo.HBaseDBOImpl;
  14. import com.apache.hadoop.hbase.client.jdo.query.DeleteQuery;
  15. import com.apache.hadoop.hbase.client.jdo.query.HBaseOrder;
  16. import com.apache.hadoop.hbase.client.jdo.query.HBaseParam;
  17. import com.apache.hadoop.hbase.client.jdo.query.InsertQuery;
  18. import com.apache.hadoop.hbase.client.jdo.query.QSearch;
  19. import com.apache.hadoop.hbase.client.jdo.query.SelectQuery;
  20. import com.apache.hadoop.hbase.client.jdo.query.UpdateQuery;
  22. /**
  23.  * Hbase JDO Example.
  24.  *
  25.  * dependency library.
  26.  * - commons-beanutils.jar
  27.  * - commons-pool-1.5.5.jar
  28.  * - hbase0.90.0-transactionl.jar
  29.  *
  30.  * you can expand Delete,Select,Update,Insert Query classes.
  31.  *
  32.  */
  33. public class HBaseExample {
  34.   public static void main(String[] args) throws Exception {
  35.     AbstractHBaseDBO dbo = new HBaseDBOImpl();
  37.     //*drop if table is already exist.*
  38.     if(dbo.isTableExist("user")){
  39.      dbo.deleteTable("user");
  40.     }
  42.     //*create table*
  43.     dbo.createTableIfNotExist("user",HBaseOrder.DESC,"account");
  44.     //dbo.createTableIfNotExist("user",HBaseOrder.ASC,"account");
  46.     //create index.
  47.     String[] cols={"id","name"};
  48.     dbo.addIndexExistingTable("user","account",cols);
  50.     //insert
  51.     InsertQuery insert = dbo.createInsertQuery("user");
  52.     UserBean bean = new UserBean();
  53.     bean.setFamily("account");
  54.     bean.setAge(20);
  55.     bean.setEmail("ncanis@gmail.com");
  56.     bean.setId("ncanis");
  57.     bean.setName("ncanis");
  58.     bean.setPassword("1111");
  59.     insert.insert(bean);
  61.     //select 1 row
  62.     SelectQuery select = dbo.createSelectQuery("user");
  63.     UserBean resultBean = (UserBean)select.select(bean.getRow(),UserBean.class);
  65.     // select column value.
  66.     String value = (String)select.selectColumn(bean.getRow(),"account","id",String.class);
  68.     // search with option (QSearch has EQUAL, NOT_EQUAL, LIKE)
  69.     // select id,password,name,email from account where id='ncanis' limit startRow,20
  70.     HBaseParam param = new HBaseParam();
  71.     param.setPage(bean.getRow(),20);
  72.     param.addColumn("id","password","name","email");
  73.     param.addSearchOption("id","ncanis",QSearch.EQUAL);
  74.     select.search("account", param, UserBean.class);
  76.     // search column value is existing.
  77.     boolean isExist = select.existColumnValue("account","id","ncanis".getBytes());
  79.     // update password.
  80.     UpdateQuery update = dbo.createUpdateQuery("user");
  81.     Hashtable<String, byte[]> colsTable = new Hashtable<String, byte[]>();
  82.     colsTable.put("password","2222".getBytes());
  83.     update.update(bean.getRow(),"account",colsTable);
  85.     //delete
  86.     DeleteQuery delete = dbo.createDeleteQuery("user");
  87.     delete.deleteRow(resultBean.getRow());
  89.     ////////////////////////////////////
  90.     // etc
  92.     // HTable pool with apache commons pool
  93.     // borrow and release. HBasePoolManager(maxActive, minIdle etc..)
  94.     IndexedTable table = dbo.getPool().borrow("user");
  95.     dbo.getPool().release(table);
  97.     // upload bigFile by hadoop directly.
  98.     HBaseBigFile bigFile = new HBaseBigFile();
  99.     File file = new File("doc/movie.avi");
  100.     FileInputStream fis = new FileInputStream(file);
  101.     Path rootPath = new Path("/files/");
  102.     String filename = "movie.avi";
  103.     bigFile.uploadFile(rootPath,filename,fis,true);
  105.     // receive file stream from hadoop.
  106.     Path p = new Path(rootPath,filename);
  107.     InputStream is = bigFile.path2Stream(p,4096);
  109.   }
  110. }

101. Scala

101.1. Setting the Classpath

To use Scala with HBase, your CLASSPATH must include HBase’s classpath as well as the Scala JARs required by your code. First, use the following command on a server running the HBase RegionServer process, to get HBase’s classpath.

  1. $ ps aux |grep regionserver| awk -F 'java.library.path=' {'print $2'} | awk {'print $1'}
  3. /usr/lib/hadoop/lib/native:/usr/lib/hbase/lib/native/Linux-amd64-64

Set the $CLASSPATH environment variable to include the path you found in the previous step, plus the path of scala-library.jar and each additional Scala-related JAR needed for your project.

  1. $ export CLASSPATH=$CLASSPATH:/usr/lib/hadoop/lib/native:/usr/lib/hbase/lib/native/Linux-amd64-64:/path/to/scala-library.jar

101.2. Scala SBT File

Your build.sbt file needs the following resolvers and libraryDependencies to work with HBase.

  1. resolvers += "Apache HBase" at "https://repository.apache.org/content/repositories/releases"
  3. resolvers += "Thrift" at "https://people.apache.org/~rawson/repo/"
  5. libraryDependencies ++= Seq(
  6.     "org.apache.hadoop" % "hadoop-core" % "0.20.2",
  7.     "org.apache.hbase" % "hbase" % "0.90.4"
  8. )

101.3. Example Scala Code

This example lists HBase tables, creates a new table, and adds a row to it.

  1. import org.apache.hadoop.hbase.HBaseConfiguration
  2. import org.apache.hadoop.hbase.client.{Connection,ConnectionFactory,HBaseAdmin,HTable,Put,Get}
  3. import org.apache.hadoop.hbase.util.Bytes
  5. val conf = new HBaseConfiguration()
  6. val connection = ConnectionFactory.createConnection(conf);
  7. val admin = connection.getAdmin();
  9. // list the tables
  10. val listtables=admin.listTables()
  11. listtables.foreach(println)
  13. // let's insert some data in 'mytable' and get the row
  15. val table = new HTable(conf, "mytable")
  17. val theput= new Put(Bytes.toBytes("rowkey1"))
  19. theput.add(Bytes.toBytes("ids"),Bytes.toBytes("id1"),Bytes.toBytes("one"))
  20. table.put(theput)
  22. val theget= new Get(Bytes.toBytes("rowkey1"))
  23. val result=table.get(theget)
  24. val value=result.value()
  25. println(Bytes.toString(value))

102. Jython

102.1. Setting the Classpath

To use Jython with HBase, your CLASSPATH must include HBase’s classpath as well as the Jython JARs required by your code.

Set the path to directory containing the jython.jar and each additional Jython-related JAR needed for your project. Then export HBASE_CLASSPATH pointing to the $JYTHON_HOME env. variable.

  1. $ export HBASE_CLASSPATH=/directory/jython.jar

Start a Jython shell with HBase and Hadoop JARs in the classpath: $ bin/hbase org.python.util.jython

102.2. Jython Code Examples

Example 27. Table Creation, Population, Get, and Delete with Jython

The following Jython code example checks for table, if it exists, deletes it and then creates it. Then it populates the table with data and fetches the data.

  1. import java.lang
  2. from org.apache.hadoop.hbase import HBaseConfiguration, HTableDescriptor, HColumnDescriptor, TableName
  3. from org.apache.hadoop.hbase.client import Admin, Connection, ConnectionFactory, Get, Put, Result, Table
  4. from org.apache.hadoop.conf import Configuration
  6. # First get a conf object.  This will read in the configuration
  7. # that is out in your hbase-*.xml files such as location of the
  8. # hbase master node.
  9. conf = HBaseConfiguration.create()
  10. connection = ConnectionFactory.createConnection(conf)
  11. admin = connection.getAdmin()
  13. # Create a table named 'test' that has a column family
  14. # named 'content'.
  15. tableName = TableName.valueOf("test")
  16. table = connection.getTable(tableName)
  18. desc = HTableDescriptor(tableName)
  19. desc.addFamily(HColumnDescriptor("content"))
  21. # Drop and recreate if it exists
  22. if admin.tableExists(tableName):
  23.     admin.disableTable(tableName)
  24.     admin.deleteTable(tableName)
  26. admin.createTable(desc)
  28. # Add content to 'column:' on a row named 'row_x'
  29. row = 'row_x'
  30. put = Put(row)
  31. put.addColumn("content", "qual", "some content")
  32. table.put(put)
  34. # Now fetch the content just added, returns a byte[]
  35. get = Get(row)
  37. result = table.get(get)
  38. data = java.lang.String(result.getValue("content", "qual"), "UTF8")
  40. print "The fetched row contains the value '%s'" % data

Example 28. Table Scan Using Jython

This example scans a table and returns the results that match a given family qualifier.

  1. import java.lang
  2. from org.apache.hadoop.hbase import TableName, HBaseConfiguration
  3. from org.apache.hadoop.hbase.client import Connection, ConnectionFactory, Result, ResultScanner, Table, Admin
  4. from org.apache.hadoop.conf import Configuration
  5. conf = HBaseConfiguration.create()
  6. connection = ConnectionFactory.createConnection(conf)
  7. admin = connection.getAdmin()
  8. tableName = TableName.valueOf('wiki')
  9. table = connection.getTable(tableName)
  11. cf = "title"
  12. attr = "attr"
  13. scanner = table.getScanner(cf)
  14. while 1:
  15.     result = scanner.next()
  16.     if not result:
  17.        break
  18.     print java.lang.String(result.row), java.lang.String(result.getValue(cf, attr))