Hexspace

Ruby client for Apache Spark SQL and Apache Hive

Installation

Add this line to your application’s Gemfile:

gem "hexspace"

Getting Started

Create a client

client = Hexspace::Client.new

Execute queries

client.execute("SELECT COUNT(*) FROM users")

Connection Options

There are a number of connection options available.

Hexspace::Client.new(
  host: "localhost",
  port: 10000,
  username: "user",
  password: "secret",
  database: "default",
  mode: :sasl,
  timeout: 10
)

Supported modes are :sasl, :nosasl, :http, and :https. Please create an issue if you need Kerberos.

The timeout is in seconds and only applies to :sasl and :nosasl.

Query Options

Set a timeout

client.execute(statement, timeout: 10)

Get a Hexspace::Result object instead of an array of hashes

result = client.execute(statement, result_object: true)
result.rows
result.columns
result.column_types
result.to_a

Spark SQL Setup

Download Apache Spark and start the Thift server.

./sbin/start-thriftserver.sh

Hive Setup

Download Apache Hive and initialize the schema.

./bin/schematool -dbType derby -initSchema

Then start the HiveServer2.

./bin/hiveserver2

It can take a minute or two to be ready. To debug, pass --hiveconf hive.root.logger=DEBUG,console.

History

View the changelog

Contributing

Everyone is encouraged to help improve this project. Here are a few ways you can help:

• Report bugs
• Fix bugs and submit pull requests
• Write, clarify, or fix documentation
• Suggest or add new features

To get started with development:

git clone https://github.com/ankane/hexspace.git
cd hexspace
bundle install

# create a database
beeline -u jdbc:hive2://localhost:10000 -e 'CREATE DATABASE hexspace_test;'

# run the tests
bundle exec rake test

Resources

• Spark SQL Reference
• Hive Language Manual
• Thrift SASL Spec