Skip to content

Spark SQL

The Spark SQL data provider allows ChartFactor to interact with Spark SQL.

1
<script src="./CFT-sparksql-provider.min.js"></script>

The Provider JSON object requires the url parameter in addition to name and provider parameters. Example:

1
2
3
4
5
6
// define providers
var providers = [{
    name:'Spark SQL',
    provider:'sparksql',
    url:'https://34.233.78.155'
}]

Then, use the setProviders() method of ChartFactor to set your data provider definitions. Example:

1
cf.setProviders(providers);

Additionally, custom headers can be passed to the spark configuration. By default it uses the following:

1
2
3
4
{
  'Content-Type': 'application/json',
  'Accept': 'application/json'
}

So, if we need to add extra headers, we should also provide the above configuration. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
var providers = [{
    ...
    headers: {
      'Authorization': 'Bearer ' + authToken,
      'Content-Type': 'application/json',
      'Accept': 'application/json'
    }
}]

cf.setProviders(providers)

This data provider assumes your Spark SQL server is fronted with an HTTP REST server with the following operations:

  • GET /tables: Returns the list of tables

  • GET /tables/{id}: Returns all fields and their types for the table specified by the {id} parameter

  • POST /query: Executes a SQL query and returns the results in JSON format

Please contact the Aktiun team if you would like more details and available options to provide a REST front to your Spark SQL server.

Supported Aggregations Out-Of-The-Box

SUM

1
    var metric = cf.Metric("amount","sum");

AVG

1
    var metric = cf.Metric("amount","avg");

MIN

1
    var metric = cf.Metric("amount","min");

MAX

1
    var metric = cf.Metric("amount","max");

PERCENTILES

1
    var metric = cf.Metric('commission', 'percentiles');

COUNT DISTINCT

1
    var metric = cf.Metric("my_attribute","unique");

Dependencies

  • None