jishan-shk/laravel-databricks

Databricks SQL client for Laravel over ODBC (Simba Spark / Databricks ODBC driver).

Compatibility

Required PHP extensions: ext-odbc, ext-pcntl. You also need the Databricks/Simba Spark ODBC driver installed on the host. Download it from Databricks: https://www.databricks.com/spark/odbc-drivers-download

Installation

composer require jishan-shk/laravel-databricks

The service provider is auto-discovered. Without package discovery, add JishanShk\Databricks\DatabricksServiceProvider::class to your providers.

Configuration

The client reads config('services.databricks'). The package fills in defaults from .env, and any databricks keys you define in config/services.php override them.

DATABRICKS_ODBC_DRIVER="Simba Spark ODBC Driver"
DATABRICKS_HOST=adb-xxxx.azuredatabricks.net
DATABRICKS_PORT=443
DATABRICKS_HTTP_PATH=/sql/1.0/warehouses/xxxx
DATABRICKS_TOKEN=dapi...
DATABRICKS_CATALOG=main
DATABRICKS_SCHEMA=default
DATABRICKS_CONNECT_TIMEOUT=20
DATABRICKS_WARMUP_QUEUE=databricks-warmup

Usage

use JishanShk\Databricks\DatabricksClient;
use JishanShk\Databricks\Support\DatabricksSql;

$rows = app(DatabricksClient::class)->select(
    'SELECT * FROM reviews WHERE created_at >= '.DatabricksSql::quoteDate('2024-01-01')
);

$affected = app(DatabricksClient::class)->statement('DELETE FROM tmp_table');

All values come back as strings (ODBC contract), so cast them where you need to.

Warming up the warehouse

use JishanShk\Databricks\Jobs\DatabricksWarmupJob;

DatabricksWarmupJob::dispatch();

The job is dispatched on DATABRICKS_WARMUP_QUEUE when that is set. Failures go to your app's save_error_log() helper if it exists, and to report() otherwise.

Testing

Bind a fake JishanShk\Databricks\OdbcDriver in the container (see tests/FakeOdbcDriver.php):

$this->app->instance(\JishanShk\Databricks\OdbcDriver::class, new MyFakeDriver);

License

MIT