Custom Driver Settings

This week we’re adding custom driver settings to Ajilius.

While our driver settings are chosen for optimal performance, there have been times when customers have asked for the ability to tailor connections to specific requirements.

We’re now supporting that request through two settings:

  • Connection Parameters. These are extensions to the connection string used by JDBC drivers. Specify connection string parameters in a single line, including delimiters after every parameter.
  • Driver Properties. These are JDBC properties which modify the behaviour of drivers through an API call instead of the connection string. Specify properties as a combination of property-value pairs, with one property per row.

 

The above example shows a modification to the connection string for the Salesforce driver, which adjusts the Salesforce API to version 36, and reduces the timeout setting to 30 seconds.

Please use custom driver settings with extreme care, as invalid settings could cause your ELT jobs to fail. We recommend you check with us before making changes regarding the specific format requirements of each driver.

Ajilius. Customisable automation.

 

Ajilius 2.4.10

This week we’ve got a mix of incremental enhancements and error corrections.
Here’s what you’ll find in Ajilius 2.4.10:
  • Enhancements:
    • Salesforce Sandbox. Ajilius now supports sandbox accounts for Salesforce.
    • Tableau Connections. Tableau connections have now been validated against SQL Server, PostgreSQL, Snowflake, Redshift, Azure SQL DW, Exasol and MariaDB ColumnStore.
    • Dedup Calculations. We now support the use of Transforms and Calculations in Dedup transformations.
    • Redshift Sort Keys. If you’re a Redshift user, you now have interleaved sortkey support to improve performance of queries against facts and dimensions.
    • Column Sorting. The Warehouse List and Extract List forms now support column sorting, but clicking the column name. We’ll roll out more sortable lists in future releases.
    • BI Lineage. We now show the lineage of a column on the tooltips in Tableau and Yellowfin.
    • Table Page Numbering. Table page numbers now show at the bottom of lists, to show your position in the list.
  • Error Corrections
    • Better Locale Support. We’ve made some adjustments that give better support for locale-specific number and date formatting.
    • Coalesce Empty. The Coalesce Empty transform, which replaces null values with an empty string, was not working correctly on Snowflake.

Registered users should already have received notification of the download location, please let us know if you’ve not received your email.

 

Better BI Lineage

Ajilius generates metadata-based models for Yellowfin, Tableau, Qlik and PowerBI. We use the descriptions and notes that you maintain in Ajilius to generate business-friendly models and documentation.

The quality of the model depends on the capabilities of the BI product. Yellowfin and Tableau have a  richer metadata capability than Qlik and PowerBI.

We’ve taken advantage of that metadata capability to add a great new feature to the models for Yellowfin and Tableau – data lineage. Note the last two lines of the highlighted tooltips in the following screenshots:

Not only do we present the documentation you provided for the column, but did you see the last two lines? We track right back through the data warehouse lineage to give you the data sources, tables and columns from which each dimension or measure was sourced.

Now there can be no arguments about the source of data in reports, it is right in the model so that users know exactly where each number and value in their report originated.

Ajilius. Better BI Governance.

Ajilius 2.4.7

This week’s release 2.4.7 brings the following new features:

  • Salesforce Adapter. Connect to Salesforce, browse and profile your data, then quickly load it to your choice of data warehouse for transformation to facts and dimensions.
  • Double-Bar delimiter. Delimited files now accept a double bar (||) as a column delimiter.

The following errors have been corrected:

  • Historic Persistent Staging. CTAS versions were not correctly updating the current row flag for this variant of persistent staging.
  • Blank lines in log. Occasional blank lines were being appended to the console, and have now been suppressed.

Registered users will receive an email over the next 1-2 days with download instructions.

Go build some data warehouses 🙂

 

Cloud Evaluations

We’re pleased to announce that the Ajilius evaluation version now includes all cloud data warehouse platforms.

Previously, we restricted evaluations to on-premise databases. This was to simplify the evaluation environment.

Now, we’re finding that more and more customers are moving their entire workload to the cloud, and there is no on-premise database. And as those workloads move, the knowledge sets of evaluators move with them.

In recognition of this shift, we now include all databases – both cloud and on-premise – in our evaluation version. You can now trial Snowflake, Redshift and Azure SQL Data Warehouse alongside favourites like SQL Server, PostgreSQL, MariaDB and Exasol.

Ajilius. Tomorrow’s data warehouse, today.

 

#CloudCred

I saw a great tweet from Snowflake today:

I laughed when I read it, it exactly mirrors the way I feel about Oracle, IBM and Teradata (among others) with their cloudwashing of legacy DW platforms.

We’re cloud-first. Cross-cloud. Multi-cloud. Even hybrid-cloud. We’ve been working with cloud data warehouses since they were born.

Ajilius. CLOUD data warehousing.

Release 2.4.2

This week’s release makes the following changes:

  • New data type “char” for single character columns. Used internally for SCD_FROM column in type-2 slowly changing dimensions.
  • New TRUNCATE option for the command line scheduler, cleans up transient data after loads to minimise storage costs on cloud platforms.
  • PostgreSQL interface on Windows was not always being fed UTF-8 encoded data. Added encoding parameter to byte conversion.
  • Deleting the active data warehouse repository on Windows would sometimes fail. Now triggering call to garbage collector to ensure closed connections are purged from memory.

Registered users will receive an email in the next 1-2 days with download instructions.

Interestingly, the last two items only showed up when we shifted our development platforms from OSX to Windows. Really liking the change – like moving back home after living on the road for a while – but it does show up the quirks of the Windows internals compared to OSX and Linux.

The list is lighter than usual because we’re working hard on new features for Version 3 … stay tuned for updates!

 

Why Data Warehouse Automation kicks the crap out of SSIS

Here’s a great explanation of how to do incremental loads in SSIS. It uses third-party components from Pragmatic Works, and a combination of hand coding and SSIS tasks.

http://blog.pragmaticworks.com/incrementally-loading-data-from-salesforce.com

See how complex it is? How long it takes?

Here’s how to do the same thing in Ajilius:

Simply check the box on the column/s that govern incremental change.

Under the covers we do almost exactly the same tasks as described in the blog post, but the difference is that we do them, not you. You make a decision in the UI, and we instantly generate all the code and logic that implements that decision.

That’s it … a perfect example of why Data Warehouse Automation kicks the crap out of  great ETL tools like SSIS, and why we deliver data warehouses in a fraction of the time spent in those products.

Ajilius. Data warehouses. Faster.