What’s New in Centerprise 7.5

New additions, enhanced performance, and improved usability – Centerprise 7.5 is better than ever!

Centerprise Data Integrator is an end-to-end ETL and data integration solution that empowers business users to perform complex data integration tasks without relying on IT. With the code-free, drag-and-drop IDE of Centerprise, we aim to flatten the learning curve for business users and make self-service data integration possible for them.

Here is everything that we’ve improved and added in Centerprise 7.5 to make it even better and a feature-rich data integration solution.

Improved server resiliency

We have greatly improved the server resiliency in Centerprise 7.5. This will ensure trouble-free server operations, better and quick recovery in the case of a database outage event, and overall improvement in the performance.

Here are some of the benefits that the users will get with this improvement:

  • Astera integration server will no longer go in a permanent state of error in case of a database outage event and will automatically recover as soon as the connection is restored.
  • Server connection will be automatically established when the connection to the repository database is available and no manual restarts will be required.
  • Activity and error tracking are greatly improved in the new release. The server writes database connection issues in the Windows event log and includes a link to the error file for easier troubleshooting.

New transformations added to the library

The transformation library in Centerprise Data Integrator is extensive and encompasses a wide range of data integration scenarios. However, we strive on making the ETL and data integration process easier for our users and thus we have included new transformations in Centerprise 7.5. These new additions are:

Switch transformation

Switch data representation to get accurate business insights

There are more than one ways of representing the same information. It is important to have the information presented in the correct representation to drive accurate business insights. Switch transformation is used to replace the data in the source file with any other representation that is more relevant to the present business case.

Learn more about how Switch transformation works in Centerprise and how it can be applied in a business scenario.

Reconcile transformation

Reconcile information for self-service data analysis

Reconcile transformation helps in reconciling information in the source data. Application of the transformation can be seen in a wide range of business use cases where merely storing data is not sufficient and monitoring the changes in data over time is essential to drive critical business decisions.

Some of the benefits of the Reconcile transformation are:

  • Can be applied to flat or hierarchical source data.
  • The transformation can be applied to inner nodes as well when the source data is in a hierarchical format.
  • Business users can perform data analysis by comparing old and new values for specific fields in the source data.
  • Provides options to view data in multiple layouts so users can choose the layout which is most in line with business requirements.

Read more about how Reconcile transformation works in Centerprise here.

Merge transformation

Merge data from disparate sources for a consolidated and unified view

As the data collection avenues are increasing, it has become a common practice for businesses to integrate data from multiple disparate sources. In some cases, the same data is contained in different sources and businesses face the need to merge the fragmented data for a consolidated view. Merge transformation in Centerprise helps in achieving this task.

What’s Next?

We have a number of new features lined up for the upcoming releases and we are excited to share the updates with you. Subscribe to our blog or join our community of data professionals to stay informed. To see Centerprise in action, schedule a discovery call with us today!

Switch Transformation in Centerprise 7.5

Overview

Switch transformation is among the newly introduced transformations that are included in Centerprise 7.5 version. It is a basic, yet an extremely useful transformation and has applications in a wide range of business use cases.

This transformation can be applied when there is a need to replace information stored in the source data with some other relevant information. Information is switched (or replaced) based on some preassigned rules.

Switch Transformation Explained with a Business Use Case

An organization maintains a database for their employees. Employee_Info table in that database contains information such as employee’s name, department, salary and employee ID.

Figure 1: Source file and data preview before the switch transformation is applied

For the ease of maintenance, the organization assigned a specific number to each department (Marketing, Sales, Finance etc.) and that information is recorded in the Department field as shown in figure 1.

Using the Switch transformation, the numeric values stored in the Department field can be easily replaced with their corresponding department names.

Basic Mode in Switch Transformation

The basic mode in Switch transformation works like a List Lookup. In this example, we want to replace the numeric information stored in the Department field with their actual corresponding departments.

Figure 2: Switch Transformation (Top: Basic Mode – Bottom: Enhanced Mode)

Drag-and-drop the transformation from the toolbox to the designer and map the required fields. Here we are mapping EmployeeID, Department, and Salary fields from the source file to the Switch transformation (BasicMode_Switch).

In the properties window of the transformation, select the field that you want to perform the switch operation on, and specify the criteria in Case Value, based on which Centerprise will look for the information and replace it with the given information in the corresponding Output Value.

Figure 3: Switch transformation properties window (Basic mode)

Now if you preview the output or write the transformed data to any destination, you will see that the numbers in the Department field are replaced with the information that is based on the rules assigned in the properties window shown in figure 3. In a case where Centerprise won’t find any matching case value, it will automatically assign the specified default output value, which is Admin in this example.

Figure 4: Data preview after the Switch transformation is applied

Enhanced Mode in Switch Transformation

The Enhanced mode in the Switch transformation provides more flexibility to the feature because it enables users to set lookup criteria by writing expressions. Figure 5 shows the properties window of the enhanced mode in Switch transformation.

Figure 5: Switch transformation properties window (Enhanced Mode)

Suppose our fictitious organization has set criteria for issuing credit cards. Employees whose salary is less than $3,500 are not eligible for the credit card. Employees with a salary between $3,500 and $5,000 are eligible for the Silver card. Other similar assignments can be seen in the Case Expression column in figure 5.

The organization has the data about employee’s salaries and the ranges for the issuance of credit cards. In this case, employee’s salary, which is the information that is stored in the database, is not relevant to the concerned department. What they’re looking for is to identify which employee is eligible for which perk. This can easily be achieved by matching the salary ranges defined in the Case Expression and switching it with the corresponding information about the credit cards specified in the Output Expression.

Figure 6 shows the output preview for the enhanced mode case in the Switch transformation.

Figure 6: Output preview of switch transformation in the enhanced mode

Benefits of the Switch Transformation

Switch transformation gives users more control over data as it enables them to transform the given information into something more relevant, which is best suited to the given business scenario. When applied to any data set, switch transformation matches source data for the criteria specified by the user, and replaces the information with the desired output (also specified in the layout) wherever the criteria is met.

Server Resiliency Improved in Centerprise 7.5

We are releasing Centerprise Data Integrator 7.5 very soon, and as with every new release, our focus is to improve the product experience for our customers. We have added new features and have made improvements to the existing ones in the upcoming build.

One of the key changes we have made in the 7.5 version is greatly improving the server resilience. This will ensure trouble-free server operations, even when connection issues occur between Astera server and the MSSQL server hosting the repository databases. Enhanced server resilience will not only improve the overall performance of Centerprise, but will also prove to be significantly beneficial in the scenarios where 24/7 operation with high uptime is required, which is the case with most of our enterprise customers.

Key Benefits of Improved Server Resilience

With improved server resilience, users will get the following benefits:

Better and quick recovery of the server

With improved server resilience, Astera server will no longer enter a permanent error state after a database outage event. Instead, it will recover as soon as the connection is restored, allowing the customer to continue with the normal operations without any downtime. The server can now survive most repository database outages without any user action required.

Auto-recovery mode – no manual restarts required

Manually restarting the server will be no longer necessary in case of lost connectivity. The server will recover automatically. If the server starts up when a connection to the repository database is not available, the server will enter a paused state waiting for the connection. When the connectivity is restored, the server will enter its normal operational state without the need for manually restarting it.

Improved performance

Improved server resilience means improved performance. If the server is running the flows and an outage event occurs, the flows will not be terminated, and instead paused waiting on connection. This means that the server will be able to complete most flows successfully even in the presence of multiple random network and/or database connection issues.

Activity/Error tracking

Logging in general, and logging of database connection issues in particular, have been greatly improved in the new release. Moreover, the server writes database connection issues in the Windows event log and includes a link to the Error file for easier troubleshooting. An entry is also added to trace the server log when the connection is restored.

 

The Dilemma of Build vs Buy – How it applies to Enterprise Software

If Shakespeare was an IT manager, the famous question ‘To be, or not to be’ would have been ‘To build or to Buy’. In fact, the phenomena of DIY-ing something or buying a commercial product is not only limited to enterprise software. IKEA is running a whole business out of providing utility to ‘build proponents’ and DIY enthusiasts. While building furniture can be fun, building an enterprise level software – not so much.

Build vs Buy

Like any other business decision, the decision to either build a software or to buy a commercial product is significantly influenced by total cost of the approach and return on investments. If you’re facing a similar dilemma, the table below summarizes the prospects and consequences of both the approaches.

         Metrics and KPIs                     Build Approach                        Buy Approach
Cost of deployment Hiring a team of developers, designers, programmers to build the solution License fee of the product and deployment costs
Time to market – Time to develop the product

– Time for performing QA analysis

– Time to fix any patches or bugs found

– Time to deploy the solution

– Product development, QA analysis and patch fixes are already taken care of by the solution provider. Therefore, solution can directly be deployed.

– Time to configure and install the product.

Ongoing maintenance and support costs A dedicated team of IT professionals should be on-board to help with ongoing product support and maintenance Updates, maintenance and customer support are handled by the solution provider. However, the solution provider might charge a fee for providing these services
Learning curve A steep learning curve is usually associated with the developed product Commercial products are developed to be used by a wide range of audience with varying levels of technical skills therefore, in most cases these solutions are designed to be more intuitive and user-friendly

 

When is ‘Building’ the right approach?

Building a software is going to be beneficial for your business if:

  • The software is going to give you sustainable competitive advantage
  • No other available solution can meet your business needs
  • The end-points from where your business collects data are not volatile or prone to frequent changes
  • You have substantial resources to cover the costs associated with building and maintaining the software

When is ‘Buying’ the right approach?

You should opt for buying a commercial software if:

  • Building a software is not the core of your business and is not going to yield you any competitive advantage
  • You have limited resources and you would rather invest them in improving your core business activities
  • There are solutions available that address the challenges your business is facing
  • You are looking for a quick solution that can be immediately deployed

IT manager at Brickell Bank, formerly known as Espirito Santo Bank, faced challenges in migrating broker data from MS Access database to IBM mainframe data warehouse. Learn more about the approach he opted for and other factors that influence build vs buy decision by downloading the free white paper.

Pushdown Optimization Mode in Centerprise Data Integrator

How does Pushdown Optimization mode work in Centerprise?

Moving data, containing millions of records, between source, ETL server, and target database can be a time-consuming process. When source and target database reside on the same server, unnecessary data movements and delays can be prevented by applying transformations to data in pushdown optimization mode.

Pushdown optimization mode pushes down the transformation logic to the source or target database. Centerprise integration server translates the applied transformation logic into automatically generated SQL queries. This eliminates the need for extracting data from the source, migrating it to staging tables on an ETL server for applying transformations, and then loading the transformed data on the target database. As a result, performance is significantly improved and data is readily made available to the end-users.

ELT, ETL, Pushdown optimization mode

Types of Pushdown Mode

There are two types of pushdown optimization modes:

  1. Full pushdown optimization mode
  2. Partial pushdown optimization mode

In full pushdown optimization mode, the Centerprise integration server executes the job completely in the pushdown mode. And in partial pushdown mode, the transformation logic is either pushed down to the target database or the source database, depending on the transformation logic and database provider.

Database Providers supported in Pushdown Mode by Centerprise

Centerprise supports following database providers:

  1. MySQL
  2. SQL
  3. Oracle
  4. Postgres
  5. MSSQL

Verify Pushdown Mode

Certain transformation logic cannot be executed in a pushdown mode. ‘Verify Pushdown Mode’ feature in Centerprise identifies the transformation logic that can be pushed down to the source or destination database.

To learn more about Pushdown Optimization mode in Centerprise and its use cases, download the white paper Centerprise Automated Pushdown Optimization.

Optimizing Business Capabilities with a Data Integration Software

Businesses are increasingly adopting a data-driven culture. The significant surge in the volume of the exchanged data indicates that the trend is creating a paradigm shift – a shift from manufacturing to an information economy. To put this in perspective, Google processes petabyte of information by the hour and The Economist recently declared data as the most valuable resource, even more than the oil.

Data integration with Centerprise

“The world’s most valuable resource is no longer oil, but data.”

-The Economist

But the true utility of any resource comes from its consumption or the value it delivers to the consumers. The same principle applies to data. To gain maximum utility out of data, businesses must be able to (quickly and reliably) integrate incoming data from disparate sources and make that information available to the relevant stakeholders, both internally and externally. Your business needs a data integration tool to perform this task efficiently.

A data integration tool can help you in following ways to optimize your current business capabilities:

By extracting data from structured and unstructured sources

Incoming data can be structured, semi-structured, poly-structured or unstructured. For instance, text-based PDF files, PDF forms and scanned PDF images are used as a medium for exchanging information by many organizations. But the data contained in PDF files is unstructured and is required to be extracted for crucial business decisions. A data integration tool can automate the data extraction process and integrate the extracted data with the internal systems for further processing and analysis.

By integrating data from hierarchical files

Integrating data from flat files is comparatively easier but business users face challenges when they try to extract, parse and integrate information from hierarchical data files such as XML, JSON, EDI and COBOL. To perform hierarchical data integration, business users rely on IT, which increases the burden on them. A data integration tool can effectively bridge this gap between business executives and IT.

Learn how Centerprise Data Integrator enables business users to work with hierarchical data, without the need for custom coding and programming, by downloading the whitepaper Hierarchical Data Integration for Business Users.

By making data readily available to business users

A data integration tool with a user-friendly interface and a comprehensive library of built-in functions can help limit the reliance on IT. It readily makes the data available to business users who can then work with the available information and get business insights without delay. Additionally, data integration tools can automate the ETL process, which eliminates the need for manual integration and significantly reduces the chances of errors.

The performance of a business is optimized when the executives are more focused on making critical business decisions rather than collecting and integrating the data.

By checking for data-quality

A data integration tool cleanses, validates and ensures the trustworthiness of the incoming data. Poor quality data can adversely affect business insights that can prove to be expensive for the business.

Overall, a data integration tool that simplifies the ETL process for the users is an investment that organizations should make to stay relevant in the current data driven business environment. It can prove to be beneficial for the business in more than one ways. By bridging the gap between IT and business executives, it helps in efficient division of workload. It empowers business users to drive insights from the data by giving them prompt access to it. And when executives delegate the task of data integration and extraction to software, they can focus on more critical aspects of the business. The result is faster and more accurate business decisions, minimized costs and increased revenue.

Astera’s Centerprise Data Integrator is a complete data integration solution that provides the mentioned benefits to its users, and more. The user-friendly interface and visual drag-and-drop environment eliminates the need for manual scripting and enables business users to work with data without relying on IT. Contact Astera’s sales and support to get more information.