Posts Tagged ‘.net data conversion’

Source Control In Centerprise

Monday, August 2nd, 2010

With Centerprise’s file system approach, putting meta data files such as transfers and dataflows under source control was always possible.  But now, we’ve added a built-in, fully-functional source control client right in the studio UI.  If you’re familiar with Microsoft’s Visual Studio, you’re probably already familiar the concept of integrated source control and know how much not having to switch between applications means to  your overall productivity.  Centerprise follows the same model and thus, now allows you to work at the file level (dataflows) as well as the project level (Centerprise projects) all within the same UI and all under source control.

Combined with Centerprise’s project feature, the new source control features are a perfect fit resulting in a more collaborative, team-based approach to data integration projects.  With source control built-in, you can now be working on a dataflow, re-factor a complicated dataflow, check-in your project, and when your colleague clicks the “get latest” button, he or she will instantly see  your work and thus be on the same page.  Some of the basic features include:

  • Check-in, check-out on all project files such as dataflows, workflows, SQL files, etc.
  • Get specific versions of files and projects including the latest version
  • View change history
  • Conflict resolution for when changes overlap.
  • Undo pending changes
  • Full source control explorer

Currently, Centerprise works with Microsoft’s Team Foundation Server for its version control back-end, but other systems such as Rational’s Clear Case and Subversion are scheduled to be included in an upcoming point release.  Please contact sales@astera.com for more info.

Tags: , , , , , , , , , ,
Posted in Centerprise Team Blog | No Comments »

Slowly Changing Dimensions in Centerprise 5

Friday, June 11th, 2010

Maintaining dimension tables in a data mart is quite a chore. We have talked to customers who spend a great deal of time writing SQL scripts, stored procedures, or other code to perform this function. Often, the code is written or duplicated for each dimension table and must be modified regularly to accommodate changing business requirements. Typically, this custom code performs poorly for all but small tables. I am sure that this brings a great deal of excitement to their working day.

Well, I guess they will have to find that excitement elsewhere because with Centerprise 5, we are taking all that fun out of dimension table maintenance. Centerprise’s Slowly Changing Dimension Write Strategy automates this function and eliminates all that work. All you need to do is to set up your SCD structure using a simple UI. SCD Strategy supports Type 1 and Type 2 slowly changing dimensions and provides support for multiple row versioning strategies including effective/expiration dates, current row flag, and version number. The following screenshot shows an example of SCD definition.

As you can see, all you have to do is specify the role that each field plays in the dimension table. These roles can be surrogate key, business key, SCD1 value, SCD2 value, effective/expiration dates, active value field, etc. That’s it. Once the roles are defined, you can use Centerprise’s preview feature to see exactly how the incoming data would be processed by the SCD strategy. Here is a preview of the dimension table update in the above screenshot.

Centerprise’s SCD Strategy is powered by a high-speed, parallel engine that compares incoming data against the data in the table and, based on the differences between the values, performs an SCD1 update, an SCD2 update/insert, a simple update, or skips the row if no material changes were found. The engine is designed to efficiently process even the largest of dimension tables.

Here is a simple dataflow

I encourage you to try this firsthand using the trial version of Centerprise and you will see the ease and speed of this component. Please contact us at sales@astera.com or 1-888-77-ASTERA begin_of_the_skype_highlighting              1-888-77-ASTERA      end_of_the_skype_highlighting for a trial download.

Tags: , , , , , , , , , , , , , , , , ,
Posted in Centerprise Team Blog | No Comments »

Centerprise 5 Beta Release is now available

Wednesday, June 2nd, 2010

I am pleased to announce that Centerprise Data Integrator 5 beta is now available for download. This release represents a major upgrade and adds sophisticated dataflow and workflow designers. We are very excited about this release and the great deal of value it can add to our customers’ business.

Centerprise 5 represents an attractive alternative to exorbitant cost of traditional data integration vendors and not-ready-for-primetime products offered at the lower end of the market.

Over the past year, we have had extensive discussions with customers from a variety of industries including financial services, pharmaceutical, healthcare, utilities, and government. Centerprise team used this feedback and other research to develop and improve product from many perspectives. Usability, always a hallmark of Centerprise, has been improved further with the addition of flow designers that provide drag and drop capabilities, unlimited undo/redo, cut/copy/paste, and parameterization, among others. Performance has been enhanced by further increasing parallelism and optimizing a number of areas including database writes and file reads. A new set of APIs enables customers and partners to extend Centerprise by adding new sources, destinations, transformations, and custom functions.

Here are the key features:

  • A Dataflow designer that supports complex data integration flows and features full complement of transformations including lookups, expressions, functions, aggregate, sort, join, normalize, denormalize, union, route, filter, and others.
  • Subflows to create reusable dataflow components that can be plugged into dataflows or other subflows.
  • Single-click WYSIWYG data view capability to preview data at any stage in data flow.
  • Integrated data quality validation and profiling.
  • Integrated drag and drop environment with unlimited undo/redo, cut/copy/paste, automatic layout building, auto map creation, one click element addition, and more.
  • A visual Workflow designer for defining job orchestration. Workflow designer provides the functionality to define job sequence, routing, and dependencies.
  • Restart capability to resume a job from the point of failure.
  • Workflow provides built-in tasks to run Dataflow, Workflow, SQL, or other programs, perform file systems actions, FTP actions, send mail, and others. Additional tasks can be created using Centerprise APIs.
  • Built-in job scheduler to start jobs at recurring intervals including hourly, daily, weekly, and monthly. Jobs can also be triggered based on file drop and through APIs.
  • High-performance parallel processing engine optimized to deliver the performance and scalability required to efficiently process very high data volumes.

If you would like to participate in Centerprise beta program, please register here:

Or you can call us at 1-888-77-ASTERA begin_of_the_skype_highlighting              1-888-77-ASTERA      end_of_the_skype_highlighting (1-805-579-004) or email sales@astera.com.

Tags: , , , , , , , , , , , , , , , , , ,
Posted in Centerprise Team Blog | No Comments »

Upcoming Centerprise Upgrade and Microsoft.Net 4.0

Tuesday, April 27th, 2010

Earlier this month, Microsoft released Visual Studio 2010. Along with VS 2010, Microsoft also released version 4.0. .Net 4.0 brings new features and improvements in a number of areas. Centerprise team has been working on a major new upgrade for some time now. Last fall, when Microsoft released preview version of Visual Studio 2010, we evaluated it and decided to develop next version of Centerprise on .Net 4.0.

For the next generation of Centerprise product, our key goals are:

• Powerful data integration functionality including enterprise grade dataflow and workflow designers, first class support for dimension and fact table loading, and extensive data quality features.
• A Highly parallel data integration engine that would scale to take advantage of increasingly larger number of CPUs and cores in today’s machine and deliver the performance and throughput that can handle very large data sets.
• An extensible platform that makes it easy to add new sources, destinations, transformations, and functions easily while providing a secure runtime environment.

Centerprise has been a parallel processing engine since version 2.0. We have always focused on performance and scalability as a key design goal. Centerprise’s parallel framework has provided an excellent foundation the integration engine. Employing the new parallel programming extensions in .Net 4.0, we have made substantial improvements to our framework and the entire product has become significantly more efficient and scalable. Parallelism pervades every aspect of the data integration engine including new multithreaded algorithms for sorting, file reading and parsing, database writes, profiling, and transformations such as join, lookup, aggregation and others.

We are planning to use .Net extensibility framework in future versions of Centerprise to provide a secure and powerful platform for customers and third party developers. This includes the ability to add new data sources, transformations, and workflow tasks.

Hermes will also feature extensive set of .Net APIs to enable our customers and partners integrate Centerprise as part of their solutions. This includes triggering and monitoring jobs, creation of function, custom transformations, addition of new data sources and destinations, and much more.

Tags: , , , , , , , , , , , , , , ,
Posted in Centerprise Team Blog | No Comments »