In this article, Ashutosh Kumar discusses the emergence of modern data solutions that have led to the development of ELT and ETL with unique features and advantages. ELT is more popular due to its ability to handle large and unstructured datasets like in data lakes. Traditional ETL has evolved into cloud-based ETL which allows rapid batch processing, scalability, savings, and simplicity while maintaining security, governance, and compliance.
Why Do We Prefer ELT Rather than ETL in the Data Lake? What is the Difference between ETL & ELT
July 4, 2023 by 1 Comment
Video Highlights: Modernize your IBM Mainframe & Netezza With Databricks Lakehouse
November 3, 2022 by Leave a Comment
In the video presentation below, learn from experts how to architect modern data pipelines to consolidate data from multiple IBM data sources into Databricks Lakehouse, using the state-of-the-art replication technique—Change Data Capture (CDC).
Databricks Announces General Availability of Delta Live Tables
April 6, 2022 by Leave a Comment
Databricks, the Data and AI company and pioneer of the data lakehouse paradigm, announced the general availability of Delta Live Tables (DLT), the first ETL framework to use a simple declarative approach to build reliable data pipelines and to automatically manage data infrastructure at scale. Turning SQL queries into production ETL pipelines often requires a lot of tedious, complicated operational work. By using modern software engineering practices to automate the most time consuming parts of data engineering, data engineers and analysts can concentrate on delivering data rather than on operating and maintaining pipelines.
Data Platforms – A journey. The Yesteryears, Today, and What Lies Ahead
November 25, 2020 by Leave a Comment
In this contributed article, Darshan Rawal, Founder and CEO of Isima, explains how the data ecosystem has exploded in the last decade to deal with multi-structured data sources. But the fundamental architecture of using queues, caches, and batches to support Enterprise Data Warehousing and BI hasn’t. This article looks at the architectural styles of the three eras of data management – pre-big data, the open-source revolution, and the cloud-native version. It will be a dive into trade-offs of each and what lies ahead. You’ll get a techno-strategic best practices of architecting data platforms as they pave the path to recovery for your organization.
How Automation Helps You Exploit the Value in Big Data
October 15, 2020 by Leave a Comment
In this sponsored post, Simon Shah spearheads marketing at Redwood Software to support continued market growth and innovation for their cloud-based IT and business process automation solutions. He believes that by using automation to collect and manage your big data processes, you will truly exploit its value for the business.
Matillion Launches Matillion ETL for Azure Synapse Empowering Users with Data Transformation Capabilities for Rapid Access to Insights
May 29, 2020 by Leave a Comment
Matillion, a leading provider of data transformation software for cloud data warehouses (CDWs), announced the availability of Matillion ETL for Azure Synapse to enable data transformations in complex IT environments, at scale. Empowering enterprises to achieve faster time to insights by loading, transforming, and joining together data, the release extends Matillion’s product portfolio to further serve Microsoft Azure customers.
Do You Actually Need a Data Lake?
November 6, 2019 by Leave a Comment
In this contributed article, Eran Levy, Director of Marketing at Upsolver, sets out to formally define “data lake” and then goes on to ask whether your organization needs a data lake by examining 5 key indicators. Data lakes have become the cornerstone of many big data initiatives, just as they offer easier and more flexible options to scale when working with high volumes of data that’s being generated at a high velocity – such as web, sensor or app activity data.
The Cloud Helped Expand ETL into Automated Data Infrastructure
April 25, 2019 by Leave a Comment
In this contributed article, Ben Bloch, CEO of Bloch Strategy, defines Extraction, Transformation and Load (ETL) as the process of taking data from one system, changing it in a way that allows it to be integrated with other data in a new system. Since it’s automated you can work with scale in more agile ways. It became a well known acronym in business software during the 1990s, due to the focus on data warehouses. Many people in the industry have tied ETL too closely to on-premise data warehousing, and consider that it has no place in the Cloud world, but that is an incorrect assumption. ETL is even more important today.
