Posts

Azure Data Factory | Copy multiple tables in Bulk with Lookup Activity & ForEach

  Shivani Mishra - Easy Azure for Everyone Introduction to Azure Data Factory Lookup and ForEach Actions In this blog, we will learn how to build dynamic pipelines in Azure Data Factory using lookup and forEach actions. The lookup action allows you to retrieve a data set from a file or table within your database, and you can use this data set to determine the list of objects that you want to work with in your pipeline. The lookup action supports most of the data sources currently supported in Data Factory, so you can use various types of databases or flat file solutions. The lookup action is often paired with the forEach action, which allows you to execute a set of activities for each object in the list returned by the lookup action. This blog will walk you through a step-by-step setup of a pipeline that lists tables from a SQL database using a lookup activity, and then iterates over the results to perform a simple copy extraction. Step-by-Step Setup Initial S...

Azure Data Factory | Introduction to ETL in Azure

Image
Shivani Mishra - Easy Azure for Everyone Data Factory: A Brief Introduction In this blog post, we will see Azure Data Factory, a service that allows you to move data between different sources. Whether you are working in the cloud or on-premises, data movement is an essential part of any project. Let's explore how Data Factory works and how you can use it in your own projects. How does Data Factory work? To understand how Data Factory works, let's use an analogy. Imagine you want to move a cabinet from a shop to your home. In this scenario, you need certain information: ·          A key and address to locate the shop and access the cabinet ·          Information about the cabinet, such as disassembly instructions or packaging details ·          Delivery details, including what to do with the package once you receive it ·    ...

The fundamentals of using Azure Secret Computing to Safeguard your data

Image
The exponential rise of datasets has led to increased scrutiny of how data is disclosed, both from the perspective of consumer data privacy and compliance. Confidential computing becomes a key tool in this setting to assist enterprises in meeting their privacy and security requirements for commercial and consumer data. Confidential computing technology encrypts data in memory and processes it only once the cloud environment has been confirmed, prohibiting data access from cloud operators, malevolent administrators, and privileged applications like the hypervisor. It aids in the protection of data throughout its lifecycle. Data is now safeguarded when in use, in addition to existing technologies for securing data at rest and in transit. Organizations all over the world can now take advantage of previously unattainable prospects because to secret computing. They can now, for example, profit from multi-party data analytics and machine learning, which aggregate datasets from parties ...