Training: Implement a data engineering solution with Azure Databricks (DP-3027)

Ref. DP-3027

Download PDF

Duration:

jour

Exam:

Non certifiant

Level:

Intermédiaire

Home > Trainings > IT Pro > Implement a data engineering solution with Azure Databricks (DP-3027)

Implement a data engineering solution with Azure Databricks Training (DP-3027)

You need to build machine learning models but want to leverage Microsoft Fabric’s integrated data science environment? Fabric combines data engineering and ML capabilities in a single platform, simplifying the path from data preparation to model deployment. Without practical skills in Fabric’s ML tools, data scientists cannot fully utilize this unified analytics environment.

This official Microsoft DP-3027 Applied Skills training enables you in 1 day to implement data science and machine learning solutions using Microsoft Fabric. Guided by MCT-certified trainers, you learn to explore data with Spark notebooks, engineer features, train models using scikit-learn and MLflow, evaluate and compare model performance, and generate batch predictions. You practice building complete ML workflows integrated with Fabric’s lakehouse. This hands-on credential validates your data science skills on Microsoft Fabric. Sessions are available in person in Geneva and Lausanne or via virtual classroom.

Participant Profiles

Data Engineer
Big Data Developer
Cloud Architect
Data Consultant
Azure Administrator
BI Manager

Objectives

Explore and prepare data using Spark notebooks
Engineer features for machine learning models
Train models using scikit-learn and MLflow
Evaluate and compare model performance
Generate batch predictions at scale
Build end-to-end ML workflows integrated with Fabric lakehouse

Prerequisites

Know how to use a computer
Understand the basics of cloud computing

Course Content

Module 1: Perform incremental processing with Spark Structured Streaming

Set up real-time data sources for incremental processing
Optimize Delta Lake for incremental processing in Azure Databricks
Handle late data and out-of-order events in incremental processing
Monitoring and performance tuning strategies for incremental processing in Azure Databricks

Module 2: Implement streaming architecture patterns with Delta Live Tables

Event driven architectures with Delta Live Tables
Ingest data with structured streaming
Maintain data consistency and reliability with structured streaming
Scale streaming workloads with Delta Live Tables

Module 3: Optimize performance with Spark and Delta Live Tables

Optimize performance with Spark and Delta Live Tables
Perform cost-based optimization and query tuning
Use change data capture (CDC)
Use enhanced autoscaling
Implement observability and data quality metrics

Module 4: Implement CI/CD workflows in Azure Databricks

Implement version control and Git integration
Perform unit testing and integration testing
Manage and configure your environment
Implement rollback and roll-forward strategies

Module 5: Automate workloads with Azure Databricks Jobs

Implement job scheduling and automation
Optimize workflows with parameters
Handle dependency management
Implement error handling and retry mechanisms
Explore best practices and guidelines

Module 6: Manage data privacy and governance with Azure Databricks

Implement data encryption techniques in Azure Databricks
Manage access controls in Azure Databricks
Implement data masking and anonymization in Azure Databricks
Use compliance frameworks and secure data sharing in Azure Databricks
Use data lineage and metadata management
Implement governance automation in Azure Databricks

Module 7: Use SQL Warehouses in Azure Databricks

Get started with SQL Warehouses
Create databases and tables
Create queries and dashboards

Module 8: Run Azure Databricks Notebooks with Azure Data Factory

Understand Azure Databricks notebooks and pipelines
Create a linked service for Azure Databricks
Use a Notebook activity in a pipeline
Use parameters in a notebook

Documentation

Access to Microsoft Learn, Microsoft’s online learning platform, offering interactive resources and educational content to deepen your knowledge and develop your technical skills.

Lab / Exercises

This course provides you with exclusive access to the official Microsoft lab, enabling you to practice your skills in a professional environment.

Complementary Courses

Additional Information

Data science in Microsoft Fabric: machine learning integrated with analytics

Microsoft Fabric provides a native data science experience that integrates seamlessly with the broader analytics platform. Using Spark notebooks, MLflow for experiment tracking and direct access to lakehouse data, data scientists can build, train and evaluate models without leaving the Fabric environment. This DP-3027 credential validates your practical ability to implement ML workflows in Fabric, an increasingly valued skill for data professionals in Switzerland.

What you practice during this 1-day training

This hands-on training focuses on implementing data science solutions in Fabric. You explore data using Spark notebooks and visualization libraries, engineer features for ML models, train models using scikit-learn, track experiments and log metrics with MLflow, evaluate model performance with standard metrics, and generate batch predictions at scale. All exercises use real datasets on Fabric environments with MCT-certified trainer guidance.

Frequently Asked Questions

What are the prerequisites for this DP-3027 training?

You should have basic Python programming skills and foundational understanding of machine learning concepts. Familiarity with Spark notebooks is helpful but not required.

Is this an Applied Skills credential or a certification?

DP-3027 is a Microsoft Applied Skills credential, validating your practical data science implementation skills through a scenario-based assessment.

How does this differ from DP-604?

DP-604 and DP-3027 cover similar ML topics within Fabric but may differ in their assessment scenarios and specific focus areas. Both validate practical data science skills on the Fabric platform. Check the latest Microsoft documentation for current differences.