Microsoft Azure, DevOps Deployment, Analytics and Cloud Design

Azure tenant subscription and resource group

A tenant is an identity boundary, a subscription is a billing boundary and a resource group is a lifecycle container for resources.

Microsoft Entra ID and Azure RBAC

Microsoft Entra ID authenticates identities and Azure RBAC authorizes them to perform actions at a defined scope.

Azure portal CLI and PowerShell

The Azure portal is a web console, while Azure CLI and PowerShell provide scriptable interfaces to the same resource-management APIs.

Azure virtual network

An Azure virtual network contains address spaces, subnets, routes, security controls.

Azure virtual machines

Azure Virtual Machines provide guest operating-system instances selected by image and size and connected to managed disks, interfaces.

Azure Storage

Azure Storage supplies durable object, file, queue, table and disk services controlled by accounts, redundancy choices, identities.

Azure SQL Database

Azure SQL Database is a managed relational database service that operates the database engine, backups.

Cosmos DB

Azure Cosmos DB is a globally distributed multi-model database with partition-key-based scaling and selectable consistency levels.

Azure Functions

Azure Functions executes event-triggered functions on managed infrastructure with bindings, scaling, identity and hosting-plan choices.

GitHub Actions and Azure DevOps

Automation platforms that run version-controlled build, test and deployment workflows on managed or self-hosted agents.

Big Data in the Cloud

AWS Big Data Services

Google Cloud Big Data

Azure Big Data

Apache Kafka

What is Kafka?

Apache Kafka is a distributed event streaming platform for:

• High-throughput, low-latency message streaming
• Real-time data pipelines
• Decoupling of data producers and consumers

Kafka Core Concepts

Kafka Use Cases

Kafka vs Traditional Messaging

Data Lake vs Data Warehouse

Data Lake

A Data Lake stores all data in its raw, native format — structured, semi-structured and unstructured.

"Store everything; process when needed"

Risk: Data Lake can become a Data Swamp without proper governance (unusable raw data).

Data Warehouse

Lakehouse

Lakehouse = Data Lake + Data Warehouse (modern approach):

• Store raw data in Data Lake
• Apply DWH features (ACID, schema enforcement) on top
• Technologies: Delta Lake (Databricks), Apache Iceberg, Apache Hudi

Azure Compute, Storage and Networking

Azure organizes resources into subscriptions and resource groups. A region contains datacenters and may provide availability zones. Virtual machines supply IaaS compute while Virtual Machine Scale Sets manage groups of similar instances. App Service hosts web applications on a managed platform. Azure Functions executes event-driven code. Container Instances run containers directly while Azure Kubernetes Service manages Kubernetes clusters.

Virtual Networks contain subnets and routes. Network security groups filter traffic. Public IP addresses expose selected resources while private endpoints connect supported services through private addresses. Load Balancer distributes transport connections and Application Gateway provides application-layer routing. VPN Gateway and ExpressRoute connect external networks.

Storage accounts expose services such as Blob object storage, Files shares, Queues and Tables. Access can be authorized through identities, scoped tokens or keys. Redundancy choices replicate within a datacenter, across zones or to another region. The choice affects durability, availability and price.

DevOps, Analytics and Design

Azure DevOps provides repositories, work tracking, build and release pipelines and package feeds. A pipeline should create one versioned artifact then promote that artifact through environments. Service connections use controlled identities. Secrets belong in a secret store and are referenced at runtime rather than committed to pipeline files. Approvals and environment checks can protect production deployment.

Azure Monitor collects metrics and logs while Application Insights traces application requests and dependencies. Log Analytics provides queryable operational data. Analytics services range from managed relational warehouses to data-lake, streaming and Spark platforms. A design chooses them according to data volume, latency, query pattern and governance.

Reliable design separates failure zones, removes single-instance dependencies and tests restoration. Identity uses least privilege and managed identities where possible. Cost controls include tags, budgets, rightsizing and lifecycle policies. Infrastructure definitions should be versioned so that environments can be reviewed and recreated.