Blog de Joe Freeman

Create resources in Azure using the Azure Portal Template UI. We created a resource group with a Namespace and individual EventHubs in another video. Then I exported the resource group contents to a JSON file. Here we load that JSON file into another Resource Group to recreate the EventHubs and their associated identities and security settings. The code for this  discussion came out of working on a different blog posting .    The exported template is mostly hardcoded. You will want to parameterize any names that might vary by environment by reuse of the template for other purposes.

Azure EventHubs can be secured via IAM Role Permissions and Resource Access Policies.  They each have their own advantages and disadvantages as discussed in a previous blog posting .  We can see how the various Authorization techniques come together in the Azure Portal .  GitHub Repository The Azure portal images in this blog were generated using the 8/2021 version of this Github repository: Azure EventHubs Example Example Security Posture Our sample uses different authorization bindings to suit different client types. It applies those bindings at different places in the resource hierarchy. Individual EventHubs and Namespacesuse Identity Access Management with a Managed Identity and  Standard Azure Roles for some use cases.  They use Shared Access Policies  and signed requests for other use cases. Permissions are applied at the Namespace  and individual EventHub levels.  Namespace

Moving into the public cloud involves balancing known techniques against cloud-native approaches.  Identity management and Authentication/Authorization is one of the areas where you can use your legacy on-prem approach or a more cloud-native approach.   Managed Identities are a powerful cloud-native identity available in Azure.  They are integrated at the Azure resource level, providing identities without the hassle of secrets management or separate lifecycle processes. Managed Identities  can act as a logical replacement firewall in cloud-only resource-to-resource topologies. Identity Types We are going to talk about 3 styles to cloud identity.  Person Identity: A user account, traditionally secured with a username and password.  User accounts are common in many systems.  Applications will often use a fake person or service account  as their identity.  Service account credentials, passwords, must be protected and rotated. This type of identity can be passed from one system to anothe

JSON and AVRO are both great serialization models.  JSON is all text, human readable, and very verbose.  AVRO is an efficient binary format.  They can serialize the same data but they can also handle schema evolution or field changes differently JSON supports field order changes because all of its fields come with their own label  in every single message.  Avro messages do not always handle field order changes. Field Order Avro serializer/deserializers operate on fields in the order they are declared. Producers and Consumers must be on a compatible schema including the field order .  Do not change the order of AVRO fields. All Producers and Consumers are must be updated at the same time if you change the field order. The AVRO 1.8 documentation says  Records   A record is encoded by encoding the values of its fields in the order that they are declared. In other words, a record is encoded as just the concatenation of the encodings of its fields. Field values are encoded per their schema

We want to protect cloud assets by giving processes the "list privilege" possible. This can be done through a combination of identities and access policies . Cloud assets are protected by access policies that describe the operations available to roles and identities. The Access Policies bind identities to permissions.  Application and system processes present their identity as part of resource requests and the Access Policies decide if access is granted.  Organizations can avoid creating "powerful" identities by creating multiple fine-grained identities, similar to roles.  Processes are assigned the minimum combination of identities required to access only the resources they required.  The Processes present the right identity when making a resource request. The Access Policies allow access based on the presented identity.   The example is implemented in Microsoft Azure.  Amazon AWS has similar capabilities. Use Case Virtual machines can need access to overlapping se

Cloud providers offer disk-optimized virtual machine types targeted at high-performance distributed document and columnar stores.  These VM types are targeted at systems like Cassandra, ElasticSearch, MongoDB. Databases can consume a disk directly without standard file systems.  Cloud providers leave it up to you to decide how the drives will be configured.  We need to encrypt these drives to protect data "at rest". Cloud providers leave it up to you to configure and encrypt the drives.   Virtual Machines and Local Storage Virtual Machines have several classes of data storage.   Network dedicated boot and mounted drives  Network shared similar to SMB or NFS. Local ephemeral SSDs primarily aimed at temporary directories or swap space. Local high-performance NVMe/SSD provided to the machines as raw devices. It is the latter type of drive that databases and caching servers can self-manage to get high performance. They will typically hold data so they need to be encrypted at res