Blog de Joe Freeman

Create resources in Azure using the Azure Portal Template UI. We created a resource group with a Namespace and individual EventHubs in another video. Then I exported the resource group contents to a JSON file. Here we load that JSON file into another Resource Group to recreate the EventHubs and their associated identities and security settings. The code for this  discussion came out of working on a different blog posting .    The exported template is mostly hardcoded. You will want to parameterize any names that might vary by environment by reuse of the template for other purposes.

Azure EventHubs can be secured via IAM Role Permissions and Resource Access Policies.  They each have their own advantages and disadvantages as discussed in a previous blog posting .  We can see how the various Authorization techniques come together in the Azure Portal .  GitHub Repository The Azure portal images in this blog were generated using the 8/2021 version of this Github repository: Azure EventHubs Example Example Security Posture Our sample uses different authorization bindings to suit different client types. It applies those bindings at different places in the resource hierarchy. Individual EventHubs and Namespacesuse Identity Access Management with a Managed Identity and  Standard Azure Roles for some use cases.  They use Shared Access Policies  and signed requests for other use cases. Permissions are applied at the Namespace  and individua...

Moving into the public cloud involves balancing known techniques against cloud-native approaches.  Identity management and Authentication/Authorization is one of the areas where you can use your legacy on-prem approach or a more cloud-native approach.   Managed Identities are a powerful cloud-native identity available in Azure.  They are integrated at the Azure resource level, providing identities without the hassle of secrets management or separate lifecycle processes. Managed Identities  can act as a logical replacement firewall in cloud-only resource-to-resource topologies. Identity Types We are going to talk about 3 styles to cloud identity.  Person Identity: A user account, traditionally secured with a username and password.  User accounts are common in many systems.  Applications will often use a fake person or service account  as their identity.  Service account credentials, passwords, must be protected and rotated. This type of...

JSON and AVRO are both great serialization models.  JSON is all text, human readable, and very verbose.  AVRO is an efficient binary format.  They can serialize the same data but they can also handle schema evolution or field changes differently JSON supports field order changes because all of its fields come with their own label  in every single message.  Avro messages do not always handle field order changes. Field Order Avro serializer/deserializers operate on fields in the order they are declared. Producers and Consumers must be on a compatible schema including the field order .  Do not change the order of AVRO fields. All Producers and Consumers are must be updated at the same time if you change the field order. The AVRO 1.8 documentation says  Records   A record is encoded by encoding the values of its fields in the order that they are declared. In other words, a record is encoded as just the concatenation of the encodings of its fields. Fi...

We want to protect cloud assets by giving processes the "list privilege" possible. This can be done through a combination of identities and access policies . Cloud assets are protected by access policies that describe the operations available to roles and identities. The Access Policies bind identities to permissions.  Application and system processes present their identity as part of resource requests and the Access Policies decide if access is granted.  Organizations can avoid creating "powerful" identities by creating multiple fine-grained identities, similar to roles.  Processes are assigned the minimum combination of identities required to access only the resources they required.  The Processes present the right identity when making a resource request. The Access Policies allow access based on the presented identity.   The example is implemented in Microsoft Azure.  Amazon AWS has similar capabilities. Use Case Virtual machines can need access to ...

Cloud providers offer disk-optimized virtual machine types targeted at high-performance distributed document and columnar stores.  These VM types are targeted at systems like Cassandra, ElasticSearch, MongoDB. Databases can consume a disk directly without standard file systems.  Cloud providers leave it up to you to decide how the drives will be configured.  We need to encrypt these drives to protect data "at rest". Cloud providers leave it up to you to configure and encrypt the drives.   Virtual Machines and Local Storage Virtual Machines have several classes of data storage.   Network dedicated boot and mounted drives  Network shared similar to SMB or NFS. Local ephemeral SSDs primarily aimed at temporary directories or swap space. Local high-performance NVMe/SSD provided to the machines as raw devices. It is the latter type of drive that databases and caching servers can self-manage to get high performance. They will typically hold data so the...

One of my favorite features of Azure DevOps is its ability to integrate Work Item updates with Office Excel.  This means you can run any canned ADO query from Excel, update the retrieved work items and then publish the updates back to ADO all from inside Excel.   Microsoft Guidance See the following pages https://docs.microsoft.com/en-us/azure/devops/boards/backlogs/office/bulk-add-modify-work-items-excel?view=azure-devops&tabs=agile-process https://docs.microsoft.com/en-us/azure/devops/boards/backlogs/office/faqs?view=azure-devops Install Azure DevOps Automation Download and install Microsoft Office Integration tools by installing 2019 dev-ops automation   This installs a TFSProtocolHelper that ADO can use to open Query results in Excel Create an ADO query that will execute and fill Excel Click on the Queries  in the Boards Menu Create a Query and save it to create a named query I tend to create two-tier queries for Features  first-level children,...

We run across issues or find problems every day. Some of them can be handled immediately or in an ad-hoc manner. We need some standard framework for investigating and solving larger or more complicated problems. There are different schools of thought on how people bring problems and when and who should do the solutioning . Just problems or problems that need solving We want to create expectations about how action will be take when a problem is found. Our best teammates raise issues or expose problems in order to try and improve the situation.  This discussion is around this group.    Other people raise issues or find problems as a way of delaying action or describing why things can't be done. They don't really intend to solve the problem. We will withhold further comments for another time. Organizational Failure. Some management don't want to acknowledge problems.  They don't want ...

Create ADRs when making significant and impactful decisions.  Retain those decisions in a way that ephemeral media like email, video calls, and IM channels do not. Architectural Decision Records (ADRs) capture architecturally significant decisions or design choices and the context and consequences  for the decisions. ADRs exist to help people in the future understand why an approach was selected over others.  This includes  future you, new team members, and future teams that take over responsibility for systems or processes that were impacted by the ADR.  The typical ADR consists of: Components Comment Problem Statement The problem statement that drove the need for an ADR Current Status Draft, approved, declined, etc.  Some ADRs will be abandon for other approaches or changes in direction. Alternatives The viable alternatives that were considered and their pros and cons. Decision The chosen solution including enough information ...