Blog de Joe Freeman

PaaS and SaaS persistence services store your data in their systems, often in their accounts or subscriptions.  The service provider protects the system and its associated storage. We need to determine our appetite for risk when deciding what additional work must be done to secure the externally hosted data. Risks Data at rest must be protected with a multi-layered approach. Identify the attacks that you wish to prevent in order to determine how much protection you want. The list below is just a sample. disk re-use  is hardware or technology-related. It can be mitigated without any application or user experience changes.  Vendor -related access issues exist because a 3rd party is hosting the data. This includes vendor staff access and the ability to remove or render unusable your data in their ho Control Plane refers to dashboards or admin screens that are vendor-provided.  Many have preview functions that let you validate the data.  The built-in user permissio...

Most of the Azure Resource Management template examples demonstrate using two JSON files. One file is the template definition that accepts a set of named parameters. The other file contains the parameter values.  The two files are accepted by the template engine and combined to create the actual definition. The Azure CLI supports providing parameters in JSON files or as name/value pairs on the command line.  Parameters are always specified using  --parameters .  The command-line option supports two different syntaxes and can be invoked multiple times to provide property values from multiple JSON files and  as command-line arguments. The middle example represents invoking the CLI with the two filenames The right-hand example represents involving the CLI with a template and a list of command-line parameter values.   Refer to https://github.com/freemansoft/vnet-p2s-vpn-bastion-azure/b...

Organizations implement controls to reduce risk, improve reliability, protect data and meet compliance objectives.  Formalizing processes and implementing controls often reduces the agility of an organization and makes it harder to innovate or experiment.  There is a constant tension between protecting the enterprise and innovation.  Note that we are talking about innovation  and not malice where people cut corners to make dates or make their lives easier. We can reduce the level of control or create special places where people can experiment and innovate.  We need to do it in a way that work done there doesn't bleed into the controlled systems and data. I worked at a place where we wanted to try a cloud service based database. We had no schema and just a Proof of Concept idea of what we wanted. It took 6 weeks of paperwork and several iterations of possible schemas to get onboarded and get access to the database for the PoC.  We knew the approved schema wa...

Cloud platforms like Azure, and AWS, and GCP provide new-user free tier accounts that help people get started in the cloud. They have no low-cost or capped offerings after that short period. They all expect you to move to uncapped fully metered pay-as-you-go services after that. This makes it hard for data scientists, developers and architects is how to stay current and innovate without running the risk of financial catastrophe.  I don't want to go bankrupt while experimenting We're talking about relatively small non-production use cases.  The dollar limits on these could be capped to relatively small amounts. I recently accidentally provisioned a dedicated Azure Event Hubs cluster that burned through my fixed $150 credit in 1 day.  The account ran up a $400 bill before Azure caught up and shut down the subscription.  The hard cap meant I was dead to Azure for a month but not dead to my spouse for spending our mortgage payment. After the time-limited free tier T...

Data Organization, build-vs-buy, transform audit, and technology choices all depend on your organization's policies, business, and compliance requirements. We are going to look at some business requirements that might put us on a different path from the parallel load, warehouse first, and lake first patterns previously discussed. Video Discussion This pattern assumes that all the primary  raw  and conformed/curated  transformations happen in one data repository with one set of tools.  The raw and conformed/curated zones are then replicated into the other repository.  Your org would choose whether the lake or the warehouse was home for transformations for those zones. 

Many companies end up creating their own cloud management  control planes on top of their cloud provider's management APIs.  These homegrown  management systems provide a central location for provisioning cloud services, for configuring SaaS offerings. They also interact with corporate compliance and control systems like artifact inventories and data catalogs. What drives companies to this effort and expense? Video Walkthrough Self Created Cloud Control Plane Companies create their own API and Web UI-driven control planes as proxies for their cloud provider, their SaaS provider, and any internal providers that must be communicated with as part of infrastructure provisioning and deployments.   Manages the company's cloud resources Categorize those resources for compliance and budget Coordinates unified deployments and configurations across other control ...