Blog de Joe Freeman

Azure PaaS services are an example of how the cloud has raised the raised the bar in the commodity platform space. More functionality is baked into the platform and less has to be built by software developers and enterprises. Some of the PaaS tenants are that it should be simple to scale-up and scale-out.  Networking should be simple and port exposure should be simple. Microsoft has at least 3 different Container services with different levels of PaaS-ness. They appear to be targeted more at Enterprise customers than the original SMB oriented PaaS services. It may be that the original ones just didn't support enterprise security, networking and other needs. Tech Comments The demonstration deploys a Tika Parser, Java Docker container running on each Azure Linux Docker PaaS services. Microsoft's move towards explicitly containerized PaaS services has both improved and degraded this model. Sample Scripts Use these scripts to experiment with Java Linux deployments on Azure ...

Agile can really burn people out with a never ending stream of sprints  or iterations .  One way to keep people fresh without breaking the team is to rotate different people though the lead roles on a Program Increment / Planning basis.  This gives people the opportunity to both reach  and recharge . Program Level We had had great success on a project over a 2 year period where the tech lead / for the Program Incremented rotated between three people.  This meant they could each do technical work 2/3 of the time while driving a PI the other 1/3.  It also made it possible for them to do some pre-planning while not trying to manage the current PI. Team Level This can also be done on a per sprint basis for intra-team responsibilities.  Story Grooming or product/technical preparation roles can be rotated every sprint. Video ...

What does it mean to move from a Relational Database style Schema on Read to  Schema on Write ? Schema on Write  is used to stage data in a consumer friendly form.  It can also be used in poor-join-performance environments to restructure and stage data in consumer read  format. It is pretty much mandatory for Document Databases.  Ingestion stores data in its original format for compliance, audit or other purposes.  This copy may be called True Source. Format Standardization converts the raw information into and agreed on standard format.  Examples include  Data Tables  in a lake or documents in a document store.  This is purely a mechanical conversion. Consumption Model are built from raw data, reference data and applies view and business rules creating a consumer ready dataset...

Data Lakes often have the some of the same performance and security decisions as past year's data warehouses.   Teams need to decide if the data in a lake is stored in producer formats or consumer formats or a combination of the two.  Storage is essentially unlimited which means we may choose to store the data in multiple consumer oriented fashion. Compute is essentially unlimited. We may decide to apply view style restrictions and access controls at read time.              Video Speaker's Notes This discussion is really only about tabular style data stored in cloud blob/object stores.  See data lakes for squares.  Record oriented data can be built up from fil...

Data lakes are for all types of data but sometimes we treat parts of our data lake as data warehouses. Fine grained access controls can be used to provide view like functionality where we can filter out columns or rows based on access rules. Fine grained access controls are implemented on top of cloud object/blob store. They are only really implemented two and half ways. Recording Row and column access controls applied to cloud blob data <Speakers notes to be added>

Your program or plan does not exist in a meaningful fashion unless it is Broadly Communicated and  Well Understood Video Presentation Content

I worked an a large company where many of our critical teams were One Deep .  There was basically one senior person or manager that really understood how that team operated. They had backup and other people but in the end the right decision or action only took place if that person was involved. The company struggled to thrive or thrive or maintain early mover advantages without them. Organizational health requires continuous attention and adjustment. Skills and responsibility depth can be an important metric. Teams need to continually groom people to move into gaps in the existing organization or gaps resulting from individual staff changes. It is easy to see how this happens with small teams, 3-10, but you can seen it with groups as large as 30-40 people. Recognition How many people understand the process? How much impact would a 6 week sabbatical have? How many people take the middle of the night call? Do many decisions depend on having one particular person? How curre...

PaaS services fit in that world in between serverless and roll-your-own infrustructure.  AWS Elastic Beanstalk is a PaaS service exposes their their service usage to teach you the right way to deploy web applications. The EB team eats their own AWS dogfood and generates a full featured environment via a generated CloudFormation template that you can examine and leverage. For this example we will deploy Apache Tika document parser as a service. We'll leverage a previous blog article PaaS Document Parsing With Tika and EBS for the instructions.  A shortened version is below Topology Discussion This diagram shows the components deployed to make an autoscaling web application that leverages Elastic Beanstalk , IAM, ALBs and other AWS goodness. Resources and Services Elastic Beanstalk uses these services to provision and deploy a simple no-database web application like the Apache Tika Parser.  We use the CLI to upload the configuration which Elastic Beanstalk the...

What should customers hear when you give a long explanation in response a question that can be answered with a yes or no?  I believe that they are usually hearing  no  when they get a long explanation . Short answers usually mean the responder understands the question and is comfortable with the answer.  Long answers are a way of applying conditions to the response.  Long answers are often an attempt to steer the question in a different direction.   By yes  I mean a positive outcome and not necessarily the word yes. Ask Questions to Drive the Conversation Ask questions until you can give a short answer as long as you don't drag it out or frustrate the other party. I worked with a great sales representative that always seemed dense in meetings.  He asked questions that made him look bad.  Later he told me that it gave the customer the opportunity to in struct and flesh out their needs. More than sales situations Asking expl...

The cost and scale of cloud storage makes it possible to store large amounts of data in almost any format.  We can build abstractions on top of this simple cheap highly available storage that lets us access it as if it were something more sophisticated. Let's divide the data into two coarse types,  unstructured  and  structured . Unstructured data, in this case, is any data where the content is un-typed, is not easily machine parse-able or does not store data with a tight schema.  Sound, media, pictures and PDF documents are examples of unstructured data for this discussion. Structured data has a deterministic content format and if often designed for machine consumption.  Let us break structured into columnar-record and non-columnar record formats.  Most columnar record formats are designed for many records per data object.  Non-columnar formats may be single record per file or may not be easily flatten able into record format. We ill...