Blog de Joe Freeman

Diagrams are a great way to communicate because most people think visually. Most diagraming tools have some type of diagram file type, vsd, vsdx, dwg, emf and some type of exported graphics format used in documents, png, jpg, gif. This means there are two files for every drawing, the drawing itself and the exported image. Dual file storage and synchronization is eliminated with drawing-embedded-png files. Some drawing tools draw.io have the ability to export an image with the drawing instructions embedded as metadata in the image. This means the PNG is both an image and a drawing. You only need to maintain one file in your document, your wiki pages, google drive, SharePoint or in your source code repository. This video walks through how to use this create and load editable PNG drawings in draw.io Saving the Diagram as a PNG with embedded markup Launch draw.io  Ccreate a diagram. Select File --> Export the diagram as png. Set a border width other than 0.  Bad image! M...

We often want to stream business events or raw logs from our applications to an analytical or operational data stores.  Kafka is currently one of the streaming APIs/platforms of choice for this.   Java applications can use their standard logging APIs and send the logs to Kafka via the Kafka Appender that can be attached to the log4j2 subsystem. We can make just a few tweaks to this and use the same logging mechanisms and Kafka platform to capture custom Business Events. Example code is availabe  https://github.com/freemansoft/spring-boot-log-to-kafka-example , It builds on previous work.   The sample code demonstrates using the logging subsystem to route logs and Audit to different destinations based on their severity or associated tags. The example send logs to the console and to a Kafka logs topics.  It sends tagged log messages to the Kafka Audit . topic.  All of this happens with the application code only ...

I migrated a repository from SourceForge to GitHub and wanted to see how many downloads were being done from GitHub.  SourceForge had a great UI for this.  GitHub does not. GitHub has a REST API that you have to script against that provides information about releases and artifacts. Need I wanted to find out the number of downloads for a given release in a specific repository. GitHub API  GitHub API docs can be found here:  https://developer.github.com/v3/   There is also v1, v2 APIs that in some cases are more different than evolutionary. Releases The releases of  portion of their API  https://developer.github.com/v3/repos/releases/  . There is a bunch of stuff to parse through. All releases for a given repository Releases Documentation This API will show you show you all the releases for a given repository.. https://api.github.com/repos/:owner/:repo/releases Example  Owner: freemansoft Repository: jacob-project https://api.github.com/repos/f...

Conway's law says systems, software, and wetware, end up shaped like the organizational structure that designed them or that they are designed for.   Organizations understand this at some level. Much of the corporate re-organization efforts are fed by people trying to change the corporate structure to push closer together the people creating a new paradigm or system.  The other main approach is to live with the partitioning that aligns with the organizational structure allowing any global redundancy or other inefficiencies. Video Images used in Video Conway's law states that software systems will be designed and built with the same alignment and partitioning as the organization that built created the software.  Teams that are closer together organizationally will build software that works better together. Teams that are farther apart will build software more loosely coupled or more error-prone.   We see the same behavior for non-software ecosystems.  C...

Kafka provides all kinds of metrics that can be used for operational support and tuning work.  We can use Telegraf/Jolokia to capture metrics from the various Confluent broker nodes and put those metrics into an Influx DB. We can then create Grafana dashboards for those metrics using Grafana Example Topology Deployed Components We run a custom Kafka Broker Docker Image built on top of the confluent/cp-server image. That image just adds the Jolokia JVM Agent jar file.   Run a docker-compose.yml to start Kafka that enables Jolokia as a Java Agent The agent URL for demontration purposes The Telegraf standalone agent conf enables the Jolokia2 input adapter.  The configuration file can be built into a new image or the configuration file can be mounted inside the standard DockerHub Telegraph image. Telegraf runs with this new configuration file. Telegraf retrieves the data from the Jolokia REST endpoints and sends it to InfluxDB Grafana provides visualization.  It u...

One of Azure's big initial innovations was their PaaS platforms.  These services were a level up  from most of Amazon's offerings. Microsoft appears to be in the process of killing of their traditional offerings. They are instead exposing the containerized underpinnings and forcing customers to do their own packaging and Docker image construction.  This makes sense for large enterprises but means that teams need to understand the underlying O/S software modules and system components that may need to be installed in their containers. PaaS PaaS makes it easier for less sophisticated teams to deploy sophisticated scalable applications. Azure PaaS lets customers deploy applications without worrying about the Operating System, System Patching, CVE security scanning , load balancing, log aggregation, or other issues.  The opacity of the underlying system meant teams had to know less and that teams couldn't know more if they wanted too. Microsoft modernized much of their Pa...

The public cloud has its' own issues around access, security and the patterns they use. You don't realize how minor they are until you move back to a data center or leased private cloud.  Sample Cloud Provider services vs Storage and Compute in data center Moving from a Public Cloud to a Data Center is like... Flying commuter prop plane for a long trip Most of your luggage got left on the tarmac because it weighed too much. Everything takes longer and involves a lot more noise. Moving back into your parents place You are forced to squeeze into the clothes you wore in high school and have to sleep in the skinny bed. You have a curfew.  Having a laptop with no battery You have no flexibility in how or where you work with all the constraints of a laptop screen. Moving from a 65" 4K to 25" 720k It has no remote so you have to get up and change the channel at the set.  Being banished to sleep in the shed out back It sort of feels like home. There is no air conditioning and...

Sometimes you have a situation where you want to push multiple tags when you push a Docker Image to Docker Hub using the Docker Hub build automation.   We will override the default behavior to do this. Video links related to this blog are available down below. Default behavior The default behavior is to build and push every time  there is a change on master.  That auto-built  image is always tagged with :latest and pushed to the repository as :latest Standard default build on master pushed with tag latest Docker Hub supports overriding its build and deploy behavior with a set of hooks. Custom build files and build hooks are stored in  /hooks  of the code repository.  Hooks are described  https://docs.docker.com/docker-hub/builds/advanced/  .   You can see an example here  https://github.com/freemansoft/cp-server-jolokia Pushing Multiple Tags to Docker Hub This is the approach we will take The following variables are avail...

Companies create Enterprise IT initiatives to support moving into new businesses, when adjusting to new regulation and audit rules, or when they want to merge systems across acquired companies or divisions. These multi-year efforts can involve hundreds of people, dozens or hundreds of cooperating systems, and millions of lines of code or configuration.  The public perception of these projects and how they actually operate can be very different.  Major software projects or IT initiatives often have more complexity than anticipated.  They can be under-sized to get funding. The project's goals almost aways change between the enthusiasm phase and the implementation  phase.  Scope changes during the project as more is learned or the market or requirements change. Planning often includes optimistic assumptions about the target state complexity or how fast the project can move. Senior management does not often have true visibility into the current state, the current go...

IT Enterprise Architecture acts as a forward-looking, unifying agent across teams and disciplines. The architects generally own the company's future direction and target state.  EA operates as one of the few groups that see all facets of a company's software development, IT Operations, governance, security, best practices, 3rd party integrations, and more. This cross-cutting responsibility can often put EA in a position where they are pushing future states or constraints that make software development and business process automation even more complicated.  Enterprise Architecture can end up being viewed as pushing solutions driven by NFRs that deliver and business may only be tangentially aware of.  They often deliver designs and mandate approaches that are based on technology and platforms that do not yet exist in the enterprise. This can cause delivery and business to view architecture as an impediment to progress.  EA are the ones th...