Create Innovation Zones Outside your Controlled Environments

Organizations implement controls to reduce risk, improve reliability, protect data and meet compliance objectives.  Formalizing processes and implementing controls often reduces the agility of an organization and makes it harder to innovate or experiment. 

There is a constant tension between protecting the enterprise and innovation.  Note that we are talking about innovation and not malice where people cut corners to make dates or make their lives easier.

We can reduce the level of control or create special places where people can experiment and innovate.  We need to do it in a way that work done there doesn't bleed into the controlled systems and data.

I worked at a place where we wanted to try a cloud service based database. We had no schema and just a Proof of Concept idea of what we wanted. It took 6 weeks of paperwork and several iterations of possible schemas to get onboarded and get access to the database for the PoC.  We knew the approved schema was wrong because we intended on expanding the use case as we learned what was possible.  This means 90% of the delay would have to be paid a second time if we decided to go forward.

Video

This 10 minute video probably contains more detail than the text in this blog post.

What do we mean by innovation?

There are all types of innovation.  In my IT experience, we tend to innovate on two axes.

Process Innovation: This is where we use new tools or technology or apply new processes.  We need access to places where we can exercise those process differences.  

Daa Innovation: This is where we experiment with data.  We get new data maybe from partners or other sources.  We try and use data we have differently. We combine existing and alternative data in new ways. We take that data and reshape it into features to use as part of ML training.

Our IT development teams may try new cloud services or work with new tools that weren't possible in the past. We might create data ponds or lakes to collect data that couldn't be used in previous system generations.

An innovation zone

Organizations should consider creating sandox or innovation zones when they see that they have problems innovating. Production restrictions often extend deep into the deployment life cycle.  Government compliance restrictions probably do not need to extend down into the most base development zones.  There is no segregation of duties requirements in nonprod places where there is no customer data. 

Innovation spaces may be your only option if you can't fix that type of problem.

• There is no place for people to fail
• There is no place for people to gain new skills
• Experimentation timelines feel like production timelines
• A full design is required when you ask to play with something new

IT in the cloud can do this by creating specialized areas that exist outside of production with no fiduciary responsibilities. 

IT Innovation Zones

Two standard SDLC / IT zones are described at the top of this table.  They represent the status quo that attempts to protect the company.

The two zones at the bottom of the table describe places where we can explore that are outside some of the standard controls.

The data exploration zone must still implement access control policies.  The data is still production data even if we are letting humans manipulate it.

Innovation zones create new problems

New tools and techniques may be absorbed into the regular processes.  They may be partially absorbed bringing in some of the technology and techniques while requiring modification to other pieces to bring them to corporate standard.

• There may be no way to bring results into production because the tools may never be approved due to security, cost, or other constraints.
• Some people will deliberately use innovation zone to force bad practices into production. They will wait until late in the process and then demand exclusions to make their dates. 
• Production guardians must be empowered to reshape the system as it was promoted from innovation into the standard SDLC.
• Data innovation lineage may be hard to reconstruct.  Data scientists may create amazing ML features that can't be audited or recreated.  Companies have to decide if they will accept experimental data as production and how ownership and lineage will be handled.
• Success in innovation may not mean success in production.  Prototyping and experimentation may use bad data or bad practices or tooling that can't be supported in production.  
• And others...

Created 9/2021