post loop
2020 Fall Online Classes at Lonestar College
Programming Fundamentals II (COSC1437)
This course focuses on the object-oriented programming paradigm, emphasizing the definition and use of classes along with fundamentals of object-oriented design. The course includes basic analysis of algorithms, searching and sorting techniques, and an introduction to software engineering processes. Students will apply techniques for testing and debugging software.
Contact me at Mark.E.Reynolds@Lonestar.edu for additional information. Or visit my college blog at http://lonestar.edu/blogs/markreynolds.
2019 Spring Online Classes at Lonestar College
Programming Fundamentals II (COSC1337)
This course focuses on the object-oriented programming paradigm, emphasizing the definition and use of classes along with fundamentals of object-oriented design. The course includes basic analysis of algorithms, searching and sorting techniques, and an introduction to software engineering processes. Students will apply techniques for testing and debugging software.
Contact me at Mark.E.Reynolds@Lonestar.edu for additional information. Or visit my college blog at http://lonestar.edu/blogs/markreynolds.
The Discipline of IIoT Engineering
IIoT vs ML vs AI vs Big Data vs Edge Computing
The Industrial Internet of Things (IIoT) is not synonym for Machine Learning (ML). Neither is it a synonym for Big Data. Neither is it a synonym for Artificial Intelligence (AI).
Generally, Machine Learning requires large amounts of data, aka Big Data. Machine Learning may exist within IIoT and utilize edge devices. But ML and AI operate on data, not devices. In fact, ML and AI should be utilizing data which has been prepared, cleaned, and verified – not on data directly out of a sensor.
The scope of the IIoT industry clearly includes Big Data, ML, and AI. But IIoT enables ML and AI to assess a task, utilizes the results from ML and AI to optimize a task, but does not necessarily require ML and AI to accomplish a task.
Frequently, interconnecting IIoT edge-point devices via networks is often grouped into the narrower genre of edge computing. However, edge computer requires computational capabilities. IIoT edge-point devices do not necessarily require edge computing. Certainly in this age to digitization and digital transformation, these distinctions are somewhat immaterial. But the distinction does exist.
IIoT edge-point in isolation is no longer relevant. As the lines of demarcation blur, we ask ourselves:
- Why would a project implement a sensor unless that sensor is producing data that should be monitored and recorded?
- If sensor devices are to be monitored and recorded, why would that project not utilize automated and autonomous acquisition, assessment, and archival?
- Once acquired, shouldn’t a project utilize that data to affect other processes, alert when anomalous situations exist, and be utilized in some way as a forecasting input?
[As an aside, the concept of the Internet of Things carries the preconception of consumer devices. The Industrial Internet of Things add clarity that the ‘things’ are industrial components, the internet is industrially focused, and the project is industrial in nature. I exclude the broader categories of IoT. Communications between the toaster and refrigerator are not, directly, relevant to the Industrial Internet of Things. “The IIoT is different from other IoT applications in that it focuses on connecting machines and devices in industries such as oil and gas, power utilities and healthcare.” (1)]
Relevance to Industry
The relevant point to be made is that IIoT , Edge Computing, ML, and AI are closely linked and often integrated. Nevertheless, disciplines focusing on IIoT, Edge Computing, and Machine Learning require different skill-sets for the practitioners, different design methodology for integration, and different back-end infrastructure.
Engineering Skill-sets Required for IIoT Projects
Industrial Internet of Things is, of necessity, comprised of end-point devices such as sensors, annunciators, actuators, and terminals. As a practical nature, IIoT also includes edge power, cabling, networks, and mobility. These skill are more decidedly focused toward the electrical engineer than other individual engineering disciplines (chemical, mechanical, etc.).
An electrical engineer must understand the capabilities of the individual devices, installation requirements of the devices, the environmental and mechanical limits of the devices, and the installation requirements of the devices. Additionally, maintainability, serviceability, reliability, and availability are key concerns.
[Perhaps I’m partial to my chosen discipline – Electrical Engineering. I make no apologies.]
The IIoT engineer should be fluent with the entire specification of the devices and components, able to design an integration of devices, and savvy in computers, networks, and security.
As a checklist, the IIoT Engineer should be demonstrably proficient in
- Sensors, sensor technology, sensor specifications, and sensor integration
- User interfaces – mobile devices, operations centers, and field annunciators
- Install-ability, Serviceability, Maintainability, Reliability, and Security
- Units of Measure, Calibration, Sampling Rates, Filtering (hardware filters and software filters)
- Data organization, storage, retrieval, and flow
This proficiency cannot exist in isolation from skills in ML, AI, Big Data, and Analytics. Finally, the IIoT must be conversant with the universe in which the IIoT must exist – especially the nature of the operation, the concerns of the field personnel, and the vernacular of the operation.
Design Methodology Required for IIoT Projects
An IIoT project is rarely completed. The first phase may come to an end, but one phase is rarely completed before the next phase begins. A design methodology is crucial.
The methodology chosen for an IIoT project should address
- First and foremost: What is the objective? Why is an organization undertaking and IIoT project of any scale and complexity? When completed, what benefit will be shown to the executive leadership?
- Don’t eat the entire elephant at one sitting. Identify specific near-term, small, demonstrable, and achievable milestones. Early success will ensure the embrace of leadership. However, nonsensical demonstrations of remote control lights and locks will equally ensure the alienation of necessary backers.
- Ensure the IIoT components are robust. In the extreme, installation or wire-wrapped demonstrator devices will be prone to fail – at the wrong time. Implement individual IIoT points, sensors, actuators, processors, gateways, etc with the care to be given productionized solutions. Particularly appropriate is the environmental fitness of the installation (explosive environment, extreme temperature, caustic chemicals)
- Address signal quality. Any electrical engineer will know that the signal to noise ratio must be of sufficient distinction to provide credible data (often an SN ratio of 3 or more is recommended). Additionally, ensure point source filtering is implemented to prevent spurious data, but not so filtered to prevent outliers.
- Consider the end user. Great ideas have failed no because of the concept, devices, quality and robustness of the design, but because the user’s ability to consider the new information source is ill-conceived. Too much data and not enough information will sink a project. If the question is “have you been watching the display?”, then the project has failed.
- What behavior change is expected of the user or the system based on the IIoT project?
- Is the solution robust, maintainable, serviceable, and autonomous?
Beyond the premier consideration of these concepts, the design methodology must iterate and expand well.
Infrastructure Required for IIoT Projects
IIoT infrastructure will cross most of the organization. Specifically:
- Field personnel to install, service, and calibrate the system.
- Communication reliable and resilient including the ability to back-fill data gaps after communication outage.
- Data retention including short-term raw volume, long-term decimated volume, behavior modelling and thumbprint, and KPI alerts.
- Streaming data processing in 4 modalities: real-time monitoring, after-action or forensic look-back analysis, pre-action or workflow planning analysis, and large-field analytics and machine learning.
- Geographic dispersion of IIoT devices, users, and systems.
Conclusion
The Discipline of IIoT Engineering is the Systems and Knowledge Engineer. This individual or functional group must be a rare mix of engineer, data engineer, integration engineer, and user interface engineer. IIoT engineering will provide the project management, architectural design, and integration skills. IIoT engineering must be conversant with the industrial subject matter experts (SME), vendors, executives, developers, and technicians.
References
(1) What is the Industrial IoT? [And why the stakes are so high], Jon Gold, Senior Writer, Network World, February 2, 2018
https://www.networkworld.com/article/3243928/internet-of-things/what-is-the-industrial-iot-and-why-the-stakes-are-so-high.html
The Journey into DevOPS as a Mindset, not a Project
DevOps as a project is a failed premise since projects complete, DevOps is ongoing, ever improving.
DevOps as a continuous improvement project is a non sequitur since the Agile / DevOps process cannot evolve from a waterfall mindset.
DevOps as a journey is inconclusive since a journey, similar to a project, has a destination. Although a journey may be construed as an experiencetial trek with an end rather than an optimized path to a goal.
DevOps as a mindset, a culture is the most illustrative since DevOps embraces the experience of not only the ascent to the summit, but also the triumph of performance.
DevOps is the most efficient and least problematic software development approach available today. DevOps may not be the best approach, but it certainly beasts the alternatives – especially the historical alternatives.
This blog attempts to chart the experiencetial journey from constrained waterfall implementation to modern DevOps mindset by exploring the journey and recognizing objectives.
Disclaimer: Waterfall is a phase, Agile is a phase, DevOps is a phase. After DevOps, the new phase beginning to gain foothold is the Lean phase. This blog willfully overlooks the Lean phase – for now.
Before Agile and DevOps
Before Agile and DevOps – Waterfall: scope, research, analyze, plan, execute, evaluate, test, deploy, assess. Waterfall produces the expected results: operations tell each other “if you want to get the project done, keep IT out of it.” I have heard this multiple times. IT, through waterfall, has earned that reputation. I regret my complacency.
MIL-STD490
I worked in the military-industrial complex in the 1980s. I was fortunate – fortunate to work as an Avionics Systems Engineer for General Dynamics on the F-16 (Lockheed Martin is now the F-16 prime contractor) and for Sikorsky Aircraft on the Seahawk Helicopter.
DevOps was not even a remote thought. Neither was Agile. In fact task estimates were based on the number of words of code (in 32K or 64K machines). Software definition and product management was defined in the MIL-STD490A specification (1). A functionality changed involved changes to a thick B-5 Software Development Specification, a Preliminary Design Freeze (PDF), a Critical Design Freeze (CDR), Development, Testing (ground testing and flight testing), and finally a C-5 Software Product Specification — in that order. To call it waterfall was being disparaging to waterfall projects. This MIL-STD490A specification is the predecessor to the Capability Maturity Mode.
Capability Maturity Model
My first encounter with the Capability Maturity Model (CMM) was during my Software Project Management class, part of my Master or Engineering degree. This business practice model was based on nearly unattainable goals of becoming efficient through regiment and control (an oxymoron even when attained). The 5 phases of the CMM are: (2)
- Initial (chaotic, ad hoc, individual heroics) – the starting point for use of a new or undocumented repeat process.
- Repeatable – the process is at least documented sufficiently such that repeating the same steps may be attempted.
- Defined – the process is defined/confirmed as a standard business process.
- Capable – the process is quantitatively managed in accordance with agreed-upon metrics.
- Efficient – process management includes deliberate process optimization/improvement.
Then along came Sarbanes-Oxley Act (3). Now, effectively, the CMM Level 4 model was thrust upon all public-stock companies.
Application Implementation Methodology
The Oracle Corporation compounded inefficiencies by providing commercial and industrial entities a new iteration of the B-5 and C-5 specifications – the Application Implementation Methodology (AIM) (4). AIM even came with its own specification templates: the MD-50 and MD-70 specifications (5):
MD50:
This document specifies various functional specification required for Techies to understand the functional requirement of Form or Report Customization and design.. This is prepared by Functional consultant to clarify the spec to Techies. For any component like Report, Interface etc we have to first get the requirements as to what is the source , destinations, what data should be moved and what functional validations would be required etc. All this would go into the MD050. Generally written by a non-technical people like Business Analysts or Functional Consultants.
MD70:
This is technical specification prepared by the programmer based on MD50. This provides tech specs, PLSQL code for the Report and Form and helps to understand and modify if required in future. MD070 is the technical document that is written to fulfill the functional requirement specified in MD050. It includes the approaches you take, pseudo code, validations, Data Sources, SQL Statements etc.
The AIM methodology was a cash-cow for consultants. AIM justified many unread documents, forgotten meetings, and development extensions. In truth, there were dozens of documents designed to keep the consultants employed.
Agile and DevOps Development Practices
Before Agile, the developer team took requirements, disappeared a few months, and re-emerged with a creatioon that, while technically correct, seldom achieved the need or expectations of the users and operators.
Agile and DevOps are not synonyms. In fact, Agile is a precursor to DevOps.
Agile Development Methodology
The Agile Manifesto has been the rallying cry for high quality, high performing software developers for more than a decade. The concept is simple: (6)
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
So simple, but entire organizations have materialized to train, coach, administer these principals. Large scale projects have been simplified and deliveries have been achieved as a result of these principals. When asked his biggest regret, the president of a large software development firm said “that I did not implement agile sooner”.
A concluding note on Agile: the project manager must embrace the MVP approach – the Minimum Viable Product. Each sprint produces a product – a viable, if not complete, product.
DevOps
In a recent Forbes article (7)
DevOps is the result of a growing need for frequent releases and disruption in the market dynamics. At the core of DevOps is continuous development and delivery. Faster time to market, successful and frequent releases, shorter lead time and steadfast recovery are some of the compelling features of DevOps, along with the ability to segment projects into fragments providing overall project visibility.
The DevOps Cycle extends the narrow Agile fixed-length scrum cycle to encompass faster time to deployment without surrendering the benefits of Agile.
But there is no manifesto for DevOps.There is no manifesto because (8):
DevOps begins with stopping the nightmare scenario pitting Ops against Dev—or any other part of the organization. But it comes so much further. It’s for a new age of computing at webscale, using tools, teamwork and techniques to get to better, happier, more engaged outcomes for customers. Some of it is so new we are still figuring out words for it.
Amazon Web Services offers a different view of DevOps (9):
Either illustration works when trying to understand DevOps. Reading further into the Amazon articled:
Software and the Internet have transformed the world and its industries, from shopping to entertainment to banking. Software no longer merely supports a business; rather it becomes an integral component of every part of a business. Companies interact with their customers through software delivered as online services or applications and on all sorts of devices. They also use software to increase operational efficiencies by transforming every part of the value chain, such as logistics, communications, and operations. In a similar way that physical goods companies transformed how they design, build, and deliver products using industrial automation throughout the 20th century, companies in today’s world must transform how they build and deliver software.
In summary, Agile is a job foreman, DevOps is an orchestra conductor. (10)
The 2017 State of DevOps Report identifies the advantages to DevOps (11). These advantages as summarized by the Neudesic presentation at the DevOps 2.0 seminar (12):
Faster Delivery
High Performing teams have 48 times lower MTTR and 3 times lower change failure rate.
Safer Delivery
High performing teams spend half as much time on rework and three times as much time on new work.
More Efficient
High performing teams have a 50% reduction in security related incidents.
Improved Satisfaction
Teams adopting CD / DevOps doubled their internal net promoter score and had tripled their customer net promoter score.
Increased Profitability
Organizations practicing CD / DevOps are 26% more profitable than traditional divisions of labor.
Conclusion
The DevOps mindset, culture, process represents the most mature improvement in development productivity available. DevOps is the progression of the waterfall mindset, to the agile mindset, into the process-centric DevOps mindset. The lean mindset is on the horizon and is practiced by some, but the DevOps mindset remains the goal for many development shops.
References
(1) MIL-STD490A Military Standard Specification Practices
https://www.product-lifecycle-management.com/download/MIL-STD-490A.pdf
(2) Capability Maturity Model
https://en.wikipedia.org/wiki/Capability_Maturity_Model
(3) Sarbanes-Oxley Act 2002
https://en.wikipedia.org/wiki/Sarbanes%E2%80%93Oxley_Act
(4) MD50, MD70 and MD120 Template
http://oracleappsdna.com/2014/12/md50-md70-and-md120-templates-free-download/
(5) Oracle List of AIM Documents
http://www.oracleerpappsguide.com/2011/03/list-of-aim-documents.html
(6) The Manifesto for Agile Software Development
http://agilemanifesto.org/
(7) Forbes, How To Implement A Successful DevOps Roadmap, October 5, 2017
https://www.forbes.com/sites/forbestechcouncil/2017/10/05/how-to-implement-a-successful-devops-roadmap/#268d664965bd
(8) Why Is There No DevOps Manifesto?, Christopher Little on May 6, 2016
https://devops.com/no-devops-manifesto/
(9) What is DevOps?
https://aws.amazon.com/devops/what-is-devops/
(10) Mark Reynolds
(11) 2017 State of DevOps Report
https://www.ipexpoeurope.com/content/download/10069/143970/file/2017-state-of-devops-report.pdf
(12) DevOps 2.0 – Beyond the Buzzword, Neudesic presentation May 9, 2017
The Data Driven Mindset
Just what is Data Driven? Or for that matter, what is a Data Driven Mindset? Why does it matter to identify Data Driven concepts? Starting from the basics, Data Driven “means that progress in an activity is compelled by data, rather than by intuition or by personal experience. Data–driven may refer to: Data–driven programming, … Data-driven testing, … Data-driven learning, … Data-driven science, … Data-driven control systems, …“(1)If only it were that simple. We could all be mind-numbed robots acting on data without regard to the end-prize.
Data Driven Concepts Explained
While researching this topic, I found a concise explanation for Data Driven by Techopedia:(2)
Being data driven means that all decisions and processes are dictated by the data. If data points to sales being down because of brand perception, then specific actions can be taken to reverse that. If data analysis reveals that users of a current generation of mobile device are leaning toward a specific feature, then the next-generation device can make use of that knowledge.
Data driven essentially means that data dictates the actions taken by the ones that execute an event or process. This is most evident in the field of big data, where data and information are the basis of all actions and gathering and analyzing of data is the core motivator. Because data is now easier to gather and inexpensive to store, big data analytics is gaining more ground as the best tool for decision making in the business world. Having so much data gives powerful insight into the world and it allows people to manipulate outcomes because of this.
Such is Data Driven in the extreme. But shouldn’t Data Driven be more tempered with the contributions of experts (Subject Matter Experts – SME) and the front-line humans most in-tune with the life of the business? Data Driven mindsets create spreadsheets and presentations which are then re-summarized until the essence of the operation is lost.
O’Reilly provided a great excerpt from the book Creating a Data-Driven Organization by Carl Anderson: “Data-Drivenness [sic] is about building tools, abilities, and, most crucially, a culture that acts on data.”(3)
Well said. I encourage you to read the entire article.
The Mindset
As with any mindset, the mindset of being Data Driven is built from the underlying business purpose, vision, and strategy.
Mindsets matter.(4)
Your mindset plays a critical role in how you cope with life’s challenges. In school, a growth mindset can contribute to greater achievement and increased effort. When facing a problem such as trying to find a new job, people with growth mindsets show greater resilience. They are more likely to persevere in the face of setbacks while those with fixed mindsets are more liable to give up.
Many companies promote their “safety mindset”. Ask many of these executives and they will tell you that safety is not “job #1”, nor is it “top priority”. Safety is what we do, who we are. If it isn’t safe, we do not proceed.
Often we hear “customer first mindset” or even “client driven”. These mindset, again, are not a priority, a task, or even a mandate. The mindset of being customer or client driven is the way a business chooses to operate.
So a mindset is an attitude, an ethos, a way of thinking. A mindset is the en grained thought pattern and process from the board room to the front line.
How Then Should We Proceed?
Getting Started
It seems like a obvious understatement, but data collection must precede data utilization. A more delineated prerequisites would be:
- Data Collection – data from terminals, IoT end-point devices, analysis logs, contextual and environmental reference, etc. Note: regardless of the source of data, that data source defines the available options toward using that data.
- Data Qualification – data that is noisy is useless. Data must be referentially relevant (especially Geo-temporally referenced), calibrated and of the same units of measure, and expungable of irrelevant data (RPM of a machine at rest is not noisy, but the fact that the machine is at rest must be available or derivable). Note: outliers in a data set may be excluded and ignored – except when they cannot be so expunged.
- Data Availability – Data must be accessible, queryable, and traceable. Too often data is stored in inaccessible blobs, held inaccessible due to governance, or unverifiable as to providence. Note: When in doubt, strive for tabular data and avoid unstructured data. Unstructured data invariably becomes structure through forethought, or artificial thought.
The sanity check – are the various data sources joinable in a query? If they are, then they probable have a common reference key, common units of measure, connectable data sources, etc.
The Data Driven Mindset in Daily Life
As explained above, Data Driven is an interesting buzzword but is, in its purest form, overly restrictive. The Data Driven Mindset is not blind adherence to data induced mandates. The Data Driven Mindset is more a Data Informed Mindset. Data is the source is Information, Knowledge, Understanding, and Wisdom.(5)
Think
Andrew Chen raises interesting points in his blog Data Informed versus Data Driven.(6) I have summarized and paraphrased his conclusions here. I encourage you to read the entire article.
- Metrics are merely a reflection of a preexisting the product or business strategy
- The task is a messy problem that may have multiple acceptable answers and be disguised with poor data
- Data is inherently biased
Build models that are based on the source, quality, and reliability of the underlying data. Build models that focus on business outcomes, not academically curious nuances. (Let the data scientist daydream about the nuances.)
Think
Build solutions and optimizations that matter. Before beginning, think. Think about the objective, the scope, the success metrics. After building these solutions and optimizations, think again. Optimization is not about achieving the most efficient, productive, or elegant solution. Optimization is more often about finding the most achievable behavior.
Think informatively. I call this Digital Transformation. Others call it Data Inquisitvity.
“A data-driven mindset means you think outside of yourself and your personal biases, and instead, you focus on what the evidence says. You may still trust your gut, but you don’t act blindly. You challenge your assumptions, collect as much data as you can, and then take action.”(7)
Conclusion
The Data Driven Mindset (Data Informed Mindset?) is a way of thinking that becomes pervasive within the focus of the individual and organization. Data queries, data charts, and data-backed presentations are not the objectives, just the end products. The Data Driven Mindset is a systems approach – data sources, data transport, data availability, and data analysis.
References
(1) Wikipedia entry for Data Driven
https://en.wikipedia.org/wiki/Data-driven
(2) Techopedia entry for Data Driven
https://www.techopedia.com/definition/18687/data-driven
(3) Creating a Data Driven Organization; Carl Anderson; Chapter 1. What Do We Mean by Data-Driven?
https://www.safaribooksonline.com/library/view/creating-a-data-driven/9781491916902/ch01.html
(4) Why Your Mindset Really Matters, Cultivating a Growth Mindset Can Boost Success;
Kendra Cherry; March 21, 2018
https://www.verywellmind.com/what-is-a-mindset-2795025
(5) Data, Information, Knowledge, Understanding, Wisdom; Mark Reynolds
http://digitaltransformation.engineer/2011/01/31/the-data-information-hierarcy/
(6) Data Informed versus Data Driven; Andrew Chen
http://andrewchen.co/know-the-difference-between-data-informed-and-versus-data-driven/
(7) How to Show You’re Data-Driven (When You’re Secretly Afraid of Numbers)
https://www.joinkoru.com/resources/how-to-show-youre-data-driven-when-youre-secretly-afraid-of-numbers/
Three Catalysts of Digital Transformation
Spring Houston TechFest
Software developers are constantly challenged with staying current and mastering new technologies. Developers who do not continually strive to learn new technologies will soon find themselves no longer relevant. Some technologies radically impact the entire industry while others are more impacting the periphery.
Houston TechFest has been a central component to the Houston software development community since September 2007. Twice rescheduled due to hurricanes, this year a second event occurred – the Spring Houston TechFest. Michael Steinberg has nurtured this labor of love since the inception at the University of Houston. The 2018 Spring Houston TechFest was held last weekend, May 5th, at the San Jacinto College in Pasadena.
A shout-out to the sponsors: Improving Houston, Code Magazine, Jet Brains, EverLeap, and GrapeCity.
Additional credit to www.UserGroup.TV. Shawn Weisfeld (https://www.linkedin.com/in/shawnweisfeld/, http://about.me/shawnweisfeld) has his own labor of love recording sessions and posting them on his website. (Find my posted topics at http://usergroup.tv/?cat=667).
Digitization, Digitalization and Digital Transformation
[updated May 17, 2018]
Three terms I’ve seen recently used together, often interchangeably and frequently incorrectly are digitization, digitalization, and digital transformation. Given the name of this blog – Digital Transformation Engineer – I have an opinion.
Digitization, Digitalization, and Digital Transformation
Digitization
Digitization is the use of digital technologies to change a business model and provide new revenue and value-producing opportunities; it is the process of moving to a digital business.(1) I disagree. I submit that a more accurate definition of digitization may be the act of moving from analog to digital processes.(2)
Examples of digitization are everywhere. Many common examples include photography, entertainment, thermometers, even toasters.
Digitization is mostly complete. Once the toaster was digitized, the hype curve was complete.
Digitalization
I do not include cell phones, social media, or other pervasive connectivity because these came after digitization was already mature. Factories, automobiles, offices similarly represent the outgrowth of digitization – digitalization.
The digitization wave has preceded the intelligent application of connectivity. But digitization is the conversion of analog processes to digital processes. Digitalization is the utilization of digitized devices to provide inter-connectivity and presumably benefit.(3) Digitalization should change the business model and capitalize on direct knowledge and control. Borrowing from military terminology, digitalization defines the restructuring of domains to revolve around communication, command, and control.
Offices have sensors, devices, access, and other connected devices monitoring, controlling, and optimizing all aspects of the office environment. Automobiles have interconnected and controlled engines, frames, climate, navigation, entertainment, and other devices that were not even contemplated a dozen years ago. Factory systems have advanced far beyond the control loop into massive data generating, centrally controlled, integrated systems. Every industry has already seen the digitalization occur.
Digital Transformation
Digitalization is the necessary forerunner and precursor to the transformation of business and industry. Digital Transformation is the planned application of technology and technologies to achieve improved business models.(4) Digital Transformation is not the catalyst, it is not the end point. Digital Transformation is determination and commitment by the leadership of a company to move into the next level. Companies consciously embracing and integrating digitization and digitalization are those in which efficiencies and effectiveness are the charge.
Restated:
- Digitization: the act of moving from analog to digital processes.(1)
- Digitalization: the utilization of digitized devices to provide inter-connectivity and presumably benefit.(3)
- Digital Transformation: the planned application of technology and technologies to achieve improved business models.(4)
Digital Transformation is not a corporate vision, it is the underpinning of all future corporate visions. And it is advancing — rapidly.
Amazon’s Alexa is one example of how the advancement and application has been phenomenally rapid and pervasive. Thankfully robust APIs have permitted many 3rd parties to utilize Alexa without costly replication of design. While passing by an Alexa station, one can ask for a weather report, baseball score, bitcoin price, and current sales volume while watching your driveway camera.
Planned and Unplanned Digital Transformation
Digital Transformation may be the planned application of technology, but planning need not be deliberate and thoughtful. The old adage that “failure to plan is planning to fail” is true here as much as ever. Ad hoc Digital Transformation results in incompatibility and insufficiency. Digital Transformation when poorly planned will produce frustration (“Which remote do I use to play I Love Lucy?”). Planned Digital Transformation will result in enhanced lifestyle (“Alexa, resume I Love Lucy on my television.). For simplicity, Digital Transformation may be segregated into planned and unplanned variations.
Unplanned Digital Transformation is uninteresting unless your enterprise is commercially focused on clean-up of the resulting failures.
So we digitize information, we digitalize processes and operations, and we engage in Digital Transformation of the business and its strategy.
“If your company has not been applying digital technologies over the last 40 years, you don’t exist.”(5)
Digital Transformation Engineering
Digital Transformation is an engineered process. When cultivated and allowed to grow, it will create completely new business concepts and frameworks. Digital Transformation is not an endpoint. As we move information systems from reactive systems to proactive systems; and proactive systems to cognitive systems, the role of the Digital Transformation Engineer takes shape. The next steps in Digital Transformation are already upon us. Analytics systems continue to mature; real-time IoT systems contribute increasing amounts of data; corporate and individual expectations drive advancements in Machine Learning (ML) and Augmented Intelligence (AI).
“Digital transformation is about more than digital products and services; it’s also about the processes that create, enable, manage, and deliver them.”(5)
Successful Digital Transformation projects require the Systems Engineer to drive vision, standards, implementation, and success. The Systems and Knowledge Engineer must keep his eye on the ball – all of them. Some of the more visible parts of the Digital Transformation include:
- Base Operations Systems
- Control Systems
- Remote Systems
- Information Systems
- Embedded Systems
- Robotic Systems
- Data Fusion
- Real-Time Systems
- Look-Back Systems
- Look-Ahead Systems
George Westerman, et. al., in their article The Nine Elements of Digital Transformation provide a great breakdown of the necessary vision and strategy of the Digital Transformation Engineer:(6)
- Transforming Customer [internal or external – mr] Experience
Element 1: Customer Understanding
Element 2: Top Line Growth
Element 3: Customer Touch Points - Transforming Operational Processes
Element 4: Process Digitization [or digitalization – mr]
Element 5: Worker Enablement
Element 6: Performance Management - Transforming Business Models
Element 7: Digitally Modified Business
Element 8: New Digital Business
Element 9: Digital Globalization
Examples
Without going into great detail, we see two examples of major Digital Transformation successes and failures today.
Netflix embraced and pushed the state of the art for the delivery of entertainment. Blockbuster focused not on the delivery of entertainment, but on the delivery of widgets.
Sears used to own the mail order business. For most of the 1900s, the Sears Catalog contained every item desired – including complete houses (in kit form), coffins, and clothing. Today, in 2018, Sears Holding Company is expected by many forecasters to turn bankrupt while Amazon may be the first publicly traded company with a market capitalization of $1 trillion – with a T.
Conclusion
Digital Transformation is the engineering behind Digitalization. Digital Transformation requires a Systems and Knowledge Engineer capable of functioning on different levels and in different systems while focusing on the basics: transforming customer experience, transforming operational processes, and transforming business models. Companies which do not plan, focus, and apply vision to the Digital Transformation task will fail to deliver, fail to be competitive, fail to be relevant.
References
(1) Gartner IT Glossary
https://www.gartner.com/it-glossary/digitalization/
(2) Mark Reynolds – definition of Digitization
(3) Mark Reynolds – definition of Digitalization
(4) Mark Reynolds – definition of Digital Transformation
(5) What digital transformation really means, Galen Gruman
https://www.infoworld.com/article/3080644/it-management/what-digital-transformation-really-means.html,
(6) The Nine Elements of Digital Transformation, George Westerman, Didier Bonnet and Andrew McAfee
https://sloanreview.mit.edu/article/the-nine-elements-of-digital-transformation/
This Digital Transformation Web Site and Blog
[updated May 17, 2018]
DigitalTransformation.Engineer
This www.DigitalTransformation.Engineer website and blog site is a personal creation; a personal labor-of-love project. Through this site and these blogs, I hope to shed a little light on the evolving industry – software, analytics, machine learning, and the digital oilfield.
The software, analytics, and digital oilfield landscape has changed in the past few years and I’ve been blessed to be a part of it.
Background
In 2010-2011, we were undertaking the 24-7 operations center. At that time, our vision was teams of operations engineers manually monitoring up to 4 active operations. We knew that value existed but the extraction of value was the unknown. The field operations did not need Big Brother but optimization and problem avoidance were low-hanging fruits. I know we were in the right spot because 3 different solutions providers told me that our company “was leading the land-based innovation in the United States.” That felt good.
I’ve witnessed an evolution over the next several years. Companies no longer considered whether to embrace 24-7 operations centers; today they consider and field operations to be invalid without the support. To be fair, not all E&P companies create their own facilities; many contract with suppliers (typically rig contractors). With this migration to ubiquitous operations centers, advancements in the capabilities were necessarily forthcoming. The industry moved from data, information, and knowledge into understanding – understanding as manifested through predictive analytics, after-event analytics, and variations such as case-based reasoning.
Recently (2015-present), prescriptive analytics, wide-area reservoir analogs, data mining, data science and machine learning are becoming the normal and serious E&P operations mandate these activities.
Digital Transformation
Digital Transformation is a new buzzword. But is very descriptive of the next step in evolution of the digital oilfield. “Digital transformation is the change associated with the application of digital technology in all aspects of human society. The transformation stage means that digital usages inherently enable new types of innovation and creativity in a particular domain, rather than simply enhance and support traditional methods.“(1)
George Westerman, principal research scientist with the MIT Sloan Initiative on the Digital Economy, is one of those who knows digital transformation when he sees it. “Digital transformation is when companies use technology to radically change the performance or reach of an enterprise,” says Westerman. The drivers tend to be disruption from market newcomers or innovation from rivals seizing the opportunity to win new customers. (2)
Digital Transformation vs Disruption
Digital Transformation is not, and cannot be, all about disruption. Disruption is chaotic and does not produce determined results. Transformation is the evolution of a process of thought. Transformation may be thought of as continuous improvement, but continuous improvement unencumbered by preexisting expectations. I’ve heard it said that continuous improvement of the candle never created the light bulb. But I submit that the light bulb did, in fact, result from the transformation of the candle into the next, logical step.
Digital Transformation is outside of the box application of evolution to good solid engineering principals. The transformation of sensors into end-point IIoT (Industrial Internet of Things) devices; the transformation of after-event forensics into pre-event assessment; the transformation of real-time look-behind into real-time look-ahead.
Why Digital Transformation Matters
Digital Transformation is a survival issue. Experienced digital emigrants (engineers and operations that know the subject intrinsically and have had to learn to embrace ubiquitous computers) are retiring and a phenomenal rate and are being replaced by a class of digital natives (those that simply expect mobility and pervasive computers and automation) who expect data to be available, processed, and providing answers before the question is asked.
According to Lumina Datamatics, “Every organization will have to embrace digital transformation at some level to remain competitive. There can be many manifestations of digital transformation, including using ubiquitous, real-time data and algorithms to processes flexible and autonomous workers. Modern processes and technologies help reduce operational overhead, enabling businesses to invest less in maintenance and more in innovation that supports rapid change.”(3)
Digital Transformation Framework
(4) Enterprise Project states “Although digital transformation will vary widely based on organizations’ specific challenges and demands, there are a few constants and common themes among existing case studies and published frameworks that all business and technology leaders should consider as they embark on digital transformation.
“For instance, these digital transformation elements are often cited:
- Customer experience
- Operational agility
- Culture and leadership
- Workforce enablement
- Digital technology integration”
Moving Digital Transformation Forward
Digital Transformation requires mandate from the top of the company hierarchy. Peter Dahlstrom, et. al. identify 7 crucial decisions which must come from the Executive Leadership Team and the Digital Transformation Team:
Decision 1: Where the business should go
Decision 2: Who will lead the effort
Decision 3: How to ‘sell’ the vision to key stakeholders
Decision 4: Where to position the firm within the digital ecosystem
Decision 5: How to decide during the transformation
Decision 6: How to allocate funds rapidly and dynamically
Decision 7: What to do when
Conclusion
Digital Transformation is here. Digital Transformation is the natural next step. Digital Transformation is not optional.
The Executive Leadership Team must set the tone and the mandate; the operations must embrace the mandate; and the Digital Transformation Engineer must be given charge. (More on the characteristics of the Digital Transformation Engineer later)
References
(1) Digital transformation – Wikipedia
https://en.wikipedia.org/wiki/Digital_transformation
(2) What is digital transformation? A necessary disruption
https://www.cio.com/article/3211428/digital-transformation/what-is-digital-transformation-a-necessary-disruption.html
(3) Why Digital Transformation Matters
http://luminadatamatics.com/content/why-digital-transformation-matters/
(4) What is digital transformation?
https://enterprisersproject.com/what-is-digital-transformation
(5) The seven decisions that matter in a digital transformation: A CEO’s guide to reinvention
Peter Dahlström, Driek Desmet, and Marc Singer
https://www.mckinsey.com/business-functions/digital-mckinsey/our-insights/the-seven-decisions-that-matter-in-a-digital-transformation
SPE Digital Energy Conference
Interesting quotes heard during the conference.
Data Mining ==> How to pull the data together, then listen to it.
Artificial intelligence is no match for natural stupidity.
Reasoning ==>
- Inductive — Platonic — Data Mining
- Deductive — Aristotle — Statistical
Neural Network according to Robert Hecht-Nielson ==> a neural network is a computing system made up of a number of simple, highly interconnected processing elements, which processes information by its dynamic state response to external inputs.
Albert Einstein ==> As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality.
Invention is the conversion of cash into ideas; innovation is the conversion of ideas into cash. (http://stuartlynn.co.uk/2010/11/16/innovation-is-the-conversion-of-ideas-into-cash-and-invention-is-the-conversion-of-cash-into-ideas/)
Interesting related definitions (found in Wikipedia)
Genetic Algorithms ==> A genetic algorithm (GA) is a search heuristic that mimics the process of natural evolution. This heuristic is routinely used to generate useful solutions to optimization and search problems.
Fuzzy Logic ==> a form of many-valued logic derived from fuzzy set theory to deal with reasoning that is fluid or approximate rather than fixed and exact.