XML to JSON

XML and XSLT used to be a very popular choice for integration transmission. However, this format proved to be messy and complex even with XSD in place. Then, JSON format comes around with the resurgence of JavaScript and REST service. It quickly overtakes XML as the preferred choice of integration format. This puts us into challenging mode as we move to convert XML to JSON.

Your Stack Setup

Historically, the usage of applications is directly linked to the type of integration format used. JSON is very geared towards web languages like JavaScript, Python, PHP and Nodejs. In contrast, XML favored Java and Dot Net languages. Thus, you will find difficulty for your legacy system to consume JSON. Even database is split for CLOB to XML. There is newer database like unstructured or NoSQL database for JSON. Therefore, a consideration of architecture changes may be required to leverage the advantage of integration format like JSON.

Your Endpoints

If you do not have budget and time to overhaul your architecture, you may consider focusing on your endpoints and gateway. It is common to setup integration gateway to manage the translation of XML to JSON. This provides backward compatibility to your legacy architecture. In another words, you gain more time to progressively upgrade your stack. The good part is that these integration gateway can serve as a command post to manage your endpoints.

The switch from XML to JSON is a time consuming effort. If you do not have legacy system with XML, you may want to consider having a technology stack that favors JSON. However, those with legacy and XML integration may want to lower risk with the setup of integration gateway to manage your endpoints and translation of XML to JSON.

Handling Outages 2021 and Beyond

We had an outages yesterday that was totally unexpected. It was so big that the major news reported them i.e. Akamai outages. It was surprising because we would have thought this type of outages were a relic of the past. How should outages being handled in a world where users cannot tolerate 5 min of downtime?

Formation of Outage Team

Do you have an outage team? It is likely majority of organisations do not have an outage team. Neither are outage drills conducted in large scale for the organisation. You can relate this like a fire drill where you will have a fire drill team for your entire buildings. This team is usually cross functional and well drilled in various skillsets from infrastructure, security and applications. Although this team could be costly, it is worthwhile to note that outages may cost a 100 times more.

Outage Management

Outage management is different from disaster recovery. Disaster recovery usually deals mostly with internal applications or related downtime. These SOP (Standard Operating Procedure) could be formal processes with your external parties. When outage happens in the old ways, it usually starts with a Sev1 (Severity 1) and manage with standard incident management process. It will take an hour or more before this is deem an outage. The slower pace and passive nature of incident management may not suit today’s outage.

Outages are no longer isolated in nature due to Cloud and networked environment. Case study like Akamai outage will lead organisations to consider a need for outage team who are highly trained and focused to manage outages swiftly.

Constant Delta

Projects are usually encountered with a constant that is consistent. I like to call it a constant delta, oxymoron intended. For the noobs, this is frown upon and recorded as issues. Instead, a positive mindset is encouraged to embrace these delta. You will notice that Agile is unable to exist in organisational context due to traditional structures, policies and roles.

Another View of Delta

Delta management is rarely emphasised because traditional project management focus on fixing scope, time and cost. Variables are deemed to be risks with negative impacts for projects. This view of delta does not change until the recent explosion of Cloud technologies with Agile. The Agile project is a 180 degree view to regard change as constant. This paradigm shift to handle changes instead of freezing scope brings upon a new view of Delta.

Human Avoid Delta

Unfortunately, organisational view remains entrenched in fixed scope. The slow rate of change is consistent with human reluctance and avoidance to change. There are no revision of organisation policies or financial tools to manage delta. This lack of knowledge and uptake will persist for the next 5 years until all applications moved fully to Cloud.

Agile is the best technique to handle constant delta in projects. For many Agile practitioners, we are frustrated by the lack of delta management in the organisations. The gap between Agile and constant will remain unless there are more efforts to focus on how to incorporate delta within organisations.

Agile Template

Agile templates are getting popular now. There is no need to build many things from scratch. You can easily setup eCommerce shop within a day. Template are developed with Agile in mind and vice versa. Thus, you should start to develop Agile Templates instead of projects.

Why Template

From many legacy applications, templates comes from the ease to reuse and the need to reduce cost. Project is a mash up and gelled need to address a focus objective. In contrast, templates address a particular story point and suit Agile perfectly. Agile team will gain most from towards templates more than previous project implementation exposure.

Shift to Template Management

The role of project manager (PM) will be on a decline as teams shift towards Agile approach. Instead, this role can move to a focus on template management. Some organisations will have similar approach like COE (Center of Excellence). Another big push is Cloud which emphasis a lot on templates for rapid deployment.

The shift in IT landscape is huge for the past few years. The COVID-19 pandemic is also a major push factor in accelerating these changes. We will see more of templates design and model from major cloud organisations. Their template libraries will be the key factor to lock in the customers, including me.

Do Not Digital Transform

Digital transformation may be the trend now with many rushing to transform their digital technologies or implement new innovations. There are many success stories reported but many have comes with trial and error. The key is to identify what not to digital transform and avoid unnecessary wastage of time, money and efforts.

Inclusion Criterion

The evaluation for digital transformation is vague in many instances due to various results and differences of business domains. What works for banking industry may not be applicable for logistics domain. Thus, you need to have clear inclusion criterion to be setup for your digital transformation program. This will allows you to pinpoint the necessary system or tools to be transform.

Exclusion Criterion

The inclusion criterion is the first step to understand what are the key things that you can digital transform. It is highly recommended to run multiple pilots to understand the nature of the tools you going to use. With deeper knowledge through realistic applications, you can form the exclusion criterion on what cannot be digital transform. For instance, voice recognition may not work in heavily accent Asian countries with different dialects.

Digital transformation could be for the good, better or worst. You need to have a clear picture and understanding of the tools that you are using. Always seek to run pilot study to know the tools well before you embark on a major digital transformation program. In this way, Agile approach is for the best.

Oracle Email Delivery Services 101

I was testing Oracle email delivery services when I realise certain constraint on the receiving end. One of them is the email security policy of the organisation. This can be quite painful where email services are blocked constantly. Thus, you should not rely on email as your only mode of communication with your customers.

The Good Part

It is pretty simple to setup and test email delivery services. Oracle provide a sample send email on GitHub to be use together with its Cloud functions. Disclaimer that I mention easy after spending days to understand Oracle Cloud Functions and API Gatway. So, I will suggest that you start studying Oracle Functions and API Gateway first before testing Email Delivery Services.

The Constraints

Some constraints I found while testing and it a stark reminder to email delivery failures. A simple list here:

  • You cannot use public email like gmail as your sender.
  • You will need access to your email configuration to update your SPF records.
  • Majority of organisations blocked relayed email.

Some improvement I will like is the ability to setup its own simple mail sender or access to public email as sender. Other features that can be added will standard email test can be trigger on the console, template for responses. The test trigger will remove the need for functions and API gateway setup.

Cloud Configuration Errors

The pain of cloud configuration is the cryptic error message. Due to the nature of many VM (virtual machines) and applications, the errors can be quite a pain in the ars*. Seriously, there is no shortcut to learning except to test drive and figure out the errors yourself. Do expect to invest a lot of hours in troubleshooting.

Photo by Ethan Kwok
Are you Stuck?

Stackoverflow is really a savior because everyone was as desperate as me in those error messages. These errors are the breadcrumbs to learn the rest of configuration syntax. You can literally search the whole error messages and stackoverflow will turn up 8 of 10 times. As a last resort, stop trying and time out. You may rethink the configuration to come up with another new approach to find the issues.

Common Mistakes

For many laughs, the mistakes are often silly and common. These are some of them.

  • Typo errors will create all kinds of error messages. Always check and recheck your typo.
  • Wrong data value is another top errors. Cloud fields are so similar than you may be mistaken to put similar data. If you are unsure, just try the similar data. Who knows, it will work.
  • Missing steps also contribute to weird error message because there are dependency with each step. Sometimes, the step are a must for the next one.
  • Outdated examples will require a change in syntax. Be prepare to tweak the example syntax for the newer version.

Cloud Configuration errors are never a breeze. If it works, it is worth the efforts. Meanwhile, be resilient and persist with guts!

Love Hate Macro

Anyway who have done macro will usually love to hate it! Macro can literally be a product of legacy achievement with headache and a consolation to excel users. Yes, this is how frustrating Macro can be. Are you someone who is trying to eliminate Excel macro?

Macro Standpoint

Macro is like a nowwhere application that is not data friendly. If you ever tried watching PiP (Picture-in-picture), you will know how macro will feel within Excel. I usually like to relate macro to RPA where we are trying to automate the wrong part of the processes with users reluctance to exit Excel. With the move of Microsoft to Cloud, macro will face a timely death in the next 5 years or so.

Macro Ocean to River

Sadly, I am a creator of macro but not a supporter of it. Building a macro is like trying to get the ocean into a river. Excel allows users flexibility to create different worksheets, rows and columns. Data is free flow and macro is trying to make sense of this ocean and empty it into a river. This is a total nightmare to your support team.

Moving forward, do reduce the reliance of macro as a low cost way of data input. Will the move to Cloud will present a shift to “Cloud macro”? Many upgrades and cloud migration are already putting a strain to these change to the macro. Where you will stand?

Cloud Dollars and Sense

Budgeting is an key aspect for many traditional IT projects. You often need to ballpark the budget estimation from previous experiences. The tricky part comes when you are doing Digital Transformation project for Cloud. How do you make dollars and sense out of something that have not been done before?

Metering

If you are moving to Cloud, it makes sense to starting metering your transactions. Think of it like your electrical bills, where your are being charged at kilowatts per hours. Metering makes it easier to convert to Cloud credits when you are planning your budgeting. The good part with Cloud is that you can adjust or stretch your credits with different Cloud features.

Running Cost

Running cost for Cloud will deviate quite a bit from your existing on premise application. This is because Cloud have factor running cost into its metrics as compared to your on premise cost. If you want to compare the running cost of on premise application, you will need to include support cost, upgrades, manpower or downtime.

Preparation for Cloud move will have to start as soon as possible. If you are still pondering over the move and trying to make sense of the dollars. You can consider to meter your existing on premise applications and sum up your running costs. This will help greatly in your comparison to Cloud costing.

Cloud or Later

Many applications have been shifting from on premise to cloud. There are many debates to whether your application should shift or not. A fact that exist is that all products will be on cloud for the next five years by 2026. This means you shall start preparations for your applications to be deployed to cloud. What does this mean to product owner?

Photo by Ethan Kwok
Cloud Architecture

The anticipated shift to Cloud will create a major change to your product architecture. This will pose a huge risk to your application when you have to move to Cloud. Another change impact is your customisation that is done to your application. In Cloud, the level of customisation is usually restricted to REST or API calls.

Ways to Cloud Safely

Rather than a Big Bang approach, you can consider to transform your customisation into cloud first. Another approach is adopt new add-ons on cloud instead of deploying them on premise. Digital Transformation of legacy applications or customisations are tedious and time consuming. Do consider building your insource cloud capabilities in preparation of your “D-Day” to Cloud. Agile is a must because Cloud will throw up surprise in your shift.

The shift to Cloud is inevitable for many. Debates will only determine if it is now or later. Rewards will goes to those who are well prepared and self reliance on their Cloud capabilities.