Information Technology at the functional level has become a commodity. People expect to come into work, sit down at their system, and it is expected to work. Just like the lights coming on or the toilets flushing. All of these things are complex and require a large amount resources to maintain at the expected level of availability the business desires. If this, commodity level service, is the desired state the business has requested then Information Technology (IT) or Information Systems (IS) is an appropriate name.

However, if the business wants to leverage the assets and investments made in technology and information systems then a change is needed. If the business wants to move forward with data and information then a different approach is needed. If a digital transformation is needed then the mindset of the group needs to change from a commodity to a service provider. Imagine if the electric company not only supplied the power but also worked with you, directly, in improving your energy usage. Some companies do this now but it is passive. I am advocating for proactive service delivery.
​
I recently have worked with one organization in making this shift. The first thing I did was change the name of the department from IS (Information Systems) to ITS, Information and Technology Services. This name is a three part name to help our team focus and let our customers, not users, know what we provided:

1. Information Services – this is a focus around data and access to data and information
2. Technology Services – this is everything from network, to end-point to Audio Visual as well as solution architecture
3. Information and Technology Services – this is the combination of the two to solve business problems. Example, when this company was having problems reaching customers on the phone (information delivery problem not technology, the phones worked) we implemented a texting solution (customer focused technology solution to deliver information).

The next things was to follow through with this service mindset and took on the challenge to implement the ITIL processes across the entire service portfolio. Including get the entire team certified at the Foundations level. To measure the effectiveness we used the Net Promoter Score methodology and established a baseline of 30.10, a decent score but room for improvement. I asked the entire company two questions:

1. On a scale of 0-10, how would you rate your overall satisfaction with the service ITS provides you?
2. Question 2 depended on Question 1 -
   
   1. If the score was a 10 - In order to provide the best service possible please take an extra minute or two to tell us why you rated ITS with a 10.
      
   2. If the score was 0-9 - In order to provide the best service possible please take an extra minute or two and tell us what we could do to improve your experience.

Through changes in practices and focusing on service and the customers, 18 months later, the organization rewarded ITS with a score of 54.76. We are continuing to survey the organization twice a year and implementing changes around what the customers tell us.

IT changed across the organization because ITS in the name.

​

​Metrics help to tell a story and tell that story to the right audience. When I present on this topic I use an image showing a child, a spouse, and grandparent. All three can ask the same question, “How was work?”, and all three will receive an answer with a different story. Each will receive details that are appropriate for their level of understanding and their background. The story that, we as CIOs and CISOs, need to tell is no different. We have different audience members that include: the Board, Executives, Auditors, and Engineers. As we tell each of these audiences the story of IT and information security we need to keep in mind their background and the ask. 
The Board is strategic and we are asking for resources. The Board might be comfortable with that functional level aggregation or they may want a single score. When a single score is needed this is usually around a maturity level and it should be coupled with a risk rating. Combined, these can tell where you are and how tight the controls are.
Engineers need the details and we are going to ask them to fix something. With the Engineers we can show them detailed tactical metrics. NIST and CIS have a great listing of tactical metrics with parameters for different levels of risk. From there, you can tie the results to technology. 
Auditors need to know we know about our environment and that we’re doing something about it. Auditors would need to need to see that we are headed in the right direction. 
Executives need actionable information, usually, by subject area and we need to answer their ask, “what’s in it for me?” Executives need to see how the security program is affecting them. We need to aggregate by topics they care about. For example, the CMO might care about Integrity and Reputation, to address concerns of report accuracy and potential reputation damage. 
Your metrics are not a burden to the job but should be a tool to help you tell a better story.

The NACD posted these questions to ask your security team:
1. Does the security team have a full, well-informed view of the organization’s security posture?
2. Is our organization resilient to attack?
3. Is the security team confident it can detect and respond quickly to security incidents?
4. How do you measure the effectiveness of our cybersecurity program and initiatives?
5. Do political or financial considerations impact your ability to protect the organization effectively?
Here is the CHICAGO Metrics™ response:
1. Yes, we know the overall CHICAGO Score™ and the average Risk and Effort ratings to drive decisions and prioritization.
2. We have advised the Board based on Risk and Effort and are at or working towards the risk posture tolerated.
3. We tactically measure multiple points to include incident response. These aggregate into the Character, Availability, Confidentiality, Integrity, and GOld scores. We actively manage to these.
4. We have 142 tactical metrics, for conversations with engineers, and aggregate them into 6 key business/risk indicators, for conversations with executives, complete with an overall CHICAGO Score™. It provides Risk and Effort indicators to provide prioritization direction. This ties into the CHICAGO Maturity Model which gives a definitive quantitative scoring model to determine your organization's maturity level.
5. We publish and manage to our Risk (Likelihood*Impact) and Effort (Time*People*Money) scores. This allows us to quantitatively provide the executive team the information they need to make an informed decision. These can also be tied to the 6 key CHICAGO Metrics™ and how they relate to each C-Suite executive. - Think: "Speak their language."

As I have sat down with some groups and discussed strategic planning, particularly the big "2020" planning, it is clear that many organizations don't have the metrics to support their plans or haven't thought about it much.  Having metrics to help develop the strategic plan provides a stronger direction as the plan is built.
There are four main stages to building a strategic plan:

1. Goal setting - this is WHAT the organization wants to accomplish.  These should be measurable.  Think SMART.  Example, become a $10B organization by 2020.
2. Objective setting - these are going to be the incremental steps, the HOW, the organization is going to take to achieve/support the goals.  These are also going to be measurable too.  Examples: increase top line growth 8% per year, and add 20 new large accounts each year.
3. Strategy planning - these statements are WHAT the organization is going to do to meet the objectives.  These high level statements are the plan and direction.  Example, Engage all prospective customers throughout their day with targeted branding.
4. Tactic development - these are the plans of HOW the organization is going to execute the strategy.  These are going to be specific, assigned accountable and responsible parties, think RACI, and most likely related to someone's performance goals.  Guess what... these are going to be measured too.  Examples, Sponsor local sporting events (measure the reach), engage in Facebook marketing (click through rates). 

When we talk with people about metrics they are usually concerned about what to measure and then how to report it out. However, one thing that rarely comes up is the continuous improvement aspect. We are not talking about the part of improving your processes to hit your goals - making your dashboard turn green. We are talking about when your metric is green for 2 or 3 reporting periods, are you adjusting them?

A dashboard that remains green all the time speaks to two scenarios: 1. A perfect operation or 2. Complacency. The first can happen but it is really hard and expensive. But, we will acknowledge that it can happen in pockets. For example, successful back-up percentages. Once this reaches 99.5% or higher, most places would consider that consistently green. Not much room to move and getting it any higher would not be smart as those dollars should probably be spent elsewhere. Now the later, complacency, is a problem. This is the manager that is proud of his all green dashboard and likes that the executives don't pay them any attention because it is all green.

If this is the goal, simply lower the thresholds. However, that is usually not the point of a metrics program. Striving to improve is the point of the program and tends to be in line with the business's objectives. Improve the business -- make more money -- continued success. The reason we have found that people struggle with this is having the period or two or three or more where the metrics are yellow or even red after they adjust. Let's pick on backup percentages again. If you start off at 85% and get to 90% then you adjust. 85% was red and 90% was green. When you adjust, now 90% is red and 95% is green. The BRAG chart is going to be like a Christmas tree but the risk profile is shrinking.

Metrics, BRAG charts, slides, and PPT decks are all tools to tell a story. The story of continuous improvement is your's to tell. Be ready to tell that story and how it impacts the business. Although the colors may look like things are not going well as long as you can tell the story your operations will move forward.

Our founder is giving a talk at CAMP IT on 5/25
​
Measuring and Managing IT Investments

This session will take a look at how to measure IT performance from tactical to strategic. It will then provide a means to determine where to place IT investments based on risk and effort. Then we’ll discuss how to know if those investments are providing the expected impact.
​
http://campconferences.com/events/2016/assetmgmt.htm

Many people have asked about what they should measure or track.  A great place to start is measure what matters.  This may sound cliché but is really holds true.

• Measuring what matters means measuring the items that pertain to your organization, group, or department.  If the measure you have doesn't change any practice in your group then it doesn't matter.  These are your tactical measures.
• Measuring what matters also means measuring things that you can change.  For example, measuring total spam messages.  While this is a nice explanatory metric it isn't a reportable metric.  It is a metric that you can't control.  However, these types of explanatory metrics should only be reported when used as a denominator to show a rate.  Another example, # of spam messages that got through / total # of spam messages.  But now this is a different metric which tells a different story and you can control it because you can tune your spam filter.
• Measuring what matters means aggregating items so they mean something to the business.  Telling executives or the Board about AV incidences, accounts in AD, or average ETL run times doesn't mean anything until they are aggregated and tied to something the business cares about.  These tend to include:
  • Is the data safe? - Confidentiality
  • Are we managing the people effectively that manage the data? - Human Resources
  • Is the data accurate and unchanged? - Integrity
  • How does this impact our reputation? - Character
  • Is the data available? - Availability
  • Are we spending wisely? - GOld