Entropy1-3
Information, Risk and Managing Risk
Sateesh K S

Adversities abound around us. There is always a potential for the “unexpected” to happen and derail our lives. The mere existence of these adversities is usually not a problem – but when they manifest and disruption occurs we are usually unprepared for the consequence. Be it in real life or in Digital life that most of us live today, risk is an integral part. Prudent practices can ensure that we are always prepared for any of the outcomes. Where we do possess relevant knowledge, we should be able to use it to reduce the chances of an undesired outcome.

Risk arises from uncertainty and uncertainty is inherent in all walks of life. Reasoning under uncertainty may appear tricky but is usually no more than a prudent application of common sense. How does risk arise from uncertainty? In this article, I will answer this in a way that helps think about Risk quantitatively.

But I will start with an incident that opened my eyes to how simple solutions often help reduce risk. This incident took place in front of a temple in India. As usual in temples, anyone who enters the temple is expected to leave their footwear outside before entering. When there is no one attending to the stall, there is a heightened risk of theft. I was standing outside waiting for my wife and observing people around me. There was a person who had an ingenious and a very simple way to reduce the risk of theft. He simply left one of the pair in one place and the other one in a completely unrelated location about 15 meters away.

Nobody can guarantee that this will eliminate the risk of theft. But it certainly lowers the risk. A thief, being in a hurry to disappear with the booty as soon as possible, would be very unlikely to look for the second of the pair. So a thief is very unlikely to target this particular piece unless they have seen the whole scene unfold in front of them. There is definitely that possibility – however, it is much smaller. Thus this simple mechanism ends up reducing the probability of theft considerably.

In general, it is possible to argue that risk arises because we have incomplete knowledge. Lack of knowledge forces us to use our intuition, experience, available data, and information, etc to make “informed” decisions. Informed decisions do not necessarily guarantee absolute security, but provides a means of mitigating risk.

For instance, scanning the network for vulnerabilities and misconfigurations is one way of getting knowledge about the system. Remediation is a definite step to mitigate any identified vulnerabilities and thereby reducing the risk faced by an organisation.

In the following, I attempt to provide an information-theoretic basis for thinking about Knowledge, Information, Uncertainty and Risk. No claims are made to mathematical rigor, although the results are mathematically rigorous I have waved my hands and given plausibility arguments to justify what is actually mathematically rigorous.

Entropy and Risk
It is not farfetched to imagine that Risk is related to lack of knowledge. If we have enough knowledge about a system, we are in a better position to make decisions that lower the risk of unwanted events occurring. Let us examine this a little further.
Any given system will evolve in time and result in one of the many possible states. Even if we knew the laws that govern the evolution and the dynamics of the system, it is, in principle impossible to predict exactly, the outcome after a certain amount of time, except in the case of simplest systems. The system may evolve into any one of the many possible states.

Let me illustrate this in the case of a coin toss. Suppose you are given a “fair” coin and asked to predict the outcome of a toss. Since fair coin means that either Heads or Tails is equally likely, the best you can do is guess.

Now suppose you are given a different coin and told that the coin is biased. Further, you are told that this coin has a probability of 80% of turning out Heads on any toss. In other words, if you tossed the coin many times 80% of the times it will land Heads up. Now if you are asked to predict the outcome of a toss, you are obviously in a better position because you have more information about the coin. In this case, the “risk” you are taking by predicting Heads is lower than if you predicted Tails. When we have more knowledge (equiprobability effectively means we know nothing about the system) we are able to make informed assertions about the outcomes.

Can we define a mathematical quantity that captures this intuition? It turns out that such a function was introduced in Physics in order to understand the second law of thermodynamics in the 19th century. It is called Entropy. Entropy was originally introduced in Physics by Ludwig Boltzmann to understand the second law of thermodynamics from a statistical mechanics perspective. In our case, Entropy should encode the above intuition. Entropy should reflect the knowledge we possess about the system.

It turns out that the amount of knowledge (information) of a system the less the Entropy is. In other words, systems that have a high degree of predictability have low Entropy and vice versa. The work of Claude Shannon showed how this can be done. I will not go into the intricacies of the Math or the Physics of it, but will only explain how it can be used in our case.
Entropy is defined (in information theory) as follows:
entropyEqn

Where Eqn0 is the probability that the event i occurs (in this case Heads or Tails). The summation goes over all possible values of the outcome.

To illustrate this with a simple example let us calculate the Entropy for a fair coin. In this case there are two outcomes either Heads or Tails with probabilities .5. H will then be equal to Eqn1 which is 1. Similarly, in the case of the biased coin with a probability of .8 for Heads the entropy will be equal to Eqn2 which is lower than 1. If the probability of Heads was .9 the entropy reduces further to .467. When an event is certain to happen, for example, if both sides of the coin was Heads and no matter how you toss it always would land on Heads, the entropy would be 0 (A matter of technicality, log (0) is undefined and we will run into trouble when we substitute 0 for the probability of Tails – but suffice it to say that Entropy takes the lowest value when there is the greatest certainty)

This function has all the desired properties necessary. Figure 1 shows the plot of the probability of getting Heads on a toss on the x-axis and the Entropy on the y-axis. Clearly, Entropy is the highest when we “know” very little or when the probability of the toss landing Heads is equal to the probability of landing Tails.

Entropy-1

Figure 1: Probability of Heads vs Entropy in a coin toss

This concept neatly generalises to more complex cases. Just to illustrate this with one other example, consider the case of fair dice. In this case, the probability of any one of the 6 numbers showing face-up (1,2,3,4,5,6) is equal. As in the case of a fair coin, equal probability is another way of saying that we have very little knowledge about the system and if we calculate the Entropy, in this case, it will be the highest. (here the entropy will be Eqn3.

If on the other hand, say we have a biased dice, with the probability of landing 3 being 1/3 instead of 1/6 and the rest equally probable (i.e. the probability of landing 1,2,4,5 or 6 is 2/15 – this follows from the requiring that the total probability of all possibilities sum up to 1), the entropy would be
Eqn4.

In general, the higher the entropy, the lower the “Knowledge” we possess about that system. Since lower knowledge, in general, leads to higher risk, we can say that higher entropy is in some sense indicative of greater risk.

While ensuring that Entropy is as low as possible says that you possess as much knowledge about the system as possible, one has to exercise that knowledge in order to reduce risk. And of course, there is always the concept of acceptable risk. Some amount of residual risk is always present, reflecting the impossibility of knowing a system perfectly.

SanerNow and Its Role in Reducing Risk
SanerNow is designed to not only provide extensive information on all systems connected to your network, it also provides you continuous visibility to the state of the connected devices in the network. With its intuitive dashboard and reporting, it provides “knowledge” about the network in a very consumable way. Having access to this knowledge is an essential aspect of creating a well-informed security posture.

Risk is not static and needs to be constantly monitored. With its ability to provide continuous visibility to the network, SanerNow equips you with the latest knowledge of all security related information. This information is critical in understanding the actions that will potentially reduce the risk posed by IT assets, processes and people using these assets to the enterprise. SanerNow also provides you a convenient interface to act on this information and reduce the risk faced by the network.

At SecPod we are working on creating an index that will tell you how close you are to having eliminated all possible sources of risk. Armed with this you can decide whether you have acceptable levels of residual risk (often it may be impractical to remove certain vulnerabilities) or you would want to further reduce risk your network poses to the organisation.

Conclusion
In conclusion, I have tried to argue that entropy is a good indicator of Risk, and risk arises mainly because of the lack of knowledge we possess about any system. When a large number of outcomes are possible any one of them may occur. However, the desired outcomes will usually be a small subset of all possible outcomes. Almost invariably the undesirable outcomes outweigh desirable outcomes, resulting in risk. It is not very surprising then that the chances of a desirable outcome occurring accidentally, when a large number of outcomes are possible, is very small. This is one overwhelming reason why one needs to take the extra step to ensure that known risks are addressed.

I have argued that entropy is a possible indicator of risk. It is possible to calculate the entropy under different circumstances and ensure that it achieves the lowest possible value. SanerNow provides all the information necessary for a security administrator to make informed decisions. This will help manage the risk posture of the organisation in a controlled way.