Blog
This is a short article to try to explain in simple non technical terms the concept of SIEM. I have been in numerous meetings where the concept is talked about and people (especially in remote working) simply nod without really understanding what the concept is about, “something to keep us safe” tends to be the shared extent of understanding. SIEM Security Information (Information Security would have been a better choice, ISEM though, is not as easy to say as “Zim”) and Event Management. The concept of SIEM is meant to be an improvement to the more traditional “log monitoring” activity we have at some point all done, mainly for Event Management when troubleshooting apps. In the field of security, monitoring allows to spot when a dodgy incoming connection has been attempted. The truth is that log data is so rich and so complete (aka verbose) that nobody really wants to have to look at it. One of our clients had to migrate away from Oracle eBusiness Suite for commercial reasons and had around 5GB of real data and 24TB of log data that had never been archived. Log information is, however hard it may be for humans to look at, an invaluable source of machine ready data that can these days be streamed and analysed in real-time and also over time to determine not only what is happening right now but also historical trends and statistical correlation for events we may not like. The rooms where these data are displayed and acted upon truly look like command centres in sci-fi movies. This practice has recently led to the concept of “Security as a Service” as this activity can be readily outsourced to specialist firms, making it easier than ever to have professional security without having to reinvent the (expensive) wheel. The way SIEM works is: Events take place in the real world during interactions with or between computer systems The OS in charge of the receiver of the event will share the event stream with a middleman or directly with the SIEM management platform The SIEM platform ingests and processes (think security data warehouse) the data The SIEM platform provides Human Computer Interaction (HCI) services to a team of people that react to the events if required Use Cases The obvious use cases are those for systems that are networked (almost everything these days) and that have a door for users, customers or support staff. It really is about who knocks at the door and more importantly “how” they do it. There are some common kinds of attempted access that raise eyebrows (and sometimes raise ransoms when successful): Brute force: what it says in the tin, really. Try and keep trying until you succeed. These are simple to spot (why would anyone try to login to an account 7,459 times). Volatile sources: logging into an account from Finland, then Moldova and finally from Turkey in under a minute is at least strange (VPNs make this possible, but still there is little rational reason to do so). File movement: statistically speaking, there is a normal byte count for accessing any system, there may also be outliers but if a threshold is hit, something is not right and your business may be leaking data. Denial of Service (and its Distributed cousin) where an overload of parallel attempts blocks the door for legitimate access. Other “odd” changes: when the normal pattern of access is used, some changes will be experienced by the system being accessed. When these parameters are exceeded, something isn’t right. All of these are patterns to look for, when the news reports that such and such company have been the victim of a “cyber attack”, it is likely that one of these have been exploited using a vulnerability or a lack of SIEM by the target of the attack. In summary S IEM can be compared with one of my favourite expressions “keeping an eye on the pennies so that the Pounds take care of themselves”. Machines can keep a vigilant eye and the whole activity can be outsourced to a third party, there are many good ones in the market (and also some that have been hacked themselves, ironically). The main difference between vendors is how cleverly they analyse the event data streams, how much Artificial Intelligence is used by the machines before the alerts reach people and suggest or actually take preventive action. In the end, the reason why there are security attacks is a mix of inefficient coordination and “if it ain’t broke, don’t fix it” mentality in businesses, combined with the fact that there are huge incentives for local and foreign actors to continue this line of activity, quite often state sponsored, but the SIEM guidelines are also published by states, for example the National Institute of Standards and Technology (NIST) in the USA. The important thing is to understand the principles and standards and know how to implement them reliably . This is what we do. AVANZ.IO
Basic Definitions Simply put, and we need to keep things simple, Master Data is one of two (or three) classes of data. Transactional Data: responds to events and typically (again, to keep it simple) is made of values: numbers that reflect how much an event generates at the point in time the event took place ( when ) Master Data: the record of who , what , why , where the event happened (e.g.: customer involved, product involved in a sale) Reference Data: slowly changing and list-based records (think of UK Counties) There are many books that deal with the simple bullets above, but this is the in a nutshell version of types of data. It gets very complex very quickly, but always responds to the questions underlined above, so it can always be broken down to simple building blocks. Transactional Data Transactional Data is usually within the limits of a range that is not fixed, like money or quantities. So it really cannot be kept in a list somewhere as the values can be anything within the logical range. The main issue this class of data can experience is inaccuracy (someone types 10000 instead of 1000, for example--fun note: around 2006 I was running the EMEA BI team for an American industrial group and we received an order for £1000000000"Australia" when the correct figure was £100K for Austria, someone was really having a bad day...). It is very hard to reconcile these values if there are inaccuracies, usually one has to go back to the sale contract, which may be out of date. The important thing is to get the data right first time every time (with logic to ensure consistency and having reconciliations). Master Data Master Data is very different, it holds values that identify things, like a customer's company name ( who ), the specific product they bought ( what ), the concept of an order or a shipment ( why ) and where the product has to be shipped or the invoice address ( where ). These values are. not normally in a small list (though they can be, for example if your company has a short product catalogue) but must always stay in "high fidelity" with the reality they represent and only change when this reality changes. Remember, data is the mirror through which you obtain information about your organisation, clearly you would not want to get the wrong image. The key issue with this class of data is usually duplicates, where a customer's company name may been spelled differently by different sales managers, like RBS vs Royal Bank of Scotland vs R.B.S. vs RBS PLC). My last project had to deal with the "Heinz Ketchup" problem: 57 different spellings of a UK organisation. This is where the infamous concept of "data cleansing" usually comes in (see remediative techniques below), always after the fact, which really is of limited help. Reference Data Reference Data is a special subclass of Master Data. Where possible, the values for these records should be sourced from external or internal single lists of permitted values. External is preferred as the problem and the solution are outsourced (for example, Companies House company registration numbers are exquisitely curated by Gov.uk and the data is free to use). Another example is the list of countries in the world, counties in the UK and so on. Even postal addresses are maintained by different organisations in the UK that can be used as "reference". The main issue with Reference Data is an insufficient control over the shared list of admissible values--or no list at all, there is nothing worse for Reference Data than "free-text" fields. These should always follow the concept of a "drop-down" when human operators are involved in the selection of values. So, if we assume the values of transactional data will be entered accurately (logical checks and balances, reconciliations, etc. to provide safeguard) and Reference Data is kept in a single central internal or external "reference" list with "drop-downs" to select from, the only data family left is Master Data, which is up to each business to keep in top shape. This is why Master Data Management (MDM) exists as a separate discipline in the data world. Who is the Master? How is data in your organisation created? I can think of two main entry points for data: machine to machine (imports, integrations) and what we love the most: people typing it (for example, a Customer Services operator receives a call from a customer to notify a change in invoice address). If we quickly rule out machine to machine ingestion as a source of problems (once designed, the data exchange will always be like-for-like), we are left with the question of people and the tools they have to do their jobs . In short, the Master in Master Data is you . So either if you have a problem of duplicates or if you don't, your current situation is and has always been up to you. This is really good news, actually, there is nothing worse than a problem over which one has no control. This control is what the Management in Master Data Management stands for, and it is a good think you can actually control your own destiny. Who does the Management? The actual management in MDM is, obviously, up to you. Of course "you" may be hundreds or thousands of people, and if any of these individuals or their information tools do not follow the same rules, then there is more than one "you" and "too many cooks spoil the broth" comes to mind. So far, we can agree one set of rules is required and these need to be implementable, as in not just a solution design, an ambition or a vision on a white paper. The management of Master Data has to be defined before you start working with Master Data (and way before you go out to buy an MDM system). Of course that is impossible if you have been in business more than five minutes, so we need to think about changing the way it works (if there is room for improvement, if your MDM is perfect, then there is little to discuss and it's all good). At this point, let's imagine that your MDM is not actually perfect and that you have, let's say, multiple instances of your customers, each for a different department (typical to service organisations) or a product catalogue that looks more like an infinite laundry list than like a proper catalogue (typical in engineering firms). What are the steps to take? There is an inevitable reality of the "historical" records that you know have a problem, there is no escaping this once it happens. And there is what you start doing today to prevent it from happening in the future. Techniques Historical Master Data (a.k.a. Data Cleansing) Let's get this out of the way quickly: data is not like dishes, you cannot "clean" data, Data Cleansing is a fallacy, a term used by vendors and contractors and sometimes unrealistic senior management. Data cannot be cleansed, in fact, data cannot be dirty. Period. What we have to think is in terms of "Quality". What is the quality of data. And this has several dimensions. We have talked about two of them earlier: accuracy and unambiguity. There are more, completeness, and a few more. For each of the dimensions the business (I will repeat this: the business ) has to define the parameters or quality that apply to that dimension of a specific column of attributes and the values that are permissible with tolerance levels. IT will not, cannot do this for you. What IT can do is implement the rules and do this consistently across your entire estate. And only when these rules are written down as specifications, should you deploy an MDM system (which is going to cost a lot). All MDM systems are "vanilla" and they do not have these out of the box (that is another fallacy I may write about in a future post). Your future starts today (cheesy, I know) It is only when you have the business rules for each of your groups of data that you can implement them to prevent these problems from happening. But here again, a word of caution: do not do this as a project. Information and data are the blood that runs through the entire business body, the minute you finish the project, it will start to go wrong. This is meant to be (following the analogy of the blog in the business body) a consistent and constant behaviour that keeps you healthy. When you stop a diet or stop your exercise regime, we all know what happens to all that weight you lost). DevOps? BAU? It does not matter what model you choose, but you have to assign the accountability and responsibility to your people, permanent people, as an ongoing effort that you should measure and, crucially, reward generously. My rule of thumb is one Data Governance Manager for every 100 employees, one Data Governance Architect for every 1000 employees and a single Chief Data Officer for the entire organisation. Rule of thumb, of course, depends on every business. But always as a constant, permanent, ongoing activity. Once you are at cruise altitude, you may even decide to outsource some of the crunching, recently Talend launched a Data Quality as a Service offering, worth looking at. The Data Bakery So, why a Bakery? Well, the way a bakery works kind of illustrates every business there is. There are three domains in every business: incoming goods, processing and outgoing goods. Costs follow that flow in that way, revenue follows the flow in the opposite way. And since data, we have said and I hope you agree by now, is the Hi-Fi reflection of the business, then data flows in the same way. So you need procurement and the data that reflects it, you need operations, and the data that flows from procurement and reflects operations and then you have the customer domain which takes the data from operations (and implicitly procurement) all the way to the point you get paid by the customer. Flour and water, bakers, bakery store where the customer buys , hopefully, the best bread around. Once you have this basic model (this is what AVANZ.IO do best) then you break it down into smaller processes, sub-processes and single activities at the business level coupled with its mirror image, then data level. Then you identify for each fine grained data element the Quality dimensions, then, as above, you define the parameters for each dimension and the admissible values and tolerances. Then you may want to set up a Dev Ops team to start it off and embed it in the fabric of your business. And if you are in doubt about any of these points, take a look at GDPR, SOX and the white papers from the UK Information Commissioner. These are all achievable by following the Bakery methodology. This is what we do. AVANZ.IO
Semantics? Recently, CFO argued with my team about the difference between information and data. His exact words were “they are the same thing” to which I politely said, "No, they are two very different things", after which he said “…semantics”. I have really never understood that answer “semantics” to imply that two things are the same. Semantics as a word collects the vast array of different values and meanings words can have, how can it be used to describe the opposite? The Basics Let’s imagine that you and I are asked to think about the same thing, a car or even better, a mobile phone. It is almost guaranteed that what you and I think of when prompted by the term “mobile phone” are going to be remarkably similar. This is simple to understand, both you and I have been exposed for a long time to mobile phones, we both understand what they are today and with a bit of luck you are old enough to agree with me on where they came from since the early nineties. Exposure to something makes it familiar and familiarity turns into intuitive knowledge where we don’t really need to think much to imagine the thing represented by the words “mobile phone”. The way this works is we associate the words (an abstraction) with the thing (an implementation).The issue with this is that there are many things, concepts that we may not both be so familiar with, sometimes everyday concepts that are not really implemented physically. These are abstract and are harder to agree on as a single definition. Information and Language When we think about our language, the one we understand and use everyday, we simply “know” it, we know that certain sentences with nouns, verbs and adjectives will be understood immediately by the person we speak with, provided they understand the same language. We take this for granted. This is because in our language (and any language) we use words that can be nouns (things), verbs (events) and adjectives (attributes or characteristics of the things object or subject to those events). Because we all know the rules and the meaning of the words, we can communicate specific ideas with the assurance that the other person will understand the information contained in the communication. The words represent something we both agree on. All words are made of letters that combine to form the familiar words. As such, letters and words are building blocks for sentences, and these words and sentences encapsulate semantic value: meaning. Now imagine that we recombine those letters into words in a different language, say Spanish. Even if the same building blocks are used, we end up with sentences that, unless we know Spanish, will not really mean anything to us. Yes, we recognise the letters but the words and the entire sentence does not mean anything to us. In this foreign language, the letters are building blocks, the words are building blocks but the complete picture is simply not there. We can say that the data is there but there is no information for us, even if the sentence in Spanish is meant to say the same thing had it been written in English instead. So a sentence with words is data, in any language we like but information is only there if we can give meaning to the sentence. This is the difference. So far this is not rocket science, but why is it so hard to wrap our heads around it when we are using the same language that we are all meant to understand? Metadata Numbers can also be combined to represent stuff. Take 12544604501 for example. Simple enough, just numbers. Do you know what 12544604501 represents? You can take a guess and say it is a US telephone number (which it is) but what if we are discussing the serial numbers of defective equipment? What tells you or me what that number represents. 12544604501 is data. GlobalStar Telephone number: +1 (254) 460-4501 is information made of data and context, metadata, without which the number is useless for any practical purpose. Glossary What is then needed to establish a language and the possibilities of information it can carry as opposed to data (I.e. written in a foreign language?). Languages have rules, vocabulary, grammar that we learn as children, and it takes time, but the rules are all around us, we are immersed in them from birth. What happens when we are expected to know a different language without having learnt it first? This “new” language is not a regional language but one specific to a single business, a language that has evolved gradually to mean something but that in reality, in many organisations is undocumented. Imagine you take a new job at a company and you come from one of their competitors. You’ll think “sure, I used to work for Vodafone, I know this”. And it will be true to a degree. However, picture this: you get sent an excel document that is commonly used in your new company, one that has two columns, one with a number as above and another one with a company name. Now you don’t really know what the document really says. So you ask for the document with column headings and you receive the document with two columns, one titled CRM Phone Number and another one called Customer. The question is: what is a customer? You may think you know (we all think we do) but is a customer: A company that placed an order for our product/service? A company that we sold something to? A company that we actually ship something to? An entry in the CRM system? Quite often, depending on who you ask, you will get a slightly different version of what a customer is. Why is this, shouldn’t a customer just be a customer? The answer is no, each organisation will have their own internal glossary if they have one, else the concept will be a different thing to different colleagues at your new role. And this is a simple example, wait until you need a definition of service and its components from a margin accounting perspective. The reason for this is that abstract concepts don’t have implementations, and most business terms are in fact purely abstract. Without a typified definition, it is impossible to paint an accurate picture of reality. And increasingly reality is the digital reflection of the world, and not the world itself. Glossaries go a long way to achieve this disambiguation, which is why terms like “single source of truth” or more to the example above, “Single Customer View” are permanent obsessions at every organisation. The issue with this simplistic view (everywhere you find the word “single”) is that it only works at mid to top management level. The problem is that what something means has to be combined with what each group of stakeholders needs to know about the thing in question. This is the Achilles heel of anything “single…”. For this reason, glossaries must be extended to contain multiple “trusted” views of the concept, in such a way that every perspective can be self-contained. For the “customer” organisations should aim at producing several “trusted customer views”. After all, what the customer themselves (as one of the groups of stakeholders) and what engineering need to see are two different perspectives of the same reality. This can be referred to as Customer-360, another popular term, but this is applicable not only to customers of course, but to everything that is an information concept for your business Business Information Model- BIM --with AVANZ.IO So before your company rushes to buy an information system to solve a problem, some groundwork needs to be done. The idea is to create a graphical information model (not a data model) that the business, everyone, can find themselves in. This model reflects “what business are we in” and should be developed following the famous Toyota Way, by asking “why” (and what, how, where, when, who) as the model is developed into deeper more granular layers all the way to the bottom. A BIM is the single most valuable artefact a company can create for several reasons. Our previous post in this blog dealt with the chain formed by: People > Process > Information > Systems > Data and information is the central link to this chain. So a BIM backed with a glossary with multiple trusted perspectives provides clarity to the business (people and process) but crucially allows you to ask IT to build or configure any system based on your information rules. People in your organisation will know their subsets of processes, perhaps even cross-team processes with hand-off points if process mapping has been done before, but the piece that is often missing is the BIM that translates the reality of the processes with the language used to capture the details and subsequently translate them to IT specifications based on unambiguous meaning and rules. This is what we do, AVANZ.IO
Culture and Data? Culture is a generic term which encapsulates the behaviour and norms found in human groups, as well as the knowledge, beliefs, capabilities, and habits of the individuals in these groups. These groups can be broad such as an entire society with cross-country shared cultural values, for example freedom of expression, equal value of individuals, the right to equal opportunity and the respect for private property. Groups can also be defined in a more narrow fashion, all the way down to a handful of individuals. Startups, for instance, are made up of a few individuals with a shared sense of purpose, which does not mean they have to share the broader cultural aspects. This is what makes the world such an innovative and exciting place: people from varied ethnic, regional and religious backgrounds can work together to achieve a narrow cultural objective they do agree upon. Despite having different cultural backgrounds, in the broad sense of the term, by embracing a hard core of objectives and ethics, small groups of people usually make a large difference within the societies they inhabit. And this drive coupled with talent represents the winning combination for any group in any society that is open to change and forward motion. This all sounds great, we all know and love a success story, and the world is full of them: Tesla, Spotify, Netflix, all started this way. But what happens when the above group does not share those core cultural values, the specific ones required for success within an established group, like a company in business? Does that mean a change is needed to align those values? Why do People do Anything? Cultural change can be a scary concept. However, the question is: why is the wrong culture almost always to blame when things go badly for organisations but only after the fact? Why is the concept of "technology will solve the problem" (a.k.a buying yourself out of trouble) even considered as a way forward before the fact? What about people, the people ? One of my favourite analogies is that technology is a bit like soap . Buying soap does not make one clean, it is the behaviour associated with the use of soap. This collective behaviour in the group is what, coupled with technology, combine to provide value. The behaviours of the people , the culture. So how do we achieve concerted behaviours in a group of people that are not inclined to do so, people who do not know why or what behaviours are needed? In human psychology, and by extension, sociology, people, all people, everyone, responds to Operant Conditioning. Operant conditioning, sometimes referred to as instrumental conditioning, is a method of learning that employs rewards and punishments for behavior. Through operant conditioning, an association is made between a behavior and a consequence (whether negative or positive) for that behaviour This concept is often called "carrot and stick", and of course, it is hard for anyone to consider it. Change is scary for most people, but especially for management, as it implies upsetting other people. We all want to be liked, we are humans, social beings, we need to be liked by the group. This is the main reason management in the Western world is reluctant to "rock the boat" too much . OK, so should we take example from authoritative societies such as China? These societies undoubtedly achieve change at dramatic speed, though sometimes the kind of change they achieve is not the intended one--people do not and cannot be coerced into doing something for very long until they resist and revolt. History is full of examples where leaders ended up rather "headless" when the rubber band of society was pulled too hard. For organisations, too much "stick" ends up bleeding talent, which can be replaced at the expense of losing hundreds of years of combined experience and knowledge. This rarely works, and worse, the ones who stay are guaranteed to become cynical and resist change in an even stronger manner. So that is one approach we can rule out. What other approaches do we have left? We are back to Operant Conditioning then. Why does anyone do anything? Sounds like a philosophical question, but it is the question. Despite everything, reward and punishment runs the world, in fact all living things respond to this and only this, when it comes to decision making. Of course not all decisions are good ones, but eventually punishment arrives in some way or other. The main issue in Western society is that we are accustomed to being shielded from consequences in the short term, it is hard to feel future pain and easy to seek instant rewards. This explains many of the problems human societies experience. And specifically, explains why organisations simply cannot change rapidly enough to adapt to the ever changing reality they operate under. Leadership and Talent Based on the above: the general rule of avoidance of pain and the pursuit of short term gratification, true leadership is meant to engineer a working environment where it seems that this is what's happening, but it only seems to be.This is a proven technique that works with people in all aspects of life where difficult paths have to be chosen instead of the easy ones. From education to personal fitness to cleanliness to work culture. So how does this social engineering happen? Rewarding Quality Deming, arguably the most influential thinker and practitioner of "quality" in the twentieth century proposed a list of 14 points: Create stable, measurable and meaningful purpose for improving products and services. Adopt the new philosophy. Cease dependence on inspection to achieve quality: train for quality End the practice of rewarding business performance on revenue alone. Reward behaviours as well. Improve constantly and forever every process for planning, production and service. Institute training on the job. Adopt, institute and reward behaviour-based leadership. Drive out fear. Break down barriers between staff areas: share the vision and the view. Eliminate slogans, exhortations and targets for the workforce. Eliminate simple numerical quotas for the workforce, especially for management. Remove barriers that rob people of pride of workmanship. Institute a vigorous program of education and self-improvement for everyone. Constantly. Put everybody in the company to work accomplishing the transformation. The above points sound a bit academic, so we can really simplify them to drive three and only three key points: Define what good looks like and reward it. Define what not-good looks like and disincentivise it. Make it real, make it believable: people see though fads, fashions and short term "we did this back in ... and did not work" People There is a quote attributed to "millennials": "they do not want to work for you, the want to work with you". This is as true for millennials as it is for everyone else. We have not have major conflict in the Western world for a long time: people do not understand threats and the inevitability of events over which they have no control. The Blitz is long forgotten. We are shielded from the consequences of bad decisions (in the Western world, again) and this makes the role of incentives all the more important. Working with people, in a way that makes people feel good for making incremental effort towards long term change is the only key to success in modern times. And this applies to business culture and specifically applies to behaviours around data based on learning, implementing, reviewing, improving, repeat, where mistakes are perfectly fine as a learning tool. Process: Bottom-Up... Positive thinking is a major factor in success. So instead of assuming that because something worked somewhere else, make sure you know how it works here. Work the bottom layer of the organisation first, ensure the people who actually do the work are listened to. They know what is in need of improvement, they do the work everyday. Management often do not have the correct understanding of what it takes to deliver results, especially in these days of running companies in Excel and PowerPoint. This, of course, should not be generalised, but there is truth in it. When foot soldiers are properly fed, they march better., fight better and win better An army runs on its stomach (c'est la soupe qui fait le soldat, "the soup makes the solider" Napoleon, though he did not take his own medicine). Once the voice of the people who do the work is heard, distilled, condensed and processes are analysed, then these messages on "how" can be taken to management with a high degree of confidence, and more importantly, with the "blessing" of the workforce. Process: ...and Top-Down At the same time, higher level objectives, strategic and tactical have to be mapped, and this happens at the top. the "why", "when" and "what" are defined at a high level and then cascaded down as processes to the point where they marry up with the "how". This is relevant to all aspects of business, but is specifically true for information and data. After all, it is data and only data that can tell whether something happened as planned or not. We do not see the customer anymore, we see the data related to them, so it better be damn good quality, else we are looking at a broken mirror for our reflection. If data is increasingly the only view of the world, all data has to be consistent with the semantic value, the meaning it ought to reflect. Management must be convinced about this, out of their own initiative or with external help. Once both streams converge into a strategy for the business that is explained in terms of information, the organisation can say they all speak the same language. Information is the language of the business, data is the bricks, what the home looks like can be shared across the entire business. It takes time to learn a language, and it is somewhat hard, but it is the only way and anyone can do it. Information Data is the mirror image of the business, and when looked at, the picture reflected back is information : meaning. Data must be connected with that meaning, the only thing we humans understand, once more: meaning. This connection starts with behaviours and uses technology to bridge the gap. When people feel good about doing something hard, the reward that comes after is the biggest incentive we humans understand. The pride in doing something we did not feel capable of doing. This feeling, in bite-sized doses, concatenated over time is the only real competitive edge organisations have at their disposal. And it has been there all along, waiting to be tapped into. PEOPLE > PROCESS > INFORMATION > SYSTEMS > DATA These are the ingredients for the "secret sauce", in that order. I often tell clients that if you get people, process and information right, you can buy systems and data takes care of itself. Working together is the only way the single language for the business, its information model, can be understood at all levels. And only when everyone understands some aspect of the vision, each from their own unique perspective, we can all work together for a set of shared goals speaking the same business language. This is what we do. AVANZ.IO
We've been helping companies transform and evolve digitally for more than 20 years.
Web by GraphikLand Design / Kaffeklubben Island