Wednesday, July 25, 2007

What to do about misrepresentation in DW and BI – not to mention ICT?

I am getting very uncomfortable about the quality and quantity of a lot of self-styled data warehousing and business intelligence professionals; organizations and individuals. The perceived and increasing degradation of professionalism and integrity in my own areas of specialization, also finds echo in other aspects of Information and Communication Technology practice, but that is outside the scope of this text.

The issues that I think need addressing are:

1. Why do some organizations take on Data Warehouse or Business Intelligence development contracts, when they are clearly ill-equipped to take on such endeavors and have little or no satisfactory experience in the field?
2. Why do some organizations make alarmingly huge and false claims about the depth and breadth of their experience and knowledge in the field of data warehousing?
3. Why do some organizations put forward “consultants” as their data warehouse experts, when these self same “consultants” barely have a clue about what data warehousing is about?
4. Why do some people consent to being touted around as professional consultants in the field of data warehousing, when clearly they are not?
5. Why do some people consider themselves professional consultants in the field of data warehousing, when clearly they are not?
6. The contribution to a fundamental lack of professionalism (including integrity and honesty) to the failures of data warehouse and business intelligence projects.
7. The contribution of professional malpractice to the business world’s view of data warehousing and business intelligence and the companies and individuals involved in these “fields”.

The problems with all of the above, is not that these organizations and individuals are doing what they do, the problem is the impact it is having on highly professional individuals and organizations:

1. Companies stung by data warehousing scams, tend to blame data warehousing, rather than the cowboys they hired to provide them with a data warehousing process.
2. The bad reputation created by lies, shoddy work and lack of professionalism can hit everyone, not just the guilty.
3. Some large organizations can buy or bluff their ways out of failure, disgrace and malpractice; small professional organizations are more prone to systemic damning of their chosen line of business.

Proposal:

1. The industry should have a professional body for accrediting data warehouse and business intelligence service providers.
2. The industry organization should be empowered to investigate potential malpractice in the profession, and to apply sanctions as appropriate.
3. This industry organization should have the teeth to name and shame cowboys in the profession – so that they either shape up, or ship out.

What do you think about the growing lack of professionalism in data warehousing and business intelligence?

Sunday, July 15, 2007

Excuse me Christopher! Your ICT sucks!

Investment in IT

As a CEO, CIO or student of Business Management you will understand that Information and Communcations Technology (ICT, was IT) has become an essential and all pervasive element in the fabric of corporate life. What I mean by IT encompasses three essential elements: Processes, Information and Technology. Indeed IT - and more importantly Information, Processes and their Management (IM) – and in my view should be viewed as essential corporate assets, they should work as contemporary business enablers, they should bring about value-adding alignment in business, and should therefore be managed as such. Although, and significantly, it typically doesn't work that way.

The tangibles and intangibles of IT, and perhaps even more importantly, IM, should not be viewed as corporate assets simply because they frequently require capital expenditure; having additional IT related assets piled on top of other traditional assets of a somewhat dubious nature and value; such as debtors, underutilized real estate or artificially high stock overheads, is not really the way to proceed with a serious evaluation of useful assets.

IT together with IM should work together as contemporary business enablers; not as a way of channeling funds into low-value and high-cost pet projects, of constraining business because of ego or hubris, or of putting up barriers to value-adding commercial progress. IT is a crucial element in providing a viable way to realize business objectives, and the intelligent and pragmatic application of IT is the best way to empower a more agile organization, in which people have the tools and technologies at their disposal which allow them to work even more effectively and efficiently towards the goals of corporate strategies, whilst ensuring the smooth running of the everyday operations and tactical decision making of the business - together with the ability to manage change as if it were second nature.

IT and IM should be aligned with the business, and never seriously attempted the other way around. The way, for example, that a business wishes to interact with its clients, prospective clients and suppliers should be driven by how the business wants the client relationship driven, the way the business is focused on the market and only then delivered using a string technological base.

In addition, if IT and IM are really assets worth having then their value must be well understood both in order to manage the assets as well as to leverage additional value from those assets.

These approaches would deliver quality IT and IM functions that almost any organization would be continually satisfied with.

Human factors

But reality bites down hard when it wants to, and more often than not when we are least expecting it. IT is seldom given importance for any valid business reasons, such as its support of operational excellence or of quality support of valuable mission critical business – and bureaucratic reasons are not necessarily business reasons - but frequently find themselves at the forefront of business life for primarily negative reasons.

It's hard to be critical of organizational IT because it's not just about technology but primarily it is about people, their careers, their work, their livelihoods, their aspirations, their level of comfort, their visions for the future, and their dignity, and so on and so forth.

Even if it is too frequently the case that some in IT management do not often consider the human factors in their decision making, that is no excuse to use that very same simplistic and inhumane instrumental reasoning in order to describe, analyse and criticize the big IT picture. Of course, and from time to time, it is sometimes necessary to be brutally honest – and from ones own perception of the situation and possible solutions, as right or wrong as they may be – in order to provoke educated debate, new ideas and agile change. Therefore the intention is of course to address the issues at hand with sound thought and the required levels of seriousness.

Landscape

From banking to airlines, through communications businesses to pharmaceutical companies, the IT landscape is littered with failures of Homeric proportions, lost opportunities and profligate waste. Across the spectrum of commercial business the IT bottom line is inevtiably familiar: in general, businesses expose themselves to unnecessary levels of disruption, and spend far too much time and money on IT projects, IT products and internal IT services that frankly suck.

Too many internal corporate IT organizations are consistently failing to deliver projects on time, or to budget or to deliver what the business sponsors have asked for. IT continually fails on deliver any tangible business benefits apart from maintaining the businesses dependency on computing.

IT will block, mismanage, exaggerate project complexity or otherwise run business projects off the rails frequently because the business requirements endanger the hidden agendas of senior managers or middle-managers in IT organisations. Relationships between internal IT staff and external vendors can reach levels whereby the IT managers may actually be appear to be more responsive to the concerns of the suppliers than the needs of their own business users. Whatever the motivation is, the outcome is that IT either invests miserably or not at all on IT projects that can provide business value.

The negative impact of a badly chosen, badly executed and badly delivered IT project has far reaching ramifications, both for the business and for the fate of future IT projects. Most of the problems regarding IT failure can be found in the IT organizations themselves. All too frequently business IT projects that only require a modest budget are shunned in favor of big-ticket failures. IT organizations do not like simple, quick and effective, they do not see any justification in small and effective projects. Many organisations feed their overfatted infrastructures on unnecessary complexity, rampant bureacracy and plain obtuseness.

IT has been left alone for far too long, usually stuck between a rock and a hard place, IT has become an activity in itself, it’s sole task is its own survival, and its benefits to the business are typically negligible if not downright negative. In social terms, IT has become the rich-persons idiot child.

It’s not as if IT has run out of steam, the potential for IT in business is vast and probably open-ended. There are many areas of business that can benefit from innovative IT based applications, sometimes requiring large project development, but some of the most significant IT projects are usually smaller.

A CEO should know all of this and much more. IT in business must face a radical overhaul, it cannot be allowed to die, but neither can it be allowed to live as a purely self-serving organism. This is where the CEO must dictate how, why, when and where IT functions. A CEO must have the courage to change and to change IT, in order to ensure that IT works for the business.

To your own self be true.

Far too few CEOs regularly think about IT on a habitual basis, and especially as something that is potentially integrated into everything their companies do and could do. This is however, not at all surprising, there is a general lack of time for any CEO to be always actively involved with what IT does and they will generally have some degree of misunderstanding of both the historic roles and the relative successes and failures of IT.

So, the truth is out on the streets again. A recognition that many IT projects still fail miserably to deliver business value, that many IT organizations are at best ineffectual and that many businesses inevitably suffer from the repeated application of undesired and mediocre IT practices. The continued feebleness and intermittent malaise which is reflected throughout the rich panoply we call Information Technology means that businesses miss opportunities, teams miss deadlines and management waste resources and people – not to mention the impact it has on staff moral. IT failure is indeed avoidable, yet simple and necessary practices in IT projects and in IT organizations are time and time again, given scant attention or are simply ignored.

IT can make a difference for a business, even if it just makes life in a business much easier for everyone who works there. Currently IT has been vastly more of a disappointment than a blessing. From experience, I am convinced that IT can be changed to become a value-adding element in business, I am also convinced that effective change cannot be a question of just changing labels and moving some people around. Change has to come in vision, structure and process – this will initially push IT organizations further down the path of short-term alienation (if they are not already there) but will also bring about an era of enlightenment in IT in business. So whatever may be the harsh reality of short-term change will be more than compensated for by rapid mid-term success and deeply-seated long-term acceptability.

Anyone in the position of CEO who is satisfied with IT in their organization is either very fortunate to have a very professional IT team or is a palpable fool. There have been so many more bad cases of IT than good that it would tend to indicate that satisfaction with IT is either thanks to a very capable team, a question of lucky or can be frequently ascribed to mutual ignorance.

The reasons why IT projects and organizations fail are many and varied. The reason why many a CEO has not tried to fix the problems are not so numerous: it’s mostly a question of time, time or time. One other key reason is the lack of a substantial body of knowledge that would help a CEO get to grips with the issues rapidly and be in a position to order the implementation of an organization change program to change the face of IT in the business.

If you think that many aspects of IT in your organization sucks then this book will help you to turn things around. It will help you to approach these problems because the time factor will be dramatically reduced and it will also provide a core body of knowledge to help avoid reinventing proven practices.

Intentional Dilemma - Before honor comes humility

The original intention was not to write this book but a short essay, a hard-hitting criticism of the litany of IT failures. Of course this is rarely enough, it wasn’t enough because it identified the issues without provoking debate, and it highlighted common issues without providing guideline suggestions, which meant that it appeared upon reflection to come across as too much like a hi-tech hit and run exercise, beating up on IT failures without actually offering tangible solutions.

Therefore, the motivation in providing this material is in an honest attempt to correct a serious problem and not specifically to apportion blame or to identify scapegoats, this is never helpful in learning from the lessons of the past. Most of us know that it’s relatively easy to criticize failure but much harder to explain how those failures can be avoided and how success can be achieved.

Subsequently my thoughts turned to explaining to executives in business how to understand the issues in order to radically improve the contribution of IT in the shortest possible time, this I knew could be done – based on past experience and knowledge of IT and Business.

But what about all the people currently working in IT, obviously any hard criticism and radical suggestions for change would not be too welcome, and here I was, first bashing people in IT for failure, telling executives in their organization that their IT probably sucks and then showing them how to purposefully change the situation for the better.

This was really the real dilemma that had to be overcome in writing this book. This is a question of:

§ Identifying the problems
§ Identifying the reason for the problems
§ Identifying how those problems effect business
§ Gauging the expediency of minimizing or eradicating the identifiable problems
§ Identifying needed changes
§ Identifying processes and structures needed to bring about change
§ Identifying roles and responsibilities in the process of change
§ Providing a commercial and moral foundation for that change
§ Articulating a cohesive and coherent rationale for change

I really felt as though I had something to say, something to write about, a positive contribution with unavoidable negative aspects. This created a dilemma for me – which lead to the dialectic, which would define the purpose, the content and the structure of the book.

Compound Fracture

Twenty years ago, you wanted a new bathroom, so you hired a building contractor to come in and build a bathroom to your design.

Twenty years on, the building contractors are still in your house, they come and go as they please, they traipse mud and dust and paint all through your living room, they leave the toilet seat up, they smoke in your bedroom and they use all your coffee. In fact, they are a part of everyday life. Before you get righteously outraged there is some good news. The builders have made you some nice French windows, a completely integrated cat flap and a dual use patio, finally, they have started work on making your bathroom, although with some minor specification changes, such as: ceramic instead of marble; a bath instead of a bath and shower; a TV instead of a window and, as the icing on the cake, the cold water “feature” now works - well, most of the time.

But you are still hopping mad, you have been getting progressively angrier over a period six months, ever since you threatened to throw them all out and get in different builders to finish the job. But they tell you not to worry, all you need to do is pay an additional 200% and then everything will be just perfect, and indeed, they can now almost guarantee that they will deliver, what you might want, at some time in the future. That was the when the incumbent cunningly convinced you that they had finally come up with the perfect solution: a new twelve-step method, which they called Building Scientific Value (BSH).

They say that building for them has now become a philosophy, that they are entering the age of building enlightenment, and the mistakes of the past were because they were unduly influenced by building tradition and unfamiliar tools and materials.

They welcome you to step out of the darkness and bask in the sunshine of their newly acquired illumination. Naturally, they see skepticism, after all, the builders are the ones who not only failed to deliver what was promised, but also have gone half way to destroying your house. But they say, don’t fear, BSH has arrived in the nick of time. The Builders Science company developed BSH from all they know about building and working, and making coffee. They learned from working on your house, they tell you that their new method is a flexible framework that can be applied to any household – it’s a reliable method for making complicated building value judgments.

They have impressed you so much you listen to them articulate their 12-point method.

This piece is dedicated to possibly the worst senior IT management morons, idiots and imbeciles that I have I have had the distinct misfortune to come across "all in one place" and "at the same time" ... in my entire life! Naming no names - names witheld to protect the guilty; one more time - but would you trust these people with the custody of your most precious assets?

Saturday, June 2, 2007