11 problems with AWS – Part 1

In my previous post I espoused why IT infrastructure sucks. Many of you pointed out that “The Cloud” was the answer. Amazon Web Services is the market leader for IaaS clouds so I’m writing a series of posts documenting the hurdles (I think I’ve found eleven!) to overcome if you want to use AWS and experience the wonders of a truly on-demand infrastructure.

The first is the issue of…

Data sovereignty

passport by One-Fat-Man (via Flickr)

Data sovereignty is basically concerned with where data copies are located, how it is used, and your legal control over it.

Company data sent to a cloud provider can be located in, or transmitted through, multiple countries with different laws. The data has a different legal posture at each stop along the way.

Countries typically have laws governing Data Privacy, Compliance, Retention, Availability, Breach Notification and Data Subpoena powers. But these laws rarely match up with each other.

Privacy generally relates to personal and confidential information and examples include health and financial information. Different countries have different laws for protecting privacy of data records and different requirements of its people and organisations.

Compliance refers to standards and laws such as Sarbanes-Oxley, PCI-DSS and corporations laws. The effect of compliance is to encourage best security practice on an IT operation including physical security, security processes, system configuration and auditing. Compliance is difficult enough to achieve on-premises. Achieving this across a global cloud infrastructure can be brain-numbingly complex and difficult to audit. See this article about PCI-DSS compliance in the cloud.

Data Retention is about being able to recover company records from a “long” time ago.  Typically an application used to create the data records will have been retired or upgraded. The data records will therefore not be retrievable without firstly recovering the version of the application that generated them. This problem has been solved in some respects by data-archiving systems that regularly migrate data in a readable/standard format. But currently archiving is immature on Cloud platforms like AWS. Cloud applications are inherently upgraded in a rapid cycle and end-users won’t have the ability to recover old versions of the platform for data access purposes. The questions to be asked are: Can I get my data out regularly in a standard format? And will I be able to access this information many years down the track?

Data Availability is more complicated in a cloud environment primarily because there are more entities involved in providing a cloud platform. The legal availability requirements of data need to be understood and contracts with providers carefully assessed so that it can be met. For example if a customer’s banking records were hosted in an overseas AWS region and became unavailable, where is the accountability?

Breach notification laws vary around the world. If your data is stored in Amazon and there is a breach, does Amazon have to disclose the fact by law? If your CIO comes and asks you if your cloud data was breached, would you shrug your shoulders and say, “I don’t know”? The US and Europe have had breach notification laws for several years while other countries like Australia have lagged.

Data Subpoena is a most interesting sovereignty issue. Nation states have used economic reconnaissance for advantage for eons. States and their corporations are rightfully concerned that another sovereign state could access their data. A current example is the US concern about Chinese hacking efforts. Another is that Amazon, being a US company, falls under the Patriot Act and must secretly disclose information to the US government if requested, even if the data is not physically located in the US. Other countries have similar laws for national security reasons. There is reasonable suspicion that economic espionage will be used for advantage and therefore represents a sovereign risk.

Data is the currency of business. When your data leaves your premises, you potentially lose control, oversight and therefore business advantage. How do you solve this problem? What controls can you put in place? Please share you thoughts

Why IT Infrastructure sucks

There’s a young hipster making his way up through the ranks. He’s the “great hope” and is given an ambitious project to run. The project will make the company a bucket of cash. He’ll win awards etc.

He assembles his team. They draw wire diagrams, make project plans and hack code on a few old PCs. He assembles a “business case”: a PowerPoint with impressively opaque “business language”. He asks for money. Something like:

  • 10 people (project manager, developers, testers, designers etc.)
  • 6 months or 110 work days (A year is typically 220 workdays)
  • $1000/person/day (damned consultants!)
  • Multiplication gives a $1.1million budget

The budget expectation is set and he goes about getting approval (going for coffee) from management.
Imagine the sinking feeling when he gets to the IT Infrastructure team. The IT infrastructure guy hits him with annoying questions like:

How many hits will your site get? What are the growth projections? What is the impact if the site goes down? Does it run our middleware? How important is the data?

He answers as best he can:

The site must never go down of course. We better have a backup site. We’ll need somewhere to develop and test. What do you mean I have to have a performance test site?

The IT infrastructure comes back in two weeks with the following high-level costing and design:

"IT Infrastructure" "virtualization" "cloud" "web application"

  • 16 servers, 5 databases
  • Resourcing: ~200-300 days – ~$350,000
  • Hardware & Software: $100,000
  • Total ~$450,000

The “great hope” flinches! An extra half a million dollars! But computers are so cheap on eBay! His mate runs a start-up uses hosting that costs a few bucks a month.
He flies professionally through the stages of grief: Denial, Anger, Bargaining, Depression and Acceptance. He’ll have to re-set budget expectations.

When can this be delivered, he asks?

The IT Infrastructure guy:

We can’t start your project for 2-4 weeks because noone is available. You should have told us about your project 6 months ago.  It’ll take 3 months to get you the platforms.

Is this true of your IT world? Any stories to share? Or is your IT shop squeaky clean? Leave a comment below:

Introduction

In year 11 my German teacher marked me with a 66% score. It was too much for my pride to bear. My self-image was that of a nerd, always getting 90% plus. The thought of doing German in year 12 and “failing” dizzied me. I dropped German and chose Computer Science. In year 12 I achieved 100% in Computer Science.

I ambled along though a Computer Engineering degree. Each morning at Monash university I would proudly start the day with a coffee scroll and Pepsi from the Engineering faculty cafe.

I’ve worked about 8 different jobs both permanent and as a contractor, worked in England and Australia, and done my share of geeky other things.

Most of the systems I’ve helped put in have been replaced. There’s not much to show for 18 years in the field. But over that time I’ve grown a fascination for how different businesses use IT, make money and become successful.

This blog is a place to leave a trail and state my opinions about technology and business and for other geeks to shoot them down!

I will write about new technology, people in IT, organisational dysfunction, and my own life-learnings as an IT geek.

I’ll try to write lean and jargon free. I’ll even use diagrams where possible. I’ll try to simplify. Basically the opposite of every other tech blog.

Oh… and I still wish I knew how to speak a second language

Load more