It was only a year ago, but oh how the polish seems to have worn off. Those wild, heady days where the Payment Card Industry Data Security Standard was ‘my new religion‘ were some of the happiest. It didn’t matter who I was talking to – and it certainly didn’t matter if they were interested in the topic – I would stop at nothing to educate on the wonders and achievements that were the Standard.
In the web hosting world, where I spend far too much of my time, the Standard sets out a list of requirements for the hosting provider to guide them on how best to secure their systems. More importantly, it tells the provider how to pass the requirements of the Standard so that their customers are not in breach of their existing contracts with their bank. That’s right, every merchant agreement requires that the merchant be compliant with the Standard, or they could be liable for any losses resulting from a security breach.
When I became manager of a web hosting company 18 months ago, I set out to implement a PCI DSS product for our customers to use. Any business approaching us to host an e-commerce site would be offered the most resilient, all-complying hosting solution money could buy. My suprise? Nobody wanted to buy it.
Defending the Standard
PCI DSS really is great stuff, and I can’t be clear enough on that. It lays out best-practices for an industry (web hosting) that doesn’t really have any. I’d be surprised if any business wants to host their customer’s or their own data in a manner other than what the DSS recommends. If you are compliant with the Standard’s requirements, then the chances of you experiencing a security breach are significantly reduced.
The best way to defend the Standard is simply to say that it’s common sense. And it is! If you doubt that, and many do, consider the following examples of PCI DSS’ requirements. Would you drop any of them and feel just as secure as you did before?
- Database servers not accessible to the Internet
- Web servers behind an Intrusion Prevention Firewall
- Up-to-date virus scanners and vulnerability management
- Police background checks on anyone with high-level access to the hosting environment
- Regular security scanning and network penetration tests
- Proper information security and response policies
What surprised me when I first started looking at this as a product to develop for my customers was the sheer lack of any competitors. From countless searches, I could come up with only 2 other Australian hosting providers who said “we are PCI DSS compliant”. But… every e-commerce site must be PCI compliant, right? This is in their contract with the bank. Here’s an entirely untapped, niche market. It all seemed to good to be true.
It was…
The problem with the Standard
The (only) problem with the Standard is, ironically, a financial one. It may be best to explain this in practical terms…
Imagine yourself as the (Australian) operator of the medium-sized e-commerce site, fancycheeseshop.com, and you host that site in Australia. Your site’s revenue is $500,000/year, or ‘medium-sized’, and your net profit every year is $100,000 or 20%.
Sounds pretty decent for a small-to-medium e-commerce site to make 500k/y. Now consider the cost of three different hosting solutions:
- El-Chepo U.S. Hosting Co. $5,000/yr. Just a basic virtual server hosted in the US. No added security or features.
- Standard Australian Dedicated Hosting. $15,000/yr. Dedicated combined web/db server. No PCI compliance, but good enough for most.
- Elite Australian PCI DSS Compliant Hosting. $48,000/yr. Everything you need to meet your contractual obligations to the bank.
These numbers are quite realistic, so I’m sure you see the problem here. The cost of being PCI Compliant (in Australia at least) is prohibitive. Would you spent half of your yearly profit to satisfy the Standard? I would not.
But why is PCI DSS so expensive? More than anything, it comes down to the cost of operating an environment that needs to be a lot more complicated than your ‘average’ hosting, where your host doesn’t care what you’re hosting. Here’s just a few of the things that we provide our PCI DSS customers that runs up the bill quite a bit:
- Centralised log collection software with tamper-proofing.
- Physically isolated web and database servers (read: duplication of infrastructure)
- SSL-offloading Intrusion Prevention System (that’s one really expensive piece of equipment)
- External security scans and auditor fees
- Housing in a PCI-compliant data centre (about 25-40% more expensive than a regular data centre)
My experience in offering this solution to our customers for 12 months has been rather disappointing. I believe that our adherence to the DSS has actually lost us business. When an e-commerce customer approaches us after spending 200k on a shop, and I tell them that I’ll charge them 25% of that every year. Well, of course they talk to my competitors.
My competitors don’t care if you’re PCI compliant or not. Many of them don’t even know what it is. When they propose a regular hosting solution, it is invariably 30% of what I proposed.
What’s The Solution?
I really don’t know. I will continue to offer PCI DSS compliant hosting to those that want it, but can I afford to be proposing such expensive solutions when none of my competitors are doing the same? Or what my prospective client isn’t the least bit interested? Should I simply stop offering it unless someone big enough knows they need it?
Personally I feel these things are irresponsible, and I believe that’s what makes our hosting company a bit different to all the rest. We try our best to make sure the client is doing the right thing in this space, which they are generally unfamiliar with. As another example, do we wait for a major corporation to ask for Disaster Recovery Plan (generally when it’s too late), or do we offer it them up-front?
Perhaps the governing body, the Payment Card Industry Security Standards Council, needs to be doing more to raise awareness of DSS and their other standards? Or maybe our own governments should be legislating more to protect not just credit card data, but all personal data online?
I welcome your comments and feedback on this subject.
I received the following feedback from Adrian Sanabria (@sawaba), a QSA over in the US. I wanted to share his feedback with anyone else coming along because I think he raises some really good points. ”
—
You were correct that PCI compliant web hosting will be more expensive than regular hosting. There is no way around that, and I’m not arguing that. However, it sounds to me like you took PCI as an excuse to geek out a bit on the security. To make an analogy, it sounds to me like when your employer told you that you’d need “reliable transportation” for this job, you went out and bought a Holden Monaro.
Before I get into specific examples, I want to make it clear that I am not up on hosting costs, equipment costs, or any of that. I can’t tell you how much a year of PCI compliant hosting should cost, but here’s where I think you went a bit overboard.
* Centralised log collection software with tamper-proofing.
YES, agreed here, though you could offer to ship the customer their logs if they already have a log collection system (make this optional, in other words)
* Physically isolated web and database servers (read: duplication of infrastructure).
That is the traditional way of handling it – you can use logical isolation in virtual infrastructure to satisfy the segregation requirement though, which should drastically drop power and hardware costs.
* SSL-offloading Intrusion Prevention System (that’s one really expensive piece of equipment)
PCI requires a basic network IDS, not an SSL-offloading IPS. What is the cost difference between the SSL-offloading IPS, and a standard Sourcefire or homemade Snort box? Some customers would certainly appreciate the extra security, but it is WAY overkill for checking the PCI box under requirement 11.
* External security scans and auditor fees
External scans are required, but this could be optional, again making the package cheaper for the customer, though they’d still need to pay an ASV to do the scans one way or another. As for auditor fees, I don’t think anyone at the $100,000 profit level is going to be required to have an external audit for PCI. Most at that profit level are Level 3 merchants.
* Housing in a PCI-compliant data centre (about 25-40% more expensive than a regular data centre) I’d really like to see a breakdown of the reasoning for the additional expense. 99% of the time I find a Data Center non-compliant, it is because they don’t have a log book for visitor sign-in. A three-ring binder and some forms are really all you need, as long as you already have some sort of access control in place for the data center (prox cards, biometrics, etc.), and I’ve never seen a data center without some sort of physical access control.
I hope that, perhaps, a lot of the extra expense you’re seeing is a result of misinterpretations of the PCI requirements, and you can still salvage the PCI hosting product idea. I think it is a really good idea, and it would make my job a lot easier if service providers were already prepared for PCI when I walk in (I am a QSA, by the way).
Honestly, in my opinion, you may get some additional customers if you can drop the price of the hosting package some, but in my experience, only one in fifty or more will volunteer themselves for the PCI-level hosting unless an auditor tells them they must. The rub is, once you get to the level 2 and level 1 merchants, less of them are using hosting, and are doing the web hosting out of their own data centers.”
—-
I agree a lot with Adrian’s comments above. I do have a tendency to geek out at every opportunity. Taking the SSL IPS stuff as an example, while there isn’t a requirement to specifically have an IPS that can decrypt and inspect SSL traffic, does it not defeat the purpose? E-commerce sites will be encrypted, so a channel could exist there for an attacker to try all kinds of attacks against the web-app without being hindered by an IPS that implicitly trusts encrypted traffic. There are, of course, cheaper alternatives to the way we’ve done this however.
In Australia, which is all I can really lend experience on, the average data centre is definitely not PCI DSS compliant, and makes no effort to be. The average cost of a rack in these environments is between $1,500 and $1,700/month. We have a very (very) limited number of higher-end facilities that are PCI DSS compliant, where a rack costs from $2,000 to $2,500/month. There isn’t really a breakdown I can show Adrian other than my own procurement experience.
In essence I take the point, PCI DSS would always be more expensive, but I’ve made it more expensive than it needed to be by ‘geeking out’ on it
“while there isn’t a requirement to specifically have an IPS that can decrypt and inspect SSL traffic, does it not defeat the purpose?”
This is an easy trap to fall into. When talking to clients, I always make them aware of the best choice from a security perspective, versus what is required to satisfy the PCI requirement. I always recommend going above and beyond the requirements, as they only represent a bare minimum in security.
However, Mike, in your case, we’re talking about the difference between someone choosing PCI compliant hosting over non-compliant hosting. Perhaps it would make sense to have a PCI hosting package that just meets the requirements, and another that surpasses them – a PCI “Plus” package?
It seems to me that what you’ve developed is a well secured hosting environment that would probably help to keep companies out of the newspapers. I think you did what any good exerienced security practitioner would do.
I think the issue is with your positioning of the solution as PCI DSS compliant hosting. We have a big problem right now in that PCI is the goal for many organizations. Talk to the PCI SSC; talk to Mastercard; talk to Visa. They will all tell you the same thing that most that work in this field will tell you. PCI DSS is a baseline.
But in the industries that would be your target customers… many are still simply shooting for “what does PCI say we need to do” and that’s about it.
@ken5m1th
I’m not so sure on some of the cost differentials.
As a QSA in my other life, I have visited around 18 different data centres for shared hosting.
With 1 exception, all of them fail PCI for the lack of CCTV on the customer’s data systems (not just covering the door to a floor with hundreds of customer staff wandering around). 4-8 additional cameras, record on motion cannot be a 25% cost differential.
Firewalling web and database – a physical port on an existing firewall shouldn’t be that expensive.
Scanning is cheap – a few dollars per IP per year.
I know margins are low in hosting, but I’d not expect a 300% cost differential
Maybe partnering with managed services on some of this cost items is a way to reduce cost of ownership….
Lyal
I think what you’ve done sounds normal for meeting PCI requirements.
The pain you have in finding customers for such services is really the same as internal IT teams have for selling security and PCI even in our own companies. It truly takes an ultimatum/hard requirement to get anyone to do more than lip service.
For the SSL-offloading, I think you’re correct. While PCI won’t specifically say you need to offload SSL and/or inspect it, anyone with security chops will agree with you that the spirit of inspection requirements is to include visibility into 443 traffic. In many common situations, there is no other way to effectively put a web app behind a web app firewall or IDS monitor (think: Windows/IIS).
I bristle only slightly at suggestions of Snort and such. Those are suggestions that brush aside capital costs but instead replace them with operational and staff support costs. A dangerous habit to get into… I love me those tools, but there are 2 sides to that coin in an enterprise environment.
“I’d be surprised if any business wants to host their customer’s or their own data in a manner other than what the DSS recommends.”
Spoken like one of us security geeks!
Sadly, it’s not reality for so many firms. It’s not that they *want* to host their customer’s data in an insecure manner, they just don’t think about it. Instead, they do whatever provides the least resistance (just like any app or process is created in most orgs). Ignorance may sometimes be actively chosen, but way too many people stay wrapped in ignorance passively.
And so far meeting PCI for many companies is still half lip service half smoke-and-mirrors. That’s not a diss on the QSAs doing the auditing, but there is way too much grey in between them and the people who live in the environment every day, and not enough time or resources to truly *prove* most compliance (if you ask me).