The High Cost of Failure
高昂的失败成本
DevOps leaders talk about “failing fast and failing early,” “leaning
into failure,” and “celebrating failure” in order to keep learning.
Facebook is famous for its “hacker culture” and its motto, “Move
Fast and Break Things.” Failure isn’t celebrated in the financial
industry. Regulators and bank customers don’t like it when things
break, so financial organizations spend a lot of time and money trying
to prevent failures from happening.
DevOps的领导者们谈论“快速失败和早期失败”，“融入失败”以及“庆祝失败”则是为了不断地学习。Facebook以其“黑客文化”和“快速前进打破陈规”的座右铭而闻名。失败在金融界是不被庆祝地的。当问题出现时，监管机构和银行客户是不会喜欢的。所以金融机构花费大量的时间和金钱来尝试防止故障发生。

Amazon is widely known for the high velocity of changes that it
makes to its infrastructure. According to data from 2011 (the last
time that Amazon publicly disclosed this information), Amazon
deploys changes to its production infrastructure every 11.6 seconds.
Each of these deployments is made to an average of 10,000 hosts,
and only .001% of these changes lead to an outage.
At this rate of change, this still means that failures happen quite
often. But because most of the changes made are small, it doesn’t
take long to figure out what went wrong, or to recover from failures
—most of the time.
亚马逊因其对基础设施的高速变更而广为人知。根据2011年的数据（最后一次亚马逊公开披露该信息的时间），亚马逊每11.6秒就对生产基础结构部署一次变更。每个部署的平均主机数为10000台，只有0.001%的变化会导致宕机。
在这种变更(成功)率下，这仍然意味着失败会发生经常。但是，因为所做的变更大部分都很小，所以没有花很长时间去弄清楚出了什么问题，或者从失败中恢复过来。-大部分时间是这样的。
Sometimes even small changes can have unexpected, disastrous consequences.
Amazon EC2’s worst outage, on April 21, 2011, was caused by a mistake made during a routine network change. While Netflix and Heroku survived this accident, it took out many online companies, including Reddit and Foursquare, part of the New York Times website, and several smaller sites, for a day or more. Amazon was still working on recovery four days later, and some customers
permanently lost data.2
有时候，即使是微小的变化也会带来意想不到的灾难性后果。2011年4月21日，亚马逊EC2最严重的宕机事故是由于在常规网络变更过程中出错而导致的。虽然Netflix和Heroku在这次事故中幸免于难，还是有很多互联网公司因此受到影响，包括Reddit和的纽约时报的子网站Foursquare，以及几个较小的网站。这些公司的受到的影响持续了一天以上一天。亚马逊在事件发生后的第四天后仍在进行恢复工作，一些客户则永久丢失的数据。
When companies like Amazon or Google suffer an outage, they lose
online service revenue, of course. There is also a knock-on effect on
the customers relying on their services as they lose online revenue
too, and a resulting loss of customer trust, which could lead to more
lost revenue as customers find alternatives. If the failure is bad enough that service-level agreements are violated, that means more
money credited back to customers, and harm to the company brand
through bad publicity and damage to reputation. All of this adds up
fast, on the order of several million dollars per hour: one estimate is
that when Amazon went down for 30 minutes in 2013, it lost
$66,240 per minute.
当像亚马逊或谷歌这样的公司遭遇停电时，他们会输
当然，在线服务收入。还有一种撞击效应
客户由于失去在线收入而依赖于他们的服务
也会导致客户信任的丧失，从而导致更多
由于客户寻找替代品，收入损失。如果故障严重到违反服务级别协议，则意味着更多
把钱还给顾客，损害公司品牌
通过不良的宣传和名誉受损。所有这些加起来
快，每小时几百万美元的订单：一个估计是
当亚马逊在2013年下跌30分钟时，它输了
每分钟66240美元。

This is expensive—but not when compared to a failure of a major
financial system, where hundreds of millions of dollars can be lost.
The knock-on effects can extend across an entire financial market,
potentially impacting the national (or even global) economy, and
negatively affecting investor confidence over an extended period of
time.
这是昂贵的，但与金融系统的一次重大故障不能相提并论。后者的损失规模动辄就是几亿美元。连锁反应可以延伸到整个金融市场，可能影响国家（甚至全球）经济，以及在相当长时间内对投资者信心产生负面影响。
Then there are follow-on costs, including regulatory fines and lawsuits,
and of course the costs to clean up what went wrong and make
sure that the same problem won’t happen again. This could—and
often does—include bringing in outside experts to review systems
and procedures, firing management and replacing the technology,
and starting again. As an example, in the 2000s the London Stock
Exchange went through two CIOs and a CEO, and threw out two
expensive trading systems that cost tens of millions of pounds to
develop, because of high-profile system outages. These outages,
which occurred eight years apart, each cost the UK financial industry
hundreds of millions of pounds in lost commissions.
然后是后续成本，包括监管罚款和诉讼，当然还有解决问题并确保同样的问题肯定不会再发生的成本。作为惯例，这通常会包括聘请外部专家来审查系统和规程，解雇管理层并替换技术系统以重新开始。例如，在2000年代，因为广受关注的系统宕机事件，伦敦股票交易所经历了两个首席信息官和一个首席执行官，并替换了两个花费数千万英镑开发的昂贵交易系统。这两次事件，它们相隔八年，每一次都让英国金融业付出了数亿英镑佣金的代价。

The risks and costs of major failures, and the regulatory requirements
that have been put in place to help prevent or mitigate these
failures, significantly slow down the speed of development and
delivery in financial systems.
重大故障的风险和成本，以及为了预防这些失败或者减轻由此造成的影响所引入的监管要求，大大降低了金融系统开发和交付的速度。