Compare commits

...

No commits in common. "7946be9e1c048b31aedb9325dad949e9a69b0713" and "7c928c901c59a600fc7c4a35ea71f579af6d3b90" have entirely different histories.

34 changed files with 93 additions and 1353 deletions

23
.gitignore vendored Normal file
View File

@ -0,0 +1,23 @@
# ---> Go
# If you prefer the allow list template instead of the deny list, see community template:
# https://github.com/github/gitignore/blob/main/community/Golang/Go.AllowList.gitignore
#
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib
# Test binary, built with `go test -c`
*.test
# Output of the go coverage tool, specifically when used with LiteIDE
*.out
# Dependency directories (remove the comment below to include it)
# vendor/
# Go workspace file
go.work

235
LICENSE
View File

@ -1,201 +1,100 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
GNU GENERAL PUBLIC LICENSE
Version 1, February 1989
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
Copyright (C) 1989 Free Software Foundation, Inc. 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
1. Definitions.
Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
Preamble
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
The license agreements of most software companies try to keep users at the mercy of those companies. By contrast, our General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. The General Public License applies to the Free Software Foundation's software and to any other program whose authors commit to using it. You can use it for your programs, too.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
When we speak of free software, we are referring to freedom, not price. Specifically, the General Public License is designed to make sure that you have the freedom to give away or sell copies of free software, that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
For example, if you distribute copies of a such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must tell them their rights.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations.
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
The precise terms and conditions for copying, distribution and modification follow.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
0. This License Agreement applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any work containing the Program or a portion of it, either verbatim or with modifications. Each licensee is addressed as "you".
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this General Public License and to the absence of any warranty; and give any other recipients of the Program a copy of this General Public License along with the Program. You may charge a fee for the physical act of transferring a copy.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
2. You may modify your copy or copies of the Program or any portion of it, and copy and distribute such modifications under the terms of Paragraph 1 above, provided that you also do the following:
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
a) cause the modified files to carry prominent notices stating that you changed the files and the date of any change; and
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
b) cause the whole of any work that you distribute or publish, that in whole or in part contains the Program or any part thereof, either with or without modifications, to be licensed at no charge to all third parties under the terms of this General Public License (except that you may choose to grant warranty protection to some or all third parties, at your option).
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the simplest and most usual way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this General Public License.
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
d) You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee.
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
Mere aggregation of another independent work with the Program (or its derivative) on a volume of a storage or distribution medium does not bring the other work under the scope of these terms.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
3. You may copy and distribute the Program (or a portion or derivative of it, under Paragraph 2) in object code or executable form under the terms of Paragraphs 1 and 2 above provided that you also do one of the following:
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
a) accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Paragraphs 1 and 2 above; or,
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
b) accompany it with a written offer, valid for at least three years, to give any third party free (except for a nominal charge for the cost of distribution) a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Paragraphs 1 and 2 above; or,
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
c) accompany it with the information you received as to where the corresponding source code may be obtained. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form alone.)
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
Source code for a work means the preferred form of the work for making modifications to it. For an executable file, complete source code means all the source code for all modules it contains; but, as a special exception, it need not include source code for modules which are standard libraries that accompany the operating system on which the executable file runs, or for standard header files or definitions files that accompany that operating system.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
4. You may not copy, modify, sublicense, distribute or transfer the Program except as expressly provided under this General Public License. Any attempt otherwise to copy, modify, sublicense, distribute or transfer the Program is void, and will automatically terminate your rights to use the Program under this License. However, parties who have received copies, or rights to use copies, from you under this General Public License will not have their licenses terminated so long as such parties remain in full compliance.
5. By copying, distributing or modifying the Program (or any work based on the Program) you indicate your acceptance of this license to do so, and all its terms and conditions.
6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein.
7. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the Program specifies a version number of the license which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the license, you may choose any version ever published by the Free Software Foundation.
8. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally.
NO WARRANTY
9. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
10. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
Appendix: How to Apply These Terms to Your New Programs
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
If you develop a new program, and you want it to be of the greatest possible use to humanity, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms.
Copyright [yyyy] [name of copyright owner]
To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
readnotes Copyright (C) 19yy bing
http://www.apache.org/licenses/LICENSE-2.0
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 1, or (at your option) any later version.
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) 19xx name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program.
You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (a program to direct compilers to make passes at assemblers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989 Ty Coon, President of Vice
That's all there is to it!

View File

@ -1,2 +1,2 @@
# readingnotes
个人读书笔记
# readnotes

View File

@ -1,111 +0,0 @@
# 第一章 SRE与Devops之间的关系
运维是一门很难的学科。
不应将运维视为一个成本中心,否则想要取得实质性的改进变得举步维艰。
## 关于DevOps的背景情况
DevOps是一套松散的实践理论、指导原则和工作文化旨在打破IT组织中开发、运维、网络和安全各自为政的局面。
### CA(L)MS
**C**ultrue 文化
**A**utomation 自动化
**L**ean continuous 精益持续改进
**M**etrics 监控指标
**S**haring 分享
### DevOps核心思想
- 不再各自为政,即打穿门墙。
- 意外乃兵家常事。 意外不仅仅是某个人的孤立行为而导致的,而是因为在发生了不可避免地意外故障时,保障措施的缺位。
- 变更要小步快跑。每次的变更要小一点,做的次数要多一点。这种小步快跑的模式,再结合自动测试,出错回滚,就可以实现变更管理方式的转型了,变迁到持续集成(CI)和持续交付或部署(CD)的模式
- 工具与文化休戚与共。 工具是DevOps非常重要的组成部分但企业文化更应受到高度重视。良好的文化可以解决糟糕的工具造成的麻烦。
- 准确的度量至关重要。 在业务的视角中度量显得尤为重要,尤其是在避免各自为政和解决事故方面。
## 关于SRE的背景情况
SRE是一个工作岗位是一系列工作的实现方式以及一些使这些工作实践充满动力的信念。如果DevOps是一种理念和工作方法SRE实现了DevOps中所描述的部分理念而且比DevOps所定义的工作更加具体、清晰。
### SRE构成原理
- **运维的痛点也是软件问题** SRE的基本信条做好运维是一个软件开发问题。
- **以服务质量目标(SLO)为准绳** SRE不会企图提供100%的可用性。产品团队和SRE团队会为服务的用户群体选择合理的可用性目标并以SLO为准绳来管理该服务。
- **尽量减少琐事** 对于需要由机器来做的运维任务,通常情况下就应该让机器去完成。琐事绝对不是工作,项目工作才能帮助我们让服务更可靠,更具扩展性。
***生产的智慧:通过在生产环境中运行事物而获得的智慧***
- **确定本年度要自动化的工作** 要明确哪些内容需要自动化,在什么条件下自动化,如何自动化。
- **故障解决的越快,进度就越快** SRE参与工作的主要收益不只是提高可靠性还改善了产品开发的输出。
- **与开发人员同舟共济(Share ownership)** SRE倾向于关注生产问题而不是业务逻辑问题。理想情况下产品研发和SRE团队都应该对产品技术栈有整体的了解。
- 岗位虽不同,工具可统一。
## 比较与对照
**DevOps与SRE的共性**
- 实施改变是进步的源泉。
- 协作与共同负责制
- 小步快跑的做变更
- 使用正确的工具
- 度量
- 对事不对人
- 一种整体行为: 通过特别的方式共同合作,使整个团队变得更好。
**DevOps与SRE的差异**
DevOps是一种更宽泛的理念和文化。SRE的职责定义就相对狭窄其工作通常是面向服务而非面向整体业务。
SRE与DevOps所认可的东西相同但认可的原因略有不同。
## 因地制宜才能事半功倍
有效的运维方法都是相似的,而出问题的方式各不相同。
- **片面、刻板的激励机制会阻碍成功** 不要将激励措施与发布相关的,可靠性相关的成果联系在一起。要允许回撤那些难以驾驭,无可救药的产品支持。
- **解铃还须系铃人,勿怨他人** 请完全消除将生产事故或系统故障归咎于其他团队的念头。
- 应当允许并鼓励在产品需要时修改代码和配置。同时授予职责范围内较高的自由度
- 支持对事不对人的事后总结报告。
- **维护可靠性是专业化角色** SRE和产品开发可以是两个独立的部门每个团队都可以有自己的工作重点、优先事项和管理模式而不必和其他的团队一一对应。这两个团队是相辅相成的。
- **毋庸斟酌是否,只需推敲时机**
- **尽量在职业发展和物质激励上一视同仁**
## 小结
推荐书籍
- Site Reliability Engineering
- Effective DevOps
- The Phoneix Project
- The Practice of Cloud System Administration: DevOps and SRE Practices for Web Service, Volume 2
- Accelerate: The Science if Lean Software and DevOps

View File

@ -1,5 +0,0 @@
# Google SRE 工作手册
目录
[第一章 SRE与Devops之间的关系](1.md)

View File

@ -1,2 +0,0 @@
# 第一章
本章暂缺。

View File

@ -1,13 +0,0 @@
# 第十章 基于时间序列数据进行有效报警
监控处于整个生产环境需求金字塔模型(参见[第三部分 具体实践](p3.md))的最底层。监控是运营一个可靠的稳定服务不可缺少的部分。
***本章主要讲Google的内部监控系统`borgmon`, `Prometheus`是`borgmon`的一个开源实现***
## 应用软件的监控埋点
Google内部产生的每个二进制文件中都默认包含一个HTTP服务同时每种主流编程语言都提供了一个编程接口可以自动注入暴露指定的程序变量。软件服务器作者可以随后对这个变量进行简单地增加操作或者直接修改这个变量的值。Go语言中的expvar标准库和他的JSON输出格式也提供了类似的API。
## 监控指标的收集
一套服务发现体系可以降低监控系统的配置维护难度,允许监控系统自动扩展。

View File

@ -1,71 +0,0 @@
# 第十一章 on-call轮值
on-call轮值是很多运维和研发团队的重要职责这项任务的目标是保障服务的可靠性和可用性。
on-call工程师的响应时间通常跟业务的可靠性有关。不同的SLO响应时间不同。
## on-call工作平衡
SRE团队对on-call工作的质量(工作压力)和数量有明确的要求,这两个维度必须保持在一个可持续的水平上。
### 数量上保持平衡
SRE团队50%的时间花在软件工程上,其余的时间中, 不超过25%的时间来on-call另外25%时间来处理其他运维工作。
SRE更多地推荐团队
1. 避免夜间任务影响健康(多地时差)
2. 限制人员数量,保障对生产环境的熟悉程度
### 质量上保持平衡
每次on-call值班过程中轮值工程师必须有足够的时间处理紧急事件和后续跟进工作,如写事后报告。
一系列根本原因或相关的事件和报警信息并且这些事件应该在同一个事后报告中讨论,这些事件称为紧急事件。
平均处理一个生产环境事件需要约6小时所以每12小时最多2个紧急事件。
### 补贴措施
为保障团队按需参与on-call同时避免过量on-call带来的问题(如项目开发时间不够或疲劳过度), 管理层需要考虑针对工作时间之外的on-call工作给予合理的补贴。
## 安全感
为确保on-call工程师保持理性、专注、有意识地进行认知类活动而不是依赖直觉、自动化、快速行动 必须减轻on-call所带来的压力感。
让on-call可以使用以下资源来寻求外部帮助以减轻on-call压力
- 清晰的问题升级路线
- 清晰定义的应急事件处理步骤
- 无指责,对事不对人的文化氛围
生产系统对应的开发团队也会参与on-call轮值SRE团队永远可以寻求这些伙伴团队的帮助。
对于非常复杂,需要多团队参与,或经过一段时间调查仍不能确定恢复时间的问题,应考虑启用某种正式的应急事务流程。
在应急事件处理解释时,应总结处理事件过程中做的好与不好的地方, 并采取措施避免再次发生同样的问题。
## 避免运维压力过大
当SRE团队需要50%以上的时间来处理运维工作时那么SRE团队的运维工作压力就过大。
通过以下途径解决运维工作压力过大的问题:
- SRE团队和管理层负责在季度工作计划中加入一些应对措施如从其他团队临时抽调一名有经验的SRE。
- 运维压力过载应该是基于数据且可量化的。
- 确保监控配置没有错误报警策略必须跟服务的SLO目标一致。
- 控制on-call工程师收到的针对同一起事故的报警数量。
- SRE和研发团队设立共同的目标解决运维压力。
- 极端情况下SRE可选择停止支持某个服务由开发团队负责on-call轮值或者与开发团队共同承担运维压力直至运维压力降低。
## 避免运维压力不够
运维压力不够也是一个不良现象应控制SRE团队的大小保证每个工程师每个季度必须至少参与一次(最好两次)on-call同时每年举办一次灾难恢复演习(DiRT)

View File

@ -1,103 +0,0 @@
# 第十二章 有效的故障排查手段
值得警惕的是,理解一个系统应该如何工作并不能使人成为专家,只能靠调查系统为何不能正常工作才行。 ---- Brian Redman
系统正常,只是该系统无数异常情况下的一种特例。 ---- John Allspaw
## 故障排查理想模型
![avtar](img/12.1.guzhangpaichaliucheng.png)
## 常见故障排查陷阱
- 关注了错误的现象或者错误地理解了系统现象的含义,在错误方向上浪费时间
- 不能正确修改系统的配置信息、输入信息或者系统运行环境,造成不能安全有效地测试假设
- 将问题过早地归结为极为不可能的因素或念念不忘之前曾发生过的系统问题,认为一旦发生过一次,就有可能再发生
- 试图解决与当前系统问题相关的一些问题,却没有认识到这些其实只是巧合,或者这些问题其实是由当前系统问题造成的
如何避免这些陷阱
- 要避免前两点,需要更详细地学习系统的运行原理,同时了解分布式系统运行的基本模式
- 要避免第三点,需要记住不是所有失败情况出现的概率都相同,当所有可能都存在,应该优先最简单的。
- 要避免第四点,需要记住,相关性不等于因果关系
## 实践
在实践中,故障排查过程不一定与理想模型一致。
### 故障报告
有效的故障报告应该写清楚预期是什么实际结果是什么以及如何重现。报告应使用统一格式并存储进故障库应该为每一个错误报告提交一个BUG而不是直接报告给具体的人。
### 定位
收到一个错误报告时,应先明确其严重程度,对于大型问题,首先要尽最大可能让系统恢复服务并保存故障现场,如日志等。
如果一个BUG有可能导致不可恢复的数据损失停止整个系统比让系统继续运行更好。
### 检查
必须能够检查系统中每个组件的工作状态以便了解整个系统是不是在正常工作:
1. 查看监控记录
2. 查看日志
3. 暴露目前的系统状态
4. 需要一个使用该系统的真是客户端
### 诊断
1. 将问题分解,逐个检查系统的每个组件,知道系统的最底层,对于大型系统可以使用对分法。
2. 找出系统目前正在执行什么,然后通过询问系统为什么正在做这些操作,以及系统的资源都被用在了哪里。
3. 检查最近对系统的修改
4. 使用针对具体系统开发的诊断工具和诊断系统
### 测试和修复
设计测试的一些考量
- 一个理想的测试应该具有互斥性,通过这个测试可以将一组假设推翻,同时确认另一组假设。
- 先测试最可能的情况,按照可能发生的顺序执行测试,同时考虑该测试对系统的危险性
- 某项测试可能产生误导的结果
- 执行测试可能带来副作用
- 某些测试无法得出准确的结论,只是建议性的
明确记录执行了哪些测试以及测试的结果。
#### 神奇的负面结果
负面结果指一项实验的预期结果没有出现,也就是该实验没有成功。
- 负面结果不应被忽略或被轻视。
- 一项实验中出现的负面结果可以明确高速我们生产环境中的信息,设计理念对错或现有系统的性能极限,并可能被再次或不同团队使用。
- 工具和方法可能超越目前的试验,为未来的工作提供帮助。
- 公布负面结果有助于提升整个行业的数据驱动风气,公布拟的实验结果,哪怕它是失败的。
### 治愈
当你最终确定了某个因素是问题根源时,应该将系统中出错的部分、定位问题的过程、修复问题的方法和防止问题再次发生的策略等记录下来。
## 使故障排查更简单
简化和加速故障排查过程的最基本方法是:
- 增加可观察性,在实现之初就给每个组件增加白盒监控指标和结构化日志
- 利用成熟的,观察性好的组件接口设计系统

View File

@ -1,21 +0,0 @@
# 第十三章 紧急事件响应
东西总是要坏的,这就是生活。
## 紧急事件响应的几点原则
1. 不要害怕问题,所有问题都有解决方案
2. 当想不到解决办法时,及时快速的问一切能给你提供帮助的人求助
3. 为事故保留记录,记录一定要诚实,要事无巨细
4. 提出关键问题,寻找如何能在战术及战略上避免事故的发生
5. 公布和维护事故报告,分享你从中学到的知识
6. 事故结束后,确保自己和他人切实完成事故中总结的待办事项
7. 提出那些大的,甚至不可能的问题
8. 鼓励主动测试

View File

@ -1,67 +0,0 @@
# 第十四章 紧急事故管理
有效的紧急事故管理是控制事故影响和迅速恢复运营的关键因素。
## 紧急事故的流程管理要素
紧急事故流程管理的技巧和手段都是为了让这些富有热情的人能够真正帮上忙。
Incident Command System是一个紧急事故管理系统。
### 嵌套式职责分离
在事故处理中,让每个人清楚自己的职责是非常重要的,系统中有如下角色可以分配:
1. 事故总控。 负责掌握此次事故的概要信息,组建事务处理团队,分配并协调处理工作
2. 事务处理团队。 团队负责人与总控沟通并指挥团队具体执行合适的事务来解决问题,事务处理团队是一次事故中唯一能对系统做修改的团队
3. 发言人。 本次事故处理团队的公众发言人,职责包括向相关人员发送周期性通知,同时维护目前的事故文档,保证其正确性和信息及时性。
4. 规划负责人。 为事务处理团队提供支持。负责一些持续性的工作,同时负责记录在处理过程中对系统进行的特殊操作,以便未来事故结束后可以复原。
### 控制中心
让处理问题的全部成员到同一个地方办公并使用聊天工具和e-mail进行远程沟通。
### 实时事故状态文档
事故总控负责人维护一个实时事故文档,该文档最好可以多人同时编辑。
### 明确公开的职责交接
当超出工作时间以后,事故负责人的职责必须明确公开的进行交接,交接结果应该宣布给其他正在处理事故的人,明确目前的事故处理负责人
## 什么时候对外宣布事故
当下面任何一条条件满足时,事故应该被及时公布
- 需要引入第二个团队来帮助处理问题
- 事故影响了最终用户
- 在集中分析一小时后,依然没有得到解决
## 事故流程管理的最佳实践
- 划分优先级
控制影响范围,恢复服务,同时为根源调查保存现场
- 事前准备
事先和所有事故处理参与者一起准备一套流程
- 信任
充分相信每个事故处理参与者,分配职责后让他们自主行动
- 反思
在事故处理过程种注意自己的情绪和精神状态,如果发现自己开始惊慌失措或者感到压力难以承受,应该寻求更多帮助。
- 考虑替代方案
周期性的审视目前的情况,重新评估目前的工作是应该继续执行还是需要执行其他更重要或更紧急的事情
- 练习
平时不断地使用这项流程,直到习惯成自然
- 换位思考
每次故障可以换一个不同的职责试试,鼓励每个团队成员熟悉流程中的其他角色

View File

@ -1,55 +0,0 @@
# 第十五章 事后总结:从失败中学习
书写事后总结的主要目的是为了保证该事故被记录下来,理清所有根源问题,确保实施有效的措施使得未来重现的几率和影响降低甚至避免重现。
书写事后总结不是一种惩罚措施,而是给整个公司一个学习的机会。
## 基本事后总结条件
事后总结应对事不对人,重点关注如何定位造成这次事件的根本问题。
- 用户可见的宕机时间或者服务质量降级程度达到一级标准
- 任何类型的数据丢失
- on-call工程师需要人工介入的事故,包括回滚、切换用户流量等
- 问题解决耗时超过一定限制
- 监控问题,预示着问题是由人工发现而非报警系统
## 协作和只是共享
事后总结工作流程的每一步都包括团队协作和知识共享。
事后总结的过程还包括正式的评审和发布过程,团队首先内部发布,同时依据如下条件评审:
- 关键的灾难数据是否已被收集并保存
- 本次事故的影响评估是否完整
- 造成事故的根源问题是否足够深入
- 文档中记录的任务优先级是否合理,能否及时解决根源问题
- 事故处理的过程是否共享给了所有相关部门
## 建立事后总结文化
SRE通过组织一些集体活动来更好的建立事后总结文化
- 分享本月最佳事后总结
- 组建事后总结讨论小组
- 组织事后总结阅读俱乐部
- 参见命运之轮游戏--再现事后总结场景,扮演其中的角色进行演习活动。
对投入产出比的质疑是对引入事后总结机制的最大阻力,以下策略来帮助面对这个挑战:
- 逐渐引入事后总结流程,首先进行几次完整和成功的事后总结,证明其价值并确定书写的条件
- 确保有效的书面总结,提供奖励和庆祝
- 鼓励公司高级管理层认可和参与其中

View File

@ -1,29 +0,0 @@
# 第十六章 跟踪故障
提高可靠性的唯一可靠方法论是建立一个基线,同时不断跟踪改变。
系统性地从过去发生过的问题中学习是服务运维的必要手段。
## Outalator
outalator是一个故障跟踪工具。
### 聚合
将多个报警信息“聚合”成一个单独的故障能够有效解决一个故障触发多个报警的问题。
### 加标签
不是每个报警信息都代表一次故障,可以添加标签给报警信息来进行区分
### 分析
SRE不仅仅是响应和处理故障分析历史数据对响应某个故障非常有帮助数据分析的下一步是找到影响更广泛的问题而不就仅仅是简单的计数。
### 报告和公告
选择一系列故障,将所有故障信息发给其他工程师
### 未预料到的好处
能够将某条报警或者某系列报警跟其他某个故障连接起来,可以加快检查速度和通过确认目前的确有某项故障以降低其他团队的压力。

View File

@ -1,2 +0,0 @@
# 第二章
本章暂缺

View File

@ -1,70 +0,0 @@
# 第三章 拥抱风险
不必追求100%的服务可靠性,过分追求服务的可靠性会带来成本的巨大提升,同时减少了新功能的数量及更新频率。
用户本身设备就有可靠性损失99.99%与99.999%在用户体验上并无太大区别。
## 服务可靠性成本
高昂的成本主要存在以下两个维度:
- 冗余服务器/计算资源成本。
- 机会成本。 构建减少风险的系统或功能,而非用户直接可用的功能的工程资源成本。
## 度量服务的风险
度量服务的风险从两个维度,基于时间和基于请求成功率。
### 基于时间的可用性公式
可用性=系统正常时间/(系统正常运行时间+停机时间)
### 基于请求成功率的可用性公式
**此公式对于分布式服务或定时任务更有意义**
可用性=成功请求数/总请求数
## 服务的风险容忍度
服务的风险容忍度通常是直接根据基本产品或服务的定义建立的。
### 消费者服务
评价服务风险容忍度时,有以下几个因素:
- 可用性目标
根据功能及市场定位来制定,快速发展的服务较低而企业服务较高。
- 故障类型
不同故障对服务的影响不同
- 成本
提升可靠性会带来一定收益并投入成本,当收益高于成本就是合理投资
- 其他服务指标
比如服务延迟
### 基础设施服务
基础设施服务与消费者服务的一个根本区别是,基础设施组件有多个用户,而他们通常有很多不同的需求。
评价服务风险容忍度时,有以下几个因素:
- 可用性目标水平
需要分析每个用户的不同需求来划分服务水平
- 故障类型
- 成本
依据不同的服务水平分割成多个基础设施服务,在多个独立的服务水平上提供该服务,是一种在符合成本效益条件下满足这些竞争性约束的方式。
## 使用错误预算
使用错误预算的好处是能够激励产品研发和SRE一起找出创新和可靠性之间合理的平衡点。
1. 定义一个服务水平目标(SLO), 确定正常服务时间
2. 通过监控系统测算实际在线时间
3. 正常服务目标时间-实际在线时间=不可靠预算
4. 只要正常在线时间高于SLO就可以发布新版本

View File

@ -1,101 +0,0 @@
# 第四章 服务质量目标
本章描述了SRE团队在指标建模、指标选择以及指标分析上采用的基本框架。
## 服务质量术语
- 指标 `SLI(Service Level Indicator)`
服务质量指标是一个服务的某项服务质量的具体量化指标。
- 目标 `SLO(Service Level Objective)`
服务质量目标是服务的某个SLI的目标值或目标范围。SLO的定义是 SLI≤目标值或范围下限≤SLI≤范围上限。
- 协议 `SLA(Service Level Agreement)`
服务质量协议是服务与用户之间的一个明确的或者不明确的协议。描述在达到或者未达到SLO之后的后果。
## 指标`SLI`在实践中的应用
常见服务的几大类SLI
- 用户可见的服务系统
通常关心可用性、延迟、吞吐量
- 存储系统
通常强调: 延迟、可用性、数据持久性
- 大数据系统
通常关心吞吐量和端到端延迟
- 所有系统
都关注正确性
### 收集指标
指标收集分为服务端和客户端。服务端主要依靠监控系统(promethues)和日志分析系统,客户端主要依靠页面埋点和其他收集系统(如专门收集手机app指标的系统)。
对于大部分指标,汇总时应当以“分布”而不是平均值来定义。
### 指标标准化
应该标准化一些常见的SLI任何符合标准定义模板的服务可以直接使用而不需要再次评估定义。
- 汇总间隔
- 汇总范围
- 度量频率
- 包含的请求
- 数据获取方式
- 数据访问延迟
## 目标`SLO`在实践中的应用
应从用户最关心的方面入手而不是从能度量什么入手。先制定目标,从而反推出指标`SLI`。
### 目标定义
定义SLO的几个要素
- SLO应该具体指出其是如何度量的以及有效条件。
- 应该使用错误预算而不是要求SLO能被100%满足。
- 通过每日(周)对SLO达标成都的监控可以展示一个趋势在重大问题发生器得到预警。
- SLO不达标的频率与错误预算进行对比利用差值来指导新版本的发布。
### 目标的选择
选择目标SLO不是一个纯粹的技术活动因为这里还涉及产品及业务层的决策SLI和SLO(甚至SLA)的选择都应直接反应该决策
选择目标的几点要素:
- 不要仅以当前状态为基础选择目标
- 保持简单
- 避免绝对值
- SLO越少越好
- 不追求完美
- 小心使用SLO
### SLI和SLO作为控制手段
SLI和SLO在决策系统运维是非常有用
1. 监控并度量系统的SLI
2. 比较SLI与SLO以决定是否需要执行操作
3. 如果需要操作决定执行什么操作以满足目标SLO
4. 执行操作
### 使用SLO建立用户预期
用户经常希望知道他们可以预期的服务质量以便理解服务是否能够满足他们的需求通过公布SLO可以设置用户对系统的预期。
使用以下策略让用户拥有正确的预期:
- 留出一定的安全区
- 实际SLO不要过高
## 协议`SLA`在实践中的应用
由业务部门和法务部门选择合适的后果条款SRE帮主这些部门理解SLA和SLO达标的概率和困难程度。
最好在用户宣传方面偏于保守。

View File

@ -1,57 +0,0 @@
# 第五章 减少琐事
***如果系统正常运转中需要人工干预应该将此视为一种BUG。***
***“正常”的定义会随系统的进步而不断改变。***
SRE要把更多的时间花费在长期项目研发上而非琐事(日常运维)中。
## 琐事的定义
琐事不仅仅代表“我不喜欢的工作”。
琐事就是运维服务中手动性的、重复性的、可以被自动化的、战术性的、没有持久价值的、与服务呈线性关系增长的工作。
流程开销不是琐事如团队会议、目标的建立和评估、每周总结以及HR的书面工作。
## SRE的工作时间分配
SRE的一个公开目标是保持每个SRE工作时间中的运维工作(琐事)的比例低于50%。SRE至少花50%的时间在工程项目上,以减少未来的琐事或增加服务功能(提高可用性、利用率、性能)。
### 什么是工程工作
工程工作是一种新颖的,本质上需要主观判断的,符合长期战略的,会对你的服务进行长久性改善的工作。
## SRE的活动分类
典型的SRE活动分为以下几类
- 软件工程
编写或修改代码,以及所有其他相关性设计和文档的工作
- 系统工程
配置生产系统,修改现存配置,或者用一种通过一次性工作产生持久的改进的方法来书写系统文档。
- 琐事
与运维服务相关的重复性的、手工的劳动
- 流程负担
与运维服务不直接相关的行政工作
## 琐事的有害因素
在SRE工作种一定量的琐事是不可避免的如果琐事特别繁重那就应该非常担忧大声抱怨。在许多琐事有害的原因种以下因素要被考虑
- 职业停滞
- 士气低落
- 造成误解
- 进展缓慢
- 开创先例,造成琐事越来越多
- 促进摩擦产生
- 违反承诺
## 总结
让我们多创新,少干琐事吧。

View File

@ -1,71 +0,0 @@
# 第六章 分布式系统的监控
## 术语定义
关于监控的一些术语定义。
### 监控(monitoring)
收集、处理、汇总,并且显示关于某个系统的实时量化数据。
### 白盒监控(white-box monitoring)
依靠系统内部暴露的一些性能指标进行监控。
### 黑盒监控(black-box monitoring)
通过测试某种外部用户可见的系统行为进行监控。
### 监控台页面(dashboard)
提供某个服务核心指标一览服务的应用程序。
### 警报(alert)
目标对象是某个人发向某个系统地址的一个通知。
### 根源问题(root cause)
指系统(软件或流程)中的某个缺陷,如果这个缺陷被修复,就可以保证这个问题不会再以同样的方式发生。
### 节点或者机器(node/machine)
指物理机、虚拟机或者容器内运行的某个实例。
### 推送(push)
关于某个服务正在运行的软件或者其配置文件的任何改动。
## 监控系统的四个黄金指标
1. 延迟
服务处理某个请求需要的时间。
2. 流量
使用系统中某个高层次的指标对系统负载需求所进行的度量。
3. 错误
请求失败的速率(包括显示或隐式失败)。
4. 饱和度
服务容量有多满。通常是系统中目前最为受限的某种资源的某个具体指标的度量。
## 监控系统设计理念
监控系统设计的核心理念是简化,直到不能再简化。
- 对于最能反映真是故障的规则应该越简单越好,可预测性强,非常可靠
- 对于不常用的数据收集汇总以及警报配置应该定时删除
- 收集到的信息,但是没有暴露给任何监控台或者没有被任喝警报规则使用的,应该定时删除。
度量指标时应使用合适的精度,系统的不同部分应该以不同的精度度量。
## 应对警报的深层理念
1. 紧急警报应该是立即需要进行某种操作的,每天只能进入紧急状态几次,避免导致狼来了效应。
2. 紧急警报应该是可以具体操作的
3. 紧急警报的回复都应该需要某种智力分析过程。对于只需要一个固定的机械动作的警报,不应成为紧急警报。
4. 紧急警报都应该是关于某个新问题的,不应该彼此重叠

View File

@ -1,29 +0,0 @@
# Google的自动化系统的演进
“黑科技”之外,就只剩下自动化和机械化了。 ---- Federico Garcia Lorca
## 自动化的价值
- 一致性
一致性地执行范围明确、步骤已知的程序是自动化的首要价值。
- 平台性
通过正确地设计和实现自动化系统可以提供一个可以扩展的,广泛适用的,甚至可能带来额外收益的平台。
一个平台也将错误集中化。
- 修复速度更快
成功运行的自动化可以降低一些常见故障的平均修复时间(MTTR)。
- 行动速度更快
机器的反应速度快过人工。
- 节省时间
## 自动化的演进路径
1. 没有自动化,纯手工操作
2. 外部维护的系统特定的自动化
3. 外部维护的通用自动化系统
4. 内部维护的通用自动化系统
5. 不需要任何自动化的系统

View File

@ -1,41 +0,0 @@
# 第八章 发布工程
发布工程(Release Engineering)是软件工程内部一个专注于构建和交付软件的学科。
## 发布工程哲学
发布工程师的日常工作是由下列四个主要的工程与服务哲学指导的。
### 自助服务模型
发布工程师开发工具,制定最佳实践,以便让产品研发团队可以自己掌握和执行自己的发布流程。
### 追求速度
频繁的发布可以使每个版本之间的变更更少,这种方式使测试和调试变得简单。
### 密闭性
构建工具必须确保一致性和可重复性,构建过程使密闭的,不应受到构建服务器上安装的第三方类库或者其他软件工程的影响。
### 强调策略和流程
多层安全和访问控制机制可以确保发布过程中只有指定的人才能执行指定的操作。
主要关注以下操作:
- 批准源代码改动----通过源代码仓库中的配置文件决定
- 指定发布流程中需要执行的具体操作
- 创建新的发布版本
- 批准初始的继承请求以及后续的cherry picking请求
- 实际部署某个发布(release)版本
- 修改某个项目的构建配置文件
## Google的持续集成方案
参见 78-82 页。

View File

@ -1,49 +0,0 @@
# 第九章 简单化
可靠性只有靠对最大程度的简化不断追求而得到。 ---- C.A.R.Hoare, Turing Award lecture
软件系统本质上是动态和不稳定的。SRE的工作室在系统的灵活和稳定性上维持平衡。
## 系统的灵活性与稳定性
有时候为灵活性而牺牲稳定性是有意义的。
SRE通过创造流程、实践以及工具来提高软件的可靠性同时SER需要最小化这些工作对于开发人员灵活性造成的影响。
## 乏味是一种美德
我们不想要自发性和有趣的程序,我们希望这些程序按设计执行,可以预见性的完成商业目标。
关注必要复杂度和意外复杂度之间的区别非常关键,必要复杂度是一个给定情况所固有的复杂度,不能从该问题的定义中移除;意外复杂度是不固定的,可以通过工程上的努力来解决。
为最小化意外复杂度SRE团队应该
- 在他们所负责的系统中引入意外复杂度时,及时提出抗议
- 不断努力消除正在接手的和已经负责运维的系统的复杂度
## 我绝不放弃我的代码
SRE推崇保证所有的代码都有必须存在的目的的实践。
## 负代码行作为一个指标
添加到项目中的每行代码都可能引入新的缺陷和错误。
较小的项目容易理解,也更容易测试,而且通常缺陷也少。应该按需删除无用的代码。
对于增加新功能的需求,应该持保守态度。
## 最小API
不是在不能添加更多的时候,而是在没有什么可以去掉的时候,才能达到完美。
书写一个明确的最小的API是管理软件系统管理简单性必要的部分。
## 模块化
在二进制文件之间或二进制文件与配置之间推行松耦合,是一种同时提高开发人员灵活性和系统稳定性的简化模式。
模块化同样适用用分布式系统与数据的设计。
## 发布简单化
简单的发布流程要比复杂的发布流程更好。如果发布是按更小的批次进行的,我们就可以更有信心地进行更快的发布,因为每个变更在系统中的影响都可以独立理解。

Binary file not shown.

Before

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 113 KiB

View File

@ -1,5 +0,0 @@
# 第三部分 具体实践
## 服务可靠度层级模型
![avatar](img/srlm.png)

View File

@ -1,51 +0,0 @@
# Google SRE 运维解密读书笔记
# 目录
## 第一部分 概览
- [第一章 介绍](1.md)
- [第二章 Google生产环境 SRE视角](2.md)
## 第二部分 指导思想
- [第三章 拥抱风险](3.md)
- [第四章 服务质量目标](4.md)
- [第五章 减少琐事](5.md)
- [第六章 分布式系统的监控](6.md)
- [第七章 Google的自动化系统的演进](7.md)
- [第八章 发布工程](8.md)
- [第九章 简单化](9.md)
## 第三部分 具体实践
- [第十章 基于时间序列数据进行有效报警](10.md)
- [第十一章 on-call轮值](11.md)
- [第十二章 有效的故障排查手段](12.md)
- [第十三章 紧急事件响应](13.md)
- [第十四章 紧急事故处理](14.md)
- [第十五章 事后总结:从失败中学习](15.md)
- [第十六章 跟踪故障](16.md)
- [第十七章 测试可靠性](17.md)
- [第十八章 SRE部门中的软件工程实践](18.md)
- [第十九章 前端服务器的负载均衡](19.md)
- [第二十章 数据中心内部的负载均衡系统](20.md)
- [第二十一章 应对过载](21.md)
- [第二十二章 处理连锁故障](22.md)
- [第二十三章 管理关键状态:利用分布式共识来提高可靠性](23.md)
- [第二十四章 分布式周期性任务系统](24.md)
- [第二十五章 数据处理流水线](25.md)
- [第二十六章 数据完整性:读写一致](26.md)
- [第二十七章 可靠地进行产品的大规模发布](27.md)
## 第四部分 管理
- [第二十八章 迅速培养SRE加入on-call](28.md)
- [第二十九章 处理中断性任务](29.md)
- [第三十章 通过嵌入SRE的方式帮助团队从运维过载中恢复](30.md)
- [第三十一章 SRE与其他团队的沟通与协作](31.md)
- [第三十二章 SRE参与模式的演进历程](32.md)
## 第五部分 结束语
- [第三十三章 其他行业的实践经验](33.md)

View File

@ -1,9 +0,0 @@
# 第一章 什么是Prometheus
Prometheus是一个开源的、基于指标的监控系统。
Prometheus是指标拉取型监控系统
## Prometheus 架构
![avtar](img/prometheus.png)

Binary file not shown.

Before

Width:  |  Height:  |  Size: 150 KiB

View File

@ -1,27 +0,0 @@
# PromQL
promQL的一些示例与解析。
#### `up{instance="localhost:9090", job="prometheus"}`
up 是一个指标名
instance 是一个标签,表示被抓取的目标地址
job 是一个标签,来自文件`prometheus.yml`的配置项`job_name`
#### `process_resident_memory_bytes`
gauge 类型, 服务当前使用的内存,单位字节
gauge类型表示当前绝对值很重要
#### `prometheus_tsdb_head_samples_appended_total`
counter类型跟踪事件发生的数量
`rate(prometheus_tsdb_head_samples_appended_total[1m])`
计算一分钟内的平均采样数
rate是一个函数这里用来计算counter每秒增长的速度
[1m] 表示取一分钟内的值

View File

@ -1,2 +0,0 @@
# Prometheus 学习手册

View File

@ -1,4 +0,0 @@
python:
python3 main.py
go:
go run main.go

View File

@ -1,24 +0,0 @@
package main
import (
"log"
"net/http"
"github.com/prometheus/client_golang/prometheus/promhttp"
)
func runHTTPServer()error{
http.Handle("/metrics", promhttp.Handler())
http.HandleFunc("/", func (w http.ResponseWriter, r *http.Request){
w.Write([]byte("hello world!"))
})
return http.ListenAndServe("localhost:8080", nil)
}
func main(){
err := runHTTPServer()
if err != nil {
log.Fatal(err)
}
}

View File

@ -1,20 +0,0 @@
import http.server
from prometheus_client import start_http_server
from prometheus_client import Counter
REQUESTS=Counter('hello_worlds_total', 'hellow worlds requested.')
EXCEPTIONS=Counter("hello_world_exceptions_total", 'exceptions serving hello world.')
class MyHandler(http.server.BaseHTTPRequestHandler):
@EXCEPTIONS.count_exceptions()
def do_GET(self):
REQUESTS.inc()
self.send_response(200)
self.end_headers()
self.wfile.write(b"hello world")
if __name__ == "__main__":
start_http_server(8000)
server = http.server.HTTPServer(('localhost', 8001), MyHandler)
server.serve_forever()

5
go.mod
View File

@ -1,5 +0,0 @@
module readnotes.bing89.com
go 1.16
require github.com/prometheus/client_golang v1.11.0

138
go.sum
View File

@ -1,138 +0,0 @@
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cespare/xxhash/v2 v2.1.1 h1:6MnRN8NT7+YBpUIWxHtefFZOKTAPgGjpQSxqLNn0+qY=
github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY=
github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8=
github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA=
github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs=
github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w=
github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0=
github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/golang/protobuf v1.4.3 h1:JjCZWpVbqXDqFVmTfYWEVTMIYrL/NPdPSCHPJ0T/raM=
github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI=
github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4=
github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/matttproud/golang_protobuf_extensions v1.0.1 h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo=
github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M=
github.com/prometheus/client_golang v1.11.0 h1:HNkLOAEQMIDv/K+04rukrLx6ch7msSRwf3/SASFAGtQ=
github.com/prometheus/client_golang v1.11.0/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0=
github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.2.0 h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M=
github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo=
github.com/prometheus/common v0.26.0 h1:iMAkS2TDoNWnKM+Kopnx/8tnEStIfpYA0ur0xQzzhMQ=
github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc=
github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU=
github.com/prometheus/procfs v0.6.0 h1:mxy4L2jP6qMonqmq+aTtOx1ifVWUgG/TAmntgbh3xv4=
github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE=
github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA=
golang.org/x/oauth2 v0.0.0-20190226205417-e64efc72b421/go.mod h1:gOpvHmFTYa4IltrdGE7lF6nIHvwfUNPOp7c8zoXwtLw=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40 h1:JWgyZ1qgdTaF3N3oxC+MdTV7qvEEgHo3otj+HB5CM7Q=
golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE=
google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo=
google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU=
google.golang.org/protobuf v1.26.0-rc.1 h1:7QnIQpGRHE5RnLKnESfDoxm2dTapTZua5a0kS0A+VXQ=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=