2020下半年书桌整理及脑图
立个flag。希望早日清空。
Frank
[AWS][Python]使用Python自动生成CloudFormation等实现批量自动化部署AWS Canaries
需求
业务需要，想要监控一些API Endpoints(部分在AWS内网)。工具选型可参看[AWS][Elasticsearch]API Synthetics
Monitoring工具的简单比较 [https://hanzhu.dev/api-synthetics-monitoring/]。最后选择的是AWS
Cloudwatch Synthetics。用ApiCanaries探测，并且就success
rate和duration两个metrics设置报警阈值，通知SNS。希望使用CloudFormation来Codify整个过程。
任务拆解与分析
Canary的实现
经过对手动生成的Canary的观察，发现Canary是借由lambda实现的，在console手动添加Canary后，会生成一个js的canary
lambda, 而另一个相关配置的canary script (也是js, 可以自动生成或者自己写)会以layer的形式加载在js canary
lambda之上，让js canary lambda来call它。
// Call customer's execution
[AWS][ELB]ALB redirect for HTTPS and Multi-Sites
原始架构是AWS Fargate with ALB，DNS（不在Route53上）指向ALB。
HTTPS Redirect
先来看一下最基本的HTTPS跳转。此时DNS record是指向ALB的。
此时ALB有两个listener。Port 80 listener 的rule是进行一个redirecting, 其余不变: Default:
redirecting to HTTPS://#{host}:443/#{path}?#{query}。Port 443
listener是foward到对应的target group, 完成请求传递。如下图：
Multi-Site Redirect
原始需求
让去XXX.sg [http://]的http和https请求跳转至XXX.com.sg(which is on https)。ALB的原始设置就是HTTPS
Redirect的情形。
思路分析
首先要让添加DNS Record,
[AWS]S3 Bucket Notification
应内部客户要求，尝试了一下S3 bucket的通知推送，这里记录一下。
分别使用了推送到SNS Topic和推送到Lambda（之后消费Gchat webhook）。
SNS Topic
S3-->Properties-->Events, Sent to SNS Topic.
需要注意的是，在SNS Topic里，需要修改policy让S3推消息，比如：
{
"Version": "2012-10-17",
"Id": "__default_policy_ID",
"Statement": [
{
"Sid": "publish-from-s3",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "SNS:Publish",
"Resource": "arn:aws:sns:ap-southeast-1:xxxxxxxxx:testing-notification-hz
[Terraform]Terraform技巧:direnv; Backend, State, Workspace
使用direnv来解决自动AWS Profile导入
1. 先设置AWS profile
2. 安装direnv
3. 在所需目录下新加.envrc文件, 里面放比方：
export AWS_PROFILE=hzhu-nmdev
export TF_VAR_trendmicro_policy_id=3601
4. 最后别忘了在.gitignore里添加**/.envrc，ignore所有.envrc。双*的作用是"to match directories
anywhere in the repository."
References:
direnv [https://direnv.net/]
Automatically Switch AWS Profiles When Entering Your Git
[Terraform][AWS]Terraform Module使用技巧
关于一些先置的技巧，比如Profile自动导入和Backend同步State File, 请移步这里
[https://hanzhu.dev/terraform-get-prepared/]。
顺便温故一下Terraform传统艺能：
cd terraform/resources/common/dev
terraform init -backend-config=./backend.tfvars
terraform plan -out dev-$(date +%s).plan
terraform apply "dev-1591347481.plan"
Modules的基本介绍
这个视频介绍得非常清楚，演讲者就是的作者,建议先行收看
How to Build Reusable, Composable, Battle tested Terraform Modules
[https://www.youtube.
[AWS][Elasticsearch]API Synthetics Monitoring工具的简单比较
针对API Synthetics Monitoring (主要是Endpoint Uptime),
可以选择Elastic的Heartbeat或者AWS新推出的Cloudwatch Synthetics Canaries.
AWS Cloudwatch Canaries
定义如下即可：
* Method: GET/POST
* Application or Endpoint URL
* Headers Key Value Pairs
* Request data
它会提供一个js的runtime, 新建一个lambda来跑，因而可以定义它所在的VPC, Subnet和SecurityGroup
* Data retention period
* Data storage, 其实就是S3。默认自动生成。
* IAM role, 默认自动生成，也可以使用existing role, "need your permission to put artifacts
into S3, and
[AWS][Log]Jsonify Apache日志
目的是日志结构化，方便推送到ES。
References
https://httpd.apache.org/docs/2.4/logs.html
http://httpd.apache.org/docs/current/mod/mod_log_config.html
https://www.loggly.com/ultimate-guide/apache-logging-basics/
AWS上的应用：
https://aws.amazon.com/premiumsupport/knowledge-center/elb-capture-client-ip-addresses/
https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/x-forwarded-headers.html#x-forwarded-for
操作实例:
Raw
[AWS]ELB-sticky sessions
Sticky sessions is an advanced load balancing method that allows you to bind a
user's session to a specific EC2 instance.
Ensures all requests from that session are sent to the same instance.
Typically utilized with a Classic LB.
Can be enabled for ALB though can only be set on
[AWS][Log]Docker输出日志
Fargate, 想输出固定日志到cloudwatch，参考了
An amazing recipe is given in the nginx Dockerfile
[https://github.com/nginxinc/docker-nginx/blob/8921999083def7ba43a06fabd5f80e4406651353/mainline/jessie/Dockerfile#L21-L23]
:
RUN ln -sf /dev/stdout /var/log/nginx/access.log \
&& ln -sf /dev/stderr /var/log/nginx/error.log
Simply, the app can continue writing to it