楚天乐的全栈之路 - 运维开发
type: channel
link: https://blog.shyclouds.net/category/运维开发/
description: 运维开发
items: 使用Vault管理服务器各种密码
items: windows环境pip无法安装dlib库的终极解决
items: Windows WLS2使用本机ss代理访问github
items: 写个dockerfile自动部署hugo
items: cat命令提取json中的key
items: centos安装java 8
items: 阿里云centos环境python的pyppeteer无法下载依赖chromium组件
items: 从git历史删除中恢复某一个文件
items: Centos 7升级内核5.x
items: godaddy生成ssl证书

使用Vault管理服务器各种密码
type: item
date: 2023-01-05T01:08:01+08:00
description: vault简介 安装 在centos上安装 $ sudo yum install -y yum-utils $ sudo yum-config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo $ sudo yum -y install vault 在ubuntu上安装 $ sudo apt update && sudo apt install gpg $ wget -O- https://apt.releases.hashicorp.com/gpg | gpg --dearmor | sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg >/dev/null $ gpg --no-default-keyring --keyring /usr/share/keyrings/hashicorp-archive-keyring.gpg --fingerprint $ echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list sudo apt update && sudo apt install vault mac上安装 $ brew tap hashicorp/tap $ brew install hashicorp/tap/vault $ brew upgrade hashicorp/tap/vault # 升级 windows上安装 choco install vault 测试环境启动vault服务 这种用法仅限于测试环境，生产环境后面我们在说 开发模式启动 $ vault server -dev WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory and starts unsealed with a single unseal key. The root token is already authenticated to the CLI, so you can immediately begin using Vault. You may need to set the following environment variables: $ export VAULT_ADDR='http://127.0.0.1:8200' The unseal key and root token are displayed below in case you want to seal/unseal the Vault or re-authenticate. Unseal Key: 910jSSp6agJyY0RNm2QnamUtv3IZNcdJp0DejDE3OvI= Root Token: hvs.Wlq6tyxrx2kS5IQSJI1F7b1l ！！！！！重要重要重要！！！！！ Development mode should NOT be used in production installations! $ export VAULT_ADDR='http://127.0.0.1:8200' # 把vault服务地址写入环境变量 $ export VAULT_TOKEN='hvs.Wlq6tyxrx2kS5IQSJI1F7b1l' # 把root token写入环境变量 可以开始测试vault了 这里我们先使用简单的kv引擎,kv存储引擎会以明文方式把信息放在内存里。vault还支持其他的secrets engine，后面再说。 写入信息 $ vault kv put -mount=secret hello foo=world # 向hello写入一个kv foo=world == Secret Path == secret/data/hello ======= Metadata ======= Key Value --- ----- created_time 2023-01-04T05:39:57.252132156Z custom_metadata <nil> deletion_time n/a destroyed false version 1 $ vault kv put -m…

windows环境pip无法安装dlib库的终极解决
type: item
date: 2022-06-07T23:38:00+08:00
description: 问题 机器学习需要安装dlib库，网上搜索 解决方案在最底部 网上搜索的解决方案 pip install cmake pip install boost pip install dlib 然而，在安装dlib的时候，错误原因不仅限于没有安装cmake。 报错如下 Collecting dlib Using cached dlib-19.24.0.tar.gz (3.2 MB) Preparing metadata (setup.py) ... done Building wheels for collected packages: dlib Building wheel for dlib (setup.py) ... error error: subprocess-exited-with-error × python setup.py bdist_wheel did not run successfully. │ exit code: 1 ╰─> [320 lines of output] running bdist_wheel running build running build_py package init file 'tools\python\dlib\__init__.py' not found (or not a regular file) running build_ext Building extension for Python 3.8.8 (default, Apr 13 2021, 15:08:03) [MSC v.1916 64 bit (AMD64)] Invoking CMake setup: 'cmake C:\Users\shyan\AppData\Local\Temp\pip-install-g2xfkj5d\dlib_6582d86d1c13456bb92e2e45ecaaae84\tools\python -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=C:\Users\shyan\AppData\Local\Temp\pip-install-g2xfkj5d\dlib_6582d86d1c13456bb92e2e45ecaaae84\build\lib.win-amd64-3.8 -DPYTHON_EXECUTABLE=C:\ProgramData\Anaconda3\python.exe -DCMAKE_LIBRARY_OUTPUT_DIRECTORY_RELEASE=C:\Users\shyan\AppData\Local\Temp\pip-install-g2xfkj5d\dlib_6582d86d1c13456bb92e2e45ecaaae84\build\lib.win-amd64-3.8 -A x64' -- Building for: Visual Studio 17 2022 -- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.19044. -- The C compiler identification is MSVC 19.32.31329.0 -- The CXX compiler identification is MSVC 19.32.31329.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.32.31326/bin/Hostx64/x64/cl.exe - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.32.31326/bin/Hostx64/x64/cl.exe - skipped -- Detecting CXX compile features -- Detectin…

Windows WLS2使用本机ss代理访问github
type: item
date: 2022-06-05T02:04:09+08:00
description: 关键问题 WSL每次启动ip不一样，所以不能用静态方法设置http代理。 WSL会把宿主ip地址写入 获得当前宿主ip地址的方法 cat /etc/resolv.conf | grep nameserver | awk '{ print $2 }' 手动设置代理的方法 $ cat /etc/resolv.conf | grep nameserver | awk '{ print $2 }' 172.29.176.1 $ export https_proxy="http://172.29.190.80:10000" # 下载github库 $ wget https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life/releases/download/v1.0/face_checkpoints.zip --2022-06-05 01:46:58-- https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life/releases/download/v1.0/face_checkpoints.zip 正在连接 172.29.176.1:10000... 已连接。 已发出 Proxy 请求，正在等待回应... 302 Found 位置：https://objects.githubusercontent.com/github-production-release-asset-2e65be/274594200/49cb1e00-e34c-11eb-82bf-3c592a7d16da?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220604%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220604T174658Z&X-Amz-Expires=300&X-Amz-Signature=5ce75d01ba6ee4cb9c9ca3190282bb603552cc90a8e9cb64797eb9e19e6cc6ba&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=274594200&response-content-disposition=attachment%3B%20filename%3Dface_checkpoints.zip&response-content-type=application%2Foctet-stream [跟随至新的 URL] --2022-06-05 01:46:58-- https://objects.githubusercontent.com/github-production-release-asset-2e65be/274594200/49cb1e00-e34c-11eb-82bf-3c592a7d16da?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20220604%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220604T174658Z&X-Amz-Expires=300&X-Amz-Signature=5ce75d01ba6ee4cb9c9ca3190282bb603552cc90a8e9cb64797eb9e19e6cc6ba&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=274594200&response-content-disposition=attachment%3B%20filename%3Dface_checkpoints.zip&response-content-type=application%2Foctet-stream 正在连接 172.29.176.1:10000... 已连接。 已发出 Proxy 请求，正在等待回应... 200 OK 长度： 684354563 (653M) [application/octet-stream] 正在保存至: ‘face_checkpoints.zip’ face_checkpoints.zip 100%[===========================================================================================================…

写个dockerfile自动部署hugo
type: item
date: 2022-03-05T22:08:00+08:00
description: 创建dockerfile FROM golang:1.17-alpine # Optionally set HUGO_BUILD_TAGS to "extended" or "nodeploy" when building like so: # docker build --build-arg HUGO_BUILD_TAGS=extended . ARG HUGO_BUILD_TAGS ARG CGO=1 ENV CGO_ENABLED=${CGO} ENV GOOS=linux ENV GO111MODULE=on ENV HUGO_VERSION=0.93.3 ENV HUGO_USER=hugo ENV BASE_DIRECORY=/home/hugo ENV SITE_NAME=hugosite ENV HUGO_SITE_DIRECORY=${BASE_DIRECORY}/${SITE_NAME} ENV HUGO_URL=https://github.com/gohugoio/hugo/releases/download/v${HUGO_VERSION}/hugo_extended_${HUGO_VERSION}_Linux-64bit.tar.gz #hugo_extended_0.93.3_Linux-64bit.tar.gz ENV GLIBC_VER=2.34-r0 ENV GLIBC_NAME=glibc-${GLIBC_VER}.apk ENV THEME_NAME=book ENV THEME_URL=https://github.com/alex-shpak/hugo-book RUN set -ex && \ apk --update add libstdc++ curl ca-certificates git # glibc ENV GLIBC_REPO=https://github.com/sgerrand/alpine-pkg-glibc ENV GLIBC_VERSION=2.30-r0 RUN set -ex && \ apk --update add libstdc++ curl ca-certificates && \ for pkg in glibc-${GLIBC_VERSION} glibc-bin-${GLIBC_VERSION}; \ do curl -sSL ${GLIBC_REPO}/releases/download/${GLIBC_VERSION}/${pkg}.apk -o /tmp/${pkg}.apk; done && \ apk add --allow-untrusted /tmp/*.apk && \ rm -v /tmp/*.apk && \ /usr/glibc-compat/sbin/ldconfig /lib /usr/glibc-compat/lib # 下载安装 ADD ${HUGO_URL} /tmp/ RUN tar -xzf /tmp/*.tar.gz -C /tmp && \ file="$(echo HUGO_URL)" && echo $file && \ file="$(ls -ll /tmp/hugo)" && echo $file && \ mv /tmp/hugo /usr/bin/ && \ #rm -rf /tmp/* && \ file="$(ls -ll /usr/bin/hugo)" && echo $file && \ adduser ${HUGO_USER} -D #CMD sh WORKDIR ${BASE_DIRECORY} #VOLUME ["$BASE_DIRECORY", "$BASE_DIRECORY"] #CMD ["ls","-a"] # 安装 RUN hugo new site ${SITE_NAME} # && file="$(ls -ll /home/hugo/hugosite)" && echo $file && echo $file && file="$(pwd)" && echo $file WORKDIR ${HUGO_SITE_DIRECORY} RUN git init \ && git submodule add ${THEME_URL} themes/${THEME_NAME} \ && cp -R themes/${THEME_NAME}/exampleSite/content . \ && echo -e "baseURL = 'http://example.org/' \r\nlanguageCode = 'en-us' \r\ntitle = 'My New Hug…

cat命令提取json中的key
type: item
date: 2021-03-26T11:09:05+08:00
description: cat连用grep进行筛选，筛选出的json还可以提取key 距离提取满足grep筛选条件的记录，并提取MessageID字段内容 cat log-20210326-* | grep xxx | grep MessageID | grep -Po 'MessageID[" :]+\K[^"]+'

centos安装java 8
type: item
date: 2020-09-08T13:37:00+08:00
description: 查看当前linux自带jdk是否安装,以及jdk版本 java -version rpm -qa | grep java rpm -qa | grep gcj rpm -qa | grep jdk # 如果已经有jdk, 会显示如下 java-x.x.x-gcj-compat-x.x.x.x-xxjpp.xxx java-x.x.x-openjdk-x.x.x.x-x.x.bxx.exx 如果有已安装的jdk，则挨个卸载，没有则跳过这一步 rpm -e –nodeps java-x.x.x-gcj-compat-x.x.x.x-xxjpp.xxx rpm -e –nodeps java-x.x.x-openjdk-x.x.x.x-x.x.bxx.exx 如果找不到openjdk source，还可以如下方法卸载 yum -y remove java java-x.x.x-gcj-compat-x.x.x.x-xxjpp.xxx yum -y remove java java-x.x.x-openjdk-x.x.x.x-x.x.bxx.exx 官网下载jdk，并安装。 https://www.oracle.com/java/technologies/javase/javase-jdk8-downloads.html 我们使用：Linux x64 Compressed Archive. # 下载之后解压缩 tar -zxvf jdk-8u261-linux-x64.tar.gz # 移动到/usr/local/ mv jdk1.8.0_261 /usr/local/ 设置环境变量 vim /etc/profile # 环境变量设置 #JDK1.8 export JAVA_HOME=/usr/local/jdk1.8.0_261 export JAVA_BIN=/usr/local/jdk1.8.0_261/bin export PATH=$PATH:$JAVA_HOME/bin export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar export JAVA_HOME JAVA_BIN PATH CLASSPATH 测试安装 执行java -version和javac

阿里云centos环境python的pyppeteer无法下载依赖chromium组件
type: item
date: 2020-09-07T16:02:00+08:00
description: pypetter能做什么 自行百度 为什么没法下载 省略原因，只有mmp 解决方法 1 既然无法下载，那我们就手动下载然后放到指定位置么。问题来了 从哪下载 放到哪里 从哪里下载 找这个文件：find / -name chromium_downloader.py python 3.8环境下，找到文件：/usr/local/lib/python3.8/site-packages/pyppeteer/chromium_downloader.py 备份py库文件稍后恢复，/usr/local/lib/python3.8/site-packages/pyppeteer/chromium_downloader.py 编辑 vim /usr/local/lib/python3.8/site-packages/pyppeteer/chromium_downloader.py 备份 mv /usr/local/lib/python3.8/site-packages/pyppeteer/chromium_downloader.py.bak /usr/local/lib/python3.8/site-packages/pyppeteer/chromium_downloader.p 稍后恢复 mv /usr/local/lib/python3.8/site-packages/pyppeteer/chromium_downloader.py.bak /usr/local/lib/python3.8/site-packages/pyppeteer/chromium_downloader.py def download_zip(url: str) -> BytesIO: """Download data from url.""" logger.warning('start chromium download.\n' 'Download may take a few minutes.') print("下载地址" + url) # 打印下载地址 运行python程序，pyppeteer输出下载链接 https://storage.googleapis.com/chromium-browser-snapshots/Linux_x64/588429/chrome-linux.zip 有了url，方便下载了吧。再不会那就要去搜别的主题了。 放到哪里 同上打开上面那个py文件 在chromium_executable()函数中增加print打印目录 def chromium_executable() -> Path: """Get path of the chromium executable.""" print(chromiumExecutable[current_platform()]) return chromiumExecutable[current_platform()] 得到存储位置 /root/.local/share/pyppeteer/local-chromium/588429/chrome-linux/chrome 对应存储目录：/root/.local/share/pyppeteer/local-chromium/588429/chrome-linux/ 创建子目录，放入chromium运行测试 ls /root/.local/share/pyppeteer/ 发现目录存在，ok，逐级创建子目录即可 进入目录 cd /root/.local/share/pyppeteer/local-chromium/588429/chrome-linux 解压缩上面下载的chrome-linux.zip 把chrome-linux文件夹下所有内容上传到/root/.local/share/pyppeteer/local-chromium/588429/chrome-linux 给运行权限 cd /root/.local/share/pyppeteer/local-chromium/588429/chrome-linux/ chmod +x chrome chmod +x chrome_sandbox 运行chrome /root/.local/share/pyppeteer/local-chromium/588429/chrome-linux/chrome 报错： error while loading shared libraries: libXss.so.1: cannot open shared object file: No such file or directory 解决错误 yum install pango.x86_64 libXcompo…

从git历史删除中恢复某一个文件
type: item
date: 2020-04-27T02:01:05+08:00
description: 有个功能点之前不需要，删掉了，现在又需要还得把他找回来。确定git能完成这项任务，但是学会之后并没有实操，早已经不记得了。 这里记录一下操作过程 git命令查看文件删除hash戳 git log --all --full-history upload\xxx.php 输出(已做处理，不要照抄): commit d6cd1e2bd19e03a81132a23b2025920577f84e37 Author: zzz zzz@qq.com Date: Wed Apr 11 11:11:11 2020 +0800 修改记录。。。。。 ok，我们明确知道删除改文件的commit是d6cd1e2bd19e03a81132a23b2025920577f84e37 2. 查看该提交文件记录 git show d6cd1e2bd19e03a81132a23b2025920577f84e37 upload\xxx.php 3. 找回文件,git checkout 后面指定hash，紧接着加一个"~1"，表示从删除前的commit获取 git checkout d6cd1e2bd19e03a81132a23b2025920577f84e37~1 upload\xxx.php all done

Centos 7升级内核5.x
type: item
date: 2020-03-28T18:10:59+08:00
description: 当前环境查看 uname -r 3.10.0-514.el7.x86_64 安装新版本内核 rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm yum --enablerepo=elrepo-kernel install kernel-ml -y 查看当前已安装内核 查看已安装版本 rpm -qa | grep kernel kernel-3.10.0-1062.18.1.el7.x86_64 kernel-tools-libs-3.10.0-1062.18.1.el7.x86_64 kernel-tools-3.10.0-1062.18.1.el7.x86_64 kernel-ml-devel-5.5.13-1.el7.elrepo.x86_64 kernel-3.10.0-957.el7.x86_64 kernel-headers-3.10.0-1062.18.1.el7.x86_64 kernel-ml-5.5.13-1.el7.elrepo.x86_64 卸载安装的内核 rpm -e kernel-ml-devel-5.5.13-1.el7.elrepo.x86_64 rpm -e kernel-ml-5.5.13-1.el7.elrepo.x86_64 查看启动项目 awk -F\' '$1=="menuentry " {print i++ " : " $2}' /etc/grub2.cfg 0 : CentOS Linux (5.5.13-1.el7.elrepo.x86_64) 7 (Core) 1 : CentOS Linux (3.10.0-1062.18.1.el7.x86_64) 7 (Core) 2 : CentOS Linux (3.10.0-957.el7.x86_64) 7 (Core) 3 : CentOS Linux (0-rescue-27c6b54f6ca241c6884f74f0193db5e9) 7 (Core) 选择0号，5.5内核启动 grub2-set-default 0 #重启 reboot 错误 选择5.5内核启动后，开机报错无法进入系统 /dev/centos/root does not exist 解决方法 1. 启动时候选择'linux rescue' 2. 使用命令 dracut -f 开启bbr sysctl -a|grep tcp_congestion_control net.ipv4.tcp_congestion_control = cubic // 当前拥塞控制算法是cubic

godaddy生成ssl证书
type: item
date: 2019-02-28T05:40:43+08:00
description: 生成crt和private key openssl req -new -newkey rsa:2048 -nodes -keyout your_domain_name.key -out your_domain_name.csr 打开godaddy把生成的your_domain_name.csr提供给godaddy。godaddy会去生成证书。 下载godaddy证书。两个文件，一个大概叫做85814333f334c3c5d.crt,另一个大概叫做叫做gd_bundle-g2-g1.crt。把两个文件上传到服务器某一目录下。执行如下操作合并两个文件 cat 85814333f334c3c5d.crt gd_bundle-g2-g1.crt >> your_domain_name.cst 打开nginx网站配置文件，加入 listen 443 ssl http2; ssl on; ssl_certificate /存储路径/your_domain_name; ssl_certificate_key /存储路径/your_domain_name.cst; ssl_session_timeout 5m; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_prefer_server_ciphers on; ssl_ciphers "EECDH+CHACHA20:EECDH+CHACHA20-draft:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5"; ssl_session_cache builtin:1000 shared:SSL:10m; 重启nginx搞定