How to use Let’s Encrypt DNS challenge validation?

Currently it is possible to perform a DNS validation also with the certbot LetsEncrypt client in the manual mode. Automation is also possible (see below).

Manual plugin

You can either perform a manual verification – with the manual plugin.

certbot -d bristol3.pki.enigmabridge.com --manual --preferred-challenges dns certonly

Certbot will then provide you an instructions to manually update a TXT record for the domain in order to proceed with the validation.

Please deploy a DNS TXT record under the name
_acme-challenge.bristol3.pki.enigmabridge.com with the following value:

667drNmQL3vX6bu8YZlgy0wKNBlCny8yrjF1lSaUndc

Once this is deployed,
Press ENTER to continue

Once you have updated DNS record, press Enter, certbot will continue and if LetsEncrypt CA verifies the challenge, certificate is issued as normally.

You may also use a command with more options to minimize interactivity and answering certbot questions. Note that manual plugin does not yet support non-interactive mode.

certbot --text --agree-tos --email [email protected] -d bristol3.pki.enigmabridge.com --manual --preferred-challenges dns --expand --renew-by-default  --manual-public-ip-logging-ok certonly

The renewal does not work with the manual plugin as it runs in non-interactive mode. More info at official Certbot documentation.

Update: manual hooks

In the new Certbot version you can use hooks, e.g. --manual-auth-hook, --manual-cleanup-hook. The hooks are external scripts executed by Certbot to perform the task.

Information is passed in environment variables – e.g., domain to validate, challenge token. Vars: CERTBOT_DOMAIN, CERTBOT_VALIDATION, CERTBOT_TOKEN.

certbot certonly --manual --preferred-challenges=dns --manual-auth-hook /path/to/dns/authenticator.sh --manual-cleanup-hook /path/to/dns/cleanup.sh -d secure.example.com

You can write your own handler or use already existing, there are many available, e.g., for Cloudflare DNS.

More info on official Certbot hooks documentation

Automation, Renewal, Scripting

If you would like to automate DNS challenges validation it is not currently possible with vanila certbot. Update: some automation is possible with the Certbot hooks.

We thus created a simple plugin that supports scripting with DNS automation. It’s available as certbot-external-auth.

pip install certbot-external-auth

It supports DNS, HTTP, TLS-SNI validation methods. You can either use it in the handler mode or JSON output mode.

Handler mode

In the handler mode, the certbot + plugin calls external hooks (a program, shell script, python, …) to perform the validation and installation. In practice you write a simple handler/shell script which gets input arguments – domain, token and makes the change in DNS. When handler finishes, certbot proceeds with validation as usual.

This gives you an extra flexibility, renewal is also possible.

Handler mode is also compatible with Dehydrated DNS hooks (former letsencrypt.sh). There are already many DNS hooks for common providers (e.g., CloudFlare, GoDaddy, AWS). In the repository there is a README with extensive examples and example handlers.

Example with Dehydrated DNS hook:

certbot \
    --text --agree-tos --email [email protected] \
    --expand --renew-by-default \
    --configurator certbot-external-auth:out \
    --certbot-external-auth:out-public-ip-logging-ok \
    -d "bristol3.pki.enigmabridge.com" \
    --preferred-challenges dns \
    --certbot-external-auth:out-handler ./dehydrated-example.sh \
    --certbot-external-auth:out-dehydrated-dns \
    run 

JSON mode

Another plugin mode is JSON mode. It produces one JSON object per line. This enables a more complicated integration – e.g., Ansible or some deployment manager is calling certbot. Communication is performed via STDOUT and STDIN. Cerbot produces JSON object with data to perform the validation, e.g.,

certbot \
    --text --agree-tos --email [email protected] \
    --expand --renew-by-default \
    --configurator certbot-external-auth:out \
    --certbot-external-auth:out-public-ip-logging-ok \
    -d "bristol3.pki.enigmabridge.com" \
    --preferred-challenges dns \
    certonly 2>/dev/null

{"cmd": "perform_challenge", "type": "dns-01", "domain": "bs3.pki.enigmabridge.com", "token": "3gJ87yANDpmuuKVL2ktfQ0_qURQ3mN0IfqgbTU_AGS4", "validation": "ejEDZXYEeYHUxqBAiX4csh8GKkeVX7utK6BBOBshZ1Y", "txt_domain": "_acme-challenge.bs3.pki.enigmabridge.com", "key_auth": "3gJ87yANDpmuuKVL2ktfQ0_qURQ3mN0IfqgbTU_AGS4.tRQM98JsABZRm5-NiotcgD212RAUPPbyeDP30Ob_7-0"}

Once DNS is updated, caller sends new-line character to STDIN of the certbot to signalize it can continue with validation.

This enables automation and certificate management from the central management server. For installation you can deploy certificates over SSH.

For more info please refer to the readme and examples on certbot-external-auth GitHub.

EDIT: There is also a new blog post describing the DNS validation problem and the plugin usage.

EDIT: we currently work on Ansible 2-step validation, will be soon off.

thrift多服务

java服务端

package com.xx.thrift;


import org.apache.thrift.TMultiplexedProcessor;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.server.TServer;
import org.apache.thrift.server.TThreadPoolServer;
import org.apache.thrift.transport.TServerSocket;
import org.apache.thrift.transport.TTransportException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Configuration;

import javax.annotation.PostConstruct;
import java.io.IOException;
import java.net.ServerSocket;

@Configuration
public class Server {

    @Value("${thrift.port}")
    private int servicePort;
    @Autowired
    private PdfSignService.Iface pdfService;
    @Autowired
    private HkVideoService.Iface videoService;

    @PostConstruct
    public void startServer() {
        try {
            System.out.println(servicePort);
            ServerSocket socket = null;
            try {
                socket = new ServerSocket(servicePort);
            } catch (IOException e) {
                e.printStackTrace();
            }
            TServerSocket serverTransport = new TServerSocket(socket);

            // thrift多服务
            TMultiplexedProcessor multiplexedProcessor = new TMultiplexedProcessor();
            // 服务1
            multiplexedProcessor.registerProcessor("PdfService", new PdfSignService.Processor<>(pdfService));
            // 服务2
            multiplexedProcessor.registerProcessor("HkVideoService", new HkVideoService.Processor<>(videoService));

            TThreadPoolServer.Args serverArgs = new TThreadPoolServer.Args(serverTransport);
            TBinaryProtocol.Factory proFactory = new TBinaryProtocol.Factory();

            serverArgs.processor(multiplexedProcessor);
            serverArgs.protocolFactory(proFactory);

            TServer server = new TThreadPoolServer(serverArgs);
            System.out.println("Start server on port : " + servicePort);

            server.serve();
        } catch (TTransportException e) {
            e.printStackTrace();
        }
    }
}


python客户端

@contextmanager
def get_pdf_client():
    transport = None
    try:
        transport = TSocket.TSocket(THRIFT_HOST, THRIFT_PORT)
        protocol = TBinaryProtocol.TBinaryProtocol(transport)
        transport.open()
        pdf_protocol = TMultiplexedProtocol.TMultiplexedProtocol(protocol, "PdfService")
        pdf_client = PdfClient(pdf_protocol)
        yield pdf_client
    finally:
        if transport is not None:
            transport.close()

fatal: early EOF fatal: index-pack failed

First, turn off compression:

git config --global core.compression 0

Next, let’s do a partial clone to truncate the amount of info coming down:

git clone --depth 1 <repo_URI>

When that works, go into the new directory and retrieve the rest of the clone:

git fetch --unshallow 

or, alternately,

git fetch --depth=2147483647

Now, do a regular pull:

git pull --all

I think there is a glitch with msysgit in the 1.8.x versions that exacerbates these symptoms, so another option is to try with an earlier version of git (<= 1.8.3, I think).

via: fatal: early EOF fatal: index-pack failed

spring-boot+mybatis+pagehelper

几个重要部分:

添加依赖

       <!--MYSQL-->
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>5.1.46</version>
        </dependency>
        <!--mybatis-->
        <dependency>
            <groupId>org.mybatis.spring.boot</groupId>
            <artifactId>mybatis-spring-boot-starter</artifactId>
            <version>1.2.0</version>
        </dependency>
        <!-- mybatis 分页-->
        <dependency>
            <groupId>com.github.pagehelper</groupId>
            <artifactId>pagehelper-spring-boot-starter</artifactId>
            <version>1.2.12</version>
        </dependency>

增加几条配置

appplication.properties

pagehelper.autoDialect=true
pagehelper.closeConn=true

直接在service层方法中使用

@Override
    public ServerResponse<PageInfo<MovieVo>> selectMovieList(String categoryId, String keyWord, Integer page, Integer pageSize) {
        System.out.print(pageSize);
        PageHelper.startPage(page, pageSize);
        List<Movie> movies = movieMapper.selectMovieList(categoryId, keyWord);
        ArrayList<MovieVo> movieVoList = newArrayList();
        for (Movie movie: movies) {
            movieVoList.add(convertToMovieVo(movie));
        }
        PageInfo<MovieVo> pageInfo = new PageInfo(movies);
        pageInfo.setList(movieVoList);
        return ServerResponse.createBySuccess(pageInfo);
    }
  • 响应为:

{ "status":0, "msg":null, "data":{ "total":4, "list":[ { "id":29, "categoryId":null, "name":"Haier/海尔HJ100-1HU1 10公斤滚筒洗衣机全自动带烘干家用大容量 洗烘一体", "subtitle":"门店机型 德邦送货", "mainImage":null }, { "id":28, "categoryId":null, "name":"4+64G送手环/Huawei/华为 nova 手机P9/P10plus青春", "subtitle":"NOVA青春版1999元", "mainImage":null } ], "pageNum":1, "pageSize":2, "size":2, "startRow":1, "endRow":2, "pages":2, "prePage":0, "nextPage":2, "isFirstPage":true, "isLastPage":false, "hasPreviousPage":false, "hasNextPage":true, "navigatePages":8, "navigatepageNums":[ 1, 2 ], "navigateFirstPage":1, "navigateLastPage":2 }, "success":true }

自签名ssl

# 1.创建 CA 私钥
openssl genrsa -out ca.key 2048

# 2.生成 CA 的自签名证书
openssl req -subj "/C=CN/ST=HZ/L=HZ/O=Mocha/OU=HZWukt/CN=Server CA/[email protected]" -new -x509 -days 3650 -key ca.key -out ca.crt

# 3.生成需要颁发证书的私钥
openssl genrsa -out server.key 2048

# 4.生成要颁发证书的证书签名请求,证书签名请求当中的 Common Name 必须区别于 CA 的证书里面的 Common Name
openssl req -subj "/C=CN/ST=HZ/L=HZ/O=Mocha/OU=HZWukt Software/CN=*.wktadmin.com/[email protected]" -new -key server.key -out server.csr

# 5.用 2 创建的 CA 证书给 4 生成的 签名请求 进行签名
openssl x509 -req -days 3650 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt

参考:

https://ningyu1.github.io/site/post/51-ssl-cert/
https://www.jianshu.com/p/e5f46dcf4664

supervisorctl无法杀死进程/supervisorctl无法kill celery进程

使用supervisorctl shutdown 无法杀死进程.

解决方法1.

stopasgroup=true 选项配置到 supervisord.conf 文件中。因为不仅要杀死父进程,还要杀死子进程。

原文参考: https://stackoverflow.com/questions/24297683/cant-kill-celery-processes-started-by-supervisor

解决方法2.

手动kill

ps -ef | grep celery | grep -v grep | awk '{print $2}' | xargs kill -9