Introduction
In the previous chapter, we introduced how to build an HTTPS server in Golang. In this post, we will answer the question "How to do an HTTPS request with a bad certificate in Golang?". First of all, we need to remind what is the certificate in HTTPS protocol:
SSL certificates are what enable websites to move from HTTP to HTTPS, which is more secure. An SSL certificate is a data file hosted in a website's origin server. SSL certificates make SSL/TLS encryption possible, and they contain the website's public key and the website's identity, along with related information. Devices attempting to communicate with the origin server will reference this file to obtain the public key and verify the server's identity. The private key is kept secret and secure.
Some problems may cause the invalid certificate
- The site does not have an SSL certificate installed
- The SSL certificate has expired
- The SSL certificate is self-signed
- The SSL certificate is invalid (for example, the name of the domain does not match the name on the certificate).
Let’s see how to solve the problem of invalid SSL certificate.
Send HTTPS request using a bad certificate
Example 1: Request to a website with an expired SSL certificate
The example below will make some HTTPS requests to a website whose SSL certificate is expired:
package main
import (
"log"
"net/http"
)
func main() {
// expired certificate
_, err := http.Get("https://expired.badssl.com/")
if err != nil {
log.Fatal(err)
}
}
Output:
2022/12/28 00:02:42 Get "https://expired.badssl.com/": x509: certificate has expired or is not yet valid:
exit status 1
Example 2: Request to a website with a self-signed SSL certificate
The example below will make some HTTPS requests to a website whose self-signed SSL certificate:
package main
import (
"log"
"net/http"
)
func main() {
// self-signed certificate
_, err := http.Get("https://self-signed.badssl.com/")
if err != nil {
log.Fatal(err)
}
}Output:
2022/12/28 00:03:38 Get "https://self-signed.badssl.com/": x509: certificate signed by unknown authority
exit status 1
We saw that our attempt to make an HTTPS request using bad certificate using not working. Now let's try to overcome this in next set of examples by turning of the security and making insecure connection.
Globally turn off the security checks for HTTPS Server
For all requests from the default client, you have the option to disable security checks by modifying the DefaultTransport. DefaultTransport is the default implementation of Transport and is used by DefaultClient. It establishes network connections as needed and caches them for reuse by subsequent calls.
defaultTransport := http.DefaultTransport.(*http.Transport)
// Create new Transport that ignores self-signed SSL
customTransport := &http.Transport{
Proxy: defaultTransport.Proxy,
DialContext: defaultTransport.DialContext,
MaxIdleConns: defaultTransport.MaxIdleConns,
IdleConnTimeout: defaultTransport.IdleConnTimeout,
ExpectContinueTimeout: defaultTransport.ExpectContinueTimeout,
TLSHandshakeTimeout: defaultTransport.TLSHandshakeTimeout,
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: customTransport}We can set the TLSClientConfig with InsecureSkipVerify option to true to skip security checks as the code shown below:
package main
import (
"crypto/tls"
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
)
func main() {
http.DefaultTransport.(*http.Transport).TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
// the self-signed certificate
res, err := http.Get("https://self-signed.badssl.com/")
if err != nil {
log.Fatal(err)
}
resBody, err := ioutil.ReadAll(res.Body)
if err != nil {
fmt.Printf("client: could not read response body: %s\n", err)
os.Exit(1)
}
fmt.Printf("client: response body: %s\n", resBody)
}Output:
client: response body: <!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="shortcut icon" href="/icons/favicon-red.ico"/>
<link rel="apple-touch-icon" href="/icons/icon-red.png"/>
<title>self-signed.badssl.com</title>
<link rel="stylesheet" href="/style.css">
<style>body { background: red; }</style>
</head>
<body>
<div id="content">
<h1 style="font-size: 12vw;">
self-signed.<br>badssl.com
</h1>
</div>
</body>
</html>
Turn off security checks for client
With the same idea as the above example, in this example, we are going to disable the security check for a client:
package main
import (
"crypto/tls"
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
)
func main() {
tr := &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: tr}
// the expired certificate
res, err := client.Get("https://expired.badssl.com/")
if err != nil {
log.Fatal(err)
}
resBody, err := ioutil.ReadAll(res.Body)
if err != nil {
fmt.Printf("client: could not read response body: %s\n", err)
os.Exit(1)
}
fmt.Printf("client: response body: %s\n", resBody)
}Output:
client: response body: <!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="shortcut icon" href="/icons/favicon-red.ico"/>
<link rel="apple-touch-icon" href="/icons/icon-red.png"/>
<title>expired.badssl.com</title>
<link rel="stylesheet" href="/style.css">
<style>body { background: red; }</style>
</head>
<body>
<div id="content">
<h1 style="font-size: 12vw;">
expired.<br>badssl.com
</h1>
</div>
</body>
</html>
Summary
I need to emphasize that: We should not disable security checks in a production environment. To resolve the certificate problem, we should update the system certificate store on the client to include the missing intermediate certificate or check the SSL on the server.
References
https://www.cloudflare.com/learning/ssl/what-is-an-ssl-certificate/
https://stackoverflow.com/questions/12122159/how-to-do-a-https-request-with-bad-certificate
