How to do https request with bad certificate? [SOLVED]


Written By - Tuan Nguyen

Introduction

In the previous chapter, we introduced how to build an HTTPS server in Golang. In this post, we will answer the question "How to do an HTTPS request with a bad certificate in Golang?". First of all, we need to remind what is the certificate in HTTPS protocol:

SSL certificates are what enable websites to move from HTTP to HTTPS, which is more secure. An SSL certificate is a data file hosted in a website's origin server. SSL certificates make SSL/TLS encryption possible, and they contain the website's public key and the website's identity, along with related information. Devices attempting to communicate with the origin server will reference this file to obtain the public key and verify the server's identity. The private key is kept secret and secure.

Some problems may cause the invalid certificate

  • The site does not have an SSL certificate installed
  • The SSL certificate has expired
  • The SSL certificate is self-signed
  • The SSL certificate is invalid (for example, the name of the domain does not match the name on the certificate).

Let’s see how to solve the problem of invalid SSL certificate.

 

Send HTTPS request using a bad certificate

Example 1: Request to a website with an expired SSL certificate

The example below will make some HTTPS requests to a website whose SSL certificate is expired:

package main

import (
	"log"
	"net/http"
)

func main() {
	// expired certificate
	_, err := http.Get("https://expired.badssl.com/")
	if err != nil {
		log.Fatal(err)
	}
}

Output:

2022/12/28 00:02:42 Get "https://expired.badssl.com/": x509: certificate has expired or is not yet valid: 
exit status 1

 

Example 2: Request to a website with a self-signed SSL certificate

The example below will make some HTTPS requests to a website whose self-signed SSL certificate:

package main

import (
	"log"
	"net/http"
)

func main() {
	// self-signed certificate
	_, err := http.Get("https://self-signed.badssl.com/")
	if err != nil {
		log.Fatal(err)
	}
}

Output:

2022/12/28 00:03:38 Get "https://self-signed.badssl.com/": x509: certificate signed by unknown authority
exit status 1

We saw that our attempt to make an HTTPS request using bad certificate using not working. Now let's try to overcome this in next set of examples by turning of the security and making insecure connection.

 

Globally turn off the security checks for HTTPS Server

WARNING:

It is so risky to disable security checks and you should not do it.

For all requests from the default client, you have the option to disable security checks by modifying the DefaultTransport. DefaultTransport is the default implementation of Transport and is used by DefaultClient. It establishes network connections as needed and caches them for reuse by subsequent calls.

defaultTransport := http.DefaultTransport.(*http.Transport)

// Create new Transport that ignores self-signed SSL
customTransport := &http.Transport{
  Proxy:                 defaultTransport.Proxy,
  DialContext:           defaultTransport.DialContext,
  MaxIdleConns:          defaultTransport.MaxIdleConns,
  IdleConnTimeout:       defaultTransport.IdleConnTimeout,
  ExpectContinueTimeout: defaultTransport.ExpectContinueTimeout,
  TLSHandshakeTimeout:   defaultTransport.TLSHandshakeTimeout,
  TLSClientConfig:       &tls.Config{InsecureSkipVerify: true},
}
client := &http.Client{Transport: customTransport}

We can set the TLSClientConfig with InsecureSkipVerify option to true to skip security checks as the code shown below:

package main

import (
	"crypto/tls"
	"fmt"
	"io/ioutil"
	"log"
	"net/http"
	"os"
)

func main() {

	http.DefaultTransport.(*http.Transport).TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
        // the self-signed certificate
	res, err := http.Get("https://self-signed.badssl.com/")
	if err != nil {
		log.Fatal(err)
	}

	resBody, err := ioutil.ReadAll(res.Body)
	if err != nil {
		fmt.Printf("client: could not read response body: %s\n", err)
		os.Exit(1)
	}
	fmt.Printf("client: response body: %s\n", resBody)
}

Output:

client: response body: <!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="shortcut icon" href="/icons/favicon-red.ico"/>
  <link rel="apple-touch-icon" href="/icons/icon-red.png"/>
  <title>self-signed.badssl.com</title>
  <link rel="stylesheet" href="/style.css">
  <style>body { background: red; }</style>
</head>
<body>
<div id="content">
  <h1 style="font-size: 12vw;">
    self-signed.<br>badssl.com
  </h1>
</div>

</body>
</html>

 

Turn off security checks for client

With the same idea as the above example, in this example, we are going to disable the security check for a client:

package main

import (
	"crypto/tls"
	"fmt"
	"io/ioutil"
	"log"
	"net/http"
	"os"
)

func main() {
	tr := &http.Transport{
		TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
	}
	client := &http.Client{Transport: tr}
	// the expired certificate
	res, err := client.Get("https://expired.badssl.com/")
	if err != nil {
		log.Fatal(err)
	}

	resBody, err := ioutil.ReadAll(res.Body)
	if err != nil {
		fmt.Printf("client: could not read response body: %s\n", err)
		os.Exit(1)
	}
	fmt.Printf("client: response body: %s\n", resBody)
}

Output:

client: response body: <!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="shortcut icon" href="/icons/favicon-red.ico"/>
  <link rel="apple-touch-icon" href="/icons/icon-red.png"/>
  <title>expired.badssl.com</title>
  <link rel="stylesheet" href="/style.css">
  <style>body { background: red; }</style>
</head>
<body>
<div id="content">
  <h1 style="font-size: 12vw;">
    expired.<br>badssl.com
  </h1>
</div>

</body>
</html>

 

Summary

I need to emphasize that: We should not disable security checks in a production environment. To resolve the certificate problem, we should update the system certificate store on the client to include the missing intermediate certificate or check the SSL on the server.

 

References

https://www.cloudflare.com/learning/ssl/what-is-an-ssl-certificate/
https://stackoverflow.com/questions/12122159/how-to-do-a-https-request-with-bad-certificate

 

Categories GO

Can't find what you're searching for? Let us assist you.

Enter your query below, and we'll provide instant results tailored to your needs.

If my articles on GoLinuxCloud has helped you, kindly consider buying me a coffee as a token of appreciation.

Buy GoLinuxCloud a Coffee

For any other feedbacks or questions you can send mail to admin@golinuxcloud.com

Thank You for your support!!