Introduction to URL
In this tutorial, we will discover how to URL Encode a query string or path segment in Golang. URLs can only contain a limited set of characters from the US-ASCII character set. These characters include digits (0-9), letters(A-Z, a-z), and a few special characters (-, ., _, ~). Any character outside of this character set needs to be encoded or escaped before it can be used inside a URL. URL encoding commonly referred to as percent-encoding, transforms a string made up of non-ASCII, reserved, or unprintable characters into a widely used format for internet transmission. URL The MIME format application/x-www-form-urlencoded is another name for encoded data.
A URL consists of multiple parts (Domain segment, Path segment, Query string, etc). Here is an example of an url:
protocol://host:port/path?query
In this tutorial we will encode and decode path as well as query section of a URL.
What is the need to encode a URL in Golang?
It is possible that a URL we are using inside our GO code contains special characters, white spaces etc so to be able to use these URLs inside our function we need to escape encode the URL. This is done using QueryEscape and PathEscape for query and path respectively. We can further decode the respective URL sections using QueryUnescape and PathUnescape function.
How about we want to query information about "Killing Eve" or "Harry&Potter"? If we change the bookName to "Killing Eve" or "Harry&Potter", the output will be "Bad request" or "Can not find the book" because the Query string contains special characters. We can use the QueryEscape() and QueryUnescape() function to resolve that problem. From the client side, we use the QueryEscape() function to encode the parameter, the server side, use the QueryUnescape() to decode.
We will see some practical examples to cover for URL encode and decode.
URL Encode Examples in Golang
In this section we will use QueryEscape and PathEscape to encode some strings to demonstrate how URL encode works in golang.
func QueryEscape(s string) string: QueryEscape escapes the string so it can be safely placed inside a URL query.
func PathEscape(s string) string: PathEscape escapes the string so it can be safely placed inside a URL path segment, replacing special characters (including /) with %XX sequences as needed.
Example-1: Using url.QueryEscape
Here is a sample code where we are converting some strings to encoded format which further can be used inside a URL:
package main
import (
"fmt"
"net/url"
)
func main() {
q1 := url.QueryEscape("ab+c") // + is converted with %2B
q2 := url.QueryEscape("de$f") // $ is converted with %24
q3 := url.QueryEscape("gh i") // Whitespace is converted with +
fmt.Println(q1) // ab%2Bc
fmt.Println(q2) // de%24f
fmt.Println(q3) // gh+i
}
Example-2: Using url.PathEscape
Here is another example where we are converting some more set of special characters into encoded format using url.PathEscape:
package main
import (
"fmt"
"net/url"
)
func main() {
q1 := url.PathEscape("ab?c") // ? is converted with %3F
q2 := url.PathEscape("de#f") // # is converted with %23
q3 := url.PathEscape("gh i") // Whitespace is converted with %20
fmt.Println(q1) // ab%3Fc
fmt.Println(q2) // de%23f
fmt.Println(q3) // gh%20i
}
Example-3: Create encoded key-value query string in URL
func (v Values) Encode() string: Encode encodes the values into “URL encoded” form ("bar=baz&foo=quux") sorted by key.
func ParseQuery(query string) (Values, error): ParseQuery parses the URL-encoded query string and returns a map listing the values specified for each key. ParseQuery always returns a non-nil map containing all the valid query parameters found; err describes the first decoding error encountered, if any.
Let’s see an example with multiple keys to be URL encoded:
package main
import (
"fmt"
"net/url"
)
func main() {
var Url *url.URL
Url, err := url.Parse("http://www.example.com/say?")
if err != nil {
panic("failed")
}
params := url.Values{}
params.Add("hello", "w@rl(|")
params.Add("author", "G()L!nuxCl@u$")
Url.RawQuery = params.Encode()
fmt.Printf("Encoded URL is %q\n", Url.String())
}Output:
Encoded URL is "http://www.example.com/say?author=G%28%29L%21nuxCl%40u%24&hello=w%40rl%28%7C"
URL Decode Examples in Golang
func QueryUnescape(s string) (string, error): QueryUnescape does the inverse transformation of QueryEscape, converting each 3-byte encoded substring of the form "%AB" into the hex-decoded byte 0xAB. It returns an error if any % is not followed by two hexadecimal digits.
Example-1: Using url.QueryUnescape
Let's try to decode the strings which we encoded in the previous section using url.QueryEscape:
package main
import (
"fmt"
"net/url"
)
func main() {
q1, err := url.QueryUnescape("ab%2Bc") // %2B is converted with +
if err != nil {
panic(err)
}
q2, err := url.QueryUnescape("de%24f") // %24 is converted with $
if err != nil {
panic(err)
}
q3, err := url.QueryUnescape("gh+i") // + is converted with Whitespace
if err != nil {
panic(err)
}
fmt.Println(q1) // ab+c
fmt.Println(q2) // de$f
fmt.Println(q3) // gh i
}
Example-2: Using url.PathUnescape
Let's try to decode the strings which we encoded in the previous section using url.PathEscape:
package main
import (
"fmt"
"net/url"
)
func main() {
q1, err := url.PathUnescape("ab%3Fc") // %2B is converted with +
if err != nil {
panic(err)
}
q2, err := url.PathUnescape("de%23f") // %24 is converted with $
if err != nil {
panic(err)
}
q3, err := url.PathUnescape("gh%20i") // + is converted with Whitespace
if err != nil {
panic(err)
}
fmt.Println(q1) // ab?c
fmt.Println(q2) // de#f
fmt.Println(q3) // gh i
}
Example-3: Decode key-value query string from URL
In this section we will decode the key-value query string we encoded in the previous section:
package main
import (
"fmt"
"net/url"
)
func main() {
var Url *url.URL
Url, err := url.Parse("http://www.example.com/say?author=G%28%29L%21nuxCl%40u%24&hello=w%40rl%28%7C")
if err != nil {
panic("failed")
}
queries := Url.Query()
fmt.Println("Query Strings: ")
for key, value := range queries {
fmt.Printf(" %v = %v\n", key, value)
}
fmt.Println("Path: ", Url.Path)
}Here we use url.Parse to parse the complete URL and then use url.Query to capture the query string in the URL. This is because our key-value string is present inside the query string so we use for loop to loop through the list of key-value pairs and print them on the console. Lastly we use url.Path to print the path section of the URL.
Here is the output from this code. You can see the key value pair are the same which we had encoded in the previous example:
Query Strings:
author = [G()L!nuxCl@u$]
hello = [w@rl(|]
Path: /say
Summary
In this article, I showed you some examples of working with url in http request as well as how to encode and decode request string in communication between client and sever. I hope you enjoyed the article and learned how to URL encode (or percent-encode) strings in Go.
References
