Introduction to the Python struct module
The struct module in Python is used to convert native Python data types such as strings and numbers into a string of bytes and vice versa. It is used mostly for handling binary data stored in files or from network connections, among other sources. Python struct is a built-in module, which means we don't need to manually install the struct module on our system.
In this tutorial, we will learn about the python struct module. We will cover various functions that are available in this module, for example, struct.pack()
, struct.unpack()
, struct.unpack_form()
, struct.calcsize()
, and struct.pack_into()
along with various examples. In a nutshell, this tutorial will give you a general idea about the python struct module and its various methods by solving different examples.
Getting started with Python struct module
The Python struct module is used to provide a simple Pythonic interface to access and manipulate C’s structure datatype. This module can convert Python values to a C structure and vice-versa. The C structure is used as a Python bytes object since there is nothing called an object in C; only byte-sized data structures. The following table shows some of the notations that are used in C and Python to represent the data types. See the table below:
Format | C type | Python type |
---|---|---|
c | char | bytes of length 1 |
b | signed char | integer |
B | unsigned char | integer |
h | short | integer |
i | int | integer |
d | double | float |
f | float | float |
s | char[] | bytes |
? | _Bool | bool |
Python struct.pack() method
The python struct.pack()
is used to pack elements into a Python byte-string (byte object). The simple syntax of struct.pack()
method is as follows:
struct.pack(format, value1, value2, …)
Here value1
, value2
, … are the values that will be packed into the byte object. The format
refers to the format of the packing. This is needed since we need to specify the datatype of the byte-string, as it is used with C code.
Example of Python struct.pack() method
Now let us take an example and see how we can use the python struct.pack()
method to pack the given values into the byte object. See the example below:
# importing struct module
import struct
# packing 3 integers, so 'iii' is required
variable = struct.pack('iii', 1, 2, 3)
# printing the type of packed variable
print(type(variable))
# printing
print(variable)
Output:
<class 'bytes'>
b'\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00'
Notice that the type of packed variable is bytes. Now let us try to understand the code. As you can see in the above program, it stores the 3 integers 1, 2, and 3 in a byte object using pack()
method. Since the size of an integer is 4 bytes, so you see 3 blocks of 4 bytes in the output, which correspond to 3 integers in C. If we will provide the wrong formatting for the given values, the program will return an error. See the following python program.
# importing struct module
import struct
# packing 3 integers, so 'iii' is required
variable = struct.pack('ccc', 1, 2, 3)
# printing the type of packed variable
print(type(variable))
# printing print(variable)
Output:
Notice that we use "ccc" format which is used for char data type but we provide integers so the python gave the above output.
Python struct.unpack() method
The struct.unpack()
method, unpacks the packed value into its original representation according to an appropriate format. This returns a tuple of size equal to the number of values passed since the byte object is unpacked to give the elements. The simple syntax of the struct.unpack()
method is as follows:
struct.unpack(format, string)
This is actually the reverse of struct.pack()
method and unpacks the byte string according to the format specifier.
Example of struct.unpack() method
Now, let us take an example and see how struct.unpack()
method actually works. See the example below:
# import struct
import struct
# creating string type of bytes
String_bytes = b'\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00'
# python strucn unpack() method
Numeric_values = struct.unpack('iii', String_bytes)
# printing
print(Numeric_values)
Output:
(1, 2, 3)
Notice that the python struct.unpack()
method converts the bytes into a tuple of values. If we will not provide the required format for the given bytes of the string, again we will get an error. See the example below:
# import struct
import struct
# creating string type of bytes
String_bytes = b'\x01\x00\x00\x00\x02\x00\x00\x00\x03\x00\x00\x00'
# python strucn unpack() method
Numeric_values = struct.unpack('ccc', String_bytes)
# printing
print(Numeric_values)
Output:
Notice that we provide the format for char characters while the given string_bytes was of integers types that is why we get the above error.
Python struct.calcsize() function
The Python struct.calcsize()
function returns the total size of the String representation of the struct using a given format specifier, to retrieve the types of the data and calculate the size. For example, this function will return 4 for integers and 1 for char characters as they contain 4 nd 1 bytes respectively. The simple syntax of the struct.calcsize()
is shown below:
struct.calcsize(Format)
This function takes only on the argument, which is the format of the data type, and based on the argument, this function returns the size of the specified data types in bytes.
Example of Python struct.calcsize() function
Now let us take an example and see how the python struct.calcsize()
function. See the example below which prints the size of different specified data types using the python struct.calcsize()
function.
# importing struct
import struct
# printing the size of integer
print("The size of an integer is :", struct.calcsize('i'))
# printing the size of char
print("The size of a char is :", struct.calcsize('c'))
# printing the size of float
print("The size of a float is :", struct.calcsize('f'))
Output:
The size of an integer is : 4
The size of a char is : 1
The size of a float is : 4
Notice that the python struct.calcsize()
method returns the size of the specified data type. Moreover, this method can also be used to find the size of multiple data types at once. This method will add all the sizes of data types and returns the sum. See the example below:
# importing struct
import struct
# printing the size of integer
print("The size of 3 integer is :", struct.calcsize('iii'))
# printing the size of char
print("The size of 5 char is :", struct.calcsize('ccccc'))
# printing the size of mixed data type
print("The total size is :", struct.calcsize('ffiicc'))
Output:
The size of 3 integer is : 12
The size of 5 char is : 5
The total size is : 18
Notice that the size of three integer data types in total is 12 and we get the same output.
Python struct.pack_into() function
The struct.pack_into()
function is used to pack values into a Python string buffer, available in the ctypes
module. The following is the simple syntax of the python struct.pack_into()
function.
struct.pack_into(Format, buffer, offset, value1, value2, …)
In the above syntax, Format refers to the format specifier, as always. buffer
is the string buffer which will contain the specified packed values. We can also specify an offset
location from the base address from which packing will occur.
Example of struct.pack_into() function
Now let us take an example and see how the python struct.pack_into() function actually works. Before writing our program, we have to first import ctypes because this function uses the pack values which are available in the ctypes. See the example below:
# import struct module
import struct
# import ctypes module
import ctypes
# creating string buffer of size 8
buffer_size = struct.calcsize('ii')
# Create the string buffer
Buffer = ctypes.create_string_buffer(buffer_size)
# struct.pack() returns the packed data
struct.pack_into('ii', Buffer,0, 1, 2,)
# Display the contents of the buffer
print(Buffer[:])
Output:
b'\x01\x00\x00\x00\x02\x00\x00\x00'
Notice that in the above code, we first created a buffer size string and we get our packed values in the buffer string. If we will provide more format sizes or format specifiers than provided values, the program will return an error. See the example below:
# import struct module
import struct
# import ctypes module
import ctypes
# creating string buffer of size 8
buffer_size = struct.calcsize('ii')
# Create the string buffer
Buffer = ctypes.create_string_buffer(buffer_size)
# struct.pack() returns the packed data
struct.pack_into('iiiii', Buffer,0, 1, 2,)
# Display the contents of the buffer
print(Buffer[:])
Output:
Notice that we only provided two values and gave format size for 5, so the program returns the above error.
Python struct.unpack_form() function
It is similar to python struct.unpack()
method and the python struct.unpack_form()
does the reverse of struct.pack_into()
. The following is the simple syntax:
struct.unpack_from(Format, buffer, offset)
This function returns a tuple of values, similar to the python struct.unpack()
.
Example of Python struct.unpack_form() function
Now let us take an example of python struct.unpack_form() method and see how it actually works in the python program. See the example below:
# importing struct
import struct
# importing ctypes
import ctypes
# creating string buffer size of 12
buffer_size = struct.calcsize('iii')
# Create the string buffer
Buffer = ctypes.create_string_buffer(buffer_size)
# struct.pack() returns the packed data
struct.pack_into('iii', Buffer, 0, 1, 2, 3)
# priting the python struct unpact_form ()
print(struct.unpack_from('iii', Buffer, 0))
Output:
(1, 2, 3)
Notice that the python struct.unpack_form()
method had returned a tuple of values.
Summary
The struct module in Python is used to convert native Python data types such as strings and numbers into a string of bytes and vice versa. In this tutorial, we learned about the Python struct module. We learned how we can find the size and convert different data types into bytes. At the same time, we cover various functions that are available in this module including python struct.pack()
, python struct.unpack()
, python struct.calcsize()
, struct.pack_into()
and struct.unpack_form()
by solving different examples.
We also discussed some of the common errors that can occur while using the python struct module and its various functions. To summarize, this tutorial contains all the necessary examples and functions that are needed in order to start working with the python struct module.
Further Reading
Python struct module documentation
Python bytes
Python data types