We are now ISO/IEC 27001:2022 certified. Read PR here.
Ignitarium is Great Place to Work® – Certified. Read PR here.
Ignitarium is Great Place to Work® – Certified. Read PR here.

info@ignitarium.webindia.com

Japanese
In the ever-evolving landscape of programming languages, Python has established itself as a favorite among developers for its simplicity, readability, and extensive libraries. Moreover, we love Python. 

pybind11 — Seamless operability between C++11 and Python

Unlocking the Power of C++ with Python

In the ever-evolving landscape of programming languages, Python has established itself as a favorite among developers for its simplicity, readability, and extensive libraries. Moreover, we love Python. 

However, there are times when the raw speed and low-level control offered by languages like C++ become essential for optimizing performance-critical code. What if you could seamlessly combine the ease of Python with the raw power of C++? This remarkable synergy is precisely where Pybind11 takes centre stage. With Pybind11, a seamless integration of Python’s user-friendly elegance with the robust might of C++ unfolds before us.

Pybind11 or  Boost.Python ?

Pybind11 is a header-only library, meaning that you don’t need to compile a separate shared library, where as boost python has been around for a long time and may involve more complex syntax and setup.

Compared to Boost, pybind11 requires less boilerplate code. Its syntax is simpler and more concise. It has strong support for Python3, which is becoming the standard choice for python development. 

For detailed information on using pybind11, you can refer to the official documentation. Here, we are focussing more on how we can effectively use Pybind11 for some AI-based applications.

Topics We Will Cover

  • Getting started

  • Exposing C++ Class

  • Handling NumPy Arrays

  • Pybind11’s Pythonic Oasis

  • Common challenges in wrapping C++ APIs

  • Wrap-up

Getting Started

Before diving into the world of pybind11, you’ll need to set up your development environment.

Installing Python 3.10 Using Conda

  • Create Python 3.10 environment

  • Clone the Pybind11 repository

conda create -n pybind_env python=3.10
git clone https://github.com/pybind/pybind11.git

Exposing C++ Class

We’ll start by defining a C++ class using Pybind11 that encapsulates our image processing operations. Pybind11’s intuitive syntax makes it a breeze to create this bridge between Python and C++

Make sure you have OpenCV installed in your machine. If not, either build from source, or simply:

sudo apt-get install libopencv-dev

Define CmakeList:

cmake_minimum_required(VERSION 3.10)

set(PROJECT_NAME "myLib")
project(${PROJECT_NAME})
set(CMAKE_CXX_STANDARD 20)
set(PYBIND11_PYTHON_VERSION 3.10)
# set(CMAKE_CXX_FLAGS "-Wall -Wextra -fPIC")
add_subdirectory(pybind11)
find_package(OpenCV REQUIRED)
pybind11_add_module(${PROJECT_NAME} process.cpp)
target_include_directories(${PROJECT_NAME} PUBLIC
${OpenCV_INCLUDE_DIRS})

target_link_libraries(${PROJECT_NAME} PUBLIC
${OpenCV_LIBS})
target_compile_definitions(${PROJECT_NAME}
PRIVATE VERSION_INFO=${EXAMPLE_VERSION_INFO})

#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
#include <opencv2/opencv.hpp>
#include <iostream>
namespace py = pybind11;
class CVutils
{
private:
    std::string filename;

public:
    py::array_t<uint8_t> process_image(std::string &filename, int width, intheight)
{
    cv::Mat input_image = cv::imread(filename, cv::IMREAD_COLOR);
    // Perform image processing, such as resizing
    cv::Mat resized_image;
    cv::resize(input_image, resized_image, cv::Size(width, height));
    return py::array({height, width,             static_cast<int>(resized_image.channels())}, resized_image.data);
}

Define a simple image utils class
Build

mkdir build && cd build
cmake ..
make

├── CMakeCache.txt
├── CMakeFiles
├── cmake_install.cmake
├── Makefile
├── myLib.cpython-310-x86_64-linux-gnu.so
└── pybind11

Python Test Suite

from build.myLib import CVutils
cv_obj = CVutils()
py_list_obj = PyListInt()
processed_image = cv_obj.process_image("flower.jpeg", 640, 480)
print("Resized: ", processed_image.shape)

Handling NumPy Arrays

A simple array addition

#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
#include <opencv2/opencv.hpp>
#include <iostream>
namespace py = pybind11;
using PyArrayFloat = py::array_t<double>;
PyArrayFloat add_arrays(PyArrayFloat input1, PyArrayFloat input2)
{
    py::buffer_info buf1 = input1.request();
    py::buffer_info buf2 = input2.request();
    if (buf1.ndim != 1 || buf2.ndim != 1)
        throw std::runtime_error("Number of dimensions must be one");
    if (buf1.size != buf2.size)
        throw std::runtime_error("Input shapes must match");
    PyArrayFloat result = PyArrayFloat(buf1.size);
    py::buffer_info buf3 = result.request();
    double *ptr1 = static_cast<double *>(buf1.ptr);
    double *ptr2 = static_cast<double *>(buf2.ptr);
    double *ptr3 = static_cast<double *>(buf3.ptr);
    for (size_t idx = 0; idx < buf1.shape[0]; idx++)
    ptr3[idx] = ptr1[idx] + ptr2[idx];
        return result;
}
};

Python Test Suite

import numpy as np
from build.myLib import add_arrays

arr1 = np.array([1, 2])
arr2 = np.array([5, 6])
print(add_arrays(arr1, arr2))

Pybind11’s Pythonic Oasis

#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>
#include <opencv2/opencv.hpp>
#include <iostream>

namespace py = pybind11;
using PyListInt = std::vector<int64_t>;
py::class_<PyListInt>(m, "MyList")
    .def(py::init<>())
    .def("append", (void(PyListInt::*)(const int &)) & PyListInt::push_back)
    .def("clear", (void(PyListInt::*)()) & PyListInt::clear)
    .def("pop", (void(PyListInt::*)()) & PyListInt::pop_back)
    .def("__len__", [](const PyListInt &v)
        { return v.size(); })
    .def("__get__", [](const PyListInt &v, int idx)
        { return v[idx]; })

    .def("print", [](const PyListInt &l)
    {
        fprintf(stderr, "[");
        auto it = l.begin();
        if (it != l.end()) {
            fprintf(stderr, "%d", *it++);            
}
        for (; it != l.end(); ++it) {
            fprintf(stderr, ", %d", *it);
            }

        fprintf(stderr, "]\n");   
})

Python Test Suite

from build.myLib import MyList
my_list = MyList()
my_list.append(1)
my_list.append(19)
my_list.print()
my_list.pop()
my_list.print()

Common challenges in wrapping C++ APIs

We might encounter several challenges which will require careful consideration and attention. These are some of the challanges you may face:

Type RepresentationPybind11 provides tools for handling custom data structrures, Its important to understand them before using
Exception HandlingC++ exceptions needs to be properly translated into python exceptions and vice versa
Callbacks and LamdasExecution context and memory management can differ
Performance OptimizationAchieving performance rewuire additional efforts, such as minimizing data copies / leveraging multi threading

These challanges offer a glimpse into the intricacies of binding C++ APIs with Pybind11.

Wrap-up

The examples presented here provide a foundational understanding; this merely scratches the surface of what PyBind11 can do. From the initial steps of setting up the environment and understanding the fundamental concepts to some more useful cases like handling NumPy and OpenCV libraries, you might have gained a comprehensive understanding of how Pybind11 bridges the gap between Python and C++.

The key takeaway is that Pybind11 empowers the user to harness the strengths of both C++ and Python, enabling to create high-performance, versatile and accessible software solutions. Especially in cases like developing AI applications, game engines, data analysis tools, or any other projects that demands the best of both languages, Pybind11 is a formidable ally.

References

  1. https://pybind11.readthedocs.io/en/stable/advanced/classes.html

  2. https://www.boost.org/doc/libs/1_81_0/libs/python/doc/html/index.html

  3. https://en.cppreference.com/w/cpp/language/reference

  4. https://chat.openai.com/chat

Stay informed

NEWS & VIEWS

Join our team

APPLY

Want to know more about us?

PRIVACY POLICY

©2024 Ignitarium Technology Solutions, All Rights Reserved

An ISO 9001:2015 certified company

Great Place to Work® Certified

Scroll to Top