Skip to content

[src,build] Add minimal example to demonstrate pybind11 binding #3668

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 15 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions src/makefiles/default_rules.mk
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@

SHELL := /bin/bash


ifeq ($(KALDI_FLAVOR), dynamic)
ifeq ($(shell uname), Darwin)
ifdef LIBNAME
Expand All @@ -25,6 +26,13 @@ else
XDEPENDS = $(ADDLIBS)
endif

ifeq ($(LIBFILE_EXTENSION),)
else
# This will be reached if compiling in the pybind11 directory,
# which uses a special extension name for the library.
LIBFILE=$(LIBNAME)$(LIBFILE_EXTENSION)
endif

all: $(LIBFILE) $(BINFILES)

$(LIBFILE): $(OBJFILES)
Expand Down
43 changes: 43 additions & 0 deletions src/pybind/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@

# make "all" the target.
all:



# Disable linking math libs because not needed here. Just for compilation speed.
# no, it's now needed for context-fst-test.
# MATHLIB = NONE

EXTRA_CXXFLAGS = -Wno-sign-compare

include ../kaldi.mk


ifeq ($(KALDI_FLAVOR),static)
$(error You cannot build the pybind directory unless the build was dynamic; reconfigure with --shared option.)
endif

PYBIND_INCLUDES=$(shell python3 -m pybind11 --includes)
CXXFLAGS += $(PYBIND_INCLUDES)
PYBIND_EXTENSION=$(shell python3-config --extension-suffix)
LIBFILE=kaldi-pybind
LIBFILE_EXTENSION=$(PYBIND_EXTENSION)

# pybind11 is heavily templated and generates code that is bloated before optimization.
# -flto is link time optimization which apparently is important.
CXXFLAGS += -O3 -flto
LDFLAGS += -flto
# TODO:

ifeq ($(shell uname),Darwin)
LDFLAGS += -undefined dynamic_lookup
endif

OBJFILES = kaldi_pybind.o

LIBNAME = kaldi_pybind

# Note: the following will actually get turned into dynamic library names.
ADDLIBS = ../lib/kaldi-util.so ../lib/kaldi-matrix.so ../lib/kaldi-base.so

include ../makefiles/default_rules.mk
14 changes: 14 additions & 0 deletions src/pybind/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
This was tested as follows. After configuring with --shared and making the rest of Kaldi,
you can cd to here and do:
```
# make
# python3
>>> import kaldi_pybind as k
>>> import numpy as np
>>> a = k.FloatVector(10)
>>> b = np.array(a, copy_data = False)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

copy_data is an invalid argument.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be copy

>>> b[5:10] = 2.0
>>> str(a)
[ 0 0 0 0 0 2 2 2 2 2 ]

```
63 changes: 63 additions & 0 deletions src/pybind/kaldi_pybind.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
// pybind/kaldi_pybind.cc

// Copyright 2019 Daniel Povey

// See ../../COPYING for clarification regarding multiple authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// THIS CODE IS PROVIDED *AS IS* BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION ANY IMPLIED
// WARRANTIES OR CONDITIONS OF TITLE, FITNESS FOR A PARTICULAR PURPOSE,
// MERCHANTABLITY OR NON-INFRINGEMENT.
// See the Apache 2 License for the specific language governing permissions and
// limitations under the License.

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
namespace py = pybind11;
#include "matrix/kaldi-matrix.h"

using namespace kaldi;

PYBIND11_MODULE(kaldi_pybind, m) {
m.doc() = "pybind11 binding of some things from kaldi's src/matrix directory. "
"Source is in $(KALDI_ROOT)/src/pybind/matrix-lib.cc";

py::enum_<MatrixResizeType>(m, "MatrixResizeType", py::arithmetic(), "Matrix initialization policies")
.value("kSetZero", kSetZero, "Set to zero")
.value("kUndefined", kUndefined, "Leave undefined")
.value("kCopyData", kCopyData, "Copy any previously existing data")
.export_values();


py::class_<Vector<float> >(m, "FloatVector", pybind11::buffer_protocol())
.def_buffer([](const Vector<float> &v) -> pybind11::buffer_info {
return pybind11::buffer_info(
(void*)v.Data(),
sizeof(float),
pybind11::format_descriptor<float>::format(),
1, // num-axes
{ v.Dim() },
{ 4 }); // strides (in chars)
})
.def("Dim", &Vector<float>::Dim, "Return the dimension of the vector")
.def("__repr__",
[] (const Vector<float> &a) -> std::string {
std::ostringstream str; a.Write(str, false); return str.str();
})
.def(py::init<const MatrixIndexT, MatrixResizeType>(),
py::arg("size"), py::arg("resize_type") = kSetZero);

}