CGo – Referencing C library in Go

At Marlin, we build networking libraries that are expected to be used by blockchain implementations written in a variety of languages - Go, Python, Rust, C++ etc. A lot of these libraries have been developed in C/C++ as reference implementations in order to be compatible with hardware routers. While in the long term, we expect alternate implementations of these libraries to be developed by the community, in the short term we need a way for blockchain developers (almost none of whom code in C/C++) to be able to use these libraries.

As a result, we have developed wrappers to encapsulate the C code and only expose interfaces in popular languages used by blockchain devs. One such wrapper is designed for Go. During the implementation of the Go wrapper, we realized that the Official CGo page is tough to read and thus compiled a simpler doc to share with the rest of the team.

Happy to share the same here to help developers out there who are interested in using a C-library in Go using CGo while spending the least amount of time in learning how to do so.

Snippets are taken from a mock version of marlinSdk specifically written for the purpose of this blog.

Basics

By importing the pseudo-package “C”, the Go code can then refer to types such as C.size_t, variables such as C.stdout, or functions such as C.putchar.

The C headers, libraries and even any ad hoc C code to be referenced in the Go code is placed in a comment section immediately preceding “import C” called the preamble. This is used as a header when compiling the C parts of the package.

Including C header files

In the example given below, header file “MockMulticastSdk.h” is included in the C preamble which allows the user to refer any of the variables and methods in it.

Note: The library corresponding to the header file also needs to be included which is illustrated in the next subsection.

File main.go:

/*
#include <stdlib.h>
#include "./cfiles/MockMulticastSdk.h"
*/
import "C"

Including C libraries

File main.go:

/*
#...
#cgo LDFLAGS: -lstdc++ -lm
#cgo LDFLAGS: ./cfiles/libMockMulticastSdk.a
#...
*/
import "C"

Variables

C types in go

- C.char, C.schar (signed char), C.uchar (unsigned char)
- C.short, C.ushort (unsigned short)
- C.int, C.uint (unsigned int)
- C.long, C.ulong (unsigned long),
- C.longlong (long long), C.ulonglong (unsigned long long)
- C.float, C.double
- C.struct_<name_of_C_Struct>
- C.union_<name_of_C_Union>
- C.enum_<name_of_C_Enum>

Void*

The C type void* is represented by Go’s unsafe.Pointer.

cs := C.CString("Hello from stdio")
C.free(unsafe.Pointer(cs))

Conversion between Go <-> C types by making copies of the data

Go types like strings need to be converted to C types before they can be passed as arguments to C functions. Here are some functions that can be used to convert between Go <-> C.

Go type to C type:

// The C string is allocated in the C heap using malloc.
// It is the caller's responsibility to arrange for it to be
// freed, such as by calling C.free (be sure to include stdlib.h)
func C.CString(string) *C.char

// Go []byte slice to C array
// The C array is allocated in the C heap using malloc.
// It is the caller's responsibility to arrange for it to be
// freed, such as by calling C.free (be sure to include stdlib.h)
func C.CBytes([]byte) unsafe.Pointer

C type to Go type:

// C string to Go string
func C.GoString(*C.char) string

// C data with explicit length to Go string
func C.GoStringN(*C.char, C.int) string

// C data with explicit length (in bytes) to Go []byte
func C.GoBytes(unsafe.Pointer, C.int) []byte

Using C structs in Go

File MockMulticastSdk.h:

// C Struct

typedef struct MockMarlinSdk MockMarlinSdk;

struct MockMarlinSdk {
	int foo;
	int bar;

	did_recv_message_func cb_recv_message;
};

File main.go:

// different ways of referencing the C struct in Go

var msdk_pointer *C.struct_MockMarlinSdk

msdk2 := C.struct_MockMarlinSdk {
		foo: 3,
		bar: 4,
	}

Calling C Functions

Finally we reach the point where we call the C functions. After importing the header file and linking with appropriate libraries, all we need to do is call C.<CFunction>

File MockMulticastSdk.h:

// Creator functions for C structs which can be called from Go to get a struct pointer in Go

MockMarlinSdk* sdk_create (int foo, int bar);

File main.go:

var msdk_pointer *C.struct_MockMarlinSdk
msdk_pointer = C.sdk_create(C.int(1), C.int(2))

Advanced

Passing a function pointer to C

Goal

To be able to pass Go function pointers to C code

Problem

A Go function cannot be passed directly to a C function as a delegate/callback because of Go pointer passing rules which state the following: “Go code may pass a Go pointer to C provided the Go memory to which it points does not contain any Go pointers”

Therefore indirection is required.

Example Scenario

The MockMulticastSdk struct has a callback function which can be set up by passing a function pointer argument of type did_recv_message_func to the sdk_set_recv_cb function. In the solution section we will try to setup this callback to a function defined in Go.

File MockMulticastSdk.h:

typedef struct MockMarlinSdk MockMarlinSdk;

typedef void (*did_recv_message_func) (
	MockMarlinSdk* client,
	const char* message
);

struct MockMarlinSdk {
	int foo;
	int bar;

    // callback function pointer which needs to be set
	did_recv_message_func cb_recv_message;
};


// function declaration to setup the callback func pointer
void sdk_set_recv_cb (MockMarlinSdk* msdk, did_recv_message_func cb_recv_message);

File MockMulticastSdk.c:

// function definition to setup the callback func pointer
void sdk_set_recv_cb (MockMarlinSdk* msdk, did_recv_message_func cb_recv_message) {
	msdk->cb_recv_message = cb_recv_message;
}

Solution

Step 1: Declaring the Gateway functions

The gateway function is a C function declared in the preamble. A pointer referencing this gateway function can then be passed to any instance expecting a C function pointer.

File main.go

package main

/*
#include <stdlib.h>
#include "./cfiles/MockMulticastSdk.h"
#cgo LDFLAGS: ./cfiles/libMockMulticastSdk.a

// Forward declaration of gateway function
extern void did_recv_message_cgo (
	MockMarlinSdk* client,
	const char* message);
*/
import "C"

Step 2: Calling the actual Go callback functions inside Gateway function definition

The actual Go function is then called inside this C gateway function.

Note - The definition of the function in preamble needs to be provided in a separate file otherwise it throws an error about multiple definition. So cfuncs.go file is created to define the gateway function.

File cfuncs.go

package main

// CALLING THE GO FUNCTION FROM THE GATEWAY FUNCTION

/*
#include "./cfiles/MockMulticastSdk.h"

extern void go_did_recv_message (
	MockMarlinSdk* client,
	const char* message);

// Gateway function defintion
void did_recv_message_cgo (
	MockMarlinSdk* client,
	const char* message) {
 	go_did_recv_message(client, message);
}
*/
import "C"

Step 3: Defining the go callback function

File main.go:

//export go_did_recv_message
func go_did_recv_message (client *C.struct_MockMarlinSdk, message *C.char) {
	fmt.Printf("Hello\n")
}

Step 4: Passing a pointer to the gateway function where we needed to pass a pointer to go function

Here we pass the pointer to our gateway function to the C.sdk_set_recv_cb

File main.go:

func main() {
	var msdk_pointer *C.struct_MockMarlinSdk
	msdk_pointer = C.sdk_create(C.int(1), C.int(2))
    
    // NOTE we need to cast did_recv_message_cgo to 
    // did_recv_message_func type
    
	C.sdk_set_recv_cb(
		msdk_pointer,
		C.did_recv_message_func(C.did_recv_message_cgo),
	)
}

Resources and References

https://gist.github.com/zchee/b9c99695463d8902cd33

https://golang.org/cmd/cgo/

https://blog.golang.org/c-go-cgo

Feel free to share your views in our research forum. Do not forget to subscribe to our blog to receive more interesting content.

Our other official social media channels:

Twitter | Telegram Announcements | Telegram Chat | Discord | Website