FlatBuffers Android Tutorial

FlatBuffers is an efficient cross platform serialization library for C++, Java, C#, Go, Python and JavaScript.
It was originally created at Google for game development and other performance-critical applications.
FlatBuffers is Open Source (Apache license V2) and available on GitHub.
It's currently used by:

•  Cocos2d-x, the open source mobile game engine and used to serialize the game data.
• Facebook uses it for client-server communication in the Android app (see the article).
• Fun Propulsion Labs at Google in most of libraries and games.

Solution overview 

• The schema will be defind in JSON format, then it will be converted to FlatBuffer format outside the application
• The Java classes of the Data model will be generated manually using flatc (FlatBuffer compiler)

Step 1: Build FlatBuffers

Download the source code in Google’s flatbuffers repository. 

The build process is described on Google's documentation FlatBuffers Building. 

On MacOS for example:

1. Open the xcode project FlatBuffers.xcodeproj
2. Build the project
3. flatc executable will appear in root directory.

Now we’re able generate model classes for a schema in different languages (Java, C#, Python, GO and C++) or convert JSON to FlatBuffer binary file.

Step 2: Prepare the schema file

Now we have to prepare schema file that defines the structure of the data we want to de-/serialize.
It will be used using flatc to generate Java data model.

I used http://www.json-generator.com/ to generate a bigger JSON file (~4mb).
The file is here.

Following the used schema:

	table PeopleList {
	peoples : [People];
	}
	table People {
	id : string;
	index : long;
	guid : string;
	name : string;
	gender : string;
	company : string;
	email : string;
	friends : [Friend];
	}
	table Friend {
	id : long;
	name : string;
	}
	root_type PeopleList;

view raw sample_schema.fbs hosted with ❤ by GitHub

Step 3: Generate FlatBuffers files

Now we will,

1. Convert the json file (sample_json.json) to FlatBuffers binary file
2. Generate Java Data Models

The command is:

./flatc -j -b sample_schema.fbs sample_json.json

view raw flatc_command hosted with ❤ by GitHub

The Java classes are:

• PeopleList.java
• People.java
• Friend.java

Step 4:Create the Android application

Following the app/build.graddle file of the application:

	apply plugin: 'com.android.application'
	android {
	compileSdkVersion 23
	buildToolsVersion "23.0.2"
	defaultConfig {
	applicationId "anaware.flatbufferssample"
	minSdkVersion 16
	targetSdkVersion 23
	versionCode 1
	versionName "1.0"
	}
	buildTypes {
	release {
	minifyEnabled false
	proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
	}
	}
	}
	dependencies {
	compile fileTree(dir: 'libs', include: ['*.jar'])
	testCompile 'junit:junit:4.12'
	compile 'com.android.support:appcompat-v7:23.1.1'
	compile 'com.google.code.gson:gson:2.3.1'
	}

view raw build.gradle hosted with ❤ by GitHub

Use FlatBuffers Java library

flatbuffers-java-1.2.0-SNAPSHOT.jar file can be used to handle the data format in java.

All we need is to put that jar inside the app/libs folder.

Android main activity 

Following the code of the main activity:

	package anaware.flatbufferssample;
	import android.os.AsyncTask;
	import android.support.v7.app.AppCompatActivity;
	import android.os.Bundle;
	import android.view.View;
	import android.widget.Button;
	import android.widget.ProgressBar;
	import android.widget.TextView;
	import com.google.gson.Gson;
	import java.io.IOException;
	import java.io.InputStream;
	import java.nio.ByteBuffer;
	import anaware.flatbufferssample.flatbuffersgen.People;
	import anaware.flatbufferssample.flatbuffersgen.PeopleList;
	import anaware.flatbufferssample.json.PeopleJson;
	import anaware.flatbufferssample.json.PeopleListJson;
	public class MainActivity extends AppCompatActivity {
	// UI Elements
	private View form;
	private ProgressBar progressBar;
	private TextView jsonStatus;
	private TextView flatbufferStatus;
	private Button jsonButton;
	private Button flatbufferButton;
	@Override
	protected void onCreate(Bundle savedInstanceState) {
	super.onCreate(savedInstanceState);
	setContentView(R.layout.activity_main);
	// Retrieve UI components
	form = (View)findViewById(R.id.form);
	progressBar = (ProgressBar)findViewById(R.id.progressBar);
	jsonStatus = (TextView)findViewById(R.id.jsonStatus);
	flatbufferStatus = (TextView)findViewById(R.id.flatbufferStatus);
	jsonButton = (Button)findViewById(R.id.jsonButton);
	jsonButton.setOnClickListener(new View.OnClickListener() {
	@Override
	public void onClick(View v) {
	parseJson();
	}
	});
	flatbufferButton = (Button)findViewById(R.id.flatbufferButton);
	flatbufferButton.setOnClickListener(new View.OnClickListener() {
	@Override
	public void onClick(View v) {
	parseFlatBuffer();
	}
	});
	}
	private void parseFlatBuffer() {
	progressBar.setVisibility(View.VISIBLE);
	form.setVisibility(View.GONE);
	new FlatParsing().execute(readRawResource(R.raw.sample_flatbuffers));
	}
	private void parseJson() {
	progressBar.setVisibility(View.VISIBLE);
	form.setVisibility(View.GONE);
	new JsonParsing().execute(readRawResource(R.raw.sample_json));
	}
	protected byte[] readRawResource(int resId) {
	InputStream stream = null;
	byte[] buffer = null;
	try {
	stream = getResources().openRawResource(resId);
	buffer = new byte[stream.available()];
	while (stream.read(buffer) != -1);
	} catch (IOException e) {
	} finally {
	if (stream != null) {
	try {
	stream.close();
	} catch (IOException e) {
	}
	}
	}
	return buffer;
	}
	private class JsonParsing extends AsyncTask<Object, Void, String> {
	@Override
	protected String doInBackground(Object... params) {
	String jsonText = new String((byte[])params[0]);
	long startTime = System.currentTimeMillis();
	PeopleListJson peopleList = new Gson().fromJson(jsonText, PeopleListJson.class);
	for (int i = 0; i < peopleList.peoples.size(); i++) {
	PeopleJson people = peopleList.peoples.get(i);
	}
	long endTime = System.currentTimeMillis() - startTime;
	String textToShow = "Elements: " + peopleList.peoples.size() + ": load time: " + endTime + "ms";
	return textToShow;
	}
	@Override
	protected void onPostExecute(String result) {
	jsonStatus.setText(result);
	progressBar.setVisibility(View.GONE);
	form.setVisibility(View.VISIBLE);
	}
	@Override
	protected void onPreExecute() {}
	@Override
	protected void onProgressUpdate(Void... values) {}
	}
	private class FlatParsing extends AsyncTask<Object, Void, String> {
	@Override
	protected String doInBackground(Object... params) {
	byte[] buffer = (byte[]) params[0];
	long startTime = System.currentTimeMillis();
	ByteBuffer bb = ByteBuffer.wrap(buffer);
	PeopleList peopleList = PeopleList.getRootAsPeopleList(bb);
	int length = peopleList.peoplesLength();
	for (int i = 0; i < length; i++) {
	People people = peopleList.peoples(i);
	}
	long endTime = System.currentTimeMillis() - startTime;
	String textToShow = "Elements: " + peopleList.peoplesLength() + ": load time: " + endTime + "ms";
	return textToShow;
	}
	@Override
	protected void onPostExecute(String result) {
	flatbufferStatus.setText(result);
	progressBar.setVisibility(View.GONE);
	form.setVisibility(View.VISIBLE);
	}
	@Override
	protected void onPreExecute() {}
	@Override
	protected void onProgressUpdate(Void... values) {}
	}
	}

view raw MainActivity.java hosted with ❤ by GitHub

Results

Following a screenshot of the application after parsing JSON as well as FlatBuffers data.

 

We can notice a huge difference between the two methods:

JSON parsing takes 1784ms while the FlatBuffers takes only 4ms!

Source code

Full source code can be found on Github.