GAE/J datastorage persistence with Objectify framework in practice

2010-03-08T23:29:00.000-08:00

I am quite a newcomer for Google App Engine platform and just getting familiar with the basic concepts. Google App Engine is a platform for developing and hosting web applications in Google-managed data centers. One of the most important features in App Engine is using the DatastorageService where developers can persistet their application objects in BigTable clusters. Many who have worked previously with JDO and JPA might find the Datanucleus implementations worth trying but I come from PHP background which lacks those high level abstractions anyway. JDO, and especially JPA is designed for managing traditional relational database models which fights against the design philosophy of Google Datastorage. Moreover, both persistence interfaces are developed over a long period and provide backward compatibility increasing the historical burden of their implementations.

Google Datastorage is built on BigTable which is addresses highly scalability and geographically dispersed locations. In constrast to RDBMS'es, Google Datastorage has other principles one must take into account when persisting objects in the cloud. Google provides DatastoreService, but it has more machine-friendly than human-friendly query interface¹.

Fortunately, there is at least three frameworks providing higher abstraction for using the DatastoreService. Objectify, Twig and SimpleDS address the complexity of DatastoreService by providing a human-friendly query interface for applications objects' persistence.

Selecting between those three took me a few hours as I was browsing through their discussion groups and evaluating the quality of Wiki. Especially, effort the developer Jeff Schnitzer has put on speaking in favor² of Objectify and defending its design principles against other candidates made me an impression that the project is really worth trying :)

Let's get started with Objectify

The following tutorial is written for developers who have no previous experience with Google App Engine. If you feel you're more experienced, just leap the steps you feel not suitable for your purposes.

This stupid app demonstrates a simple one-to-many relationship between a car and its owner, and how to manage their persistence with Objectify.

Relational domain model

1. Install Eclipse and Google Plugin for Eclipse

2. Create new GAE project

Name your project and package as you wish but deselect Use Google Web Toolkit and then click Finish

3. Download Objectify 2.0.2 and copy the jar into lib-path and add it also to build path

4. You can test run your application and get the "hello world" printed on browser screen

Everything seems to be ok, so let's proceed with some real code

5. Let's create an entity class Person which stores the car owner instance

import javax.persistence.Id;
import com.googlecode.objectify.annotation.Entity;

@Entity
public class Person {
@Id private Long id;
private String name;
/* setters and getters omitted for brevity */
}

Notes:

@Entity is not obligatory for Objectify but good for distinguishing entities easily
@Id is the only required parameter and can be Long, long or String. I prefer Long, because id is auto generated if entity does not exist

6. Create Car entity class


import javax.persistence.Id;
import com.googlecode.objectify.Key;
import com.googlecode.objectify.annotation.Entity;
import com.googlecode.objectify.annotation.Parent;

@Entity
public class Car {
@Id public Long id;
private String license;

@Parent public Key<Person> owner;

private Car() {}
public Car(Key<Person> owner) {
this();
this.owner = owner;
}
/* setters and getters omitted for brevity */
}

Notes:

@Parent informs that the entity is in many-one relationship with Person (unlike in JDO/JPA, only children entities annotate their parent entities). Parent must be a type of Objectify Key data type.
Car constructor must be created with Key<person> argument to set the owner

7. Now we're going to implement some logic. Open ObjectifyCarsServlet.java and replace the code printing "Hello, world"


public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws IOException {

resp.setContentType("text/plain");
resp.getWriter().println("Hello, world");

}

...with...


public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws IOException {

resp.setContentType("text/html");

String html = "<form>Owner id (blank for new)<input name="id" type="text"> "+
"Car license <input name="license" type="text"><input type="submit"></form>";
resp.getWriter().println(html);

if( req.getParameter("license") == null ) {
return;
}
if( req.getParameter("license").length() == 0 ) {
resp.getWriter().println("License not set");
return;
}

}

Notes:

You should have basic form with car id and license fields when you restart the application and browse it
Owner id's are auto-generated by Objectify so you can not give distinctive name by yourself

8. Then... we must register Objectify entities. Write a static block for ObjectifyCarServlet-class:


@SuppressWarnings("serial")
public class ObjectifyCarsServlet extends HttpServlet {

static {
ObjectifyService.register(Person.class);
ObjectifyService.register(Car.class);
}

public void doGet(HttpServletRequest req, HttpServletResponse resp)
...

Notes:

For further reading about registering entities, go to Objectify Best Practices

9. Now let's make some logic for GET-requests. Paste the following code block bottom of doGet()-method:


Objectify ofy = ObjectifyService.begin();

Person owner = new Person();
if( req.getParameter("id") != null && req.getParameter("id").length() > 0 ) {
owner = ofy.find(Person.class, Long.parseLong(req.getParameter("id")) );
}
else
ofy.put(owner);

Notes:

Create Objectify object, ofy, for persistence manager
Then, we either find an existing owner or create a new one

10. Continue pasting with...


Key<Person> ownerKey = new Key<Person>(Person.class, owner.getId());
Car car = new Car(ownerKey);
car.setLicense(req.getParameter("license"));
ofy.put(car);

Notes:

Declare new ownerKey data-storage key for the owner object
Create a car instance for the owner
Persist the car instance with Objectify.put()

11. Finally, we display all cars stored in data-storage:


Query<Car> q = ofy.query(Car.class).ancestor(ownerKey);
resp.getWriter().println("<pre>Cars for owner id "+owner.getId()+":");
for( Car c : q )
resp.getWriter().println(c.getLicense());

Notes:

ofy.query(Car.class) searches all Car.class entities from the database
ancestor(ownerKey) omits all Car entities which don't have the requested owner from the result set (it took me quite a while to find out that Objectify Wiki lacks an example of using Query.ancestor() method)
Query is a collection type so it can be iterated through and strip down to single entities

There you go, the result could look something like this with browser after populating a few entries to the data-storage...

Doppler image animation with PHP and ImageMagick

2009-07-11T03:48:00.000-07:00

Gnome Weather panel provides a radar map using an image URL to show a current weather forecast. Radar map has a support for animated gifs which drove me to develop a little PHP script to make a doppler animation.

I found out that Foreca provides a set of dopper images showing the next 7 hours forecast in Finland. Images have well formed filenames, so there's an easy way to make a script to download them periodically.

Dependencies: ImageMagick and a truetype font

 // Nice little script I made courtesy to Gnome Weather Report panel's radar image

chdir(dirname(__FILE__));

exec('rm *.gif');
// download doppler images (foreca provides next 7 hours ahead)
for( $i = 0; $i < 7; $i++ ) {
$hour = date('H')+$i;
$hour = $hour%24;
$hour = str_pad($hour, 2, "0", STR_PAD_LEFT);

$date = date('Ymd').$hour;
$file = 'fin-'.$date.'.gif';

// thanks to taloussanomat :)
$url = 'http://http.foreca.com/xfer/taloussanomat/precipanim/'.$file;
exec( 'wget '.$url );

$date_str = $hour.':00';
// make a date image
exec( 'convert -size 50x20 xc:transparent -font '.dirname(__FILE__).'/tahoma.ttf -fill red '.
  '-pointsize 14 -draw "text 5,15 \''.$date_str.'\'" date.png ' );
// add date image on doppler image
exec( 'composite -gravity southeast date.png '.$file.' '.$file );
}

// generate animation (1s delay, infinite loop)
exec( 'convert -delay 100 -loop 0 fin-*.gif doppleranimation.gif' );

The script can be added to crontab, to make the doppler animation refresh once an hour. Working animation can be seen here http://panic.fi/~tero/weather/doppleranimation.gif

Tero Nurminen

GAE/J datastorage persistence with Objectify framework in practice

Doppler image animation with PHP and ImageMagick