Google App Engine provides some remote API functionality out of the box. One of the remote API features  is to download data from the development server. After downloading, then you can upload the downloaded data to your development server, effectively mirroring the content of the production server to your local development server. This is very useful if you are working CMS, sites, etc. where you want to test new layout or views locally against the old data before putting them to the production.

First enable remote API in app.yaml:

- url: /remote_api
  script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py
  login: admin

Note: Using builtins app.yaml directive didn’t work for me some reason, so I had to specify remote API URI manually.

After this you should be able to download data. Here I am using appcfg.py global installation on OSX. Below is the command and sample output.

appcfg.py -e yourgoogleaccount@gmail.com download_data --url=http://yourappid.appspot.com/remote_api --filename=data.sqlite3
...
Downloading data records.
[INFO    ] Logging to bulkloader-log-20110313.222523
[INFO    ] Throttling transfers:
[INFO    ] Bandwidth: 250000 bytes/second
[INFO    ] HTTP connections: 8/second
[INFO    ] Entities inserted/fetched/modified: 20/second
[INFO    ] Batch Size: 10
...
[INFO    ] Have 1803 entities, 0 previously transferred
[INFO    ] 1803 entities (972883 bytes) transferred in 91.0 seconds

data.sqlite3 is your production database dump in SQLite 3 binary format (used internally by the development server).

If you have sqlite command line tool installed you can explore around the data dump there:

sqlite3 data.sqlite
SQLite version 3.7.5
Enter ".help" for instructions
Enter SQL statements terminated with a ";"

sqlite> .tables
Apps                                   your-app!Model1!Entities
IdSeq                                  your-app!Model1!EntitiesByProperty
Namespaces                             your-app!Model2!Entities
bulkloader_database_signature          your-app!Model2!EntitiesByProperty
your-app!!Entities                     result
your-app!!EntitiesByProperty

Now you can upload data.

Note: Even though there exists option –use_sqlite for dev_appserver.py looks like it cannot directly use the database file produced by download_data. You cannot just swap database files, you need upload the downloaded data to the development server.

Start your development server:

dev_appserver.py .

In another terminal, go to downloaded data.sqlite folder and give the command:

appcfg.py upload_data --url http://localhost:8080/remote_api --file=data.sqlite --application=yourappid

It will ask you for credentials, but it seems that any username and password is accepted for the local development server.

Now you can login to your local development server to explore the data:

http://localhost:8080/_ah/admin

Ensure your data got copied over using Data Viewer:

http://localhost:8080/_ah/admin

 Subscribe to RSS feed Follow me on Twitter Follow me on Facebook Follow me Google+