Photo Sphere formats and metadata

I recently got a Nexus 4 and the panoramic Photo Sphere feature is really nice. Supposedly it comes from the Google Street View technology, but the the Android UI for creating panoramas is really simple and quick to use, and produces nice results. 

The panorama is automatically stitched and cropped on the Android device and saved as a single, large JPEG.

Displaying these images inside a sphere instead of as a flat panorama is possible on Google+ and Google Maps (and doesn’t require Flash!). There’s a gallery of existing panoramas on Google Maps, for those who wish to upload their own panoramas — though this requires that you enable geo-tagging of your photos beforehand. A recent very-quickly-taken effort of mine (inside the TUI Arena in Hannover) can be opened on the right. You may then need to click on the sphere icon at the top-left to get the full view.

In order to display Photo Sphere images correctly, metadata is added to each image. The original JPEG for the above panorama includes the following XMP metadata:

<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP Core 5.1.0-jc003">
  <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
    <rdf:Description rdf:about=""
        xmlns:GPano="http://ns.google.com/photos/1.0/panorama/"
      GPano:UsePanoramaViewer="True"
      GPano:ProjectionType="equirectangular"
      GPano:PoseHeadingDegrees="171.0"
      GPano:CroppedAreaLeftPixels="1535"
      GPano:FullPanoWidthPixels="6366"
      GPano:FirstPhotoDate="2012-12-11T19:52:38.215Z"
      GPano:CroppedAreaImageHeightPixels="1614"
      GPano:FullPanoHeightPixels="3183"
      GPano:SourcePhotosCount="13"
      GPano:CroppedAreaImageWidthPixels="3228"
      GPano:LastPhotoDate="2012-12-11T19:53:17.283Z"
      GPano:CroppedAreaTopPixels="1265"
      GPano:LargestValidInteriorRectLeft="0"
      GPano:LargestValidInteriorRectTop="0"
      GPano:LargestValidInteriorRectWidth="3228"
      GPano:LargestValidInteriorRectHeight="1614"/>
  </rdf:RDF>
</x:xmpmeta>

This format is documented on the Google Developers site.

Google Maps also provides a web service which adds the appropriate XMP metadata to semi-spherical panoramas created manually, so long as you can provide the values for the horizontal and vertical field of view.

If you want to accept Photo Sphere images in your own Android apps, you can add the custom MIME type application/vnd.google.panorama360+jpg to the relevant IntentFilter in your app. I tried adding this MIME type to the full panorama image on my webserver, but currently there are no Android apps which combine the https scheme with this MIME type, so the Photo Sphere isn’t displayed automatically.

So it seems that Photo Spheres are entirely self-contained. I couldn’t find the individual images that make up the panorama on my phone, except for what seemed to be temporary files hidden away in a non-user-accessible directory at <shared-storage>/Android/data/com.google.android.gallery3d/files/panorama_sessions/. This included the individual images taken via the Photo Sphere UI, plus the following two files, orientations.txt:

-0.0034921998 0.9999939 -5.448116E-17 -0.999945 -0.003492029 0.009890969 0.009890908 3.4541237E-5 0.99995106 1.0128322
-0.010889014 0.89141405 0.45305902 -0.999578 -0.021906279 0.019077305 0.026930613 -0.4526601 0.89127636 0.79672396
-0.007397307 0.60263807 0.7979803 -0.9990442 -0.038834233 0.020066598 0.04308185 -0.79706913 0.60234934 0.22377136
-0.02910276 0.9068215 -0.420509 -0.99957347 -0.027429868 0.010026784 -0.0024420042 0.42062145 0.90723294 0.76564556

And session.meta:

first_photo_time,1355384611933
last_photo_time,1355384624478
source_photos_count,4
pose_heading,88

The former would appear to be positional data about each of the four images I had taken, while the latter file contains the metadata we saw in the XMP above.

However, the panorama_sessions directory only contained data for two recent Photo Spheres, and not ones taken more than a couple of days ago — so I guess these really are just temporary files and the original, individual images aren’t intended to be kept. As stitching is done automatically by the device, this means there’s no opportunity to import the individual images into software like the excellent Hugin, should you wish to improve the result.

As for displaying on Google+, once you upload a Photo Sphere image it is split into a “pyramid” structure of multiple tiles and and zoom levels (similar to Google Maps etc.). The XML file loaded by Google+ to display the above panorama looks like this, and appears pretty self-explanatory:

<?xml version="1.0" encoding="UTF-8"?>
<TileInfo
  tilewidth="512" tileheight="512"
  fullpyramiddepth="4" origin="TOPLEFT" version_number="495"
  baseurl="lh4.ggpht.com/-pHEkwelf6FY/UMfOmPj0UMI/AAAAAAAAAe8/8FC5DinBKI8/2012%2B-%2B1">

    <pyramidlevel numtilesx="1" numtilesy="1" inversescale="8"
      emptypelsx="199" emptypelsy="356"/>
    <pyramidlevel numtilesx="2" numtilesy="1" inversescale="4"
      emptypelsx="397" emptypelsy="199"/>
    <pyramidlevel numtilesx="3" numtilesy="2" inversescale="2"
      emptypelsx="282" emptypelsy="397"/>
    <pyramidlevel numtilesx="5" numtilesy="3" inversescale="1"
      emptypelsx="52" emptypelsy="282"/>

</TileInfo>

The viewer seems to use the panorama metadata, the “pyramid” info, plus presumably a ton of JavaScript to display the Photo Sphere on an HTML <canvas> element.

Though with the existing viewer heavily intertwined in the JavaScript of Google+, there doesn’t yet seem to be a solution for people to display Photo Spheres on a non-Google site…