Send comments and corrections to <[email protected]>
This is an unofficial, community-generated guide for developing applications that use 360 media from RICOH THETA cameras. This is not authorized by RICOH and is based on publicly available information.
An application for the RICOH THETA can focus on any of these categories.
-
Ideas for Apps to Control the Camera
-
Example: build a hardware remote to set a timer, use a small hardware device to add GPS information or other tags. Media is manually uploaded to YouTube or Facebook by photographer or script automatically uploads media using YouTube API.
-
-
Ideas for Back-end Server Media Management
-
Example: mobile app takes image then sends information to server to process and display on webpage using embedded third-party image viewer or plays video on existing site such as YouTube, Facebook.
-
-
-
Example: Viewing and then controlling the 360 degree media with an application you build yourself.
-
There’s many types of applications you can build to control the camera. If you’re just getting started with 360 images, I recommend you focus on building an application to control the camera. You can extend the application to move media from the camera to an external storage device for processing later or to a third-party site such as YouTube, Facebook, or Google+.
Here are some ideas from the community.
-
Security: Take 360 images (not video) at set intervals. Transfer images from camera to external storage device. The image is already in equirectangular format. Apply your own algorithm to identify objects.
-
Accident: Attach THETA to a camera platform rigged to a commercial truck or a sign post near a dangerous crossing such as a train crossing. In the event of a traffic accident, a 360 degree image taken.
-
Historical Site: Use a drone or autonomous vehicle to take images of sites and apply tagging.
-
Additional Functionality
-
Add a timer delay to the camera
-
Create a remote control feature
-
Take four images at once using four cameras and four small devices like a Raspberry Pi, each with a WiFi USB adapter. Trigger the images simultaneously using the GPIO pins of the Pi.
-
In this scenario, you build a web site that hosts 360 images or video. You then build a mobile app that connects to THETA and takes the image and uploads it to your special server. Example uses are real estate, sporting event, music festival, travel.
A basic mockup of this concept built by a high school student is available here. The purpose is to show you how easy it is to start building your own site with a JavaScript open source viewer and a basic Bootstrap template. The site, including the source code for the viewer software, is available as a GitHub repository, gh-pages branch.
The mockup of the controller is based on an existing Python/Pygame application that was described in the Unofficial API Guide. Full source code available.
|
Important
|
The mobile app will not be able to connect to the THETA and Internet WiFi at the same time. You may need to disconnect the phone to THETA connection before connecting to the Internet. |
|
Note
|
The THETA becomes a WiFi server with IP address 192.168.1.1 The IP address is not designed to be changed. |
If your application has a mobile component, you should download the official RICOH SDK for Android or iOS applications.
You can extend this concept to make the images or video viewable inside of a headset for an immersive experience. The easiest way to get a demo with 360 video up and running is to use an existing application such as YouTube to display the media.
If you want to embed a 360 image into your web application, you can look at one of these existing applications:
-
Pano2VR: Commerical. Pano2VR is an application to convert spherical or cylindrical panoramic images into HTML5, Flash, or QuickTime VR (QTVR) panoramas with features such as customizable skins, tour building, multiresolution (gigapixel panorama), hotspots and directional sound. There is a demo available. The image with the demo will have a watermark.
-
HoloBuilder: Free. HoloBuilder is a f ree web platform that lets you create 360° and virtual reality content. Build virtual tours from your 360° photospheres and enhance them with 3D objects. Embed images right into your website using holo player. Based on WebGL, HoloBuilder runs on any modern web browser on any device. HoloBuilder was the RICOH THETA developer content grand prize winner. The application is free to use.
-
krpano: Commercial. Viewer for panoramic images and interactive virtual tours. The viewer is available as Flash and HTML5 application. The viewer is designed for the usage inside the Browser on Desktop (Windows, Mac, Linux) and on Mobiles/Tablets (iPhone, iPad, Android, …). In addition to the krpano Viewer there are the krpano Tools - small tools and droplets which help to prepare the panoramic images for viewing and make them ready-to-use. Making a pano or a tour is possible just by drag-and-drop. Commercial product. Demo available.
Other ideas. The camera could be distributed to influencers at a travel event like Las Vegas casino opening and a special web platform could aggregate and create 360 experiences using an existing front-end system.
The THETA cannot stream video over WiFi. You can use
camera._getLivePreview
to get a live view. This only works in still image capture mode, not video mode.
You can use
camera.takePicture to
take the picture and
camera.getImage to
transfer the image from the camera to the headset.
In most cases, the headset will be a mobile phone that is inside of a cardboard holder. If you’re targeting a mobile phone, you can use the examples in the official RICOH SDK.
|
Important
|
In most cases, your application should target a still image. Although the THETA can stream live video from the camera’s USB port, the output is in dual-fisheye mode (two spheres). Desktop software on Mac or PC can stitch the dual-fisheye videos into equirectangular format so that it can be viewed on the mobile phone. This is a manual process and the video is saved to disk. You could build a mobile app that displays a catalog of pre-recorded videos for display in a headset. |
|
Note
|
At CES, RICOH demonstrated live streaming of stiched video. This software is not yet available for public use (as of 1/27). RICOH is not able to disclose lens distortion data. |
The RICOH THETA S does support
camera.getLivePreview
in still image mode. This outputs binary equirectangular MotionJPEG for Live View.
According the RICOH site, the image size for Live View is 640x320 and the frame rate is 10fps.
It might be theoretically possible to use LivePreview to show MotionJPEG at low resolution and then trigger a high-resolution 360 image for transfer to the headset. You may encounter problems with the delay.
If you’re building a mobile app or using Java, you should definitely download and study the official SDK.
The easiest place to get started is to look at the network section of the SDK.
For Android, the docs are in:
RICOH_THETA_v2_SDK_for_Android.0.1.0/RICOH_THETA_v2_SDK_for_Android.0.1.0/ricoh-theta-sample-for-androidv2/doc/index.html
Here’s a taste of what the SDK offers for controlling the camera.
ImageData getImage(java.lang.String fileId, HttpDownloadListener listener) Acquire raw data of specified image
ImageSize getImageSize() Acquire currently set shooting size
java.util.ArrayList<ImageInfo> getList() Acquire list of media files on device
private java.util.ArrayList<ImageInfo> getListInternal(int maxReceiveEntry, java.lang.String token) Acquire media file list (limited number of items)
java.io.InputStream getLivePreview() Acquire live view stream
private java.lang.String getState() Acquire device status
StorageInfo getStorageInfo() Acquire storage information of device
Bitmap getThumb(java.lang.String fileId) Acquire thumbnail image
Hopefully, at this stage I’ve enticed you to open up the official SDK and see if it meets your needs.
The SDK covers display and navigation of media. It does not cover video streaming.
There are two versions of the THETA API. Version 1 works with m15 cameras. Version 2 works with the THETA S. It is an HTTP API that is compliant with the Open Spherical Camera API. You should focus on the v2 API.
The API only works over WiFi. As of 1/27, use of the API over USB is not supported.
If you are new to the RICOH API, we’ve produced a gentle introduction.
The official RICOH SDK is a great place to start to understand how to work with 360 degree media.
To help you start your amazing app quickly, we’ve also started collecting community information in an unofficial media guide.
These are ideas brought up by the community. We do not know if they are feasible. As far as we know, no one has successfully built these applications.
This concept was submitted by a professional VR photographer, but has not been built yet. I’ve adapted the concept with my own idea of using Raspberry Pi units.
Materials:
-
4 Raspberry Pi units. Cheap ones fine. Processing power not important.
-
4 USB WiFi dongles
-
8 wires to connect to the GPIO pins of the Raspberry Pi
-
Frame or board to mount the Raspberry Pi units together as a single transportable unit
-
Physical electrical button
-
Portable power supply for the Raspberry Pis (battery)
Pointers:
-
_wlanChanneloption to set WiFi channel of camera if you encounter interference -
Unofficial API guide with Raspberry Pi section and Python code examples
The winner of the last TechCrunch Disrupt Hackathon won the RICOH THETA prize for streaming video using an Android device and sending the video to an external server for stitching.
I built a similar device using a Raspberry Pi that could stream dual-fisheye video. Streaming dual-fisheye video to an external server over WiFi from a device connected to the THETA is fairly easy. It will probably take you less than an hour. Different groups of people have then got the stitching to work, though there are gaps in the edges of the video that are noticeable. You probably can’t get rid of the gaps without confidential information that RICOH can’t release to the public.
The other thing to consider is that RICOH demoed live streaming and stitched video at CES in January over USB. Thus, it seems likely there will be an official solution coming out in the future that solves the stitching problem.
However, this is still an interesting idea that could work to prototype a concept as you wait for the supporting technology to mature.
It’s important to note that the resolution of the stitched video may be lower than what you need for actual use. It’s for experimentation and prototyping with equipment you may already have around your studio.
Update: 2/10/2016. The Unofficial Media Guide has an image and video of a demo of the official RICOH live streaming USB driver with stitching. The driver is not available to the public yet and may not be available at the Feb 13 hackathon.
If you’re interested in testing out the API, I suggest you read the Unofficial API Guide. Have fun!