Fabio Biondi Blog » Blog Archive » Augmented Reality using Papervision3D and Flex

Home » 3d parts libraries, ActionScript 3.0, Adobe, Flex 3, Papervision 3D, _Featured

Augmented Reality using Papervision3D and Flex

8 October 2009 3 Comments

Augmented reality (AR) is a term for a live direct or indirect view of a physical real-world environment whose elements are merged with (or augmented by) virtual computer-generated imagery - creating a mixed reality.
The augmentation is conventionally in real-time and in semantic context with environmental elements, like for example sports scores on TV during a match.

(from Wikipedia)

Following the final result of this exercise, using in Flex 3 and Papervision 3D:

Recently ActionScript 3.0 developers are knowing Augmented Reality thanks to FLARToolkit, an AS3 library that allows to localize an image (a “marker” defined from developers) in a 3D environment with the possibility to overlay 3d items over it.
A good video tutorial, that explain how to configure your Flex environment, create custom markers (the image that will be recognize from the webcam) and overlay some 3d static cubes, is available on gotoAndLearn.
But in this sample, we’ll use the FLARToolkit to create 3d animated objects over our custom marker every time it will be recognize from our webcam.

To understand this code is required ActionScript 3.0 and Papervision basic knowlege and i really suggest to view the gotoAndLearn video tutorial.

The final output is visible on the video above but following you can test the live demo, printing the custom marker (available on this URL) and moving it in front of your webcam.

This movie requires Flash Player 10

Following the complete source code:

package {
	import flash.display.BitmapData;
	import flash.display.Sprite;
	import flash.events.Event;
	import flash.media.Camera;
	import flash.media.Video;
	import flash.utils.ByteArray;
 
	import org.libspark.flartoolkit.core.FLARCode;
	import org.libspark.flartoolkit.core.param.FLARParam;
	import org.libspark.flartoolkit.core.raster.rgb.FLARRgbRaster_BitmapData;
	import org.libspark.flartoolkit.core.transmat.FLARTransMatResult;
	import org.libspark.flartoolkit.detector.FLARSingleMarkerDetector;
	import org.libspark.flartoolkit.support.pv3d.FLARBaseNode;
	import org.libspark.flartoolkit.support.pv3d.FLARCamera3D;
	import org.papervision3d.materials.BitmapFileMaterial;
	import org.papervision3d.objects.primitives.Sphere;
	import org.papervision3d.render.BasicRenderEngine;
	import org.papervision3d.scenes.Scene3D;
	import org.papervision3d.view.Viewport3D;
 
	[SWF(width="640", height="480", frameRate="40", backgroundColor="#FFFFFF")]
 
	public class EarthAndSat extends Sprite
	{
		[Embed(source="FBPattern.pat", mimeType="application/octet-stream")]
		private var pattern:Class;
 
		[Embed(source="camera_para.dat", mimeType="application/octet-stream")]
		private var params:Class;
 
		private var fparams:FLARParam;
		private var mpattern:FLARCode;
		private var vid:Video;
		private var cam:Camera;
		private var bmd:BitmapData;
		private var raster:FLARRgbRaster_BitmapData;
		private var detector:FLARSingleMarkerDetector;
		private var scene:Scene3D;
		private var camera:FLARCamera3D;
		private var container:FLARBaseNode;
		private var vp:Viewport3D;
		private var bre:BasicRenderEngine;
		private var trans:FLARTransMatResult;
 
		private var earthSphere:Sphere
		private var satellite:Sphere
 
 
                private var angle:Number = 0;
		private var centerX:Number = 0;
		private var centerY:Number = 100;
		private var radius:Number = 100;
		private var speed:Number = .1
 
 
		public function EarthAndSat()
		{
			setupFLAR();
			setupCamera();
			setupBitmap();
			setupPV3D();
			addEventListener(Event.ENTER_FRAME, loop);
		}
 
		private function setupFLAR():void
		{
			fparams = new FLARParam();
			fparams.loadARParam(new params() as ByteArray);
			mpattern = new FLARCode(16, 16);
			mpattern.loadARPatt(new pattern());
		}	
 
		private function setupCamera():void
		{
			vid = new Video(640, 480);
			cam = Camera.getCamera();
			cam.setMode(640, 480, 30);
			vid.attachCamera(cam);
			addChild(vid);
		}	
 
		private function setupBitmap():void
		{
			bmd = new BitmapData(640, 480);
			bmd.draw(vid);
			raster = new FLARRgbRaster_BitmapData(bmd);
			detector = new FLARSingleMarkerDetector(fparams, mpattern, 80);
		}	
 
		private function setupPV3D():void
		{
			scene = new Scene3D();
			camera = new FLARCamera3D(fparams);
			container = new FLARBaseNode();
			scene.addChild(container);
 
 
			//  Create Earth
			var material:BitmapFileMaterial = new BitmapFileMaterial("assets/earth.jpg");
			material.precise = true;
 
			earthSphere = new Sphere(material,50,24,16);
			earthSphere.rotationX = 90;
			earthSphere.z = 100;
			container.addChild(earthSphere);
 
 
			// Create Satellite
			var materiaSat:BitmapFileMaterial = new BitmapFileMaterial("assets/mars.jpg");
			materiaSat.precise = true;
 
			satellite = new Sphere(materiaSat,15,24,16);
			satellite.z = 80;
			satellite.x = 100;
			container.addChild(satellite);
 
 
			// Hide container 
			container.visible = false;
 
			// Add ViewPort
			vp = new Viewport3D();
			addChild(vp);
 
			// PV3D Render Engine
			bre = new BasicRenderEngine();
 
			// FLAIR Trans
			trans = new FLARTransMatResult();
		}	
 
 
 
 
 
		private function loop(e:Event):void
		{
 
			// Move Satellitte around Earth
			satellite.x= centerX + Math.sin(angle) * radius;
                        satellite.z= centerY + Math.cos(angle) * radius;
 
			bmd.draw(vid);
 
			try
			{
				// Detect if the marker is displayed on the camera
				if(detector.detectMarkerLite(raster, 80) && detector.getConfidence() > 0.5)
				{
					// Earth rotation
					earthSphere.localRotationY +=3;
 
					// Satellite Angle
					angle +=speed
 
					detector.getTransformMatrix(trans);
					container.setTransformMatrix(trans);
 
					// Show Container (only first time) 
					if (!container.visible) 
						container.visible = true;
 
 
				} else {
					// Pattern not found (i.e. out of stage)
				}
			}
			catch(e:Error){}
 
			// Render scene
			bre.renderScene(scene, camera, vp);
		}
	}
}

Let’s check the main differences with gotoAndLearn tutorial.
First of all we create two spheres, defining their materials and positions.

// Earth
var material:BitmapFileMaterial = new BitmapFileMaterial("assets/earth.jpg");
material.precise = true;
earthSphere = new Sphere(material,50,24,16);
earthSphere.rotationX = 90;
earthSphere.z = 100;
container.addChild(earthSphere);
 
// Satellite
var materiaSat:BitmapFileMaterial = new BitmapFileMaterial("assets/mars.jpg");
materiaSat.precise = true;
satellite = new Sphere(materiaSat,15,24,16);
satellite.z = 80;
satellite.x = 100;
container.addChild(satellite);

We’ll change the satellite position(the little sphere) so it will be moved around the Terra (the big sphere), initializing some variables and using a simple formula inside the loop to create an ellipse.

private var angle:Number = 0;
private var centerX:Number = 0;
private var centerY:Number = 100;
private var radius:Number = 100;
private var speed:Number = .1
 
private function loop(e:Event):void
{
// Move Satellitte around Earth
satellite.x= centerX + Math.sin(angle) * radius;
satellite.z= centerY + Math.cos(angle) * radius;
…

However, often happens that webcam doesn’t recognize our marker (for bad lights or a wrong marker position in front of your webcam), generating some position sphere issues when it will be identified again.
To avoid this problem we’ll move the sphere only when the marker is really recognized.

if(detector.detectMarkerLite(raster, 80) && detector.getConfidence() > 0.5)
{
	// Earth rotation
	earthSphere.localRotationY +=3;
 
	// Satellite Angle
	angle +=speed	
	detector.getTransformMatrix(trans);
	container.setTransformMatrix(trans);		
}

Download Source Code
(In the zip you will find a txt file with guidelines to configure your project)

3 Comments »

Andrea Collet: Introduction to Augmented Reality and Papervision 3D >> http://is.gd/44Q70 | Augmented Reality said:

[...] Collet Introduction to Augmented Reality and Papervision 3D >> http://www.fabiobiondi.com/blog... 12 minutes ago from Twitter - Comment - Like [...]

# 8 October 2009 at 14:57

Michael said:

The code you gave do not work at all. It neither open the cam. Do you have any explanation? I get all the last version of the sw.
Did you tried? Could you descrive better the organization of the library and especially why do not open the cam?
thk you a lot upfront for your help.

# 12 October 2009 at 22:16

Fabio Biondi (author) said:

Hi Michael,
the file organization and the libraries i have used are explained in detail on the gotoAndlearn tutorial. Please check it.

I only created a variant but the logic is the same

# 30 November 2009 at 15:51

Augmented Reality using Papervision3D and Flex

3 Comments »

Leave your response!

Archives

Categories

Blogs

Recent Posts

Most Commented

Recent Comments