September 16, 2014

Dust

Creating Ephemeral Gestures On iOS8 #

Last week, Apple announced the Watch and one of the features I was really interested in was the gesture based communication tool which lets people sketch an ephemeral message to other Watch users.

I decided on Friday that I wanted that feature now, so I wrote it and here is the breakdown of what I’m calling Dust. Dust uses Apple’s Multipeer Connectivity framework, but could build this on any realtime peer to peer networking system including Firebase or ZeroMQ. I chose Multipeer Connectivity so I could run the app offline only using a Bluetooth connection between peers.

Core Animation’s Particle Emitter and the Multipeer Connectivity Framework are the two major components that make Dust work. The first component I’m going to talk about is the gesture based particle emitter. First, create two gestures and add the gestures so the view recognizes them.

// Pan Gesture
let panGesture = UIPanGestureRecognizer(target: self, action: "handlePanGesture:")
panGesture.minimumNumberOfTouches = 1
panGesture.delegate = self

// Double Tap Gesture
let tapGesture = UITapGestureRecognizer(target: self, action: "handleTapGesture:")
tapGesture.numberOfTouchesRequired = 1
tapGesture.numberOfTapsRequired = 2
tapGesture.delegate = self

view.addGestureRecognizer(panGesture)
view.addGestureRecognizer(tapGesture)

Next, add some boilerplate code to advertise our Dust app to peers and also set up a browser to find peers nearby. The Multipeer Connectivity Framework has an easy way to start advertising and discover other nearby users without doing much work.

// Create the session that peers will be invited/join into.
session = MCSession(peer: self.localPeerID, securityIdentity: nil, encryptionPreference: .None)
session.delegate = self

// Apples Predefined View Controller To Find Peers
browserViewController = MCBrowserViewController(serviceType: dustServiceType, session: session)
browserViewController.delegate = self

// Apples Predefined Class To Advertise Yourself To Nearby Peers    
advertiserAssistant = MCAdvertiserAssistant(serviceType: dustServiceType, discoveryInfo: nil, session: session)
advertiserAssistant.delegate = self
advertiserAssistant.start()

Now that the gestures and networking are setup, the next step is to create particles. I used an app called Particle X which is $1.99 in the AppStore because it gives a great visual representation of what’s happening as you play with the Core Animation’s Particle Emitter settings. Here is the code for the particle emitter.

var newEmitterLayer = CAEmitterLayer()
newEmitterLayer.emitterPosition = CGPointMake(0,0)
newEmitterLayer.emitterSize = CGSizeMake(60.0,1)
newEmitterLayer.emitterMode = kCAEmitterLayerOutline
newEmitterLayer.emitterShape = kCAEmitterLayerPoint
newEmitterLayer.renderMode = kCAEmitterLayerAdditive
newEmitterLayer.shadowOpacity = 0.0
newEmitterLayer.shadowRadius = 0.0
newEmitterLayer.shadowOffset = CGSizeMake(0,0)
newEmitterLayer.shadowColor = UIColor.whiteColor().CGColor

var newEmitterCell = CAEmitterCell()
newEmitterCell.name = "dustCell"
newEmitterCell.birthRate = 1000
newEmitterCell.lifetime = 6.0
newEmitterCell.lifetimeRange = 0.5
newEmitterCell.color = UIColor.redColor().CGColor
newEmitterCell.redSpeed = 0.000
newEmitterCell.greenSpeed = 0.000
newEmitterCell.blueSpeed = 0.000
newEmitterCell.alphaSpeed = 0.000
newEmitterCell.redRange = 0.581
newEmitterCell.greenRange = 0.000
newEmitterCell.blueRange = 0.000
newEmitterCell.alphaRange = 0.000
newEmitterCell.contents = UIImage(named: "particle").CGImage
newEmitterCell.emissionRange = CGFloat(2.000*M_PI)
newEmitterCell.emissionLatitude = CGFloat(0.000*M_PI)
newEmitterCell.emissionLongitude = CGFloat(0.000*M_PI)
newEmitterCell.velocity = 1
newEmitterCell.velocityRange = 1
newEmitterCell.xAcceleration = 0
newEmitterCell.yAcceleration = 0
newEmitterCell.spin = CGFloat(0.0*M_PI)
newEmitterCell.spinRange = CGFloat(0.01*M_PI)
newEmitterCell.scale = 3.0/UIScreen.mainScreen().scale
newEmitterCell.scaleSpeed = 0.0
newEmitterCell.scaleRange = 5.0

Once the particle emitter is created, its location needs to be bound to the coordinates of a pan gesture handled by the handlePanGesture: action. After the particle emitter is bound, the particle trail will follow a finger, stay on screen for a few seconds, and then disappear. This is what the particle emitter looks like when it’s following your gestures.

The second component required to bring Dust to life is peer to peer communication. With Apple’s Multipeer Connectivity framework, you have three options: Messages, Streams, and Resources. Dust uses Streams — an open channel of information used to continuously transfer data like audio, video, or real-time sensor events.

Streams are tricky because instead of having a predefined message format, you’re creating your own format that can be an arbitrary byte size. Since Dust is sending real-time gesture data over bluetooth or wifi, I wanted to make the data packets as small as possible. I was able to get the messages down to 10 bytes (2 floats and 2 enums).

x: Float - x position percentage of the screen where peer touches
y: Float - y position percentage of the screen where peer touches
type: Enum - .Pan for pan gesture and .Tap for double tap gesture
color: Enum - Red, Orange, Yellow, Green, Blue, or Violet

Once MCSessionState is Connected between two devices, initialize and open the output stream.

outputStream = session.startStreamWithName("dust-stream", toPeer: peerID, error: nil)
outputStream.delegate = self;
outputStream.scheduleInRunLoop(NSRunLoop.mainRunLoop(), forMode: NSDefaultRunLoopMode)
outputStream.open()

We also have to create and open our input stream, which we handle when the MCSessionDelegate calls the didReceiveStream delegate method.

stream.delegate = self;
stream.scheduleInRunLoop(NSRunLoop.mainRunLoop(), forMode: NSDefaultRunLoopMode)
stream.open()

After the connections streams are set up, we can send and receive data. The next code block builds a message and sends it to your peer via the outputStream.

var sendData = NSMutableData()
var percentX = Float(point.x/view.bounds.width)
var percentY = Float(point.y/view.bounds.height)
var type = gestureType
var color: DustColors = DustColors.fromRaw(colorSegmentControl.selectedSegmentIndex)!

sendData.appendBytes(&percentX, length: sizeof(Float))
sendData.appendBytes(&percentY, length: sizeof(Float))
sendData.appendBytes(&type, length: sizeof(GestureType))
sendData.appendBytes(&color, length: sizeof(DustColors))

if session.connectedPeers.count == 0 { return }
writeBytesToAllOutputStreams(sendData.copy() as NSData)

The recipient pair gets gets the custom message via the inputStream and extracts the data in the same order it was written in.

var inputStream = aStream as NSInputStream

var buffer = [UInt8](count: 10, repeatedValue: 0)
let result: Int = inputStream.read(&buffer, maxLength: buffer.count)

switch result {
case 10:
var data = NSData(bytes: buffer, length: result)
var (percentX, percentY, type, color) = (Float(), Float(), GestureType.Pan, DustColors.Red)
data.getBytes(&percentX, range: NSMakeRange(0, 4))
data.getBytes(&percentY, range: NSMakeRange(4, 4))
data.getBytes(&type, range: NSMakeRange(8, 1))
data.getBytes(&color, range: NSMakeRange(9, 1))

let x = CGFloat(percentX * Float(view.bounds.width))
let y = CGFloat(percentY * Float(view.bounds.height))
...

Once all of the delegates are setup correctly, this is what Dust looks like. I’ve done the demo with 2 iOS devices but it can run on up to 8 devices concurrently.

.Source - For Developers

The iOS Simulator is not GPU accelerated so for the best performance, run this code on a device

AppStore - For All To Play With (Requires iOS8)

Kudos

Dust

Creating Ephemeral Gestures On iOS8 #

Now read this

Contextual iOS Software