Upload Camera Images To Firebase Using Ionic Framework

phonegap-spain-ionic-framework-angular-js.jpg
In my opinion, Firebase and Ionic Framework are two of the coolest things around at the moment when it comes to mobile app development with cloud service integration.

What if we wanted to create an image service like Instagram, Google Picasa, or Imgur where users could take pictures and save them to the cloud for later?

In this particular tutorial we are going to see how to take pictures on a mobile device and bind them immediately to Firebase to be seen everywhere we are signed in. In case you didn’t know, images can be saved to Firebase instead of only text.

Creating a New Ionic Framework Project

The Prerequisites

We’re going to need a few more things than my other Firebase tutorial if you’ve been keeping up with my posts. The list of our requirements is as follows:

Creating Our Project

To start things off, we’re going to create a fresh Ionic project using our Terminal (Mac and Linux) or Command Prompt (Windows):

ionic start ImageApp blank
cd ImageApp
ionic platform add android
ionic platform add ios

Note, if you’re not using a Mac with Xcode installed, you cannot add and build for the iOS platform.

At this point we’re left with a new Android and iOS project using the Ionic Frameworkblank template.

Adding Our Plugins

This project will be using one external plugin that is supported by Apache Cordova. The camera plugin will allow us to access the device or simulator camera for taking pictures.

With your Ionic project as the current working directory in your Terminal or Command Prompt, run the following:

cordova plugin add org.apache.cordova.camera

This will install the plugin for both Android and iOS. For more information on seeing this plugin in action, you can visit one of my previous tutorials on the topic.

Adding Our Libraries

In our Ionic project’s www/index.html we need to add all the various prerequisite libraries. The file’s should look something like the following:

<head>
    <meta charset="utf-8">
    <meta name="viewport" content="initial-scale=1, maximum-scale=1, user-scalable=no, width=device-width">
    <title></title>
    <link href="lib/ionic/css/ionic.css" rel="stylesheet">
    <link href="css/style.css" rel="stylesheet">
    <script src="lib/ionic/js/ionic.bundle.js"></script>
    <script src="js/ng-cordova.min.js"></script>
    <script src="cordova.js"></script>
    <script src="js/firebase.js"></script>
    <script src="js/angularfire.min.js"></script>
    <script src="js/app.js"></script>
</head>

It is very important to note that ng-cordova.min.js must be placed abovecordova.js. If you don’t do this, you’re going to get strange results.

Preparing Our JavaScript File

For simplicity, we are going to be doing all custom JavaScript coding in ourwww/js/app.js file. It will contain all our controllers, factories, and prototypes.

Naming Our AngularJS Module

For cleanliness, we are going to give our AngularJS module a name.

var imageApp = angular.module("starter", ["ionic", "ngCordova"]);

In this case we’ve named our module imageApp and it will be used for all controllers and factories in our application. Also take note that we’ve added ngCordova to the list.

Getting Firebase Started

We’re not quite ready to talk about Firebase, but we are at a point where it would be a good idea to initialize it. At the top of your www/js/app.js file, outside any AngularJS code, you want to add the following line:

var fb = new Firebase("https://INSTANCE_ID_HERE.firebaseio.com/");

Of course you need to replace INSTANCE_ID_HERE with your personal Firebase instance. By setting it globally, it will load before AngularJS and can be used throughout your application.

We also need to add firebase to our AngularJS module. In the end it will look something like this:

var imageApp = angular.module("starter", ["ionic", "ngCordova", "firebase"]);

Using The AngularJS UI-Router

Our application will have the following two screens for simplicity:

  • A screen for registering or signing into Firebase
  • A screen for showing uploaded pictures and allowing users to upload new pictures

To accomplish different screens or views in an AngularJS application we will be using the UI-Router since it is already bundled with Ionic Framework. If you’re familiar with my other tutorials, you might have already seen a previous demonstration that I’ve done.

Configuring Our States

With the AngularJS UI-Router, each view or screen is considered a state. In order to use we must configure the routes and the controllers associated with them. In your Ionic project’s www/js/app.js file, let’s add the following chunk of code which will represent the states for each of our screens:

imageApp.config(function($stateProvider, $urlRouterProvider) {
    $stateProvider
        .state("firebase", {
            url: "/firebase",
            templateUrl: "templates/firebase.html",
            controller: "FirebaseController",
            cache: false
        })
        .state("secure", {
            url: "/secure",
            templateUrl: "templates/secure.html",
            controller: "SecureController"
        });
    $urlRouterProvider.otherwise('/firebase');
});

For simplicity, every time we open our app, we will be sent to the Firebase view forcing us to sign in. If you want to get creative, you can check the authorization state and plan appropriately.

Creating Our Views

We’ve just configured all of the routing information for our states so it is now time to set up our view templates. Essentially these are the HTML pages that will represent each of our screens.

The first step is to prepare our Ionic project’s www/index.html file to use UI states. This is easy and can be accomplished in just a few lines. Navigate to the <ion-pane>lines and replace them with the following:

<ion-pane>
    <ion-nav-bar class="bar-stable"></ion-nav-bar>
    <ion-nav-view></ion-nav-view>
</ion-pane>

Time to go through the process of designing each of our views. Brace yourself. Our templates are going to be very simplistic. Feel free to be fancier in your view design.

Create and open the file called www/templates/firebase.html and add the following code:

<ion-view title="Firebase Login">
    <ion-content>
        <div>
            <div class="list list-inset">
                <label class="item item-input">
                    <input ng-model="username" type="text" placeholder="Username" />
                </label>
                <label class="item item-input">
                    <input ng-model="password" type="password" placeholder="Password" />
                </label>
            </div>
            <div class="padding-left padding-right">
                <div class="button-bar">
                    <a class="button" ng-click="login(username, password)">Login</a>
                    <a class="button" ng-click="register(username, password)">Register</a>
                </div>
            </div>
        </div>
    </ion-content>
</ion-view>

Now create and open the file called www/templates/secure.html and add the following code:

<ion-view title="My Images" ng-init="">
    <ion-nav-buttons side="right">
        <button class="button button-icon icon ion-camera" ng-click="upload()"></button>
    </ion-nav-buttons>
    <ion-content>
        <div class="row" ng-repeat="image in images" ng-if="$index % 4 === 0">
            <div class="col col-25" ng-if="$index < images.length">
                <img ng-src="data:image/jpeg;base64,{{images[$index].image}}" width="100%" />
            </div>
            <div class="col col-25" ng-if="$index + 1 < images.length">
                <img ng-src="data:image/jpeg;base64,{{images[$index + 1].image}}" width="100%" />
            </div>
            <div class="col col-25" ng-if="$index + 2 < images.length">
                <img ng-src="data:image/jpeg;base64,{{images[$index + 2].image}}" width="100%" />
            </div>
            <div class="col col-25" ng-if="$index + 3 < images.length">
                <img ng-src="data:image/jpeg;base64,{{images[$index + 3].image}}" width="100%" />
            </div>
        </div>
    </ion-content>
</ion-view>

At first glance the code we have in our www/templates/secure.html file may look nuts. It really isn’t. It is a grid system that is better explained in a previous article that I wrote.

Configuring Our Firebase Instance

Defining Permissions

Like my previous Firebase tutorials regarding building a Firebase Todo List or Creating a Password Manager, we’re going to be using the same permission strategy:

{
    "rules": {
        "users": {
            ".write": true,
            "$uid": {
                ".read": "auth != null && auth.uid == $uid"
            }
        }
    }
}

Everyone will be able to write to the users node (create a new account), but only authorized users will be able to read data. In this case data will be the users own images. You can paste the JSON rules in the Security & Rules section of the Firebase dashboard.

Allowing For Account Creation

In the Login & Auth section of the Firebase dashboard, we must enable Email & Password authentication. There are other types of authentication, but for this particular tutorial we’re going to focus on email and password based.

Enabling this will allow people to register new accounts and create a uniquesimplelogin:x user key in our NoSQL data structure where x is an auto incrementing numeric value.

Understanding The Data Structure

The data stored will be of very strict formatting. It will of course be JSON, but it will look like the following:

{
    "images": {
        "unique_image_id_here": {
            "image": ""
        }
    }
}

When storing our images, Firebase will automatically create our unique image ids. You’ll see that shortly.

Making Controllers To Pair With Our View Logic

The Firebase Controller

The logic behind the FirebaseController is as follows:

  • If the user chooses to sign in, check the email and password and redirect to the secure page if successful
  • If the user chooses to register, create a new account and then immediately sign into and redirect to the secure page
imageApp.controller("FirebaseController", function($scope, $state, $firebaseAuth) {

    var fbAuth = $firebaseAuth(fb);

    $scope.login = function(username, password) {
        fbAuth.$authWithPassword({
            email: username,
            password: password
        }).then(function(authData) {
            $state.go("secure");
        }).catch(function(error) {
            console.error("ERROR: " + error);
        });
    }

    $scope.register = function(username, password) {
        fbAuth.$createUser({email: username, password: password}).then(function(userData) {
            return fbAuth.$authWithPassword({
                email: username,
                password: password
            });
        }).then(function(authData) {
            $state.go("secure");
        }).catch(function(error) {
            console.error("ERROR: " + error);
        });
    }

});

Firebase stores sign in information on the device so that you can later make use of the getAuth() functions to see if you’re currently authenticated.

The Secure Controller

The logic behind the SecureController can be a little more complicated and is as follows:

  • Clear the history stack so the back button doesn’t bring us to the sign in screen again
  • Create an empty array of images just in case no images exist on Firebase. Don’t want any undefined errors
  • Check to see if the user is signed in
    • If not signed in, redirect back to the Firebase view
    • If signed in, pull down the array of images from the server
  • If the user chooses to upload an image, open the camera
    • If the user accepts the camera image, then immediately push it to Firebase
imageApp.controller("SecureController", function($scope, $ionicHistory, $firebaseArray, $cordovaCamera) {

    $ionicHistory.clearHistory();

    $scope.images = [];

    var fbAuth = fb.getAuth();
    if(fbAuth) {
        var userReference = fb.child("users/" + fbAuth.uid);
        var syncArray = $firebaseArray(userReference.child("images"));
        $scope.images = syncArray;
    } else {
        $state.go("firebase");
    }

    $scope.upload = function() {
        var options = {
            quality : 75,
            destinationType : Camera.DestinationType.DATA_URL,
            sourceType : Camera.PictureSourceType.CAMERA,
            allowEdit : true,
            encodingType: Camera.EncodingType.JPEG,
            popoverOptions: CameraPopoverOptions,
            targetWidth: 500,
            targetHeight: 500,
            saveToPhotoAlbum: false
        };
        $cordovaCamera.getPicture(options).then(function(imageData) {
            syncArray.$add({image: imageData}).then(function() {
                alert("Image has been uploaded");
            });
        }, function(error) {
            console.error(error);
        });
    }

});

Notice the Camera.DestinationType.DATA_URL. This means that we are going to store the image as base 64 data in Firebase.

Properly Testing Our App

Using the Web Browser

Don’t do it. This app makes use of native device plugins that are loosely compatible with the web browser. You’re setting yourself up for failure if you even attempt to open this application in a web browser.

Using a Device or Simulator

To test this application on your device or simulator, run the following in your Command Prompt or Terminal:

ionic build android
adb install -r platforms/android/ant-build/CordovaApp-debug.apk

The above will get you going on Android. If you wish to track down errors and view the logs, you can use the Android Debug Bridge (ADB) like so:

adb logcat

Conclusion

You’ve just made your own little photo repository. Is it the next Instagram? Not in its current state, but you can definitely expand upon the material described in this article.

We took the base 64 image data obtained from the device camera and immediately pushed it into our Firebase storage. This storage can be synchronized between all of your devices and platforms.


Written by Nic Raboy

Did you find this article helpful? Don’t forget to drop your feedback in the comments section below. Suggest for you:

☞ iOS Mobile App Development: Create your own Soundboard App

☞ iOS 10 Projects: Build Amazing Apps with Apple’s Newest iOS

☞ Develop iOS Game with Flappy Bird Course

☞ iOS 10 for Beginners: From Noob to Paid Pro

☞ iOS 10 & Swift 3: From Beginner to Paid Professional

13 Best iOS Apps to Create and Edit GIFs

y5.png

GIF animations are small compressed files of looping fun that are more often than not, used for memes. While you can create GIFs straight from a browser (here’s how you do it on Chrome) or from web apps like these, these days weprefer to have tools that can make or edit GIFS of our personal photos or videos.

If you are part of this group, you’re in luck, cause we have the iOS mobile apps to help you make, edit and share GIFs right from your iPhone.

These are the apps that can help you create GIFs easily and quickly — either from live shoots or photos/videos already in your gallery — as well as share them with your friends and family via social networks, messaging apps or via URL.

GifsArt

This is a powerful and comprehensive animated GIF generator made by Apple Inc. This application helps you capture images and videos with the in-app camera then lets you combine image, video, and GIF into one perfect animation. Without leaving the application, you can customize your GIF by using animated masks, effects, stickers and text.

Animated masks and stickers are imported through Giply. Users can also export different filters, effects, text and captions to their PicArt Gallery. One very good thing about this application is that users will not bothered by a watermark.

HipGif

With HipGif, you can choose many kinds of animated stickers to add to your GIF file. These stickers and popular GIFs can be mixed with your own, and the pile is updated multiple times a day. Speed up, slow down, draw on your GIFs, add a filter, text or frame to any of your GIF and share them on Instagram as a video.

You can snap multiple photos, or use any of the meme background templates available. The app is also great for making photo slideshows but best of all you can share easily to Facebook, Messenger, Twitter, WhatsApp and even via SMS.

Gifo

Share animated GIFS, memes, reactions through Gifo with your iPhone or iPad camera. It makes it easy for you to make stunning animated collages, to speed up or slow down GIFs as what you want. Furthermore, you can make 4 GIFs in one go. It also comes with cool filters, frames, colorful text and fonts that you can add to your GIFs.

Your Gifo results can be copied without saving it first into the camera roll. Justdirectly paste it into any application you want to share it to. If you do want to save it into the camera roll, you can, in GIF form or as a video file.

Lively

If you’re looking for a quick and easy way to trim, play backwards, have speed control and export your captured videos to GIF format, Lively can do all that and more. The catch is your exported GIFs will carry their watermark, which you can remove with a one-time purchase.

Lively works with an iPhone 6S/6S Plus. The app comes with 3D Touch support. Its extensive editing features not only gives you full control of what to show and not to show others, but also lets you shrink the size of your GIF files,pick a single frame from a full video, and share to iMessage, Facebook Messenger, Twitter, Slack, Tumblr and many more.

Slowmographer

Slowmographer is aimed to mix the best feature of mobile photography and video making. Instead of a photo or regular video, Slowmographer helps you create 3-second loops of slowed down videos. It doesn’t seem like a cool thing to shout about until you figure out its use in capturing tricks in and more.sports like mountain biking, skateboarding, surfing, skiing

Not only is the video-taking easy, with this app you can also add image filters, transition effects, remove flickering effect which are a pain in slow motion shots, and easily share them on your favorite social network sites. Here are some examples on Instagram.

Kanvas Lab

Kanvas Lab lets you live life creatively, by allowing you to send GIF loops of your otherwise muundane everyday life. Apart from sending a GIF file, you can alsosend a message with a GIF looping in the background of your message. I’m sure you will be able to find ways to express yourself with these two combinations.

Animated GIFs are recorded in 6 frames; if you need more, take a video message for up to 15 seconds instead. You can speed up or slow down the video message as you like. The app also has custom stickers, an artsy toolkit for additional sprucing up, and flipbook photo album that can be easily shared to your favorite social networks.

Video to GIF

Everything you can do with GIF, you can do with this app. Shoot in GIF, convertyour own videos or YouTube videos to GIF, turn photos into GIF and add animated text or effects to it, all with this handy GIF-making tool.

The app also lets you add animated text, save GIF as a video or share it to your favorite social networks. You can also get a URL for your GIF, to share with others. If you need a quick and easy way to make GIFs, you found it.

Vhoto

Vhoto will record your favorite moments on video and present them to you as a GIF. It will find the best highlight in your video before turning it into an awesome GIF. You can choose to record directly or choose an existing video to convert. For the best GIF loop result, trim your videos to find the best shot.

After make your GIF, you can share to your friend on social media or send it within a text message. This free app overall gives you easiest way to convert video into GIF in a jiffy.

ImgPlay

ImgPlay brings to you who love to shoot in diferent modes – burst photos, iOS Live Photo or normal mode – as this app can turn your shots into GIF. You can edit your GIF before publishing it, by adding captions, editing frame sector/order, control speed/direction (forward, in reverse), and loop count.

us-ipad-2-imgplay-pro-gif-maker.jpeg

In the end you can export your creation into several GIF quality levels, or you can also export it into video to share easily on multiple social networks and messaging apps.

GifLab

GifLab allows to to create GIFs on your iOS device with plenty of customization options. The app makes it easy to trim, adjust quality and speed, add text with dozens of nice fontfaces, and beautify images with its range of filters. If you are a huge fans of Instagram, then GifLab can export GIFs into videos that are upload-ready.

The GIFs can also be shared on social network sites, via messaging apps or group chats, and via URL. Previously a free app, now the app costs $1.99 to own.Watermarks and locks on filters and fonts have been removed.

GIF Toaster

GIF Toaster is handy app to convert photos or video into animated GIF. It works for both iPad and iPhone. Just select the photo or video you want to convert to GIF then set encoding options such as range (which part of the video), play back speeds, number of frames per second, size of video then click Start Encoding.

The app can convert GIF to video, photo, live photo or to new GIF. It can also do all that in batch mode (doing multiple at the same time). You can also share the resulting GIF via URL.

Gifx

If you are a creative person, then Gifx will give you space to create customized animated images. You can import photos and videos, and apply more than 200 GIF effects and over 100 masks to personalize your GIF. The files can be saved as a movie file or GIF file. You can share your GIF on popular social media you love later.

There are other features available that are locked down by purchases, including being rid of the watermark, a graphics, pack, an export feature, high resolution feature and more. Check out their tumblr page to see what the app can do.

GIF Maker

GIF Maker allows you to shoot your moments and save them in animated GIF format. You can create GIFs from your video shots from as short as 5 seconds to a maximum of 25 seconds. Alternatively, one chan choose to take a minimum of 10 photos to a maximum of 50 photos to turn into GIFs.

Users can also create GIFS from pre-existing photos or videos in their camera gallery. The app also includes some effects that make your GIF look funny, bizarre or scary plus other cool effects.
Written by Agus

Did you find this article helpful? Don’t forget to drop your feedback in the comments section below. Suggest for you:

☞Complete Beginners Guide to iOS Development – Build 10 Apps

☞How to Make a Freaking iPhone App – iOS 10 and Swift 3

☞The Complete iOS 10 Developer – Build Real Apps with Swift 3

☞Complete Swift 3 training – Get ready for XCode 8 and iOS 10

☞Swift 3 and iOS 10 The Final Course Learn to Code like a Pro

Introduction to Google Cardboard for iOS

googlecardboard.jpg

Many of the biggest tech firms in the world are investing heavily in virtual reality technologies: Facebook spent $2 billion to acquire Oculus Rift; Disney invested $65 million into a VR film company called Jaunt; Microsoft introduced its HoloLens this year and is selling the device to developers for $3000 apiece.

Apple spent $32 million to acquire the engineers from Metaio and $345 million to acquire PrimeSense, two companies of VR innovators, and pre-orders for Sony’s Playstation VR sold out even quicker than they’d expected, proving there’s a strong consumer market for VR.

Of all the exciting new VR technology being introduced, however, Google Cardboard VR has made virtual reality most accessible to the hobbyist. In fact, with just a low-cost Google Cardboard VR headset, a smartphone and your iOS skills, you can go farther than you ever thought possible.

In this intro to Google Cardboard VR tutorial, fellow jetsetters, you’ll embark on a worldwide 360° vacation by clicking through multiple 360° vacation images and pausing/playing your way through a 360° vacation video.

Note: This Google Cardboard VR tutorial assumes you know the basics of iOS and Swift development. If you’re new to iOS development and Swift, check out our “Learn to Code iOS Apps with Swift Tutorial” series first.

Getting Started

Download the starter project and open Vacation 360.xcodeproj in Xcode. In the 360 images folder in the Navigation Bar on the left-hand side of your Xcode console, you’ll see three 360° panorama images you’ll eventually display in the app. In Main.storyboard, you’ll find a VacationViewController containing a few labels, two empty UIViews and some constraints.

The interface may not look like much yet, but you’ll eventually change these two UIViews into Google Cardboard photo and video VR views. Build and run your app; you won’t see much except for the labels describing the behaviors you’ll implement in this Google Cardboard VR tutorial.

Starter app screenshot

Installing the Google Cardboard VR SDK

Before you start coding your Google Cardboard VR app, the first step is to install the Google Cardboard VR SDK with CocoaPods. Or rather, if you’ve never used CocoaPods before, the first step is to install CocoaPods and the second is to install Google Cardboard VR using CocoaPods.

As described in Joshua Green’s great tutorial on How to Use CocoaPods with Swift, install CocoaPods and the Google VR SDK using the following steps.

Open Terminal, then execute the following command:

sudo gem install cocoapods

Enter your computer’s password when requested. To install the Google VR SDK in the project, navigate to theVacation_360 starter project folder by using the cd command:

cd ~/ComputerLocation/Vacation_360

Next, create a Podfile for your project:

pod init

Then open the Vacation_360 folder, open the Podfile with a text editor and replace all of its current text with the following:

target “Vacation 360” do
pod ‘GVRSDK’
end

This tells CocoaPods that you want to include the GVRSDK, i.e. the Google VR SDK, as a dependency for your project. Save and close Podfile. Then in Terminal, in the same directory to which you navigated earlier, enter:

pod install

That’s it! You’ve installed Google’s VR SDK. As the log output states, “Please close any current Xcode sessions and use `Vacation 360.xcworkspace` for this project from now on.”

terminal

As with any app in which you install CocoaPods, you’ll be working from the .xcworkspace file instead of the.xcodeproj from this point forward.

Because the Google VR SDK is an Objective-C framework, you’ll have to use an Objective-C bridging header to access it from your Swift code. Go to File\New\File…, select iOS\Source\Header File and then click Next. Name it Vacation 360-Bridging-Header, select the Vacation 360 folder for the Group, then hit Create.

Create Bridging Header

Next, select the Vacation 360 project in the Project Navigator, then select Vacation 360 under TARGETS. Go toBuild Settings, look for Objective-C Bridging Header under the Swift Compiler – Code Generation section and enter “Vacation 360/Vacation 360-Bridging-Header.h”.

Bridging Header Build Setting - Google Cardboard VR

As you might have guessed, this tells the compiler where to look for your bridging header.

Replace the contents of Vacation 360-Bridging-Header.h with the following:

#import “GVRPanoramaView.h”
#import “GVRWidgetView.h”
#import “GVRVideoView.h”

Now you have access to the three Google Cardboard VR SDK classes you’ll be using in this tutorial.

 

Coding the Groundwork

The Google Cardboard VR SDK has three VR view types: GVRCardboardView, GVRPanoramaView, andGVRVideoView. GVRCardboardView is by far the most powerful of the three, since in VR mode it lets you determine the user’s head and eye positions, layout 3D audio, and dynamically alter the landscape. Unfortunately,GVRCardboardView requires complex OpenGL rendering beyond the scope of this tutorial; but GVRPanoramaView, Google VR’s image viewer, and GVRVideoView, Google VR’s video viewer, are more than adequate vessels for an international adventure.

Before you start diving into the Google VR methods, you’ll have to lay some groundwork. First, you’ll connect and load the Google Cardboard 360° panoramic photo and video views.

Navigate to Main.storyboard and select the transparent UIView directly under the Click through Sindhu Beach, Grand Canyon, & Underwater Photos label. In the Utilities menu on the right hand side of Xcode, navigate to the Identity Inspector, then in the Custom Class section, enter “GVRPanoramaView” in the Class field. Similarly, select the UIView directly below Play/Pause “Living with Elephants” Safari Video by Photos of Africa and enter “GVRVideoView” in the Class field.

Using the Assistant Editor, control-drag from these two views in Interface Builder to the top of theVacationViewController class in VacationViewController.swift and create Outlets. For GVRPanoramaView, use the name imageVRView and for GVRVideoView use videoVRView. Similarly connect the label outlets to the top ofVacationViewController. Name the top label imageLabel and the bottom label videoLabel.

Connecting GVRVideoView outlet - Google Cardboard VR

Next, make the media file URLs accessible by defining them in an enumeration. Directly under the IBOutlets, define enum Media as follows:

enum Media {
static var photoArray = [“sindhu_beach.jpg”, “grand_canyon.jpg”, “underwater.jpg”]
static let videoURL = “https://s3.amazonaws.com/ray.wenderlich/elephant_safari.mp4&#8221;
}

In this enumeration, photoArray holds the file names of the 360° images stored in the bundle and videoUrl holds the URL of the 360 video you’ll display. The photoArray is variable and not constant so you can alter the array to easily cycle through the images later on.

Monoscopic 360° of Sindhu Beach, Indonesia by Eggy Sayoga

Set the initial photo and video by adding the following in viewDidLoad() just below super.viewDidLoad():

imageVRView.loadImage(UIImage(named: Media.photoArray.first!),
ofType: GVRPanoramaImageType.Mono)
videoVRView.loadFromUrl(NSURL(string: Media.videoURL))

You load imageVRView‘s UIImage with Media.photoArray.first! while passing the typeGVRPanoramaImageType.Mono to indicate that the image is monoscopic.

Then you load videoVRView with an NSURL created from the Media.videoURL string.

There are two format types for 360° images and videos: stereoscopic and monoscopic – stereo and mono for short. Although both formats render 360° media by converting a rectangular panorama into a spherical layout, the stereoscopic format adds some depth to the viewing experience by including two slightly offset images or videos, one stacked above the other, that mimic the viewer’s perspective.

In VR mode, even monoscopic media appears stereoscopic. - Google Cardboard VR

Monoscopic media features the image or video from a single point of view. Google Cardboard VR supports both formats, but displays either format as stereoscopic. When viewing either format in fullscreen VR mode, the viewer shows two side-by-side images or videos slightly offset from one another. You can find the ideal specifications for both mono and stereo here.

Stereoscopic 360° of Sindhi Beach, Indonesia by Eggy Sayoga

Build and run the app without your Google Cardboard VR headset to see what these views provide with minimal configuration. You’ll see the 360° image in the top view and, as long as you’re connected to the internet and thus able to load from the web URL, you’ll see the 360° video in the bottom view.

Rotating the device will rotate both embedded views. Tap the info symbol at the bottom left corner of either VR view and you’ll be directed to a Google Cardboard VR help webpage. Tap elsewhere on the VR views, and nothing will happen.

Initial VR View - Google Cardboard VR

In order to enable the fullscreen mode and fullscreen VR mode, you’ll need to enable the corresponding buttons on the GVRPanoramaView and GVRVideoView. Add the following below the line in viewDidLoad() where you loadimageVRView‘s image:

imageVRView.enableCardboardButton = true
imageVRView.enableFullscreenButton = true

Next, add the following below the line where you load videoVRView‘s video:

videoVRView.enableCardboardButton = true
videoVRView.enableFullscreenButton = true

Build and run again; two new buttons should appear on the bottom right of each VR view. Tap the outer right button, i.e. the Fullscreen Button, on either VR view, and the view will resize to fit the full screen. While in full screen, tap the back button on the top left corner to return to the main view.

Next tap the inner right button, i.e. the Cardboard Button, and a view should appear with instructions on how to connect your viewer. To view the media in fullscreen VR mode, insert your device into your Google Cardboard VR headset as instructed such that the top of the device is on the left side of the viewer and the bottom is on the right:

(1) Main embedded view (2) Fullscreen view (3) Google Cardboard instruction view - Google Cardboard VR

Note: Tapping the Switch button at the bottom left corner of the Google Cardboard instructions screen will allow you to sync your app to your current VR headset by capturing the QR code on your viewer.

To get the full immersive VR experience from this tutorial, I suggest you use a Google Cardboard compatible device. You can purchase a headset on the Google Cardboard website for as little as $14.95.

Or you can even make your own! To do this, just sit back and order a pizza. … What? You’re still VR-less 30 minutes later? Oh right – you also have to cut up that box, add some lenses, a magnet, velcro, and a rubber band to MacGyver your way into your worldwide vacation. The do-it-yourself instructions are available at that same Google Cardboard website link towards the bottom of the page.

You can build your own Google Cardboard! - Google Cardboard VR

But if you’re content with viewing a 360° panorama from your device without being fully immersed, this tutorial will still work for you without a viewer.

Making the VR Views Interactive

In Vacation 360, the user can click through images while viewing imageVRView and can play and pause video while viewing videoVRView. The backing classes for each of these inherit from GVRWidgetView, which implements the GVRWidgetViewDelegate protocol. This protocol lets GVRWidgetViewnotify its delegate of various state changes and interactions, which will come in handy while customizing the VR view behaviors.

Directly underneath the closing bracket of the VacationViewController class, add the following:

extension VacationViewController: GVRWidgetViewDelegate {
func widgetView(widgetView: GVRWidgetView!, didLoadContent content: AnyObject!) {
}
 
func widgetView(widgetView: GVRWidgetView!, didFailToLoadContent content: AnyObject!,
withErrorMessage errorMessage: String!) {
}
 
func widgetView(widgetView: GVRWidgetView!, didChangeDisplayMode displayMode: GVRWidgetDisplayMode) {
}
 
func widgetViewDidTap(widgetView: GVRWidgetView!) {
}
}

You’ve implemented the GVRWidgetViewDelegate and all of its methods in an extension. It’s time to fill in each method one by one.

widgetView(_:didLoadContent:) is called when a VR view has loaded its content successfully. What’s that you say? How about using this method to only reveal elements of the view once they have loaded? Sounds like a great idea! :]

First, head back to viewDidLoad() and add the following just below super.viewDidLoad():

imageLabel.hidden = true
imageVRView.hidden = true
videoLabel.hidden = true
videoVRView.hidden = true

This will hide all of the labels and VR views initially, so that they won’t appear until the content loads.

Now, grouped with the other imageVRView code in viewDidLoad(), add the following:

imageVRView.delegate = self

Similarly, add the following with the videoVRView code:

videoVRView.delegate = self

This sets both view’s GVRWidgetViewDelegates to the VacationViewController.

You can now unhide the labels and views once their corresponding content has loaded. Back inwidgetView(_:didLoadContent:), add the following:

if content is UIImage {
imageVRView.hidden = false
imageLabel.hidden = false
} else if content is NSURL {
videoVRView.hidden = false
videoLabel.hidden = false
}

widgetView(_:didLoadContent:) is passed the loaded content object – in this case a UIImage forGVRPanoramaViews and an NSURL for GVRVideoViews. For the UIImage, unhide the imageVRView and its corresponding label. For an NSURL, unhide the videoVRView and its corresponding label.

Build and run; you’ll see that each VR view and label now only appears when its corresponding content has loaded:

The VR views now appear after they load. (1) The image VR view has loaded. (2) The video VR view has also loaded.

Add the following to widgetView(_:didFailToLoadContent:withErrorMessage:):

print(errorMessage)

This is called when there’s an issue loading the content. In that case, you simply print the passed errorMessage.

Next you’ll use widgetView(_:didChangeDisplayMode:) to store the current display mode and selected view, so that tapping on the widgets will trigger the appropriate actions only when a VR view is in fullscreen or VR mode. This method is called each time you switch between embedded, fullscreen, or VR mode and passes the involvedwidgetView.

First, in the main VacationViewController class below the enum Media declaration block, add the following:

var currentView: UIView?
var currentDisplayMode = GVRWidgetDisplayMode.Embedded

currentView will maintain a reference to the view currently displayed in fullscreen or VR mode.currentDisplayMode is used to hold the current display mode. It is initialized to GVRWidgetDisplayMode.Embeddedwhich represents the initial display mode when the VR views are embedded in the VacationViewController.

Then in widgetView(_:didChangeDisplayMode:), add the following:

currentView = widgetView
currentDisplayMode = displayMode

widgetView(_:didChangeDisplayMode:) passes the new displayMode as well as the widgetView where the mode was changed. These are stored in currentDisplayMode and currentView, respectively, for later reference.

Knowing the display mode and displayed view, you can finally start to implement the interactive behaviors while in fullscreen or VR mode. Tapping the magnetic widget button on Cardboard or directly tapping the screen should cycle through the photos in Media.photoArray when the imageVRView is in fullscreen or VR mode. To implement this behavior, add the following to widgetViewDidTap(_:):

// 1
guard currentDisplayMode != GVRWidgetDisplayMode.Embedded else {return}
// 2
if currentView == imageVRView {
Media.photoArray.append(Media.photoArray.removeFirst())
imageVRView?.loadImage(UIImage(named: Media.photoArray.first!),
ofType: GVRPanoramaImageType.Mono)
}
  1. Since you only want to handle taps while in fullscreen or VR mode, a currentDisplayMode of .Embeddedtriggers an early return.
  2. If currentView is imageVRView, advance the Media.photoArray queue by removing the first element of the array and appending it to the end. Since Media.photoArray.removeFirst() returns the removed element, you can remove the first element and append it to the end in a single line of code. Then you loadimageVRView‘s UIImage using the file now in the first index of the photo array.

Build and run your project; open the image VR view into either fullscreen or VR fullscreen, then tap either the device screen or the widget button respectively. You should be able to click through the images.

Clicking through images in fullscreen - Google Cardboard VR

You may notice one minor issue: the other elements of the view show through between each click. To prevent that, hide the elements whenever you’re not viewing the embedded display mode by adding the following to the end of widgetView(_:didChangeDisplayMode:):

if currentView == imageVRView && currentDisplayMode != GVRWidgetDisplayMode.Embedded {
view.hidden = true
} else {
view.hidden = false
}

If the currentView is the imageVRView and the currentDisplayMode isn’t embedded, this means you’re viewing the image view in fullscreen or VR mode. In that case, hide the view controller’s root view; otherwise, unhide it.

Build and run; click through the images again, and the transition between images should be almost seamless:

Clicking through images in fullscreen

Note: Even after setting view.hidden to true, the GVRPanoramaView stays unhidden because it and theGVRVideoView are not subviews of the VacationViewController‘s view when in fullscreen or VR mode.

Congrats on successfully creating a 360 image slideshow!

congrats!

But don’t get too lost in your vacation yet. The interactions are only halfway done!

What? It gets even better??

Video View Interaction

You still have to implement the video view interaction which will let you pause and play the “Living with Elephants” video by Photos of Africa so that each moment of your vacation can last even longer.

Back in the main VacationViewController class, add the following new variable underneath the var currentDisplayMode declaration:

var isPaused = true

There is no property inherent to the GVRVideoView that indicates whether the video is paused or playing, so the variable you’ve just created can serve as a state marker.

Also create a method to set the initial play state behavior dependent on the display mode. UnderneathviewDidLoad() add:

func refreshVideoPlayStatus() {
// 1
if currentView == videoVRView && currentDisplayMode != GVRWidgetDisplayMode.Embedded {
videoVRView?.resume()
isPaused = false
}
// 2
else {
videoVRView?.pause()
isPaused = true
}
}
  1. If videoVRView is in fullscreen or VR mode, calling this method plays the video and appropriately setsisPaused to false.
  2. Otherwise, if the videoVRView‘s display mode is in its embedded state or the imageVRView is in fullscreen, calling this method pauses the video and appropriately sets isPaused to true.

Then within widgetView(_:didLoadContent:)‘s else if block, add:

refreshVideoPlayStatus()

This pauses the video as soon as it loads.

Now in widgetView(_:didChangeDisplayMode:), just before the if block and after setting currentDisplayMode, insert the following:

refreshVideoPlayStatus()

By calling refreshVideoPlayStatus() whenever the display mode changes, the video will play when entering fullscreen or VR mode, and pause otherwise.

Now that you’ve set videoVRView‘s initial display mode behaviors, you can implement a play/pause toggle triggered by user interaction. In widgetViewDidTap(_:), after the if block, add:

else {
if isPaused {
videoVRView?.resume()
} else {
videoVRView?.pause()
}
isPaused = !isPaused
}

You hit this else clause when you tap a videoVRView. If the video is paused, resume playing; if the video is playing, pause it. Then you update isPaused to reflect the new play state.

Build and run your app, open the video into fullscreen or VR mode, then tap the screen or widget button and the video should toggle between play and pause states:

Play/pause while watching the video by Photos of Africa - Google Cardboard VR

Note: There is currently a minor bug in the SDK that causes the video to unpause when the VR mode is selected, but not actually entered. In other words, if you back out of the VR view without actually tilting the device to start playback, the video will unpause.

You don’t want your vacation to stop after a few minutes though, do you? Of course not! So therefore you’ll loop the video to the beginning once it reaches the end.

Add a GVRVideoViewDelegate extension below the final bracket of the GVRWidgetViewDelegate extension to implement the looping behavior:

extension VacationViewController: GVRVideoViewDelegate {
func videoView(videoView: GVRVideoView!, didUpdatePosition position: NSTimeInterval) {
if position >= videoView.duration() {
videoView.seekTo(0)
videoView.resume()
}
}
}

videoView(_:didUpdatePosition:) is called at approximately one-second intervals as the video plays. Once theNSTimeInterval position of the video is greater than or equal to the duration of the video, the video has reached its end. videoView.seekTo(0) then sets the position back to the beginning before resuming the video.

Note: There is no need to set VacationViewController as the GVRVideoViewDelegate because…it’s already set! GVRVideoViewDelegate inherits from GVRWidgetViewDelegate, which you’ve already adopted and set up in viewDidLoad() of VacationViewController.

There you have it! Now you can build, run, sit back and enjoy the worldwide vacation you’ve just created with your bare hands. Isn’t iOS development spectacular?

kick back with cardboard - Google Cardboard VR

Where to Go From Here?

Download the finished project here, and check out Google’s sample projects here.

The Google Cardboard mobile SDK had originally only been available for Android devices until Google graciously opened up its Cardboard SDK to iOS earlier this year. Google VR for iOS is still very new and will evolve over time, so keep an eye on raywenderlich.com for Google Cardboard VR tutorials using the enthralling new features Google will undoubtedly roll out in the near future.
Written by Lyndsey Scott

Did you find this article helpful? Don’t forget to drop your feedback in the comments section below. Suggest for you:

☞ iOS 10 & Swift 3: From Beginner to Paid Professional

☞ Master iOS 9 – 10 – Xcode 8 and Swift 3

☞ Complete Beginners Guide to iOS Development – Build 10 Apps

☞ How to Make a Freaking iPhone App – iOS 10 and Swift 3

☞ The Complete iOS 10 Developer – Build Real Apps with Swift 3

Demystifying iOS 9 Spotlight Search for Developers

There are three search points built into iOS: Siri, Spotlight Search, and Safari search. Siri, one of the most iconic features of iOS, is something that most iPhone users are already familiar with. But many people are not aware of Spotlight search – something that has been an integral part of iOS long before Siri existed. Spotlight Search can be accessed when you slide the home screen downwards, or new to iOS 9, swipe right from your personal home screen. This reveals a search bar at the top of the screen.

The focus of this search tool in iOS 8 and earlier was searching the phone itself, so its results page lists apps that are on your phone, as well as emails, messages, and other private items within the Apple apps. It will also show a Wikipedia definition if it seems relevant. Finally, it offers an option to search the web via Safari as an additional step.

In contrast to Spotlight Search, Safari is all about the world outside your phone. In iOS 8, the user can choose which search engine to use for Safari search: Google, Yahoo, Bing, and DuckDuckGo. Naturally Google handles the majority of Safari search requests. Regardless of the search engine chosen, pre-filled search terms are presented in Safari, potentially with a Wikipedia match.

toptal-blog-image-1449667180627-3ad04eccf6a8c04644b1f7e78aad5174.jpg

In iOS 9, Spotlight Search is much more prominent and more widely scoped than before. In fact, the first thing you see are “Siri Suggestions” based on your usage patterns. For example, if you use Safari regularly in the afternoon, Siri will suggest it for you around that time.

And, while it is still a place to find “things on your phone”, it is becoming a direct portal for “things not on your phone”. Already you see things nearby and news headlines sourced from the web via Apple’s own search engine, and other web-based results are just starting to appear now.

Yes, Apple has created a web search engine for iOS 9, which is a very big step. Say hello to Applebot.

In a similar vein to Spotlight, Apple’s stated aim for Safari is also to surface results and suggestions from Apple’s search index, before offering the ability to search the web with Google (or other chosen provider). As of iOS 9.1, Safari seems to be lagging behind Spotlight search by continuing to serve results from the user’s chosen search provider — perhaps to give Apple a slower roll-out of its search features and more time to fine-tune its algorithms.

What is particularly exciting about the launch of iOS 9 is that for the first time, it gives iOS developers the ability to hook into these search functions and include content from within apps. Furthermore, the promise is that search will ultimately include results not on the user’s phone. This is a huge step.

Let’s just put this into context. Google’s search engine has spawned an entire industry built around “SEO” — the process of optimizing your assets around Google’s search. When fully launched, Apple’s search engine will own roughly 50% of the mobile search market, which is 50% (and growing) of overall search traffic. Applebot is therefore big, which means “Apple Mobile Search Optimization” will also be big.

Apple Mobile Search Optimization efforts will have a much bigger impact on app discovery by mobile users than regular SEO.

Welcome to the newborn AMSO industry.

The bad news is that implementing all these features involves many new technologies from Apple as well as enhancement to old ones, and the combination of these elements can be confusing. The goal of this series is to walk through the various pieces and clarify how they might be implemented. Let’s start with something basic and build on it.

toptal-blog-image-1449667857096-62b7ea5da79535f65c954cfd5d897422.jpg

In support of this series, I have created a simple app for reference throughout. The project can be found here:https://github.com/rwforsythe/iOS9-Search. It will also be updated as this series continues to cover other elements of the search toolkit.

The Foundation: CoreSpotlight Framework

This framework, completely new in iOS 9, allows you to contribute items to the iPhone’s local search index for discovery by the user. For example, before iOS 9, only content within the official Apple apps, such as Calendar, could be found via Spotlight search. Now, any app that publishes its calendar events through the CoreSpotlight framework can be found via Spotlight Search and even Siri.

Note that the focus here is on personal information; you don’t want your personal calendar events being found by users on another phone! Apple is very clear that the CoreSpotlight framework interacts exclusively with the private index on each person’s phone. It is not possible to use CoreSpotlight API to publish content outside the phone’s own index.

That does not mean that CoreSpotlight cannot be used to index already public information. It is certainly possible to use CoreSpotlight to pull information from the cloud and place it into the phone’s search index. It is simply a guarantee from Apple that if personal and confidential information is indexed with CoreSpotlight, it will remain confidential.

The framework comes in two parts: the CSSearchableItemAttributeSet object that allows the app to describe each item in detail, and a CSSearchableItem object that is used to give the item a unique ID. Each Spotlight entry is made up of a pair of these objects. The reason for using two objects instead of one is thatCSSearchableItemAttributeSet objects are used for a different kind of search interaction (described later).

It’s pretty easy to do the basics.

For those referencing the sample code, setting up CoreSpotlight is done in AppDelegate.

…
@import CoreSpotlight;
@import MobileCoreServices;	
…

These two modules are needed for the new functionality.

For simplicity in the demo app, the application:didFinishLaunchingWithOptions: method is used to call a dedicated method for setting up CoreSpotlight:

if ([CSSearchableItemAttributeSet class])
    [self setUpCoreSpotlight];
//Check for iOS version that supports CoreSpotlight API

Moving down to the setUpCoreSpotlight method, we start the process by creating aCSSearchableItemAttributeSet object:

CSSearchableItemAttributeSet * attributeSet = 
    [[CSSearchableItemAttributeSet alloc]
        initWithItemContentType:(NSString *)kUTTypeItem];

The Content Type is important as it is supposed to govern the general way in which the item is treated by the search algorithms. “Supposed” is the operative word, since this is an area that is not well documented by Apple. There is a large menu of item types to pick from (image types, video types, audio types, a contact type), but trial and error is required to determine what impact (if any) it has on the user’s experience. Some types seem to display more text content than others when displayed in search results. It can also impact whether a thumbnail image is displayed along with the result. kUTTypeItem is a great place to start.

Now describe the object in more detail:

attributeSet.displayName = @"A Christmas Carol";

attributeSet.title = @"A Christmas Carol By Charles Dickens";
    //Sounds similar to displayName but is not displayed to user
    
attributeSet.contentDescription = @"Who would dare to say “Bah! Humbug” after reading A Christmas Carol? Charles Dickens wrote the novella in just six weeks before it was first published on December 19 1843 but his morality tale about a bitter old miser named Ebenezer Scrooge lives on to this day as a reminder of the importance of the Christmas spirit.";
    
attributeSet.keywords = @[@"A Christmas Carol", @"Charles Dickens", @"Victorian Literature"];
    
UIImage *image = [UIImage imageNamed:@"CC-Cover"];    
NSData *imageData = [NSData dataWithData:UIImagePNGRepresentation(image)];
attributeSet.thumbnailData = imageData;

There are lots of possible properties, specific to media types like images, videos and so on. More information is probably better, but trial and error is needed to determine what information is actually used and/or displayed in search results. At the time of writing, for example, star-ratings seem not to be displayed for a given index item, even if the attributeSet contains star rating data.

Most text properties are included in the search query, so even though the title property is not displayed, the text can be found via search. It is therefore not entirely clear how the keywords property is fundamentally different from the title property.

See Apple’s doc for more information: developer.apple.com

The final step is to package the CSSearchableItemAttributeSet with CSSearchableItem and register it with the index.

CSSearchableItem *item1 = [[CSSearchableItem alloc] initWithUniqueIdentifier:@”https://www.notestream.com/streams/564159e4e5c24” domainIdentifier:@"notestream.com" attributeSet:attributeSet]; 

Two new things here. The domainIdentifier property allows you to group items together for batch operations. For example, @“meetingItem” and @”reminderItem” in a calendar app would allow you to delete all Spotlight entries of one sort while leaving the other.

The uniqueIdentifier property has a more important role to play. First, it is passed back to your app when the user clicks on an item in the CoreSpotlight index (discussed later). Second, there are recommendations from Apple about what this should look like when using other elements of the search infrastructure. For now, we will use a string that is in fact a URL that uniquely represents this item. Technically this property could be any string, as long as it is unique to the item.

The last step: push the items you’ve created into the actual index.

[[CSSearchableIndex defaultSearchableIndex] indexSearchableItems:@[item1, item2, item3]
    completionHandler: ^(NSError * __nullable error) {
    if (!error)
        NSLog(@"Search item(s) journaled for indexing.");
}]; 

A couple of points to make here:

  1. This method takes an array of objects (and there are other methods that can be used to batch the process if there is a lot of data to index).
  2. It is important to realize that this method does not actually complete the indexing process. The completion handler is called only when your search items have been queued for indexing. Use theCSSearchableIndexDelegate to handle scenarios where the indexing process itself fails for some reason and your app needs to handle the situation.

Congratulations, you’ve added items to the phone’s CoreSpotlight search index!

toptal-blog-image-1449667704152-9e8cc8709020185afc4edf764958ab91.jpg

How Search Works in iOS 9

Naturally, Apple keeps their search algorithms close to their chest, so it is worthwhile to install the sample app and play around with keywords. For example, it quickly becomes apparent that Apple is maximizing the results presented by giving very different results as the user typed each letter:

It looks like the principle is that the results thought more appropriate will be presented after just a couple of letters have been typed. As the user types more letters, those early results are dropped and other things (originally estimated to be less likely) are presented instead.

More generally, Apple has only a few things to say about how to improve ranking under their algorithms. This page contains Apple’s recommendations: Enhance Your Search Results

One of the key take-aways from Apple’s documentation is that ranking is improved when the app uses multiple search technologies to index content. We’ll dive into another of these technologies in the next article. Meanwhile, we still need to implement code to handle the scenario when a user clicks on a search result.

When the User Clicks

We’ve created the code to populate the index with potential search results, but what happens when the user clicks on a result? The answer is that CoreSpotlight borrows the application:continueUserActivity:restorationHandler: method from the UIApplicationDelegate protocol. This was originally introduced in iOS 8 for the handoff mechanism that allows user activities to be passed from device to device (e.g. a user sees a web URL on an iWatch, uses handoff to pick it up on the iPhone Safari browser, and ultimately view it on their Mac when they get to the office.)

The first challenge is to determine if the method was called due to a Handoff activity or from Spotlight Search. The activityType property of the activity parameter will tell you, but if the code might run on iOS 8, you’ll need to avoid a crash in that case.

Here’s how some simple code might look here:

- (BOOL)application:(UIApplication *)application continueUserActivity:(NSUserActivity *)activity restorationHandler:(void (^)(NSArray *))restorationHandler
{
    
    NSString * valueCSSearchableItemActionType;
    BOOL wasHandled = NO;
   
    if ([CSSearchableItemAttributeSet class]) //iOS 9
    {     
        valueCSSearchableItemActionType = CSSearchableItemActionType;
        
    } else { 
        // iOS 8 – This method was introduced in iOS 8, so iOS 7 is not a possible scenario
        valueCSSearchableItemActionType = @"not supported";
    }
        
    if ([activity.activityType isEqual: valueCSSearchableItemActionType])
    {	
        // Invoked via CoreSpotlight, we can assume iOS 9 from now on…

        NSString * activityIdentifier = [activity.userInfo valueForKey:CSSearchableItemActivityIdentifier];

        wasHandled = YES;
        NSLog(@"Continuing user activity %@", activityIdentifier);
        
    } else {
        
        //the app was launched via Handoff protocol
        //or with a Universal Link
    }
    
    return wasHandled;
}

Note the line of code:

NSString * activityIdentifier = [activity.userInfo valueForKey:CSSearchableItemActivityIdentifier];

This extracts the unique identifier that was placed into the spotlight index when the CSSearchableItem object was created. Naturally your app should use this unique identifier to present the user with the piece of content that they selected from the search index. For simplicity here, we just NSLog the unique identifier.

Recap

So far, we’ve examined just one part of iOS 9’s search functionality. We’ve run through the core functionality needed to load the phone’s Spotlight search index with things the user might find useful: documents, calendar items, contacts and so on. When the user searches for something, sees a result from your app and clicks on it, the app will be able to handle the request and display the appropriate item for the user.

What we have not yet discussed is the technologies that relate to the public index Apple is building. This index is intended to allow users to find content in your app even without it being installed on their phone. We’ll tackle that in the next post.
Written by  RICHARD FORSYTHE

Did you find this article helpful? Don’t forget to drop your feedback in the comments section below. Suggest for you:

☞ Complete Beginners Guide to iOS Development – Build 10 Apps

☞ Intro To iOS Backend Development: Image Uploads

☞ Ionic 2 Tutorial with Two Complete Apps

☞ Professional iOS Chat Apps with Social Login using Firebase3

☞  iOS Mobile App Development: Create your own Soundboard App

What’s New in watchOS 3: Background Tasks

With the upcoming watchOS 3 update to all Apple Watch devices, the performance of many watch applications is going to significantly improve. This is mainly due to the new forms of background execution which watchOS apps can take advantage of in order to refresh their content periodically and always have the latest information ready to be viewed by the user.

In this tutorial I am going to show you how you can utilise the new WatchKit APIs in watchOS 3 to have your own application use these new background tasks. It is important to note that, even if your app is set up properly to support these background refreshes, depending on a particular user’s configuration, the system may not let your app run in the background.

This tutorial requires that you are running Xcode 8 on OS X El Capitan or later.

The first important new class in watchOS 3 is the WKRefreshBackgroundTask class. This is an abstract class which you should not subclass or create instances of yourself. It defines only one property,userInfo, which can store extra information relating to the background task. It also defines a single method, setTaskCompleted(), which you must call yourself to tell the system that the task is complete. If you do not call this then the system will spend the maximum amount of time budgeted to your app trying to complete the task, which in turn wastes battery and hurts the performance of other apps.

When your application should handle a background task being completed, the system will call your watchOS extension delegate’s handle(_:) method. This method’s single parameter is a set ofWKRefreshBackgroundTask objects which you need to loop through and process accordingly.

Within the WatchKit framework, Apple provides the following four concrete subclasses; watchOS will pass your extension delegate an instance of the appropriate class depending on the task.

  • WKApplicationRefreshBackgroundTask is a general background task type that is not specialised for a particular refresh type. The most common use case for this type is for scheduling other tasks. For example, you could schedule a background refresh for your app at a specific time of day each day. When the corresponding WKApplicationRefreshBackgroundTask object is then passed into your extension delegate’s handle(_:) method, you could create a background network task to download some data to refresh your app’s content.
  • WKSnapshotRefreshBackgroundTask is a task type specifically for when your app needs to update its snapshot. We will discuss snapshots later on in this tutorial. All you need to know for now is that an app’s snapshot is used as its launch image and can also be shown in the user’s dock (accessible by pressing the side button). Please note that this type of background task has a unique task completion method which we will also discuss later on.
  • WKWatchConnectivityRefreshBackgroundTask is a task type for when you have transferred data from your iPhone to your watch via the WatchConnectivity framework. As this framework provides a method to transfer complication data directly to the watch, this corresponding background task type is mainly used to schedule snapshot updates in response to data being received from the iPhone.
  • WKURLSessionRefreshBackgroundTask is a task type given to your extension delegate when a background networking task has completed. In this scenario, the delegate methods you set up for the URLSessionobject are still called, and it is here where you need to work with the data appropriately. Thehandle(_:) method of your extension delegate is only called so that your application can respond to the network operation being completed. The WKURLSessionRefreshBackgroundTask does not give you access to the URLSession object nor the data that was transferred. The only extra property it defines is a string called sessionIdentifier which you can use to determine what network process has been completed.

There are a number of ways that you can schedule a background refresh in watchOS 3. For a simple application or snapshot refresh respectively, you can use the following two methods on any WKExtensionobject.

When completed, this method will return a WKApplicationRefreshBackgroundTask object to your extension delegate. The watchOS system will attempt to wake your extension in the background at the time specified by the Date parameter.

Please note that it is not guaranteed that your application will be woken in the background at this precise time. Due to other circumstances (such as battery remaining, other apps running and in memory, etc.) the system may decide to wake your app at a later time.

The userInfo parameter will be accessible via any of the background task objects passed to your extension delegate. This can be particularly useful if you need to send information relating only to the current refresh. Lastly, the scheduledCompletion parameter can provide a code block which will be run once your refresh has been scheduled or has failed due to some error.

This method will return a WKSnapshotRefreshBackgroundTask object to your extension delegate. The parameters for this method are identical to when scheduling a regular application refresh. The only difference with this method is the type of refresh that you are scheduling.

The following is an example piece of code which could be used anywhere in your watchOS application to schedule a background refresh:

As mentioned earlier, a WKWatchConnectivityRefreshBackgroundTask is created in response to you sending data to the watch from its paired iPhone via the WatchConnectivity APIs.

The last type of background task, WKURLSessionRefreshBackgroundTask, is scheduled automatically by the system when you create a URLSession object with a background configuration. Doing this from anywhere within your watch app will cause the system to automatically schedule the request for you and call your extension delegate’s handle(_:) method when completed. The following example code contains two functions within an example class. The first function configures and starts a background download task. The second is a delegate method declared in the URLSessionDelegate protocol.

In watchOS 3, app glances have been removed entirely from the Apple Watch and instead have been replaced by the dock. This dock is accessed by pressing the side button on the watch and contains any number of apps which the user can configure:

IMG_0138.jpg

If your app has been placed in a user’s dock then you are responsible for keeping the visible “snapshot” shown in the dock up to date.

Your app is notified when you need to update your snapshot by a WKSnapshotBackgroundRefreshTask being passed into your extension delegate’s handle(_:) method. When you receive this notification, in addition to your WatchKit extension being woken in the background, your app’s root interface controller is also woken in the background and off screen. In this process, the following methods are called in this order:

  1. The root interface controller’s awake(withContext:) method
  2. The root interface controller’s willActivate() method
  3. The extension delegate’s handle(_:) method

At any point in these three methods, you can customise the appearance of your app’s interface to create the snapshot shown in the dock.

The final step of refreshing the snapshot happens in the extension delegate’s handle(_:) method. As with the other background task types, you must tell the system that the task has been completed. For snapshot refreshes, however, this is done by calling the setTaskCompleted(restoredDefaultState:estimatedSnapshotExpiration:userInfo:) method on theWKSnapshotBackgroundRefreshTask object. As soon as you call this method, watchOS captures the snapshot of your app’s current interface and keeps this in memory until the next refresh. To better understand this method, let’s go through all of its parameters:

  • restoredDefaultState is just a boolean value specifying whether or not your snapshot is of the first interface controller of your app. If you have transitioned to another interface then pass in false for this value.
  • estimatedSnapshotExpiration is a Date object which specifies when the system should next try to refresh your app’s snapshot. If you do not need to schedule another refresh immediately then you can pass in the new Date.distantFuture class property.
  • userInfo is a collection of custom data and will be passed into the nextWKSnapshotBackgroundRefreshTask object if you are scheduling another refresh straight away.

The WatchConnectivity framework introduced in watchOS 2 and iOS 9 is also gaining some new functionality with watchOS 3 and iOS 10.

If a user has put your app’s complication on their watch face, then you can transfer data from the phone to the watch up to 50 times in a single day. This is done, as in iOS 9 and watchOS 2, via the WCSessiontransferCurrentComplication(userInfo:) instance method. New in watchOS 3 and iOS 10 is the WCSession remainingComplicationUserInfoTransfers property, which will tell you how many transfers are remaining for that day.

Another new WCSession property added with this year’s platform updates is the hasContentPending property, which is just a boolean value signifying whether or not there is data yet to be transferred between the iPhone and Apple Watch.

Overall, the new APIs in watchOS 3 allow you to quickly and efficiently refresh your Apple Watch app’s content in the background. As more apps adopt these new APIs, the performance of watchOS apps in general will greatly improve and create a much better user experience.

The linked GitHub repo contains a watchOS 3 Xcode project with some example code showing how you can schedule and respond to background refreshes in your own apps.
Written by  Davis Allie

Did you find this article helpful? Don’t forget to drop your feedback in the comments section below. Suggest for you:

☞iOS 10 Projects: Build Amazing Apps with Apple’s Newest iOS

☞iOS 10 Message Extension App Reskinning without Coding

☞Master iOS 9 – 10 – Xcode 8 and Swift 3

☞Swift 3 and iOS 10 The Final Course Learn to Code like a Pro

☞The Complete iOS 10 Developer Course – Build 21 Apps

Apple Watch in a Nutshell: A Feature Introduction for iOS Developers

There’s been a lot of trepidation around the Apple Watch SDK, announced at the WWDC last June and released just recently.

This is just the first round, with limited functionalities. Apple Watch v2 will follow, but not that soon – soforward-thinking iOS developers should be prepared to wait for a few months.

The feeling among developers is extremely diversified. A lot of iOS developers are disappointed by many limitations imposed by Apple Watch SDK and its features. On the other side, many of us are excited, while others are not too far from a neutral position. But, after all, everybody is apparently happy to have a new toy to play with.

One thing that catches the eye, is how Apple Watch users are referred to. They are not users or mobile users, nor watch users or (with a pinch of originality) watchers. They are simply wearers. A term that sounds a bit weird, because if a user uses, implying that he interacts, a wearer wears, and that sounds like a passive usage. Of course, the Apple Watch is far from being a passive device. It combines the traditional features of a watch with more advanced options enabled by modern technology, especially when used in combination with an iPhone. It was not that long ago when phones started to become much more than just phones – history is repeating itself.

However, at the end the wearer name appears to be a smart choice. At least when writing user stories I don’t have to specify “As an Apple watch user I want to …” in full, I can just shorten that to “As a wearer, I want to …” – less ambiguity.

toptal-blog-image-1417522628761.png

Device and UI

Omitting all Apple Watch features not strictly related to development, the most important problem designers and developers have to deal with are sizes – in fact there are 2 different Apple Watch models:

  • 38mm screen with a 272×340 px resolution
  • 42mm screen with a 312×390 px resolution

with a retina display in both cases.

Needless to say, it’s on iOS developers to assure that application looks good on both versions.

Apple Watch Connectivity

The Apple Watch communicates with the companion iPhone via Bluetooth 4.0, and it’s also NFC enabled. Apparently, it has no WiFi connectivity. Although it can tether via the iPhone and use both, WiFi and cellular data connection. But in a press info Apple clearly states that it also features Wi-Fi 802.11b/g.

User Interaction

User interaction (or wearer interaction, if preferred) can occur in 4 different ways:

  • simple touch: a tap on a button is a typical example
  • force touch: activated by applying more force when tapping and holding the screen. Similar to the right mouse click on a PC it will display a contextual menu (containing up to 4 items)
  • gesture: vertical swipe to scroll, horizontal swipe to navigate between pages, left edge swipe to navigate back
  • digital crown: hardware input used for fine tuned scrolling

There are some limitations regarding the user interaction, in particular about gestures:

  • they are predetermined, so custom gestures are not supported
  • multi touch is not supported, so no pinch to zoom or similar gestures requiring more than one finger

Here is the sample of contextual menu from the Apple WatchKit Guide.

toptal-blog-image-1437603365862-0ffd048c82c4a6ee2017bf6150e03e1c.png

User Interface

Speaking of the Apple Watch UI, there are several limitations imposed by the reduced screen size, by design choices, and in many cases as a natural consequence of the underlying hardware. We expect that some of them will change for the better in future releases of the SDK, although it’s not possible to anticipate nor forecast what will actually change.

The first limitation is about how user interface is implemented: via storyboards only.

Apple Watch Navigation

WatchKit offers two mutually exclusive ways of navigating among the views of an Apple Watch app:

  • a hierarchical navigation model, similar to the one used in iOS, with views stacked up during navigation
  • a flat page based model, where the user can navigate between pages by swiping left and right

Unfortunately, combining these two navigation models in the same app is not possible. One of them must be chosen and used throughout the whole app.

However, any interface controller can present a modal interface controller, which can either be a single interface controller, or multiple interface controllers, arranged using a page navigation model.

UI Elements

There are 11 distinct interface components:

  • button
  • slider
  • image
  • label
  • separator
  • slider
  • switch
  • table
  • date: automatically displaying the current date or time
  • timer: displaying a countdown timer, automatically updated
  • group: a container for one or more child components
  • map: displaying a non interactive map, optionally with max 5 annotations

Layouts

The group component can be used to combine Apple Watch UI elements in either a vertical or horizontallayout. Elements will be automatically laid out depending on their size. Some degree of flexibility is available here, given that groups can be nested.

Unfortunately, the auto layout is not available. But, considering that the user interface is much simpler compared to the iOS counterpart, it’s probably not a big loss.

For lists, the table component creates and manages content using a single column model, just like the UITableView. However, it is not using the delegation pattern – the data source is directly provided to the table by specifying the number of rows first, and then initializing the content of each row in a for loop. On a positive note, it’s still possible to use custom table rows (i.e. cells in the iOS jargon)

Glances

Glances are read-only, single page, and non scrollable views, used for showing a snapshot of a timely and relevant app-specific data. No user interaction is allowed, except for the tap event. Tap on the glance will simply launch the app, optionally providing a context to let the app display a specific interface controller.

For example, an Apple Watch weather app can use Glance to display current temperature, location etc. By tapping it, the app opens, showing more detailed information.

Here is the sample Glance from the official Apple WatchKit Guide.

glance_lister_2x.png

To read more about Glances please check the WatchKit Programming Guide.

Apple Watch Notifications

There are 2 notification types: short look and long look.

The short look notification is used to alert the wearer that something has happened, and it is automatically dismissed when the wrist is lowered. The layout is very simple, and consists of the app icon, the app name, and the notification title.

toptal-blog-image-1437603411682-33b0f875a1c9ca9965bdf85fcbd30827.png

Keeping the wrist raised for some time, or explicitly tapping the short look notification, will turn it into a long look notification. This type of notifications provides more details, and supports user interaction via customizable buttons. These buttons can send notifications to the application. Unlike short notification, user must explicitly dismiss a long look notification by tapping a button that is automatically displayed at the bottom of the view.

The long look notification is one of the distinct Apple Watch features developers should know about.

More information about using Apple Watch Notifications can be found here.

Development

Architecture

Downloading an Xcode version supporting the WatchKit (currently the 6.2 beta version), and trying to get started with a WatchKit project without reading any documentation, a developer could feel confused because there is no WatchKit template:

Apple Watch apps can only be added as a new target to one of your existing iOS applications.

There is no standalone watch application and a Watch App can only be added as a new target to an existing iOS application.

To develop an Apple Watch app, start with your current iOS app and add the Watch app as a new target.

The reason for that relies in the architecture, and the way the watch app interacts with the related iOS app:

toptal-blog-image-1417522760181.png

When the app is launched on the watch, the companion iOS extension is automatically launched. They both work in synergy, with the watch app showing content to the wearer and responding to interactions, while iOS extension is doing everything else. Actually, no application code is executed on the watch: all processing is delegated to the iOS extension.

This model closely resembles the browser <-> web application model, where the domain logic is performed at server side, and the client side is responsible for showing content to the user and responding to user actions.

The Watch App, that is installed on the device, consists of the storyboard and all related static resources, packed in a bundle. This bundle is static, meaning that adding or changing any of the resources, views, images etc. is not possible at runtime.

This static client-server model imposes several limitations on the iOS developer that wants to add a Watch app to their repetoire, and it’s probably one of the primary reasons why some developers didn’t react positively. Another popular remark is about animations: they are simply unsupported. The only way to mimic an animation is by packing individual frames into the watch app, and then simulating the animation by displaying images in a sequence. Good news is that by naming frames sequentially you can automate the animation directly from the Interface Builder, although the same can be done from code with a single invocation of an image control method.

However Apple has already announced that a new version will be released sometimes next year, with support for native applications, for which there are no further details yet.

The Apple Watch Framework: A Quick Look

WatchKit doesn’t use, inherits or specializes UIKit. A new framework has been implemented, specifically designed, and optimized for the watch, for its tiny display and for the limited hardware. The framework consists of 15 classes only, 12 of which are UI components (label, button, etc.).

The most important class exposed by the framework is WKInterfaceController, which is the WatchKit counterpart of UIViewController. So what in iOS (and OSX) is called the view controller, in Apple Watch is aninterface controller.

Without digging too deeply, the most noticeable improvement over UIKit, from an Apple developer’s perspective, is the establishment of a pattern to pass data between interface controllers. A quick look at the initializer:

Swift init(context: AnyObject?) // context from pushing controller or nil

reveals that it’s possible to provide an arbitrary object to an interface controller instance (called the contextin all APIs), simplifying and standardizing passing data from one interface controller to another. The only downside is that the object is not of a generic type, but an opaque AnyObject which must be downcast to the expected type inside the initializer of the destination interface controller. That’s probably a trade off for still having (and probably also wanting) to keep using Objective C.

The interface controller life cycle is simplified if compared to the view controller counterpart, consisting of 3 transitions only:

  • init: invoked when the object is instantiated
  • willActivate: invoked when the content is about to be displayed
  • didDeactivate: invoked when the interface has just been hidden

Besides showing content to the user, the interface controller is also responsible for:

  • handling user interactions
  • managing a contextual menu
  • coordinate handoff activity
  • respond to notification

Is Apple WatchKit Ready?

With the several limitations, the watch kit may appear as an immature product. And in reality it is. It doesn’t support native watch applications, not to mention that there is no watch device in the market yet, so developers are still unable to test their apps and have a real feeling and feedback of how the app works.

Ok, there’s a simulator, which actually is not an independent simulator, but accessible as an external display to the iOS simulator (via the Hardware -> External Displays menu). But a simulator never replaces a real device – good during development and for debugging, but not enough.

The WatchKit is just a preview, a beta version, an appetizer, which is going to evolve with every new release. iOS developers have got the first version, let’s play with it and let’s try to get the most out of it.
Written by  ANTONIO BELLO

Did you find this article helpful? Don’t forget to drop your feedback in the comments section below. Suggest for you:

☞ iOS 10 Projects: Build Amazing Apps with Apple’s Newest iOS

☞ The Complete iOS 10 Developer Course – Build 21 Apps

☞ iOS Mobile App Development: Create your own Soundboard App

☞ iOS 10 Message Extension App Reskinning without Coding

☞ iOS 10 & Swift 3: From Beginner to Paid Professional

SpriteKit From Scratch: Fundamentals

SpriteKit, available on iOS, tvOS, and OS X, is a framework that allows developers to create high quality 2D games without worrying about the complexities of graphics APIs, such as OpenGL and Metal. In addition to handling all of the graphics for developers, SpriteKit also offers a wide range of extra functionality, including physics simulation, audio/video playback, and saving/loading games.

Throughout this series, you learn all about SpriteKit and gain skills, which you can then apply to any sort of 2D game being built with this excellent framework. In this series, we create a fully functional game in which the player controls a car that must avoid obstacles that cross its path.

This series requires that you are running Xcode 7.3 or higher, which includes support for Swift 2.2 as well as the SDKs for iOS 9.3, tvOS 9.2, and OS X 10.11.5. You also need to download the starter project fromGitHub, which contains the visual assets to be used in the game and a few lines of code to help us get started.

The graphics used for the game in this series can be found on GraphicRiver. GraphicRiver is a great source for finding artwork and graphics for your games.

The first class you need to be aware of when working with SpriteKit is the SKScene class. For any 2D game built with SpriteKit, you split the game’s content up into multiple scenes, each with their own SKScenesubclass.

While we will look at the functionality and inner workings of the SKScene class later on in this series, what you need to know for now is that a scene is what you add the game’s content to and it acts as the root node. This brings us to nodes.

The SKNode class, which SKScene is a subclass of, is used to represent any sort of item within your scene. While there are many different types of nodes available in SpriteKit, commonly referred to as sprites, they share the same key properties with the most important being:

  • position (CGPoint)
  • xScale (CGFloat): representing the horizontal scale of a node
  • yScale (CGFloat): similar to xScale but it acts in the vertical direction instead
  • alpha (CGFloat): representing the node’s transparency
  • hidden (Bool): value determining whether or not the node should be visible
  • zRotation (CGFloat): representing the angle, in radians, that the node should be rotated
  • zPosition (CGFloat): used to determine which nodes should be displayed on top of other nodes in the scene

As you can see, the above properties allow you to accurately position, rotate, and edit the basic appearance of every node in a scene.

Just as you can add subviews to any UIView object in UIKit, in SpriteKit you can add any number ofchild nodes to an existing node. This is done using the addChild(_:) method on any SKNode object.

Also similar to UIView objects, the position and scale properties of any child node are relative to the parent node that it is added to. This is how an SKScene object, being a subclass of SKNode, can act as the root node of your scene. For any nodes that you want in your scene, you need to add them as children of the scene itself using the addChild(_:) method.

The SKNode class on its own is not capable of producing any visual content. In order to do so, you must use one of the many subclasses available in SpriteKit. Some important SKNode subclasses to be aware of are:

  • SKSpriteNode: takes an image and creates a textured sprite in your scene
  • SKCameraNode: controls where your scene is viewed from
  • SKLabelNode: renders a text string
  • SKEmitterNode: used in conjunction with a particle system to render particle effects
  • SKLightNode: creates lighting and shadow effects in your scene

Throughout this series, we use these node types as well as some of the other less common ones.

To display your SpriteKit content within a working application, you need to use the SKView class. This is a subclass of UIView (or NSView on OS X), which can be easily added to any part of your application. To present a SpriteKit scene, you need to call either the presentScene(_:) or presentScene(_:transition:)method on an SKView instance. The second method is used when you want a custom animation or transition into your new scene.

In addition to just rendering your SpriteKit content, the SKView class also offers some extra and very useful functionality. This includes a Bool property called paused, which can easily be used to pause or resume a scene.

There are also several properties that are useful when debugging your game and can show information, such as the current FPS (frames per second), node count, and physics fields within your scene. If you want to have a look at all the available properties, check out the SKView class reference.

Now that we have looked over the basic classes used in SpriteKit games, it is time to create our very first scene. While scenes can be created entirely programmatically, Xcode provides an excellent scene editing tool, which allows you to easily drag and drop nodes into your scene while editing their position, appearance, and other properties.

Open the starter project and create a new file by pressing Command+N or selecting New > File… from Xcode‘s File menu. Choose the iOS > Resource > SpriteKit Scene file type and click Next.

SpriteKit Scene File

Name the file MainScene and save it. You should now see your new scene file in the Project Navigator.

MainScene File

Select MainScene.sks to open Xcode’s scene editor. At the moment, your scene is blank and has a grey background. There are no nodes in it yet. Before adding some sprites to your scene, we first need to change its size.

In the Attributes Inspector on the right, change your scene’s size to have a width (W) of 320 points and a height (H) of 480 points.

Scene Size

This scene size correlates to the smallest screen size we are targeting (3.5″ screen on iPhone 4s) so that we can size our nodes correctly. We will change the scene size to be adaptive to the current device screen size in a later tutorial.

To add nodes to your scene, open the Object Library on the right. You may need to click the icon highlighted in blue if the Object Library is not visible.

Object Library

From the Object Library, drag a Color Sprite to the middle of your scene.

Screen Shot 2016-04-11 at 7.02.25 AM.png

Open the Attributes Inspector on the right and change the Texture, Position, and Size attributes to the following values:

Sprite Properties

Your scene should now have a car in the middle as shown below.

Screen Shot 2016-04-11 at 7.09.51 AM.png

Take note that, unlike the UIKit coordinate system, in SpriteKit, the position of a node always relates to its centre from the scene’s bottom-left corner origin point (0, 0). This is why setting the car’s position to (160, 120) places the centre of the car 160 points in from the left and 120 points up from the bottom.

With your scene completed for now, open ViewController.swift and add an import statement for the SpriteKit framework at the top. This allows us to use the SpriteKit classes and APIs in ViewController.swift.

Next, update the implementation of the viewDidLoad() method of the ViewController class as follows:

With this code, we first create an SKView instance, which is the same size as the view controller’s view. Next, we load our scene from the file we created by using the SKScene(fileNamed:) initializer. We make the SKView instance present this scene and then add it as a subview to the main view.

Select the iPhone 4s simulator (so that our scene is correctly sized for the screen) and build and run your application. You should see a screen similar to the following:

Simulator Screen Shot 11 Apr 2016, 7.15.54 AM.png

In this first tutorial of SpriteKit From Scratch, you learned the basics of views, scenes, and nodes in SpriteKit. You created a basic scene with one sprite using the scene editor built into Xcode, which makes creating scenes much easier when compared to doing it all programmatically.
Written by Davis Allie

Did you find this article helpful? Don’t forget to drop your feedback in the comments section below. Suggest for you:

☞ Master iOS 9 – 10 – Xcode 8 and Swift 3

☞ Complete Swift 3 training – Get ready for XCode 8 and iOS 10

☞ The Complete iOS 10 Developer – Build Real Apps with Swift 3

☞ Swift 3 and iOS 10 The Final Course Learn to Code like a Pro

☞ How to Make a Freaking iPhone App – iOS 10 and Swift 3

 

Simplifying RESTful API Use and Data Persistence on iOS with Mantle and Realm

Every iOS developer is familiar with Core Data, an object-graph and persistence framework from Apple. Apart from persisting data locally, the framework comes with a host of advanced features, such as object change tracking and undo. These features, although useful in many cases, don’t come for free. It requires a lot of boilerplate code, and the framework as a whole has a steep learning curve.

In 2014, Realm, a mobile database, was released and took the development world by storm. If all we need is to persist data locally, Realm is a good alternative. After all, not all use cases require the advanced features of Core Data. Realm is extremely easy to use and as opposed to Core Data, requires very little boilerplate code. It is also thread safe and is said to be faster than the persistence framework from Apple.

In most modern mobile applications, persisting data solves half the problem. We often need to fetch data from a remote service, usually through a RESTful API. This is where Mantle comes into play. It is an open-source model framework for Cocoa and Cocoa Touch. Mantle significantly simplifies writing data models for interacting with APIs that use JSON as their data exchange format.

toptal-blog-image-1437736381161-4aeb77c375aea3e9e8a9eeb692d2aa01.jpg

In this article, we will be building an iOS application that fetches a list of articles along with links to them from the New York Times Article Search API v2. The list is going to be fetched using a standard HTTP GET request, with request and response models created using Mantle. We will get to see how easy it is with Mantle to handle value transformations (eg. from NSDate to string). Once the data is fetched, we will persist it locally using Realm. All this with minimal boilerplate code.

RESTful API – Getting Started

Let’s start by creating a new “Master-Detail Application” Xcode project for iOS named “RealmMantleTutorial”. We will be adding frameworks to it using CocoaPods. The podfile should resemble the following:

pod 'Mantle'
pod 'Realm'
pod 'AFNetworking'

Once the pods are installed we can open the newly created MantleRealmTutorial workspace. As you’ve noticed, the famous AFNetworking framework has been installed too. We will be using it to perform requests to the API.

As mentioned in the introduction, New York Times provides an excellent article search API. In order to use it, one needs to sign up to get an access key to the API. This can be done at http://developer.nytimes.com. With the API key in hand we’re ready to get started with coding.

Before we delve into creating Mantle data models, we need to get our network layer up and running. Let’s create a new group in Xcode and call it Network. In this group we’ll be creating two classes. Let’s call the first one SessionManager and make sure it is derived from AFHTTPSessionManager which is a session manager class from AFNetworking, the delightful networking framework. Our SessionManager class will be a singleton object that will we use to perform get requests to the API. Once the class has been created, please copy the code below into interface and implementation files respectively.

#import "AFHTTPSessionManager.h"

@interface SessionManager : AFHTTPSessionManager

+ (id)sharedManager;

@end
#import "SessionManager.h"

static NSString *const kBaseURL = @"http://api.nytimes.com";

@implementation SessionManager

- (id)init {
    self = [super initWithBaseURL:[NSURL URLWithString:kBaseURL]];
    if(!self) return nil;
    
    self.responseSerializer = [AFJSONResponseSerializer serializer];
    self.requestSerializer = [AFJSONRequestSerializer serializer];
    
    return self;
}

+ (id)sharedManager {
    static SessionManager *_sessionManager = nil;
    
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{
        _sessionManager = [[self alloc] init];
    });
    
    return _sessionManager;
}

@end

The session manager is initialized with the base URL defined in the static kBaseURL variable. It will also use JSON request and response serializers.

Now the second class that we are going to create in the Network group will be called APIManager. It shall be derived from our newly created SessionManager class. Once the necessary data models are created we will add a method to ApiManager that will be used to request a list of articles from the API.

New York Times Article Search API Overview

The official documentation for this excellent API is available at http://developer.nytimes.com/…/article_search_api_v2. What we are going to do is use the following endpoint:

http://api.nytimes.com/svc/search/v2/articlesearch

… to fetch articles found using a search query term of our choosing bounded by a date range. For instance, what we could do is ask the API to return a list of all articles that appeared in the New York Times that had anything to do with basketball in the first seven days of July 2015. According to the API documentation, to do that we need to set the following parameters in the get request to that endpoint:

Parameter Value
q “basketBall”
begin_date “20150701”
end_date “20150707”

The response from the API is quite complex. Below is the response for a request with the above parameters limited to just one article (one item in docs array) with numerous fields omitted for clarity.

{
  "response": {
    "docs": [
      {
        "web_url": "http://www.nytimes.com/2015/07/04/sports/basketball/robin-lopez-and-knicks-are-close-to-a-deal.html",
        "lead_paragraph": "Lopez, a 7-foot center, joined Arron Afflalo, a 6-foot-5 guard, as the Knicks’ key acquisitions in free agency. He is expected to solidify the Knicks’ interior defense.",
        "abstract": null,
        "print_page": "1",
        "source": "The New York Times",
        "pub_date": "2015-07-04T00:00:00Z",
        "document_type": "article",
        "news_desk": "Sports",
        "section_name": "Sports",
        "subsection_name": "Pro Basketball",
        "type_of_material": "News",
        "_id": "5596e7ac38f0d84c0655cb28",
        "word_count": "879"
      }
    ]
  },
  "status": "OK",
  "copyright": "Copyright (c) 2013 The New York Times Company.  All Rights Reserved."
}

What we basically get in response are three fields. The first one called response contains the array docs, which in turn contains items representing articles. The two other fields are status and copyright. Now that we know how the API works, it’s time to create data models using Mantle.

Introduction to Mantle

As mentioned earlier, Mantle is an open-source framework that significantly simplifies writing data models. Let’s start by creating an article list request model. Let’s call this class ArticleListRequestModel and make sure it is derived from MTLModel, which is a class that all Mantle models should be derived from. Additionally let’s make it conform to the MTLJSONSerializing protocol. Our request model should have three properties of suitable types: query, articlesFromDate, and articlesToDate. Just to make sure our project is well organized I suggest that this class be placed in Models group.

Mantle simplifies writing data models, reduces boilerplate code.

Here’s how the interface file of ArticleListRequestModel should look:

#import "MTLModel.h"
#import "Mantle.h"

@interface ArticleListRequestModel : MTLModel <MTLJSONSerializing>

@property (nonatomic, copy) NSString *query;
@property (nonatomic, copy) NSDate *articlesFromDate;
@property (nonatomic, copy) NSDate *articlesToDate;

@end

Now if we look up the docs for our article search endpoint or have a look at the table with request parameters above, we will notice that the names of the variables in the API request differ from those in our request model. Mantle handles this efficiently using the method:

+ (NSDictionary *)JSONKeyPathsByPropertyKey.

Here’s how this method should be implemented in the implementation of our request model:

#import "ArticleListRequestModel.h"

@implementation ArticleListRequestModel

#pragma mark - Mantle JSONKeyPathsByPropertyKey

+ (NSDictionary *)JSONKeyPathsByPropertyKey {
    return @{
             @"query": @"q",
             @"articlesFromDate": @"begin_date",
             @"articlesToDate": @"end_date"
             };
}

@end

The implementation of this method specifies how the properties of the model are mapped into its JSON representations. Once the method JSONKeyPathsByPropertyKey has been implemented, we can get a JSON dictionary representation of the model with the class method +[MTLJSONAdapter JSONArrayForModels:].

One thing that is still left, as we know from list of parameters, is that the both date parameters are required to be in the format “YYYYMMDD”. This is where Mantle gets very handy. We can add custom value transformation for any property by implementing the optional method +<propertyName>JSONTransformer. By implementing it we tell Mantle how the value of a specific JSON field should be transformed during JSON deserialization. We can also implement a reversible transformer that will be used when creating a JSON from the model. Since we need to transform an NSDate object into a string, we will also make use ofNSDataFormatter class. Here is the complete implementation of ArticleListRequestModel class:

#import "ArticleListRequestModel.h"

@implementation ArticleListRequestModel

+ (NSDateFormatter *)dateFormatter {
    NSDateFormatter *dateFormatter = [[NSDateFormatter alloc] init];
    dateFormatter.dateFormat = @"yyyyMMdd";
    return dateFormatter;
}

#pragma mark - Mantle JSONKeyPathsByPropertyKey

+ (NSDictionary *)JSONKeyPathsByPropertyKey {
    return @{
             @"query": @"q",
             @"articlesFromDate": @"begin_date",
             @"articlesToDate": @"end_date"
             };
}

#pragma mark - JSON Transformers

+ (NSValueTransformer *)articlesToDateJSONTransformer {
    return [MTLValueTransformer transformerUsingForwardBlock:^id(NSString *dateString, BOOL *success, 
    NSError *__autoreleasing *error) {
        return [self.dateFormatter dateFromString:dateString];
    } reverseBlock:^id(NSDate *date, BOOL *success, NSError *__autoreleasing *error) {
        return [self.dateFormatter stringFromDate:date];
    }];
}

+ (NSValueTransformer *)articlesFromDateJSONTransformer {
    return [MTLValueTransformer transformerUsingForwardBlock:^id(NSString *dateString, BOOL *success, 
    NSError *__autoreleasing *error) {
        return [self.dateFormatter dateFromString:dateString];
    } reverseBlock:^id(NSDate *date, BOOL *success, NSError *__autoreleasing *error) {
        return [self.dateFormatter stringFromDate:date];
    }];
}

@end

Another great feature of Mantle is that all of these models conform to the NSCoding protocol, as well as implement isEqual and hash methods.

As we have already seen, the resulting JSON from the API call contains an array of objects that represent articles. If we want to model this response using Mantle, we will have to create two separate data models. One would model objects representing articles (docs array elements), and the other would model the whole JSON response except for the elements of the docs array. Now, we don’t have to map each and every property from the incoming JSON into our data models. Let’s suppose we are only interested in two fields of article objects, and those would be lead_paragraph and web_url. The ArticleModel class is rather straightforward to implement, as we can see below.

#import "MTLModel.h"
#import <Mantle/Mantle.h>

@interface ArticleModel : MTLModel <MTLJSONSerializing>

@property (nonatomic, copy) NSString *leadParagraph;
@property (nonatomic, copy) NSString *url;

@end
#import "ArticleModel.h"

@implementation ArticleModel

#pragma mark - Mantle JSONKeyPathsByPropertyKey

+ (NSDictionary *)JSONKeyPathsByPropertyKey {
    return @{
             @"leadParagraph": @"lead_paragraph",
             @"url": @"web_url"
             };
}

@end

Now that the article model has been defined, we can finish response model definition by creating a model for the article list. Here’s how the class ArticleList response model is going to look.

#import "MTLModel.h"
#import <Mantle/Mantle.h>
#import "ArticleModel.h"

@interface ArticleListResponseModel : MTLModel <MTLJSONSerializing>

@property (nonatomic, copy) NSArray *articles;
@property (nonatomic, copy) NSString *status;

@end
#import "ArticleListResponseModel.h"

@class ArticleModel;

@implementation ArticleListResponseModel

#pragma mark - Mantle JSONKeyPathsByPropertyKey

+ (NSDictionary *)JSONKeyPathsByPropertyKey {
    return @{
             @"articles" : @"response.docs",
             @"status" : @"status"
             };
}

#pragma mark - JSON Transformer

+ (NSValueTransformer *)articlesJSONTransformer {
    return [MTLJSONAdapter arrayTransformerWithModelClass:ArticleModel.class];
}

@end

This class has only two properties: status and articles. If we compare it to the response from the endpoint we will see that the third JSON attribute copyright will not be mapped into the response model. If we look at thearticlesJSONTransformer method, we will see that it returns a value transformer for an array containing objects of class ArticleModel.

It is also worth noting that in the method JSONKeyPathsByPropertyKey, the model property articlescorrespond to the array docs that is nested within JSON attribute response.

By now we should have three model classes implemented: ArticleListRequestModel, ArticleModel, and ArticleListResponseModel.

First API Request

toptal-blog-image-1437736761484-e7c512656ff5de52c619c4c71f298898.jpg

Now that we have implemented all the data models, it is time to get back to the class APIManager to implement the method that we will use to perform GET requests to the API. The method:

- (NSURLSessionDataTask *) getArticlesWithRequestModel:(ArticleListRequestModel *)requestModel success:(void (^)(ArticleListResponseModel *responseModel))success failure:(void (^)(NSError *error))failure

takes an ArticleListRequestModel request model as a parameter and returns an ArticleListResponseModel in case of success or an NSError otherwise. The implementation of this method uses AFNetworking to perform a GET request to the API. Please note that in order to make a successful API request we need to provide a key that can be obtained as mentioned earlier, by registering at http://developer.nytimes.com.

#import "SessionManager.h"
#import "ArticleListRequestModel.h"
#import "ArticleListResponseModel.h"

@interface APIManager : SessionManager

- (NSURLSessionDataTask *)getArticlesWithRequestModel:(ArticleListRequestModel *)requestModel success:(void (^)(ArticleListResponseModel *responseModel))success failure:(void (^)(NSError *error))failure;

@end
#import "APIManager.h"
#import "Mantle.h"

static NSString *const kArticlesListPath = @"/svc/search/v2/articlesearch.json";
static NSString *const kApiKey = @"replace this with your own key";

@implementation APIManager

- (NSURLSessionDataTask *)getArticlesWithRequestModel:(ArticleListRequestModel *)requestModel
                                              success:(void (^)(ArticleListResponseModel *responseModel))success
                                              failure:(void (^)(NSError *error))failure{
    
    NSDictionary *parameters = [MTLJSONAdapter JSONDictionaryFromModel:requestModel error:nil];
    NSMutableDictionary *parametersWithKey = [[NSMutableDictionary alloc] initWithDictionary:parameters];
    [parametersWithKey setObject:kApiKey forKey:@"api-key"];
    
    return [self GET:kArticlesListPath parameters:parametersWithKey
             success:^(NSURLSessionDataTask *task, id responseObject) {
        
        NSDictionary *responseDictionary = (NSDictionary *)responseObject;
        
        NSError *error;
        ArticleListResponseModel *list = [MTLJSONAdapter modelOfClass:ArticleListResponseModel.class
                                                   fromJSONDictionary:responseDictionary error:&error];
        success(list);
        
    } failure:^(NSURLSessionDataTask *task, NSError *error) {
        
        failure(error);
        
    }];
}

There are two very important things happening in the implementation of this method. First let’s take a look at this line:

NSDictionary *parameters = [MTLJSONAdapter JSONDictionaryFromModel:requestModel error:nil];

What is happening here is that using the method provided by MTLJSONAdapter class we get a NSDictionaryrepresentation of our data model. That representation mirrors the JSON that is going to be sent to the API. This is where the beauty of Mantle lies. Having implemented JSONKeyPathsByPropertyKey and +<propertyName>JSONTransformer methods in the ArticleListRequestModel class, we can get the correct JSON representation of our data model in no time with just a single line of code.

Mantle also allows us to perform transformations in the other direction too. And that’s exactly what’s happening with the data received from the API. The NSDictionary that we receive is mapped into an object of ArticleListResponseModel class using the following class method:

ArticleListResponseModel *list = [MTLJSONAdapter modelOfClass:ArticleListResponseModel.class fromJSONDictionary:responseDictionary error:&error];

Persisting Data with Realm

Now that we are able to fetch data from a remote API, it is time to persist it. As mentioned in the introduction, we will do it using Realm. Realm is a mobile database and a replacement for Core Data and SQLite. As we will see below, it is extremely easy to use.

Realm, the ultimate mobile database, is a perfect replacement for Core Data and SQLite.

To save a piece of data in Realm we first need to encapsulate an object that is derived from RLMObject class. What we need to do now is to create a model class that will store data for single articles. Here’s how easy it is to create such a class.

#import "RLMObject.h"

@interface ArticleRealm : RLMObject

@property NSString *leadParagraph;
@property NSString *url;

@end

And this could be basically it, the implementation of this class could remain empty. Please note that the properties in the model class have no attributes like nonatomic, strong, or copy. Realm takes care of those and we need not worry about them.

Since the articles that we can get are modelled with the Mante model Article it would be convenient to initialize ArticleRealm objects with objects of class Article. To do that we will add initWithMantleModel method to our Realm model. Here’s the complete implementation of ArticleRealm class.

#import "RLMObject.h"
#import "ArticleModel.h"

@interface ArticleRealm : RLMObject

@property NSString *leadParagraph;
@property NSString *url;

- (id)initWithMantleModel:(ArticleModel *)articleModel;

@end
#import "ArticleRealm.h"

@implementation ArticleRealm

- (id)initWithMantleModel:(ArticleModel *)articleModel{
    self = [super init];
    if(!self) return nil;
    
    self.leadParagraph = articleModel.leadParagraph;
    self.url = articleModel.url;
    
    return self;
}

@end

We interact with the database using objects of class RLMRealm. We can easily get a RLMRealm object by invoking the method “[RLMRealm defaultRealm]”. It is important to remember that such an object is valid only within the thread it was created on and can not be shared across threads. Writing data to Realm is quite straightforward. A single write, or a series of them, need to be done within a write transaction. Here’s a sample write to the database:

RLMRealm *realm = [RLMRealm defaultRealm];
    
ArticleRealm *articleRealm = [ArticleRealm new];
articleRealm.leadParagraph = @"abc";
articleRealm.url = @"sampleUrl";
    
[realm beginWriteTransaction];
[realm addObject:articleRealm];
[realm commitWriteTransaction];

What happens here is the following. First we create a RLMRealm object to interact with the database. Then anArticleRealm model object is created (please bear in mind that it is derived from RLMRealm class). Finally to save it, a write transaction begins, the object is added to the database, and once it is saved the write transaction is committed. As we can see, write transactions block the thread on which they are invoked. While Realm is said to be very fast, if we were to add multiple objects to the database within a single transaction on the main thread, that could lead to the UI becoming unresponsive until the transaction is finished. A natural solution to that is to perform such a write transaction on a background thread.

API Request and Persisting Response in Realm

This is all the information we need to persist articles using Realm. Let’s try to perform an API request using the method

- (NSURLSessionDataTask *) getArticlesWithRequestModel:(ArticleListRequestModel *)requestModel success:(void (^)(ArticleListResponseModel *responseModel))success failure:(void (^)(NSError *error))failure

and Mantle request and response models in order to get New York Times articles that had anything to do (as in the earlier example) with basketball and were published in the first seven days of June 2015. Once the list of such articles is available, we will persist it in Realm. Below is the code that does that. It’s placed in theviewDidLoad method of the table view controller in our app.

ArticleListRequestModel *requestModel = [ArticleListRequestModel new]; // (1)
requestModel.query = @"Basketball";
requestModel.articlesToDate = [[ArticleListRequestModel dateFormatter] dateFromString:@"20150706"];
requestModel.articlesFromDate = [[ArticleListRequestModel dateFormatter] dateFromString:@"20150701"];

[[APIManager sharedManager] getArticlesWithRequestModel:requestModel   // (2)
												success:^(ArticleListResponseModel *responseModel){
	
	dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{ // (3)
		@autoreleasepool {
			
		RLMRealm *realm = [RLMRealm defaultRealm];
		[realm beginWriteTransaction];
		[realm deleteAllObjects];
		[realm commitWriteTransaction];
		
		[realm beginWriteTransaction];
		for(ArticleModel *article in responseModel.articles){
			ArticleRealm *articleRealm = [[ArticleRealm alloc] initWithMantleModel:article]; // (4)
			[realm addObject:articleRealm];
		}
		[realm commitWriteTransaction];
	   
			dispatch_async(dispatch_get_main_queue(), ^{ // (5)
				RLMRealm *realmMainThread = [RLMRealm defaultRealm]; // (6)
				RLMResults *articles = [ArticleRealm allObjectsInRealm:realmMainThread];
				self.articles = articles; // (7)
				[self.tableView reloadData];
			});
		}
	});
	
} failure:^(NSError *error) {
	self.articles = [ArticleRealm allObjects];
	[self.tableView reloadData];
}];

First, an API call is made (2) with a request model (1), which returns a response model that contains a list of articles. In order to persist those articles using Realm we need to create Realm model objects, which takes place in the for loop (4). It is also important to notice that since multiple objects are persisted within a single write transaction, that write transaction is performed on a background thread (3). Now, once all the articles are saved in Realm, we assign them to the class property self.articles (7). Since they’re going to be accessed later on the main thread in TableView datasource methods, it is safe to retrieve them from the Realm database on the main thread too (5). Again, to access the database from a new thread, a new RLMRealm object needs to be created (6) on that thread.

toptal-blog-image-1437736736973-c122aa01a412250edee995b94ce9628c.jpg

If getting new articles from the API fails for whatever reason, the existing ones are retrieved from the local storage in the failure block.

Wrapping Up

In this tutorial we learned how to configure Mantle, a model framework for Cocoa and Cocoa Touch, in order to interact with a remote API. We also learned how to locally persist data retrieved in the form of Mantle model objects using Realm mobile database.

In case you want to try out this application, you can retrieve the source code from its GitHub repository. You will need to generate and provide your own API key before running the application.
Written by MARCIN KMIEC

Did you find this article helpful? Don’t forget to drop your feedback in the comments section below. Suggest for you:

☞ Complete Beginners Guide to iOS Development – Build 10 Apps

☞ The Complete iOS 10 Developer – Build Real Apps with Swift 3

☞ The Complete iOS 9 Developer Course – Build 18 Apps

☞ The Complete iOS 10 Developer Course – Build 21 Apps

☞ Intro To iOS Backend Development: Image Uploads

iOS Animation and Tuning for Efficiency

Building a great app is not all about looks or functionality, it’s also about how well it performs. Although hardware specifications of mobile devices are improving at a rapid pace, apps that perform poorly, stutter at every screen transition or scrolls like a slideshow can ruin the experience of its user and become a cause of frustration. In this article we will see how to measure performance of an iOS app and tune it for efficiency. For the purpose of this article, we will build a simple app with a long list of images and texts.

iOS Animation and Tuning for Efficiency

For the purposes of testing performance, I would recommend the use of real devices. If you’re serious aboutbuilding apps, and optimizing them for smooth iOS animation, simulators simply don’t cut it. Simulations can sometimes be out of step with reality. For example, the simulator may be running on your Mac which probably means the CPU (Central Processing Unit) is far more powerful than the CPU on your iPhone. Conversely, the GPU (Graphics Processing Unit) is so different between your device and your Mac that your Mac actually emulates the device’s GPU. As a result, CPU-bound operations tend to be faster on your simulator while GPU-bound operations tend to be slower.

Animating at 60 FPS

One key aspect of perceived performance is making sure your animations run at 60 FPS (frames per second), which is the refresh rate of your screen. There are some timer based animations, which we won’t discuss here. Generally speaking, if you’re running at anything greater than 50 FPS your app will look smooth and performant. If your animations are stuck between 20 and 40 FPS there will be a noticeable stutter and the user will detect a “roughness” in transitions. Anything below 20 FPS will severely affect the usability of your app.

Before we start, it’s probably worth discussing the difference between CPU bound and GPU bound operations. The GPU is a specialized chip that is optimized for drawing graphics. While the CPU can too, it’s far slower. This is why we want to offload as much of our graphics rendering, the process of generating an image from a 2D or 3D model, to the GPU. But we need to be careful, as when the GPU runs out of processing power, graphics related performance will degrade even if the CPU is relatively free.

Core Animation is a powerful framework that handles animation both inside your app, and outside it. It breaks down the process into 6 key steps:

  1. Layout: Where you arrange your layers and set their properties, such as color and their relative position
  2. Display: This is where the backing images are drawn onto a context. Any routine you wrote in drawRect:or drawLayer:inContext: is accessed here.
  3. Prepare: At this stage Core Animation, as it is about to send context to the renderer to draw on, performs some necessary tasks such as decompress images.
  4. Commit: Here Core Animation sends all this data to the render server.
  5. Deserialization: The previous 4 steps were all within your app, now the animation is being processed outside your app, the packaged layers are deserialized into a tree that the render server can understand. Everything is converted into OpenGL geometry.
  6. Draw: Renders the shapes (actually triangles).

You might have guessed that processes 1-4 are CPU operations and 5-6 are GPU operations. In reality you only have control over the first 2 steps. The biggest killer of the GPU is semi-transparent layers where the GPU has to fill the same pixel multiple times per frame. Also any offscreen drawing (several layer effects such as shadows, masks, rounded corners, or layer rasterization will force Core Animation to draw offscreen) will also affect performance. Images that are too large to be processed by the GPU will be processed by the much slower CPU instead. While shadows can be easily achieved by setting two properties directly on the layer, they can easily kill performance if you have many objects onscreen with shadows. Sometimes it’s worth considering adding these shadows as images.

Measuring iOS Animation Performance

We will begin with a simple app with 5 PNG images, and a table view. In this app, we will essentially load 5 images, but will repeat it over 10,000 rows. We will add shadows to both the images and to labels next to the images:

-(UITableViewCell*)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath {
    CustomTableViewCell *cell = [tableView dequeueReusableCellWithIdentifier:[CustomTableViewCell getCustomCellIdentifier] forIndexPath:indexPath];
    
    NSInteger index = (indexPath.row % [self.images count]);
    NSString *imageName = [self.images objectAtIndex:index];
    
    NSString *filePath = [[NSBundle mainBundle] pathForResource:imageName ofType:@"png"];
    UIImage *image = [UIImage imageWithContentsOfFile:filePath];
    
    cell.customCellImageView.image = image;
    
    
    cell.customCellImageView.layer.shadowOffset = CGSizeMake(0, 5);
    cell.customCellImageView.layer.shadowOpacity = 0.8f;
    
    cell.customCellMainLabel.text = [NSString stringWithFormat:@"Row %li", (long)(indexPath.row + 1)];
    cell.customCellMainLabel.layer.shadowOffset = CGSizeMake(0, 3);
    cell.customCellMainLabel.layer.shadowOpacity = 0.5f;
    
    return cell;
}

Images are simply recycled while labels are always different. The result is:

toptal-blog-image-1450440888864-8b5d563353f1ec12b1fcdfd844a19910.jpg

Upon swiping vertically, you are very likely to notice stutter as the view scrolls. At this point you may be thinking that the fact we are loading images in the main thread is the problem. May be if we moved this to the background thread all our problems would be solved.

Instead of making blind guesses, let’s try it out and measure the performance. It’s time for Instruments.

To use Instruments, you need to change from “Run” to “Profile”. And you should also be connected to your real device, not all instruments are available on the simulator (another reason why you shouldn’t be optimizing for performance on the simulator!). We will be primarily using “GPU Driver”, “Core Animation” and “Time Profiler” templates. A little known fact is that instead of stopping and running on a different instrument, you can drag and drop multiple instruments and run several at the same time.

Now that we have our instruments set-up, let’s measure. First let’s see if we really have a problem with our FPS.

toptal-blog-image-1450440951441-b43a8b5b9fc56c481044561c550a9ad3.jpg

Yikes, I think we’re getting 18 FPS here. Is loading images from the bundle on the main thread really that expensive and costly? Notice our renderer utilization is almost maxed out. So is our tiler utilization. Both are above 95%. And that has nothing to do with loading an image from the bundle on the main thread so let’s not look for solutions here.

Tuning for Efficiency

There is a property called shouldRasterize, and people will probably recommend you to use it here. What does shouldRasterize do exactly? It caches your layer as a flattened image. All those expensive layer drawings need to happen once. In case your frame changes frequently, there is no use for a cache, as it will need to be regenerated each time anyway.

Making a quick amendment to our code, we get:

-(UITableViewCell*)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath {
    CustomTableViewCell *cell = [tableView dequeueReusableCellWithIdentifier:[CustomTableViewCell getCustomCellIdentifier] forIndexPath:indexPath];
    
    NSInteger index = (indexPath.row % [self.images count]);
    NSString *imageName = [self.images objectAtIndex:index];
    
    NSString *filePath = [[NSBundle mainBundle] pathForResource:imageName ofType:@"png"];
    UIImage *image = [UIImage imageWithContentsOfFile:filePath];
    
    cell.customCellImageView.image = image;
    
    
    cell.customCellImageView.layer.shadowOffset = CGSizeMake(0, 5);
    cell.customCellImageView.layer.shadowOpacity = 0.8f;
    
    cell.customCellMainLabel.text = [NSString stringWithFormat:@"Row %li", (long)(indexPath.row + 1)];
    cell.customCellMainLabel.layer.shadowOffset = CGSizeMake(0, 3);
    cell.customCellMainLabel.layer.shadowOpacity = 0.5f;
    
    cell.layer.shouldRasterize = YES;
    cell.layer.rasterizationScale = [UIScreen mainScreen].scale;
    
    return cell;
}

And we measure again:

toptal-blog-image-1450441003128-836545d9086c8dbccab150b62442fdc7.jpg

With just two lines, we have improved our FPS by 2x. We are now averaging above 40 FPS. But would it help if we had moved image loading to a background thread?

-(UITableViewCell*)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath {
    CustomTableViewCell *cell = [tableView dequeueReusableCellWithIdentifier:[CustomTableViewCell getCustomCellIdentifier] forIndexPath:indexPath];
    
    NSInteger index = (indexPath.row % [self.images count]);
    NSString *imageName = [self.images objectAtIndex:index];
    
    cell.tag = indexPath.row;
    
    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
        NSString *filePath = [[NSBundle mainBundle] pathForResource:imageName ofType:@"png"];
        UIImage *image = [UIImage imageWithContentsOfFile:filePath];
        
        dispatch_async(dispatch_get_main_queue(), ^{
            if (indexPath.row == cell.tag) {
                cell.customCellImageView.image = image;
            }
        });
    });
    
    
    cell.customCellImageView.layer.shadowOffset = CGSizeMake(0, 5);
    cell.customCellImageView.layer.shadowOpacity = 0.8f;
    
    cell.customCellMainLabel.text = [NSString stringWithFormat:@"Row %li", (long)(indexPath.row + 1)];
    cell.customCellMainLabel.layer.shadowOffset = CGSizeMake(0, 3);
    cell.customCellMainLabel.layer.shadowOpacity = 0.5f;
    
//    cell.layer.shouldRasterize = YES;
//    cell.layer.rasterizationScale = [UIScreen mainScreen].scale;
    
    return cell;
}

Upon measuring, we see the performance is averaging around 18 FPS:

toptal-blog-image-1450441016252-e1ce9cb00862c36b9b4cf51936f987b6.jpg

Nothing worth celebrating. In other words, it made no improvement to our frame rate. That’s because even though clogging the main-thread is wrong, it wasn’t our bottleneck, rendering was.

Moving back to the better example, where we were averaging above 40FPS, performance is notably smoother. But we can actually do better.

Checking “Color Blended Layers” on the Core Animation Tool, we see:

toptal-blog-image-1450441043259-22b4c5ba7b6538b7eedffd8e2f2411e5.jpg

“Color Blended Layers” shows on the screen where your GPU is doing a lot of rendering. Green indicates the least amount of rendering activity while red indicates the most. But we did set shouldRasterize to YES. It is worth pointing out that “Color Blended Layers” is not the same as “Color Hits Green and Misses Red”. The later basically highlights rasterized layers in red as the cache is regenerated (a good tool to see if you’re not using the cache properly). Setting shouldRasterize to YES has no effect on the initial rendering of non-opaque layers.

This is an important point, and we need to pause for a moment to think. Regardless of whether shouldRasterize is set to YES or not, to render the framework needs to check all the views, and blend (or not) based on whether subviews are transparent or opaque. While it could make sense for your UILabel to be non-opaque, it maybe be worthless and killing your performance. For example a transparent UILabel on a white background is probably worthless. Let’s make it opaque:

toptal-blog-image-1450441058534-2285b3d050319984b634153cedebc193.jpg

This yields better performance, but our look and feel of the app has changed. Now, because our label and images are opaque, the shadow has moved around our image. No one is probably going to be fond of this change, and if we want to preserve the original look and feel with top notch performance we are not lost for hope.

To squeeze out some extra FPS while preserving the original look, it’s important to revisit two of our Core Animation phases that we have neglected so far.

  1. Prepare
  2. Commit

These may seem to be completely out of our hands, but that’s not quite true. We know for an image to be loaded it needs to be decompressed. The decompression time changes depending on image format. For PNGs decompression is a lot faster than JPEGs (though loading is longer, and this depends on image size too), so we were sort of on the right track to use PNGs, but we are doing nothing about the decompression process, and this decompression is happening at the “point of drawing”! This is the worst possible place where we can kill time – on the main thread.

There is a way to force to decompression. We could set it to the image property of a UIImageView right away. But that still decompresses the image on the main thread. Is there any better way?

There is one. Draw it into a CGContext, where the image needs to be decompressed before it can be drawn. We can do this (using the CPU) in a background thread, and give it bounds as necessary based on the size of our image view. This will optimize our image drawing process by doing it off the main thread, and save us from unneeded “preparing” calculations on the main thread.

While we’re at it, why not add in the shadows as we draw the image? We can then capture the image (and cache it) as one static, opaque image. The code is as follows:

- (UIImage*)generateImageFromName:(NSString*)imageName {
    
    //define a boudns for drawing
    CGRect imgVwBounds = CGRectMake(0, 0, 48, 48);
    
    //get the image
    NSString *filePath = [[NSBundle mainBundle] pathForResource:imageName ofType:@"png"];
    UIImage *image = [UIImage imageWithContentsOfFile:filePath];
    
    //draw in the context
    UIGraphicsBeginImageContextWithOptions(imgVwBounds.size, NO, 0); {
    
    //get context
    CGContextRef context = UIGraphicsGetCurrentContext();
    
    //shadow
    CGContextSetShadowWithColor(context, CGSizeMake(0, 3.0f), 3.0f, [UIColor blackColor].CGColor);
    
    CGContextBeginTransparencyLayer (context, NULL);
    
    [image drawInRect:imgVwBounds blendMode:kCGBlendModeNormal alpha:1.0f];
        CGContextSetRGBStrokeColor(context, 1.0, 1.0, 1.0, 1.0);
        CGContextEndTransparencyLayer(context);
    }
    
    image = UIGraphicsGetImageFromCurrentImageContext();
    UIGraphicsEndImageContext();
    
    return image;
    
}

And finally:

-(UITableViewCell*)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath {
    CustomTableViewCell *cell = [tableView dequeueReusableCellWithIdentifier:[CustomTableViewCell getCustomCellIdentifier] forIndexPath:indexPath];
    
//    NSInteger index = (indexPath.row % [self.images count]);
//    NSString *imageName = [self.images objectAtIndex:index];
//    
//    cell.tag = indexPath.row;
//    
//    dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_HIGH, 0), ^{
//        NSString *filePath = [[NSBundle mainBundle] pathForResource:imageName ofType:@"png"];
//        UIImage *image = [UIImage imageWithContentsOfFile:filePath];
//        
//        dispatch_async(dispatch_get_main_queue(), ^{
//            if (indexPath.row == cell.tag) {
//                cell.customCellImageView.image = image;
//            }
//        });
//    });
    
    cell.customCellImageView.image = [self getImageByIndexPath:indexPath];
    cell.customCellImageView.clipsToBounds = YES;
    
//    cell.customCellImageView.layer.shadowOffset = CGSizeMake(0, 5);
//    cell.customCellImageView.layer.shadowOpacity = 0.8f;
    
    cell.customCellMainLabel.text = [NSString stringWithFormat:@"Row %li", (long)(indexPath.row + 1)];
    cell.customCellMainLabel.layer.shadowOffset = CGSizeMake(0, 3);
    cell.customCellMainLabel.layer.shadowOpacity = 0.5f;
    
    cell.layer.shouldRasterize = YES;
    cell.layer.rasterizationScale = [UIScreen mainScreen].scale;
    
    return cell;
} 

And the results are:

toptal-blog-image-1450441077538-99d375390fd37ae0f3d270c6105707c3.jpg

We are now averaging above 55 FPS, and our render utilization and tiler utilization are nearly half of what they originally were.

Wrap Up

Just in case you were wondering what else we can do to crank out a few more frames per second, look no further. UILabel uses WebKit HTML to render text. We can go directly to CATextLayer and perhaps play with the shadows there too.

You may have noticed in our above implementation, we weren’t doing the image loading in a background thread, and instead we were caching it. Since there were just 5 images this worked really fast and didn’t seem to affect the overall performance (especially since all 5 images were loaded on screen before scroll). But you may want to try moving this logic to a background thread for some extra performance.

Tuning for efficiency is the difference between a world-class app and an amateur one. Performance optimization, especially when it comes to iOS animation, can be a daunting task. But with the help of Instruments, one can easily diagnose the bottlenecks in animation performance on iOS.
Written by STEFAN PROGOVAC

Did you find this article helpful? Don’t forget to drop your feedback in the comments section below. Suggest for you:

☞ Swift 3 and iOS 10 The Final Course Learn to Code like a Pro

☞ Master iOS 9 – 10 – Xcode 8 and Swift 3

☞ Complete Swift 3 training – Get ready for XCode 8 and iOS 10

☞ Learn Swift 3, Xcode 8 and iOS 10 Step by Step

☞ Professional iOS Chat Apps with Social Login using Firebase3

SpriteKit From Scratch: Visual and Audio Effects

In this tutorial, the fourth installment of the SpriteKit From Scratch series, we look at the various visual and audio features SpriteKit provides to add some more detail and variety to your games. This includes particle systems, filters, lighting, and audio.

 

In SpriteKit, the term particle system refers to a single emitter node, represented by the SKEmitterNodeclass. It defines the system’s position in your scene and all of the particles it creates. The emitter specifies the various behaviors of the particles that it generates.

Particle systems are best used in SpriteKit games where you need to generate a large number of identical or similar sprites, which don’t need to have a specific location or execute any actions.

In this tutorial, we are going to add two particle systems when the car hits an obstacle:

  • a short explosion effect that briefly appears
  • a smoke effect that remains in the scene indefinitely

While particle systems can be created programmatically, it is much easier to do so using Xcode’s built in editor. All of the properties of a particle system can be modified in this editor and the changes you make are immediately visualized. This is much easier than having to run your game after every change you make.

We are first going to create the explosion effect. Create a new file in your project and choose the iOS > Resource > SpriteKit Particle File template.

Particle File Template

From the menu that appears, select Fire as the Particle template. Name the file Explosion or something similar.

Fire Particle Template

After Xcode has created the file, you see that there are two new files in your project, Explosion.sksand spark.png.

Two Files Are Added

Explosion.sks contains the particle system and it is the file we will be working with. The second file,spark.png, is a simple image used by the Fire particle template to create its  visual effect. If you open Explosion.sks, you can see the fire animating.

Fire Particle System

The most important changes we need to make to this particle system is to have particles moving outwards from the emitter in all directions and to not continuously spawn new particles.

To make the first change, open the Attributes Inspector and, under the Particles section, change Angle Range to 360°.

Changing Angle Range

Straight away, you can see that the particles are now moving outwards in a circular shape.

Circular Fire

To stop the particle system from continuously creating new particles, we can specify a Maximum value. This value tells the particle system how many particles in total it should create. The default value of 0means that there is no maximum, which causes new particles to be created continuously.

In addition to specifying a maximum value, we are also going to change a few others properties to create a better explosion effect. In the Particles section of the Attributes Inspector, change the following values:

Explosion Custom Properties

We set Birthrate to a value that is higher than the Maximum property because it determines how many particles are created per second. We want the explosion to happen very quickly. So rather than having 1,000 particles spawning over the span of a full second, which would happen with a Birthrate of 1,000, we specify a Birthrate of 5,000. This means that all the particles are created in just 0.2 seconds.

By setting Lifetime > Start to 3 the particles live for 3 seconds. The Lifetime Range property can be used to add variation to the lifetime of the particles.

Lastly, we set Speed > Start to 200 so that the particles fly out very quickly from the emitter as would happen in a real explosion.

After making these changes, you can see that the particle system looks quite different and more like a proper explosion.

Explosion Particle System

Note that, even though the animation loops periodically in the Xcode editor, the particle system only animates once when added to your scene.

With the explosion particle system completed, it is time to move on to the smoke particle system. Create a new file, Smoke, using the same template we used for the explosion. The only difference is the Particle template, which we set to Smoke.

Smoke Particle Template

The only change we need to make to this particle system is to have the smoke move outwards in a circle rather than just straight up. To do so, change the Angle > Range property to 360° like we did for the explosion particle system. After doing so, the smoke particle system should look something like this:

Smoke Particle System

With both particle systems ready, we can add them to our scene. To do this, we load each of the files that we created as SKEmitterNode objects and then add these to the scene like a regular node. Open MainScene.swift and replace the implementation of didBeginContact(_:) with the following:

As in the previous implementation of didBeginContact(_:), we perform the same check as before to see if either of the nodes involved in the collision is the car node. We then use optional binding to get the paths to both the explosion and smoke particle system resource files. We use these paths to instantiate SKEmitterNode objects from them.

Next, we remove all actions from the camera and player nodes, and we hide the player node by removing it from the scene. We remove the car to avoid more collisions which then lead to more explosions.

We also set the position of the emitter nodes to that of the car and add them to the scene. As a result, SpriteKit immediately begins animating the particle systems as soon as they are added to the scene.

Build and run your game. You should see the explosion particle system as soon as the car hits an obstacle. This is followed by smoke once the fire has cleared.

Particles in Scene

In SpriteKit, there is a special type of node (represented by the SKEffectNode class) that can use a Core Image filter object (represented by the CIFilter class) to render its child nodes with a variety of effects. The SKScene class is also a subclass of SKEffectNode, which means that you can also apply a filter to the entire scene.

Unfortunately, at the time of writing this tutorial, there are a few issues relating to these filters and effect nodes in iOS 9. Currently, as soon as an effect is enabled for an effect node, all of its children are hidden, which results in the effect not being visible.

Even though we can’t implement this into our game and see what it looks like, we can still run through the code that would be used to create an effect. In this case, the following method is an example of adding and gradually fading a blur effect to the entire scene.

We create a CIFilter object of a particular type. If you want to look at some of the other built-in filters available to you, check out the Core Image Filter Reference. We ensure this filter has all of the default input values and then manually set inputRadius to 0.0, which means that there is initially no blur.

We then assign the filter to the filter property of the current scene and set shouldEnableEffects to true to enable it. Finally, we run a custom SKAction that gradually increases the filter’s input radius to 10.

Hopefully, in a future iOS release, the issues affecting effect nodes are fixed, because they provide a way to add some very unique and interesting effects to your SpriteKit scenes.

SpriteKit also includes an excellent lighting system that can be used to make your scenes feel more realistic. Lights are very easy to implement and are created through the use of the SKLightNode class. A light node defines certain properties, such as the light’s color (including ambient color) and its strength over distance.

In our scene, we are going to create a single white light that will be attached to the car. This light will light up the obstacles in front of the car as well as produce shadows.

Let’s start by creating a light in the didMoveToView(_:) method of your the MainScene class.

With this code, we create a new SKLightNode object, change its lightColor property to white, and lower itsfalloff property from the default value of 1 to 0.5.

Just like when setting up physics collision detecting in SpriteKit, you must specify which lights interact with which nodes in a scene through the use of bit masks. When SpriteKit renders the lights in a scene, it uses a logical AND operator on the light node’s categoryBitMask and the lightingBitMask andshadowCastBitMask of every other node to determine how that particular node should appear.

For our game, we want the obstacles to interact with the light so that they cast shadows in the scene. To do so, add the following two lines at the end of the spawnObstacle(_:) method of the MainScene class:

By setting a bit mask with all bits enabled, the obstacles interact with every light in the scene.

Build and run your app. You will see that, as your car moves through the scene, each obstacle has a dynamic shadow, which is always pointing away from the centre of the car.

Shadows in Scene

As you can see, lights in SpriteKit are very simple to use and can add nice effects to your scenes.

Finally, we are going to look at audio nodes in SpriteKit. Audio nodes are used to add sound effects to a scene. These special nodes are represented by the SKAudioNode class. Because SKAudioNode is an SKNode subclass, you can add and position them anywhere in a scene, like a regular node.

In addition to playing audio in a regular manner and sounding the same no matter how your scene is arranged (for example, background music), SpriteKit allows you to utilize positional audio to create a truly immersive effect. This is done by specifying a listener node for your scene, which is where the sound will be “heard” from.

Audio nodes are positional by default. This means that, if you don’t want to use this functionality in particular cases, you can set a specific node’s positional property to false.

While we won’t be implementing this into our game, the following is an example method of adding a background music node that loops as long as it is part of the scene. In the method, we also add an explosion sound node that start playing when we tell it to.

Note that we import the AVFoundation framework at the top. This is necessary to access and work with the avAudioNode property of an SKAudioNode object. As you can see, audio nodes are very easy to set up and work with in SpriteKit.

You should now be comfortable working with some of the more advanced effects in SpriteKit, including particle systems, filters, light, and audio. These effects combined can greatly impact the appearance of your game and how immersive it is.
Written by Davis Allie

Did you find this article helpful? Don’t forget to drop your feedback in the comments section below.Suggest for you:

☞  Learn Swift 3, Xcode 8 and iOS 10 Step by Step

☞  Professional iOS Chat Apps with Social Login using Firebase3

☞  The Complete iOS 9 Developer Course – Build 18 Apps

☞ The Complete iOS 10 Developer Course – Build 21 Apps

☞  Intro To iOS Backend Development: Image Uploads