逍遙文工作室

在研究所時代有旁聽機器學習課程，當時有親自操作SVM來產生model，可參考最近七年度以來一直很夯的文章：支持向量機器 (Support Vector Machine)。此時想要在iOS上使用SVM model要怎麼做呢？好在Apple釋出轉換model工具，再配合2017年釋出的Core ML，就能輕易在iOS上使用該model。

.

在此跳過轉換model的部分，直接進入套用已轉好的Core ML Models。不管你是阿貓還是阿狗做出來的model，都能透過Apple的轉換工具變身為飛天阿貓阿狗給App使用ＸＤ～

想要使用Core ML Framework，必須在Xcode 9和iOS 11環境下才能實作與運行。在網路上多為Swift範例，在此我特地使用Objective C來實作，做完之後就知道原來這麼容易。大致上要注意三個重點：

1. Import model
2. Input data
3. Output result

在此我以Apple提供的models為例：Places205-GoogLeNet

其他models可參考：

https://developer.apple.com/machine-learning/

Detects the scene of an image from 205 categories such as an airport terminal, bedroom, forest, coast, and more.

Apple已將model轉換為Core ML Model：GoogLeNetPlaces.mlmodel，檔案大小為24.8 MB，因為是影像麻豆，特徵非常多，所以model檔案也就肥大。

將model匯進專案後，很神奇地會幫我自動產生Objective C model class如下：

/**
 Theme: GoogLeNetPlaces model
 IDE: Xcode 9
 Language: Objective C
 Date: 107/03/20
 Author: HappyMan
 Blog: https://cg2010studio.com/
 */

//
// GoogLeNetPlaces.h
//
// This file was automatically generated and should not be edited.
//

#import 
#import 
#include 

NS_ASSUME_NONNULL_BEGIN

/// Model Prediction Input Type
API_AVAILABLE(macos(10.13), ios(11.0), watchos(4.0), tvos(11.0))
@interface GoogLeNetPlacesInput : NSObject

/// Input image of scene to be classified as color (kCVPixelFormatType_32BGRA) image buffer, 224 pixels wide by 224 pixels high
@property (readwrite, nonatomic) CVPixelBufferRef sceneImage;
- (instancetype)init NS_UNAVAILABLE;
- (instancetype)initWithSceneImage:(CVPixelBufferRef)sceneImage;
@end

/// Model Prediction Output Type
API_AVAILABLE(macos(10.13), ios(11.0), watchos(4.0), tvos(11.0))
@interface GoogLeNetPlacesOutput : NSObject

/// Probability of each scene as dictionary of strings to doubles
@property (readwrite, nonatomic) NSDictionary * sceneLabelProbs;

/// Most likely scene label as string value
@property (readwrite, nonatomic) NSString * sceneLabel;
- (instancetype)init NS_UNAVAILABLE;
- (instancetype)initWithSceneLabelProbs:(NSDictionary *)sceneLabelProbs sceneLabel:(NSString *)sceneLabel;
@end

/// Class for model loading and prediction
API_AVAILABLE(macos(10.13), ios(11.0), watchos(4.0), tvos(11.0))
@interface GoogLeNetPlaces : NSObject
@property (readonly, nonatomic, nullable) MLModel * model;
- (nullable instancetype)initWithContentsOfURL:(NSURL *)url error:(NSError * _Nullable * _Nullable)error;

/**
    Make a prediction using the standard interface
    @param input an instance of GoogLeNetPlacesInput to predict from
    @param error If an error occurs, upon return contains an NSError object that describes the problem. If you are not interested in possible errors, pass in NULL.
    @return the prediction as GoogLeNetPlacesOutput
*/
- (nullable GoogLeNetPlacesOutput *)predictionFromFeatures:(GoogLeNetPlacesInput *)input error:(NSError * _Nullable * _Nullable)error;

/**
    Make a prediction using the convenience interface
    @param sceneImage Input image of scene to be classified as color (kCVPixelFormatType_32BGRA) image buffer, 224 pixels wide by 224 pixels high:
    @param error If an error occurs, upon return contains an NSError object that describes the problem. If you are not interested in possible errors, pass in NULL.
    @return the prediction as GoogLeNetPlacesOutput
*/
- (nullable GoogLeNetPlacesOutput *)predictionFromSceneImage:(CVPixelBufferRef)sceneImage error:(NSError * _Nullable * _Nullable)error;
@end

NS_ASSUME_NONNULL_END

若要使用此model提供的功能，就import GoogLeNetPlaces.h。

使用image (224×224)當作Input。

-(void)viewWillAppear:(BOOL)animated
{
    [super viewWillAppear:animated];

    googLeNetPlacesModel = [[GoogLeNetPlaces alloc] init];
}

-(void)predictPhoto:(UIImage *)image
{
    NSError *error;
    CVPixelBufferRef pixelbuffer = [self pixelBufferFromCGImage:image.CGImage];
    GoogLeNetPlacesOutput *output = [googLeNetPlacesModel predictionFromSceneImage:pixelbuffer error:&error];

    NSString *outputStr = [NSString stringWithFormat:@"結果：%@", output.sceneLabel];
    statementLabel.text = outputStr;
    NSLog(@"output: %@", output.sceneLabel);
    NSLog(@"output detail: %@", output.sceneLabelProbs);
}

程式運行流程如圖：

編譯後執行程式，Demo圖如：

相片是我2/25和七位美女去武陵農場賞櫻。參考遊記武陵農場賞櫻、武陵農場賞櫻2。

輸出資訊範例：

2018-03-19 17:45:46.857755+0800 TestCoreML[54977:15240983] output: botanical_garden

2018-03-19 17:45:46.863412+0800 TestCoreML[54977:15240983] output detail: {

    abbey = “0.0002244384813820943″;

    “airport_terminal" = “5.5498608162452e-06″;

    alley = “0.000172614207258448″;

    amphitheater = “0.0007163180271163583″;

    “amusement_park" = “0.002700716489925981″;

    “apartment_building/outdoor" = “7.044187805149704e-05″;

    aquarium = “0.0001515663170721382″;

..

    ballroom = “0.0002574142999947071″;

    “bamboo_forest" = “0.0009810042101889849″;

    “banquet_hall" = “0.001488566980697215″;

    bar = “5.034253626945429e-05″;

    “baseball_field" = “5.577856791205704e-05″;

    basement = “2.673672133823857e-05″;

    basilica = “6.066541391192004e-05″;

    bayou = “0.000220770132727921″;

    “beauty_salon" = “6.698675861116499e-05″;

    bedroom = “0.0001197672245325521″;

    boardwalk = “0.001099746441468596″;

    “boat_deck" = “2.295863669132814e-05″;

    bookstore = “0.0003391916106920689″;

    “botanical_garden" = “0.3794905245304108″;

預測是植物園 (botanical garden)，看來還頗準確！？數值越大者表示預測越偏向該category。

可以看到列印出來的分類很長，因為此model訓練205種catogory。

可參考我放在Github上的原始專案：https://github.com/happymanx/CoreMLTest

官方教學：2017 WWDC – Introducing Core ML

Machine learning opens up opportunities for creating new and engaging experiences. Core ML is a new framework which you can use to easily integrate machine learning models into your app. See how Xcode and Core ML can help you make your app more intelligent with just a few lines of code.

最後不過卻是最重要的，就是了解我們使用Core ML在架構中所扮演的角色：

參考：

• Core ML 與Vision：iOS 11 機器學習教程
• Core ML and Vision: Machine Learning in iOS 11 Tutorial
• Google 與蘋果達成合作：TensorFlow Lite 將支援 Core ML
• Core ML Tools初學者指南：如何將Caffe模型轉換為Core ML格式
• 初探 Core ML：學習建立一個圖像識別 App