17.0

In July, 2024, Help Lightning released 17.0 of its server and client SDKs. This brings in several major new features including:

  • Session Support
  • Screen Sharing
  • Smart Source Selection
  • Enhanced Audio Controls

Server Changes

The RESTful API has several new APIs and no deprecations. The biggest API change, is that the RESTful API now supports creating and managing Workboxes.

Workbox APIs

As a user (directly or via impersonation), it is now possible to create and interact with workboxes:

An Administrator can also get a list of all the workboxes in their workspace:

For any individual workbox, you can use the token in the result and call the above Workbox methods to interact with it.

New Webhooks

A new webhook type for workbox has been created. This allows listening in real time for when workboxes are created or closed. Please see the Webhook documentation for more details.

Client SDK Changes

There have been new client SDK releases for the following platforms:

  • Web
  • Android
  • iOS
  • Xamarin

The SDKs include several new features, including:

  • Improved audio controls
  • Screen Sharing
  • Multiple images supported for quick knowledge merge overlay
  • SVG support for quick knowledge merge overlay
  • Miniview support

Web SDK

The 17.0 SDK for the web is versioned as 5.14.x with the latest version being 5.14.2.

Include Changes

It is no longer correct to include the opencv.js.

    <!-- Load help lightning JS SDK -->
    <script src="https://code.jquery.com/jquery-3.6.0.min.js"></script>
    <script src="https://helplightning.net/sdk/5.14.2/helplightning.min.js"></script>
    <script src="https://helplightning.net/sdk/5.14.2/opentok-enterprise/opentok.min.js"></script>
    <script src="https://helplightning.net/sdk/5.14.2/pdf_viewer/build/pdf.min.js"></script>

Delegate Changes

  • onActiveSessionsChanged

This delegate method is called when the active sessions change. It returns the latest active sessions.

{
    'sessions': Session[]
}
  • onWillJoinCall

This delegate method is called when the user is about to join a call.

{
    'sessionId': String
}
  • setMinimizingView

This delegate method is called when the user asks to change the minimizing view state.

{
    'minimize': Boolean
}

Icon Changes

The following are new icons since the 16.4 SDK release.
IMAGE_CAMERAMENU_CAMERA_ON_NO_HIGHLIGHT
IMAGE_CAMERAMENU_CAMERA_ON_HIGHLIGHT
IMAGE_SHARE_MENU_SCREEN_SHARING
IMAGE_SHARE_MENU_FACE_TO_FACE
IMAGE_SHARE_MENU_FACE_TO_FACE_SELECTED
IMAGE_SHARE_MENU_SCREEN_SHARING_SELECTED

iOS SDK

The 17.0 SDK for iOS is versioned as 17.0.x with the latest version being 17.0.3.

New features:

  • Improved audio controls
  • Screen Sharing
  • Multiple images supported for quick knowledge merge overlay
  • SVG support for quick knowledge merge overlay
  • Miniview support
  • Merged Reality+

Screen Sharing

Here is a tutorial to add the screen sharing feature to an app, using Help Lightning iOS SDK:

iOS Screen Sharing Tutorial

New Delegate Methods in HLClientDelegate

The following optional delegate methods are added to HLClientDelegate

/**
 The dictionary key for the app group name of the screen sharing
 */
extern NSString* const kHLCallPluginScreenSharingAppGroupName;

/**
 The dictionary key for the bundle id of the screen sharing broadcast extension
 */
extern NSString* const kHLCallPluginScreenSharingBroadcastExtensionBundleId;

/// The call needs the screen sharing information to start screen sharing
/// - Parameter call: the current call
/// - Returns: A dictionary with the screen sharing information, which should include ``kHLCallPluginScreenSharingAppGroupName`` and ``kHLCallPluginScreenSharingBroadcastExtensionBundleId``. If the dictionary is empty, the call will not show the screen sharing option in the share menu.
- (NSDictionary<NSString*, id>*)hlCallNeedScreenSharingInfo:(id<HLGenericCall>)call;

New Theme Images

extern NSString *const kHLImageShareMenuScreenSharing;
extern NSString *const kHLImageShareMenuScreenSharingSelected;
extern NSString *const kHLImageShareMenuFaceToFace;
extern NSString *const kHLImageShareMenuFaceToFaceSelected;

New HLCall API

@interface HLCall : NSObject <HLGenericCall>

// Set this new property to YES to allow Merge Reality Plus. Set it to NO to disable Merge Reality Plus
// The default value is YES.
@property (nonatomic) BOOL mergeRealityPlusAllowed;

Minimize Call View

Here are the new delegates to support minimize a call view.

/// Determine if the app supports to minimize the call view
/// - Parameter call: the current ongoging call
/// - Parameter call: the information to minimize the call view. The available keys are kHLCallPluginWorkspaceId, kHLCallPluginCallWorkboxId, kHLCallInfoSessionID, kHLCallPluginCallWorkboxId, kHLCallPluginCallContentView. But these keys are optional
/// - Returns: a Promise which should be fulfilled with ``@YES`` when the app supports to minimize the call view. If the promise is fulfilled with ``@NO``, the call will not show the call minimize button on the action bar.
- (FBLPromise*) hlCall:(id<HLGenericCall>)call canMinimizeCallViewWithCallInfo:(NSDictionary<NSString*, id>*)callInfo;

/// Request the app minimize the call view
/// - Parameter call: the current ongoging call
/// - Parameter callInfo: the information to minimize the call view. The available keys are kHLCallInfoPresentingViewController,  kHLCallPluginWorkspaceId, kHLCallPluginCallWorkboxId, kHLCallInfoSessionID, kHLCallPluginCallWorkboxId, kHLCallPluginCallContentView. But these keys are optional
/// - Returns: a Promise which should be fulfilled when the app minimize the call. If the promise is rejected, the app fails to minimize the call view.
- (FBLPromise*) hlCall:(id<HLGenericCall>)call didMinimizeCallViewWithCallInfo:(NSDictionary<NSString*, id>*)callInfo;

/// will restore minimize the call view
/// - Parameter call: the current ongoging call
/// - Parameter callInfo: the information to minimize the call view. The available keys are kHLCallInfoPresentingViewController,  kHLCallPluginWorkspaceId, kHLCallPluginCallWorkboxId, kHLCallInfoSessionID, kHLCallPluginCallWorkboxId, kHLCallPluginCallContentView. But these keys are optional
/// - Returns: a Promise which should be fulfilled when the app minimize the call. If the promise is rejected, the app fails to minimize the call view.
- (FBLPromise*) hlCall:(id<HLGenericCall>)call willRestoreCallViewWithCallInfo:(NSDictionary<NSString*, id>*)callInfo;

/// did restore minimize the call view
/// - Parameter call: the current ongoging call
/// - Parameter callInfo: the information to minimize the call view. The available keys are kHLCallInfoPresentingViewController,  kHLCallPluginWorkspaceId, kHLCallPluginCallWorkboxId, kHLCallInfoSessionID, kHLCallPluginCallWorkboxId, kHLCallPluginCallContentView. But these keys are optional
- (void) hlCall:(id<HLGenericCall>)call didRestoreCallViewWithCallInfo:(NSDictionary<NSString*, id>*)callInfo;

Capture Mode Delegate

/// a call did capture a photo
/// - Parameters:
///   - call: the call
///   - imageData: the image data
///   - mimeType: the mime type of the image data
- (void) hlCall:(id<HLGenericCall>)call didCaptureImage:(NSData*)imageData mimeType:(NSString*)mimeType;

Android SDK

The 17.0 SDK for Android is versioned as 3.0.x with the latest version being 3.0.2.

New features:

  • Improved audio controls
  • Screen Sharing
  • Multiple images supported for quick knowledge merge overlay
  • SVG support for quick knowledge merge overlay
  • Miniview support

Updated Key Libraries

Several libraries have been updated in this release including:

  • org.jetbrains.kotlin:kotlin-gradle-plugin (1.7.22 -> 1.9.22)
  • com.android.tools.build:gradle (7.3.1 -> 8.3.2)
  • gradle (7.4 -> 8.4)

To support these updates, you might need to make the following changes to your app’s build.gradle file. Add plugins:

apply plugin: 'kotlin-android'
apply plugin: 'kotlin-kapt'

Specify jvmToolchain:

kotlin {
    jvmToolchain 17
}

Add the following to the android block:

    kotlinOptions {
        freeCompilerArgs = ["-Xstring-concat=inline"]
    }

    namespace 'com.example.project'

    buildFeatures {
        buildConfig true
    }

HLCall updates

New Features:

  • Introduced the HLCall.Builder class for constructing HLCall objects, enabling a more flexible and readable way to create instances.
  • Added a new configuration option, startInMiniView, allowing calls to start in mini-view mode.

Deprecated:

  • Marked the existing HLCall constructors as deprecated. Users are encouraged to use the new HLCall.Builder class for constructing HLCall objects.

Constructor Changes:

  • The deprecated constructors now include the new startInMiniView parameter to maintain backward compatibility.

Builder Class Details:

  • The HLCall.Builder class supports the following methods for setting parameters:
    • sessionId(String sessionId)
    • sessionToken(String sessionToken)
    • userToken(String userToken)
    • gssUrl(String gssUrl)
    • localUserDisplayName(String localUserDisplayName)
    • localUserAvatarUrl(String localUserAvatarUrl) (Pass an empty string to generate an avatar with initials and a random color)
    • helplightningAPIKey(String helplightningAPIKey)
    • autoEnableCamera(boolean autoEnableCamera)
    • autoEnableMicrophone(boolean autoEnableMicrophone)
    • autoEnableAudioPlusMode(boolean autoEnableAudioPlusMode)
    • startInMiniView(boolean startInMiniView)
  • To create an HLCall instance, use the build() method.

Example Usage:

HLCall call = new HLCall.Builder()
    .sessionId("your-session-id")
    .sessionToken("your-session-token")
    .userToken("your-user-token")
    .gssUrl("your-gss-url")
    .localUserDisplayName("your-display-name")
    .localUserAvatarUrl("your-avatar-url")
    .helplightningAPIKey("your-api-key")
    .autoEnableCamera(true)
    .autoEnableMicrophone(true)
    .autoEnableAudioPlusMode(false)
    .startInMiniView(false)
    .build();

Please update your code to use the new HLCall.Builder class and benefit from the new configuration options and improved code readability.

Mini View

The mini view feature allows users to minimize an active call to continue using the rest of the app. While the call is minimized, camera and microphone continue will continue operating as if the user were still in the call Activity. Follow the mini view tutorial to learn how to use the mini view feature.

Xamarin SDK

The 17.0 SDK for iOS is versioned as 17.0.x with the latest version being 17.0.1.

New features:

  • Improved audio controls
  • Screen Sharing
  • Multiple images supported for quick knowledge merge overlay
  • SVG support for quick knowledge merge overlay
  • Miniview support
  • Merged Reality+ (iOS Only)

Screen Sharing on iOS

Please see the full tutorial to setup screen sharing

New constants for screen sharing

public static class CallClientDelegateConstants {
    ...
    public const string BroadcastExtensionBundleId = "BroadcastExtensionBundleId";
    public const string BroadcastExtensionAppGroupName = "BroadcastExtensionAppGroupName";
}

New Delegates

The following APIs are added to HelpLightning.SDK.ICallClientDelegate to support screen sharing on iOS:

/// <summary>
/// The main app should implement this method to provide information to enable screen sharing
/// </summary>
/// <param name="call">The current call</param>
/// <returns>
/// The delegate method should return a dictionary with two entries:
/// The value of <see cref="CallClientDelegateConstants.BroadcastExtensionAppGroupName"/> should be the App Group for the broadcast extension.
/// The value of <see cref="CallClientDelegateConstants..BroadcastExtensionBundleId"/> should be the broadcast extension's bundle id.
/// </returns>
IDictionary<string, object> CallNeedScreenSharingInfo(Call call);

Minimize Call View on iOS

The following APIs are added to HelpLightning.SDK.ICallClientDelegate to minimize call view on iOS:

/// <summary>
/// Determine if the app supports to minimize the call view
/// </summary>
/// <param name="call">The current ongoging call</param>
/// <param name="callInfo">The information to minimize the call view.</param>
/// <returns>A task which should be fulfilled with true when the app supports to minimize the call view.
/// If the task is fulfilled with false, the call will not show the call minimize button on the action bar.</returns>
Task<bool> CallCanMinimizeCallViewWithCallInfo(Call call, IDictionary<string, object> callInfo);

/// <summary>
/// Request the app to show the minimized call view
/// </summary>
/// <param name="call">The current ongoging call</param>
/// <param name="callInfo">The information to minimize the call view. App can get the mini call view from the key: CallPluginCallContentView</param>
/// <returns>A task which should be fulfilled when the app minimizes the call. If the app fails to minimize the call view, the promise shoud be rejected.</returns>
Task<bool> CallDidMinimizeCallViewWithCallInfo(Call call, IDictionary<string, object> callInfo);

/// <summary>
/// Notify the app when the full call view will be restored. The mini call view should be removed from the UI hierachy.
/// </summary>
/// <param name="call">The current ongoging call</param>
/// <param name="callInfo">The information to minimize the call view. App can get the mini call view from the key: CallPluginCallContentView</param>
/// <returns>A task which should be fulfilled when the app remove the mini callview. If the app fails to remove the mini call view, the promise shoud be rejected.</returns>
Task<bool> CallWillRestoreCallViewWithCallInfo(Call call, IDictionary<string, object> callInfo);

/// <summary>
/// The call view has been restored from the mini call view.
/// </summary>
/// <param name="call">The current ongoging call</param>
/// <param name="callInfo">The information to minimize the call view</param>
void CallDidRestoreCallViewWithCallInfo(Call call, IDictionary<string, object> callInfo);

/// <summary>
/// When a photo is captured from the call.
/// </summary>
/// <param name="call">The current ongoging call</param>
/// <param name="imageData">The image data. The data generally can be used to create an UIImage instance</param>
/// <param name="mimeType">The image type</param>
void CallDidCaptureImage(Call call, byte[] imageData, string mimeType);

Merge Reality Plus on iOS

The following APIs are added to HelpLightning.SDK.Call to support Merged Reality+ on iOS:

/// <summary>
/// The flag to Allow Merge Reality+ in the call. The default value is true. This flag only works on iOS
/// </summary>
/// <remarks>
/// If true, allow Merge Reality+ in the call
/// If false, disable Merge Reality+ in the call.
/// </remarks>
/// <value>Automatically enable microphone</value>
public bool MergeRealityPlusAllowed { get; }

New icons/images for iOS

//17.0 iOS Icons
public const string ImageShareMenuScreenSharingIOS = "image_share_menu_screen_sharing";
public const string ImageShareMenuScreenSharingSelectedIOS = "image_share_menu_screen_sharing_selected";
public const string ImageShareMenuFaceToFaceIOS = "image_share_menu_face_to_face";
public const string ImageShareMenuFaceToFaceSelectedIOS = "image_share_menu_face_to_face_selected";

Xamarin Android SDK Setting

  • Upgrade Android Target SDK Version to 34
  • Upgrade Android Minimum SDK version to 24

Minimize Call View on Android

The following APIs are added to HelpLightning.SDK.ICallClientDelegate and HelpLightning.SDK.ICallClient to minimize call view on Android:

/// <summary>
/// Check if the app can support to minimized a call view
/// </summary>
/// <param name="call">the call information</param>
/// <returns>true if the app supports to minimize the call view. </returns>
bool IsMinimizeCallEnabled(Call call);
AndroidX.Lifecycle.LiveData GetMinimizedCall();

void ReturnToActiveCall();

void EndActiveCall(AndroidX.Fragment.App.FragmentActivity context);

bool IsCallMinimized();

SampleXamarin/SampleAndroid in the sample repo include the update to support call view minimization: Here are the key codes:

public override View OnCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState)
{
    ...

    CallClientFactory.Instance.CallClient.GetMinimizedCall().Observe(ViewLifecycleOwner, new MiniCallViewObserver(this));
...
}

private class OngoingCallListener : Java.Lang.Object, IOngoingCallListener
{
    JoinSession self;
    internal OngoingCallListener(JoinSession self)
    {
        this.self = self;
    }
    public void OnEndCall()
    {
        CallClientFactory.Instance.CallClient.EndActiveCall(self.Activity);
    }
     public void OnMaximizeCall()
    {
        CallClientFactory.Instance.CallClient.ReturnToActiveCall();
    }
}

private class MiniCallViewObserver : Java.Lang.Object, IObserver
{
    JoinSession self;
    internal MiniCallViewObserver(JoinSession self)
    {
        this.self = self;
    }

    public void OnChanged(Java.Lang.Object value)
    {
        if (value != null && value is OngoingCallInfo callInfo)
        {
            self.ongoingCallMiniView.SetCallInfo(callInfo.MiniView, callInfo.CallTitle);
        }
        else
        {
            self.ongoingCallMiniView.SetCallInfo(null, null);
        }
    }
}