Quick Start with Mesibo Video Conferencing API for Android

• Introduction
• Create a Room
• Android
• iOS
• Javascript
• Moderation
• Open-Source Apps

Prerequisites

Before we dive into the various concepts and APIs for conferencing & streaming, you need to go through the following to familiarize yourself with mesibo APIs.

• Get Started Guide
• Tutorial on Writing your First mesibo Enabled Application
• Familiar with Real-time Group Management and Group Messaging APIs.
• Basic familiarity with Mesibo Call APIs
• It is also recommended that you refer to the source code of our open-source Android Conferencing App to quickly learn how each API is used.

Basic terminology

In this document, we will use the following terms

• Conference - Group Call, Webinar, Live streaming, etc. For this document, we will refer to all the use cases as Conference.
• Room - The group used to hold the conference. A conference room is a mesibo group. The participants are members of this group.
• Participant - A user participating in the conference. A participant can be a publisher, subscriber, or both.
• Host - The participant who created the room. The host controls who can join and participate in the room, by setting permissions.
• Stream - Video (camera, screen, etc) and Voice data.
• Publishing - Sending video/voice stream to the conference. A participant can publish any number of streams. For example, they can stream from their camera and share their screen, they can also stream from multiple cameras to build a 360 view!
• Subscribing - Receiving streams of other participants
• Publisher - The participant sending voice/video streams
• Subscriber - The participant listening to other participant’s voice or viewing video streams

Development Setup for Android

1. Start Android Studio.
2. Create a new project with a minimum SDK version 16 or higher.
3. Android Studio starts Gradle and builds your project. This may take a few seconds. For more information about creating a project in Android Studio, see https://developer.android.com/training/basics/firstapp/creating-project.html
4. Add the following dependencies in the /app/build.gradle file of your project and perform gradle sync as explained in Installation Instructions.

   ...
   dependencies {
    ...	
    implementation "com.mesibo.api:calls:${project.MesiboVersion_Calls}"
    ...
   }
   

5. You can then import Mesibo Call API as follows.

   import com.mesibo.calls.api.MesiboCall;
   

Quick Start with Conferencing API for Android

As explained before, to create a conference room, the host needs to create a mesibo group and add participants as members using the Group Management APIs. Once a room is created, participants can join the room, publish and view streams using the Conferencing APIs available for Android, iOS, or Javascript.

In this section, we will learn how to use mesibo conferencing APIs for Android, to perform typical operations in a conference room such as joining conference rooms, publishing streams, viewing streams, etc. By the end of this document, you will be ready to build a fully functional conferencing app.

We recommend that you go through the source code for the open-source conferencing app at https://github.com/mesibo/conferencing to learn more.

Follow the steps below to create a typical conferencing/streaming app in Android:

Create a Room and add Participants

You need to Create a Room and add participants using Group Management APIs.

For example, to create a basic meeting app with four participants - Alice, Bob, Carol and Dave, Create a mesibo group, Office Meeting and add four users Alice, Bob, Carol and Dave, as members to this group.

Set permissions for Alice and Bob to publish and view streams. Set permissions for Carol and Dave to only view streams. You can learn more about adding participants and setting permissions in the previous section.

Once you create the room, the Participants - Alice, Bob, Carol, and Dave can join through mobile or web apps using the Conferencing APIs.

Create a Group Call Object

The first step in joining a group call is to create a Group Call object - An instance of MesiboGroupCall. This global object represents the conference and holds all the participants & streams in the room.

// Create a group call object
MesiboCall.MesiboGroupCall mGroupcall = 
MesiboCall.getInstance().groupCall((MesiboCallActivity) getActivity(), mGid);

You need to use this object to join a room and create publishers as explained in the upcoming steps.

Implement GroupCallListener

You will need to implement GroupCallListener and pass it to the Group Call object which you created in the step above so that it can notify you when other participant joins and leaves the room by calling MesiboGroupcall_OnPublisher. It also notifies you when a participant subscribes to your video or voice streams through MesiboGroupcall_OnSubscriber.

How to get Publishers

When you join the room, you will start getting a list of publishers through MesiboGroupcall_OnPublisher. This listener will be called for each publisher in the room and also when publishers join later. Once you know who all are publishing in the room, you can subscribe to their stream.

Whenever someone starts or stops publishing a stream, mesibo will call MesiboGroupcall_OnPublisher.

public void MesiboGroupcall_OnPublisher(MesiboParticipant participant, boolean joined) {
  if (joined) {
    // call the publisher to view their stream
    // Add participant to list of publishers
  } else {
    // Remove participant from list of publishers
  }
}

How to get Subscribers

When you start publishing your stream to the group, everyone in the group will be notified about your stream through MesiboGroupcall_OnPublisher.

When anyone subscribes to your stream, you will be notified through MesiboGroupcall_OnSubscriber. You will get the information about the subscriber like name, address, etc. which you can use for typical book-keeping.

Whenever someone starts or stops subscribing to your stream, mesibo will call MesiboGroupcall_OnSubscriber.

public void MesiboGroupcall_OnSubscriber(MesiboCall.MesiboParticipant participant, boolean joined) {
  // This participant has subscribed to your stream
}

To join a conference, you need to implement GroupCallListener and pass it to the join method as explained in the next step.

Join a Room

To join the room, use the join method from the Group Call object that you created before.

For example, If you implement the GroupCallListener in your fragment like this

public class GroupCallFragment extends Fragment implements MesiboCall.GroupCallListener{

    @Override
    public void MesiboGroupcall_OnPublisher(MesiboCall.MesiboParticipant participant, boolean joined) {
    ...
    }

    @Override
    public void MesiboGroupcall_OnSubscriber(MesiboCall.MesiboParticipant p, boolean joined) {
        ...
    }

...
}

then, you can call join in the context of “this” fragment like this

mGroupCall.join(this);

If you have implemented the GroupCallListener in a separate class like this

public class MyGroupCallListener implements MesiboCall.GroupCallListener {

    @Override
    public void MesiboGroupcall_OnPublisher(MesiboCall.MesiboParticipant participant, boolean joined) {
    ...
    }

    @Override
    public void MesiboGroupcall_OnSubscriber(MesiboCall.MesiboParticipant p, boolean joined) {
        ...
    }

 ...
}

then, you can create an instance of the listener and pass it like this

MyGroupCallListener listener = new MyGroupCallListener();
mGroupCall.join(listener);

Subscribe to a stream using a Participant Object

To subscribe to a stream, you need a Participant object received through the MesiboGroupcall_OnPublisher listener.

Every stream in the conference is represented by a Participant object - An instance of MesiboParticipant. A Participant object contains details about the stream such as the name and address of the publisher, audio & video properties, mute status, talk status, etc. To perform any operation on the stream such as subscribing to a stream, publishing a stream, muting the stream, etc. you need to use the corresponding Participant object.

You will get the Participant object of publishers through MesiboGroupcall_OnPublisher. To subscribe to a publisher, you need to use the call method from the Participant object.

The call method similar to the P2P Mesibo Call API takes the following parameters:

• audio, true if you would like to receive the audio stream, false otherwise
• video, true if you would like to receive the video stream, false otherwise
• groupCallInProgressListener, An instance of GroupCallInProgressListener

So, to only listen to the audio

participant.call(true, false, this);

To view them with both video and audio,

participant.call(true, true, this);

Note that, the publisher should be publishing the corresponding video or audio stream. For example, if the publisher is publishing only the audio stream, invoking call with video parameters will not yield any video stream.

You will need to implement a GroupCallInProgressListener and pass it as a parameter to the call method to listen to Participant events such as when they mute, start talking, etc. Once you subscribe to a publisher, you will receive audio and video streams and you can display/view them.

public void MesiboGroupcall_OnPublisher(MesiboParticipant participant, boolean joined) {
  if (joined) {
    // call the publisher to subscribe to their stream
    // "this" is a GroupCallInProgressListener 
    participant.call(true, true , this);
  }
}

Implement GroupCallInProgressListener

Mesibo informs you of various participant events so that you can easily update your App UI or take necessary action, such as,

• When you make a call using the call method and you are connected, MesiboGroupcall_OnConnected will be called.
• Once you are connected, you will be notified when you receive the video stream through the listener MesiboGroupcall_OnVideo or the audio stream through MesiboGroupcall_OnAudio
• When a publisher mutes audio or video, you will be notified through MesiboGroupcall_OnMute
• When a publisher starts talking, MesiboGroupcall_OnTalking will be called (Talk Detection).
• When a publisher changes video source (say, to another camera or the screen), MesiboGroupcall_OnVideoSourceChanged will be called.
• When a publisher leaves the conference, MesiboGroupcall_OnHangup will be called

While subscribing to a stream using the call method, you need to pass an instance of GroupCallInProgressListener.

The following are important participant events that you need to handle, by implementing the GroupCallInProgressListener. For the complete list of events, refer to the reference.

When the call gets connected

MesiboGroupcall_OnConnected is called when the call to a publisher is connected, reconnected, or disconnected. Note that, you will receive video and audio streams only after getting connected.

public void MesiboGroupcall_OnConnected(MesiboCall.MesiboParticipant participant, 
  boolean connected) {
    if(connected){
        // Participant is connected or reconnected
    } else {
        // Participant has been disconnected
    }
}

When you receive the video stream

MesiboGroupcall_OnVideo is called when you receive a video from the publisher you have subscribed to. You can now display the stream. It is also called when any properties of the video change. For example, if there are changes in aspect ratio, resolution, orientation (say portrait to landscape), etc. You can accordingly update the UI to display the video or reflect changes.

public void MesiboGroupcall_OnVideo(MesiboCall.MesiboParticipant participant, 
  float aspect_ratio, boolean available) {
    // Display the video
}

When you receive the audio stream

MesiboGroupcall_OnAudio is called when you receive audio from the participant.

public void MesiboGroupcall_OnAudio(MesiboCall.MesiboParticipant participant) {
    // Receiving Audio
}

When publisher mutes audio or video

MesiboGroupcall_OnMute is called when the publisher mutes audio or video.

public void MesiboGroupcall_OnMute(MesiboCall.MesiboParticipant participant, 
  boolean audioMuted, boolean videoMuted) {
    
    if(remote){
	    // Remote participant has muted
    }

    // Check mute status
    if(audioMuted){
        // Audio Muted
    }

    if(videoMuted){
        // Video Muted
    }
}

You can also use the getMuteStatus method to know about the current mute status of a participant.

// Pass true to get video mute status, false for audio mute status

participant.getMuteStatus(true); // Get Audio Mute Status
participant.getMuteStatus(false); // Get Video Mute Status

You can also mute video and audio locally, for the streams that you are viewing. You can use the toggleAudioMute method to toggle the audio and the toggleVideoMute method to toggle the video of a stream.

For example,

participant.toggleVideoMute();
participant.toggleAudioMute();

If you mute a stream that you are publishing (you created the Participant object), other participants who are subscribing to your stream, will be notified of it through MesiboGroupcall_OnMute

When a publisher starts talking (Talk Detection)

Mesibo let you know when the publisher is talking. MesiboGroupcall_OnTalking is called when the participant starts or stops talking.

public void MesiboGroupcall_OnTalking(MesiboCall.MesiboParticipant participant, boolean talking) {
        if(talking){
                // If talking is true, participant started talking
                // if it is false, participant stopped talking
                // Handle talking. For example, Show a talking icon 
        }
    }

You can also use the isTalking method from the Participant object to check if the publisher is talking.

participant.isTalking();

When a publisher Hangs up

To hang up a stream, you need to use the hangup method from the Participant object.

participant.hangup()

When a publisher hangs up, those who are subscribing to that publisher will be notified of it through MesiboGroupcall_OnHangup

public void MesiboGroupcall_OnHangup(MesiboCall.MesiboParticipant p, int reason) {
        if(MESIBOCALL_HANGUP_REASON_REMOTE == reason){
                // Remote participant has hanged up
                // Cleanup
        }

The reason for hangup can be one of the following:

• MESIBOCALL_HANGUP_REASON_REMOTE, The remote participant hanged up
• MESIBOCALL_HANGUP_REASON_BACKGROUND, The application moved into the background
• MESIBOCALL_HANGUP_REASON_USER, The user hung up the participant
• MESIBOCALL_HANGUP_REASON_ERROR, An error occurred

Subscribing to multiple streams from a participant

A participant can publish as many streams as they wish, for example, multiple cameras, multiple audios, multiple screens, etc. If a participant is publishing multiple streams, MesiboGroupcall_OnPublisher will be called for each stream.

For example, let’s say we have a participant on mobile, sending three streams - one from the front camera, one from the rear camera, and sharing the screen simultaneously. In this case, MesiboGroupcall_OnPublisher will be called thrice with three different participant objects. You can use the call method from each of these participant objects to subscribe to each of these streams and display them.

public void MesiboGroupcall_OnPublisher(MesiboCall.MesiboParticipant participant, boolean joined) {
   if (joined) {
        // participant object for front cam 
        // or participant object for rear cam
        // or participant object for screen, ..etc.
        
        // subscribe to each stream
        participant.call(true, true, this); 
    } 
}

Publish a Stream by creating a Participant Object

In previous sections, we learned how to subscribe to streams from other publishers. In this section, we will learn about how to publish your stream so that others can subscribe to it.

As we explained before, every stream in the conference is a Participant object - An instance of MesiboParticipant. To publish a stream, you need to create a Participant object using the createPublisher method from the Group Call object. Then, use the call method to publish your stream. You also need to implement a GroupCallInProgressListener and pass it as a parameter to the call method, just like in the case of subscribing to a stream.

//Create a participant object
mParticipant = mGroupcall.createPublisher(0);

//Set the source. For example, camera
mParticipant.setVideoSource(MesiboCall.MESIBOCALL_VIDEOSOURCE_CAMERAFRONT, 0);

//Make a call. "this" is an instance of GroupCallInProgressListener 
mParticipant.call(true, true, this);

Once you publish, all other participants in the room will get your Participant object. A participant can publish multiple streams by creating multiple Participant objects.

Publishing Multiple Streams

You can create a Participant object using the createPublisher method from the Group Call Object. Each participant needs to have a unique sid or stream-id, which can be any random number(a long integer, up to 32 bits). A participant can publish multiple streams by creating multiple participant objects each with a different sid.

For example, if you want to publish three streams - one from the front camera, one from the rear camera and share the screen simultaneously, you need to create three participant objects.

MesiboParticipant mFrontCamera = mGroupcall.createPublisher(0);
MesiboParticipant mRearCamera = mGroupcall.createPublisher(1);
MesiboParticipant mScreen = mGroupcall.createPublisher(2);

Set the Video Source

A participant can publish streams from two types of video sources - camera or screen. You can specify the source, using the setVideoSource method which takes the following parameters:

• source, which can be one of the following:
  • MESIBOCALL_VIDEOSOURCE_CAMERADEFAULT Default camera to publish from
  • MESIBOCALL_VIDEOSOURCE_CAMERAFRONT to publish stream from the device’s front camera (on Mobile)
  • MESIBOCALL_VIDEOSOURCE_CAMERAREAR to publish stream from the device’s rear camera (on Mobile)
  • MESIBOCALL_VIDEOSOURCE_SCREEN to share live screen
• index, Camera index. If there are multiple(in addition to the front and rear) cameras on mobile, set the camera index.

For example,

mFrontCamera.setVideoSource(MesiboCall.MESIBOCALL_VIDEOSOURCE_CAMERAFRONT, 0);
mRearCamera.setVideoSource(MesiboCall.MESIBOCALL_VIDEOSOURCE_CAMERAREAR, 0);
mScreen.setVideoSource(MesiboCall.MESIBOCALL_VIDEOSOURCE_SCREEN, 0);

If you are not publishing a video, and you are only streaming audio, you need not set any source. You can change the video source anytime during the call.

Switching the Video Source

You can switch between the two video sources - camera and screen using the switchSource method For instance, if are publishing a camera stream, and you call switchSource on your Participant object, you will switch to streaming from the screen. This is useful when the participant is using a mobile app and at a given time they need to only publish either from the front camera or share the mobile screen.

Publishing a Stream

To publish your stream, use the call method from the Participant object. You need to pass a GroupCallInProgressListener as a parameter to it.

When you publish your stream, you will also receive your stream and MesiboGroupCall_OnVideo will be called with your Participant object. You can then display your stream. Refer to the section Implement GroupCallInProgressListener to learn more.

The call method takes the following parameters:

• audio, true if you would like to receive the audio stream, false otherwise
• video, true if you would like to receive the video stream, false otherwise
• groupCallInProgressListener, An instance of GroupCallInProgressListener

So, to publish only the audio

mParticipant.call(true, false, this);

To publish audio and video,

mParticipant.call(true, true, this);

Once you make a call, your Participant object is sent to the group. All other participants(group members) will receive your Participant object through MesiboGroupcall_OnPublisher on their end. If they subscribe to your stream ( using the call method), you will be notified of it through MesiboGroupcall_OnSubscriber.

Display Streams

Once you subscribe to a stream, you will receive the video and audio stream. When you receive the video, you will be notified by MesiboGroupCall_OnVideo and you can display the stream. When you receive the audio you will be notified by MesiboGroupCall_OnAudio

To display a video stream, you need a MesiboVideoView element.

Before, displaying the video it is a good practice to check if the stream has a video stream using the hasVideo method. Use the setVideoViewmethod from the Participant object, to display the stream in the UI element.

In Android, you can have a MesiboVideoView in your XML Layout file like below,

<com.mesibo.calls.api.MesiboVideoView
    android:id="@+id/participant_stream_view"
    android:layout_width="match_parent"
    android:layout_height="match_parent"/>

You can display the stream in this view, as follows:

if(participant.hasVideo()){
    MesiboVideoView videoView = findViewById(R.id.participant_stream_view);
    participant.setVideoView(videoView);
}

You can use various methods from MesiboVideoView, to control how the video is displayed. For example, use the enableMirror method to mirror the video, the scaleToFill method to fill video, etc. Refer to the documentation for MesiboVideoView to learn more.

Display multiple streams from a participant

Each stream published by a participant will have sid - the stream ID. The sid of a stream is arbitrary and is set by the publisher(using createPublisher). You can get the sid of a stream using the getSid method from the Participant object.

If a participant is publishing multiple streams, each stream will have a different sid. For example, let’s say we have a participant sharing three streams - two cameras and one screen simultaneously. Now, in this case, MesiboGroupcall_OnPublisher will be called thrice, with three different Participant objects for three streams, each stream with a different sid. If we subscribe to each of these streams then, we receive three different video or audio streams. To display each stream, call setVideoView for each Participant object.

Leave the room

A participant can leave the room anytime. All other participants(if they have subscribed to that participant) will be informed that the participant has hanged up through MesiboGroupcall_OnHangup.

mGroupcall.leave();

Open-Source App

Now that you have are familiar with all the basic conferencing APIs, you are ready to build a fully functional conferencing app using mesibo! To make it easier for you, we have created a fully-featured open-source Zoom-like conferencing app for mobile and web. Head over to the next section Open Source App to learn more.

Open Source App >>

API Reference

Mesibo conferencing APIs are unified APIs available for Android, iOS, and Javascript. Mesibo maintains the same API signature across all platforms. This makes it easier for you to create cross-platform apps, target multiple platforms, or port code from one platform to another.

The following is a list of all methods and functions which are part of the Conferencing API, used in this document.

MesiboGroupCall

The following are some of the methods in MesiboGroupCall class. Refer to the class reference for MesiboGroupCall for a complete list of methods and properties.

MesiboParticipant

The following are some of the methods in MesiboParticipant class. Refer to the class reference for MesiboParticipant for a complete list of methods and properties.

GroupCallListener

The following functions are part of the GroupCallListener class.

GroupCallInProgressListener

The following functions are part of the GroupCallInProgressListener listener class.

Also see:

• Quick Start - iOS
• Quick Start - Javascript
• Open Source App