Introduction to mediastreamer2 concepts.

Introduction. More...

Modules
	Howto 1: build a sample audio graph.

---

Detailed Description

Introduction.

What is mediastreamer2

mediastreamer2 is a powerful engine to make audio and video streams. mediastreamer2 is GPL (COPYING). Please understand the licencing details before using it!

Commercial support and licensing is provided by Belledonne Communications http://www.belledonne-communications.com

See also:

http://www.linphone.org/eng/documentation/dev/mediastreamer2.html

Some definitions.

Filter: A filter is a mediastreamer2 component that process data. A filter have 0 or several INPUT pins and 0 or several OUTPUT pins. Here is a list of possible use of filters:

   capture audio or video data.
   play audio or display video data.
   send or receive RTP data.
   encode or decode audio or video data.
   transform (resize video, resample audio...) data. 
   duplicate any kind of data.
   mix audio/video data.
 

Graph: A graph is a manager of filters connected together. It will transfer data from OUTPUT pins to INPUT pins and will be responsible for running filters.

How do I use mediastremer2?

Mediastreamer2 can be used for a lot of different purpose. The primary use is to manage RTP audio and video session. You will need to use the API to build filters, link them together in a graph. Then the ticker API will help you to start and stop the graph.

Basic graph sample:

  AUDIO CAPTURE   -->   ENCODE  -->     RTP
      FILTER      -->   FILTER  -->    FILTER
 

The above graph is composed of three filters. The first one has no input: tt captures audio data directly from the drivers and provide it to the OUTPUT pin. This data is sent to the INPUT pin of the encoder which of course encode the data and send it to its OUTPUT pin. This pin is connected to the INPUT pin of a filter capable to build and send RTP packets.

The modular design helps you to encode in many different format just by replacing the "ENCODE FILTER" with another one. mediastreamer2 contains internal support for g711u, g711a, speex and gsm. You can add new encoding format by implementing new filters which can then be dynamically loaded.

List of existing filters.

mediastreamer2 already provides a large set of filters. Here is a complete list of built-in filters.

 All supported platforms:
   RTP receiver
   RTP sender
   tee (duplicate data)

 Audio Filters:
   audio capture
   audio playback
   mme API (windows)
   alsa API (linux)
   oss API (linux)
   arts API (linux)
   portaudio API (macosx and other)
   iounit API (macosx and ios)
   Android sound
   macsnd API (native macosx API -please do more testing...-)

   several audio encoder/decoder: PCMU, PCMA, speex, gsm, G722
   wav file reader.
   wav file recorder.
   resampler.
   conference bridge.
   volume analyser.
   acoustic echo canceller.
   dtmf generation filter.

 Video Filters:
   video capture
     v4w API (windows)
     directshow API (windows)
     video4linux API (linux)
     Quicktime API (MACOSX)
     AV Foundation API (IOS)
     android.hardware.Camera API (Android) 
   video display
     v4w API (windows)
     SDL API (linux, macosx...)
     XV API (linux)
     OPENGL-ES (Android, IOS)
   several audio encoder/decoder: H263-1998, MP4V-ES, VP8, theora
     image resizer.
     format converter. (RBG24, I420...)

 Plugin Filters:
  iLBC decoder/encoder.
  AMR-NB decoder/encoder.
  SILK decoder/encoder.
  H264 decoder/encoder.