Stream video and audio from A20SOM-EVB

arieedzig · November 09, 2014, 08:16:13 PM

Hello all,

I created IP camera using olimex A20SOM-EVB, you can follow the tutorial from this http://taqlim.blogspot.de/2014/11/video-and-audio-streaming-from-a20som.html
The problem is, I got about 1s delay for the streaming and sometimes not smooth video.
I can manage to have no delay on video by separating the ffmpeg for each device or using arecord on audio, but the sound is not sync with the video. Any one have experienced the same problem?
Can someone suggest a workaround?

Best regards

Punith · December 06, 2014, 01:29:22 PM

HI ArieedZig,

I tried by removing -vewait but not works giving fallowing log.

<6>[CSI]sensor initial success when csi open!
[ 257.293183] [CSI]sensor initial success when csi open!
<6>[CSI_ERR]buffer count is invalid, set to 5
[ 257.523242] [CSI_ERR]buffer count is invalid, set to 5
<6>[CSI]buffer_setup, buffer count=5, size=1382400
[ 257.532950] [CSI]buffer_setup, buffer count=5, size=1382400
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: 1417865292.674912, bitrate: 165888 kb/s
Stream #0:0: Video: rawvideo (NV12 / 0x3231564E), nv12, 1280x720, 165888 kb/s, 15 fps, 15 tbr, 1000k tbn, 1000k tbc
Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, alsa, from 'sysdefault:CARD=sunxicodec':
Duration: N/A, start: 1417865292.693301, bitrate: 1536 kb/s
Stream #1:0: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s
[matroska @ 0x1baab70] Invalid stream specifier: c.
Last message repeated 3 times
Codec AVOption qp (Constant quantization parameter rate control method) specified for output file #0 (pipe:) has not been used for any stream. .
Codec AVOption b (set bitrate (in bits/s)) specified for output file #0 (pipe:) has not been used for any stream. The most likely reason is eit.
[aac @ 0x1bac480] The encoder 'aac' is experimental but experimental codecs are not enabled, add '-strict -2' if you want to use it.
[lrc @ 0x278e460] Format lrc detected only with low score of 5, misdetection possible!
Input #0, lrc, from 'pipe:':
Duration: N/A, bitrate: N/A
Stream #0:0: Subtitle: text
Output #0, flv, to 'tcp://0.0.0.0:6666':
Metadata:
streamName : livestream
Output file #0 does not contain any stream

and also i tried by using script but not able to stream , in my case i am assigning static ip.

Regards
Punith

Punith · December 08, 2014, 09:11:20 AM

Hi Arieedzig,

Hi i did fallwoing code and it is showing audio is streaming but not heard on client side.
here my command and log:

Command:

ffmpeg -f v4l2 -s hd720 -pix_fmt nv12 -i /dev/video0 -f alsa -i sysdefault:CARD=sunxicodec -pix_fmt nv12 -qp 20 -c:v cedrus264 -b:v 300k -r 30 -ar 44.1k -b:c 128k -acodec libmp3lame -ab 96k -async 1 -f matroska - | ffmpeg -i - -c:a copy -c:v copy -f flv -metadata streamName=livestream tcp://0.0.0.0:6666

Log:
outpput #0, flv, to 'tcp://0.0.0.0:6666':
Metadata:
streamName : livestream
encoder : Lavf56.2.100
Stream #0:0: Video: h264 ([7][0][0][0] / 0x0007), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 1k tbn, 1k tbc (default)
Metadata:
ENCODER : Lavc56.0.101 cedrus264
Stream #0:1: Audio: mp3 ([2][0][0][0] / 0x0002), 44100 Hz, stereo, 128 kb/s (default)
Metadata:
ENCODER : Lavc56.0.101 libmp3lame
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
frame= 1847 fps= 15 q=-1.0 size= 14181kB time=00:02:01.02 bitrate= 959.9kbits/s

But on client side i am not getting any audio and also i tried fallowing command under root but it is not able to stream at all.

Command:
ffmpeg -f v4l2 -s hd720 -pix_fmt nv12 -i /dev/video0 -f alsa -i sysdefault:CARD=sunxicodec -pix_fmt nv12 -qp 20 -c:v cedrus264 -b:v 300k -r 30 -vewait 3600 -ar 44.1k -b:c 128k -c:a aac -strict -2 -f matroska - | ffmpeg -i - -c:a copy -c:v copy -f flv -metadata streamName=livestream tcp://0.0.0.0:6666

Log:
<6>[CSI]sensor initial success when csi open!
[ 6870.532746] [CSI]sensor initial success when csi open!
<6>[CSI_ERR]buffer count is invalid, set to 5
[ 6870.772818] [CSI_ERR]buffer count is invalid, set to 5
<6>[CSI]buffer_setup, buffer count=5, size=1382400
[ 6870.782524] [CSI]buffer_setup, buffer count=5, size=1382400
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: 1418022590.563278, bitrate: 165888 kb/s
Stream #0:0: Video: rawvideo (NV12 / 0x3231564E), nv12, 1280x720, 165888 kb/s, 15 fps, 15 tbr, 1000k tbn, 1000k tbc
Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, alsa, from 'sysdefault:CARD=sunxicodec':
Duration: N/A, start: 1418022590.589010, bitrate: 1536 kb/s
Stream #1:0: Audio: pcm_s16le, 48000 Hz, 2 channels, s16, 1536 kb/s
[matroska @ 0x28f4100] Invalid stream specifier: c.
Last message repeated 3 times
Codec AVOption qp (Constant quantization parameter rate control method) specified for output file #0 (pipe:) has not been used for any stream. T
he most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was
not actually used for any stream.
Codec AVOption vewait (Time to wait if the VE is busy (default 0)) specified for output file #0 (pipe:) has not been used for any stream. The mo
st likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not a
ctually used for any stream.
Codec AVOption b (set bitrate (in bits/s)) specified for output file #0 (pipe:) has not been used for any stream. The most likely reason is eith
er wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any str
eam.
[aac @ 0x28f58c0] The encoder 'aac' is experimental but experimental codecs are not enabled, add '-strict -2' if you want to use it.
[lrc @ 0x20f9460] Format lrc detected only with low score of 5, misdetection possible!
Input #0, lrc, from 'pipe:':
Duration: N/A, bitrate: N/A
Stream #0:0: Subtitle: text
Output #0, flv, to 'tcp://0.0.0.0:6666':
Metadata:
streamName : livestream
Output file #0 does not contain any stream

I tried by changing container also but not yet works.Kindly tell me how can i stream video with audio.

Regards
Punith

arieedzig · December 08, 2014, 09:44:23 AM

Hi Punith,

Have you checked whether your mic device is working properly?
Try this command to check it but you must plug in an earphone or a speaker in your A20EVB.
arecord -f cd -D sysdefault:CARD=sunxicodec | aplay -D sysdefault:CARD=sunxicodec -
you shall hear your voice in speaker when you talk.

Regards,

Ariee

Punith · December 08, 2014, 10:22:45 AM

Hi arieedzig,

Yes its working fine. i able to hear what i spoke.

Regards
Punith

arieedzig · December 08, 2014, 11:24:50 AM

I tried your command and it works for video and audio. I used flowplayer.
Try to stream the audio only and check whether it works or not.
arecord -D sysdefault:CARD=sunxicodec -f cd | ffmpeg -i - -c:a libmp3lame -b:a 128k -f flv -metadata streamName=livestream tcp://0.0.0.0:6666
If still not working, try to record your voice directly to a file and check whether you can hear your voice.
Make sure that the codec working properly locally, if everthing is ok then something in your client side.

Punith · December 08, 2014, 12:55:58 PM

Hi Ariee,

I tried this command it is streaming fine but not getting any audio on client side.

arecord -D sysdefault:CARD=sunxicodec -f cd | ffmpeg -i - -c:a libmp3lame -b:a 128k -f flv -metadata streamName=livestream tcp://0.0.0.0:6666

LOg:
Output #0, flv, to 'tcp://0.0.0.0:6666':
Metadata:
streamName : livestream
encoder : Lavf56.2.100
Stream #0:0: Audio: mp3 (libmp3lame) ([2][0][0][0] / 0x0002), 44100 Hz, stereo, s16p, 128 kb/s
Metadata:
encoder : Lavc56.0.101 libmp3lame
Stream mapping:
Stream #0:0 -> #0:0 (pcm_s16le (native) -> mp3 (libmp3lame))
overrun!!! (at least 56.465 ms long)rate= 134.1kbits/s
size= 212kB time=00:00:13.03 bitrate= 133.3kbits/s

I tried by using vlc and ffplay but not able to get any connection.

Command:
ffplay rtsp://10.42.0.78/live/livestream
Log:
avplay version 0.8.3-6:0.8.3-6ubuntu2, Copyright (c) 2003-2012 the Libav developers
built on Oct 1 2012 12:57:14 with gcc 4.7.2
rtsp://10.42.0.78/live/livestream: Invalid data found when processing input

and also i tried by recoding only voice to a file which is working fine.

and i tried to install flowplayer in ubuntu but not succeed. Could you this in vlc?.

Regards
Punith

arieedzig · December 08, 2014, 02:18:24 PM

Hello Punith,

I think the problem is in vlc. It didn't recognize the audio stream when you encode it using libmp3lame.
But using flowplayer, there is no problem.
Try using this command:

ffmpeg -f v4l2 -s hd720 -pix_fmt nv12 -i /dev/video0 -f alsa -i sysdefault:CARD=sunxicodec -pix_fmt nv12 -qp 20 -c:v cedrus264 -b:v 300k -r 30 -vewait 3600 -ar 44.1k -b:a 128k -c:a aac -strict -2 -f matroska - | ffmpeg -i - -c:a copy -c:v copy -f flv -metadata streamName=livestream tcp://0.0.0.0:6666

I could play audio and video in vlc. My command example on my blog had configuration error and had been fixed.

Punith · December 08, 2014, 03:36:43 PM

Hi Ariee,

Its working now for both csi and usb camera with audio.

but for csi camera after some time its keep on throws error.
log: for CSI camera

Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
[aac @ 0x18e28c0] Queue input is backward in time0:01:43.20 bitrate=9483.1kbits/s
Last message repeated 2 times
[matroska @ 0x18e1100] Non-monotonous DTS in output stream 0:1; previous: 429233, current: 423394; changing to 429233. This may result in incor.
[matroska @ 0x18e1100] Non-monotonous DTS in output stream 0:1; previous: 429233, current: 415252; changing to 429233. This may result in incor.
[aac @ 0x18e28c0] Queue input is backward in time

For usb camera i used fallowing commmand:
Command:

ffmpeg -f video4linux2 -s 640x480 -pix_fmt nv12 -i /dev/video1 -f alsa -i sysdefault:CARD=Camera -pix_fmt nv12 -qp 20 -c:v cedrus264 -b:v 300k -r 30 -vewait 3600 -ar 44.1k -b:a 128k -c:a aac -strict -2 -f matroska - | ffmpeg -i - -c:a copy -c:v copy -f flv -metadata streamName=livestream tcp://0.0.0.0:6666

But log is immediately shows fallwing error:
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
aac @ 0x2425860] Queue input is backward in time
[matroska @ 0x2424340] Non-monotonous DTS in output stream 0:1; previous: 269, current: 265; changing to 269. This may result in incorrect time.
Input #0, matroska,webm, from 'pipe:':47kB time=00:00:01.99 bitrate= 190.9kbits/s dup=0 drop=16
[matroska @ 0x2424340] Non-monotonous DTS in output stream 0:1; previous: 54771, current: 54677; changing to 54771. This may result in incorrec.
[alsa @ 0x23fbda0] ALSA buffer xrun.
[alsa @ 0x23fbda0] ALSA buffer xrun.5102kB time=00:00:55.44 bitrate= 753.8kbits/s dup=0 drop=16
[alsa @ 0x23fbda0] ALSA buffer xrun.5182kB time=00:00:55.70 bitrate= 762.1kbits/s dup=0 drop=16
[alsa @ 0x23fbda0] ALSA buffer xrun.5277kB time=00:00:56.08 bitrate= 770.7kbits/s dup=0 drop=16

Regards
Punith

tmarlok · February 19, 2015, 04:11:12 PM

Quote from: arieedzig on November 09, 2014, 08:16:13 PM
I can manage to have no delay on video by separating the ffmpeg for each device or using arecord on audio, but the sound is not sync with the video.
Best regards

Hi arieedzig!

I am also working on an IP cam, but I don't need audio. The problem is, I still have 2 sec delay on the video stream without the audio. Can you help me with the ffmpeg commandline you use?

Mine looks like this now:

Code Select

ffmpeg -f v4l2 -s hd720 -pix_fmt nv12 -i /dev/video0  -pix_fmt nv12 -c:v cedrus264 -b:v 300k -r 30 -an -f matroska - | ffmpeg -i - -c:v copy -r 30 -f flv -metadata streamName=livestream tcp://localhost:6666

Without pipeing the stream into another ffmpeg, de stream always stops. Maybe it has to do something with this (dmesg), but I did not find anything useful about that.

Code Select

[ 2120.470376] [CSI]sensor initial success when csi open!
[ 2120.701119] [CSI_ERR]buffer count is invalid, set to 5
[ 2120.710783] [CSI]buffer_setup, buffer count=5, size=460800

Thanks in advance!

arieedzig · February 19, 2015, 06:14:22 PM

I do have the same message in dmesg.
This is my command with audio, you can delete the audio part:

Code Select

ffmpeg -f v4l2 -s hd720 -pix_fmt nv12 -i /dev/video0 -f alsa -i sysdefault:CARD=sunxicodec -pix_fmt nv12 -qp 20 -c:v cedrus264 -b:v 300k -r 30 -vewait 3600 -ar 44.1k -b:a 128k -c:a aac -strict -2 -f matroska - | ffmpeg -i - -c:a copy -c:v copy -f flv -metadata streamName=livestream tcp://0.0.0.0:6666

But if you only want a video, maybe it will be better to use mjpegstreamer than my method.