Tuesday, September 30, 2014

Foscam video capture server at motion detection

I have been thinking of making this program for long time and I have finally got it working. The idea is to use Foscam IP camera with Raspberry Pi. Although Foscam made a good IP camera, there is a problem that I needed to solve. There was no way for me to store the streaming video when motion is detected. The camera comes with a feature that can detect motion but it can only upload still images to FTP server. I wanted to store the video when the motion is detected by the camera. I attempted it with shell scripts but it didn't go well.

To achieve it, I needed a fake FTP server that can receive the user id and password information. When the camera detect motion, it will try to access the fake FTP server. Then I can start VLC video recording until the motion stops. That's it. It sounds simple and easy but it took for long time for me to have a program for it.

I like to start with the main function. It takes one input argument for video file name prefix.
It creates a thread safe message queue that will be used for inter-thread-communication.
A thread for VLC recording is created and it will be always ready in background. Whenever a new message is enqueued to the message queue, it will wake up and start recording the video.
A fake FTP server will keep listening a new connections. Whenever there is a valid new connection, it will store the data to the message queue so that the VLC thread can use them.

The thread safe queue is a new class that I created. It is so simple but very useful. Basically it uses std::queue but it had to be thread safe. I wish there was a simpler template utility like std::thread_safe< std::queue > but there is no such a thing. lol. I had to use std::mutex, std::unique_lock and std::condition_variable to achieve it. An interesting point in the class is that it sleeps in the function, "WaitAndPop()", until the condition_variable issues "notify_one()". It could be simply usleep( time ), but it will constantly wake up and check if there was a change. Also the time duration of the sleep will decide how fast the thread can respond. In other words, if it sleep too long, the thread will respond too slow and if it sleep too short, it will waste CPU resources to frequently check if there was a change. I think it was "Effective Java" where I learned the pattern of notify/wait.


The VLCThreadMain is the starting point for the VLC thread. It takes two input arguments, the request message queue and output file prefix that came from the program argument. Those arguments are passed in with std::bind. And also note that the first input argument of VLCThreadMain is a reference pointer. So it had to be passed with std:ref().

It is almost similar to my previous program. One of the differences is that it build up an output file name with the current time. I used three functions, time(), localtime_r() and strftime(). localtime_r is a platform dependent thread-safe version of localtime(). Output file name extension is fixed at ".avi"; I am not sure if other extensions are allowed for H.264 in VLC.

Three input values are given to VLC library through my wrapping classes, MyVLC and MyMediaPlayer. Although they are already explained in other article, I made little improvement, which I will explain in a moment.

The important part of the logic is that it has to keep sleep_for() until the message queue gets empty; or the request came from other IP. But since I have only one camera, this program is designed for only one camera. In fact, even if I have two foscam camera, I will have a dedicated Raspberry Pi for each. This logic was very difficult to program in shell script. First of all, there wasn't an easy way for me to do a safe message queue between two processes. I believe C++ program gets way more simpler than shell scripts in many cases.

I had to modify my previous fake FTP server because I found a problem with the previous approach. When the message is sent to the client, the client didn't get it. I am sure the message needed to be flushed somehow but I couldn't find a way to flush. However, tcp::iostream provides a way to flush in a traditional way. And I liked that the stream operators << and >> looks much simpler and easier to read. It wouldn't always work because it doesn't let me control how many bytes I am expecting to receive but I guess it will be fine with casual use.

As I mentioned, I modified VLC wrapping classes. First of all, it now uses an interface for MyMediaPlayer so that users will need to deal with the interface instead of the implementation class, MyMediaPlayer. Originally I wanted to make MyMediaPlayer a inner class of MyVLC and make the constructor private. But then I had to declare MyVLC as a friend class to MyMediaPlayer; I don't like "friend", although I love my friends. lol.

The reason why MyMediaPlayerInterface is inherited as virtual is because it is interface. Let me give you an example of the cases where it has to be inherited as virtual. Let's say there are two interfaces, A and B, and two implementations, C and D. The interface A inherits from the interface B. The implementation C implements the interface A. The implementation D implements the interface B. Now it turned out that the implementation has to inherit from the implementation C. D is screwed because it will end up inheriting the interface A twice. This is a common problem on multiple inheritance. Although we can treat a pure virtual class as an interface, they are still multiple inheritance in C++ world. As far as I know there is no way for us to predetermine which interface will be inherited only once. Interfaces are supposed to have no implementation so it should be safe to inherit more than once but there is no way for us to do it in C++. So whenever an interface is inherited, they will have to be inherited as virtual.

As I discussed in my previous article, I think it is a better idea to utilize stack memory than heap memory. A new template class, std::aligned_storage, in C++0x provides a way for us to do it without the complex calculation of alignment. It didn't work out nicely for me at first try. It took one hour for me to figure out that I had to use "::type" after aligned_storage. "aligned_storage" class itself has only 1 byte size. The member typedef, "::type", is what we want to use. Then the allocated stack memory space is used with placement new. Then the memory is assigned to unique_ptr in order to make sure it will call the destructor properly. Note that since the memory is allocated from stack memory, we should not "delete" it. We simply have to skip the memory de-allocation step yet the destructor must be called. That's what the class, PlacementDelete, does.

As you can see from the screenshot above, the compiler warns me about an assempler command "swp{b}". I have no idea how to avoid it yet. I think it is coming from boost implementation somewhere. Apart from the minor issue, it works. I had to kill the process for the above case because it spams a lot of text messages.

Now I am thinking of put back my foscam IP camera with the dedicated Raspberry Pi for video recording. But... it will fill up the SD memory card pretty quick. I am not sure what I should do about it.

3 comments:

  1. I look forward to seeing the finished product! I found your blog post a while ago about recording foscam without using iSpy (http://wrice.blogspot.com/2013/06/recording-surveillance-video-from.html). I was using that to record my foscam but have stopped using it. I just got a Raspberry Pi and think it would be neat to get this setup.

    ReplyDelete
  2. Thanks for the comment. But I realized that VLC is not using the hardware H.264 encoder. I am trying to recompile VLC with openMax but it is very time consuming especially on RaspberryPi.
    I am also thinking of making another program that does the same thing with GStreamer sooner or later.

    ReplyDelete
  3. Finally I was able to compile VLC. It took four days. However the generated libvlc.so and libvlccore.so didn't have functions that I needed to use. It was strange. Probably I was missing something but I didn't want to go deeper in the dark tunnel.

    Instead, I got GStreamer working with H.264 encoder: http://raspberrypiprogramming.blogspot.com/2014/10/foscam-video-capture-server-with.html

    For the record, I paste the VLC configuration that worked for me: ./configure '--enable-static' '--prefix=/home/pi/prog/vlc_installed' '--enable-libmpeg2' '--enable-x264' '--enable-omxil' '--enable-omxil-vout' '--enable-rpi-omxil' '--disable-ogg' '--disable-mux_ogg' '--disable-archive'

    ReplyDelete

About Me

My photo
Tomorrow may not come, so I want to do my best now.