Video displaying and overlays with IP cameras
Hello, I'm new to the forums, so I apologize if I'm posting in the wrong place or for anything else that may be wrong. It is kind of a broad question, so this seemed like the right place to put it.
I am having a problem with working with video. I have a relatively simple concept; basically, I take several video streams and display/hide them as they are needed. I have no problems with this part. The tricky part is that I need to modify one of the video streams to add overlays to it. Again, not a problem. The problem comes in with the latest change to the overall system. All previous systems had been using analog cameras and converted to digital through USB. This works great with DirectShow, I could do everything I needed. The newest system uses IP cameras instead of analog cameras. Now all data is transferred over the network. I cannot, for the life of me, get this to work with DirectShow. I have tried several other approaches as well:
1. I tried using the ActiveX component supplied by the camera manufacturer (Grandstream Networks). This works great for displaying the video, uses up very little memory, and minimizes the complexity of the overall code. The problem here is I cannot create overlays. I can put picture boxes over the image, but I need to make parts of each overlay clear, and when I do that, the picturebox does not redraw the video, just the background of the ActiveX control. I tried accessing the ActiveX controls graphic, but I cannot access its paint event, so If I add the overlay, it disappears with the next frame. This ended up being a dead end.
2. I tried accessing the video stream directly and displaying it frame by frame, then modifying each frame before it is displayed. The obvious problem here is that as the image takes up more and more of the screen, more and more CPU is consumed. At about half of the required screen size, I am at 100% CPU. This approach is also not desired because it requires me to convert the data stream to mjpeg instead of the much smaller H264. Again, I hit a dead end. I also looked at using DirectX to display the images, I figured if I can unload the image processing to the graphics card, it would save the CPU and I could continue running the rest of the program. However, I am new to DirectX, and good documentation is rather hard to find, so it has been slow going. This approach may work, I just haven't figured out how to do it.
3. I tried to create a source filter for DirectShow to be able to keep all of my old code, and just modify the source. I cannot find any documents on how to do this, and all of the filters I have downloaded and tried do not work. It seams like this should be as easy as telling a standard source filter that the source is a URL and inserting a converting filter, but I have had zero luck with this approach.
If anyone has any advice on how I should continue, it would be much appreciated. I cannot waste anymore time just to find another dead end.
The "easiest" way to do this is through the DirectX SDK, or the XNA framework. But it's not really easy.
There might actually be an easier way, through the Media Foundation and DXVA, but it's not really mature...
How many different video streams do you have to handle, what is their bitrate, and how many threads have you tried working with? In these instances, modern 8-core CPUs I'm sure could handle at least 16 streams of moderate bitrate (e.g. not DVD quality). How many network cards are receiving the data and in what format?
Last edited by Herman; 07-23-2012 at 9:28 PM.
Hi Herman, thanks for the reply. There will be a maximum of 8 video streams on the network, only 4 have to be displayed at a time, but they want all 8 running in the background so there is no activation delay when they switch. There are two possible formats, H264 (prefered) or MJPEG, I've found a bit rate of 4096 at 30 Hz is acceptable for MJPEG. Also, while the cameras have been updated, the target PC has not; it is a single core industrial PC with a pretty good (though not great) video card. I have the DirectX SDK, so I'll look further into that. I'll also check out XNA (I'm not really sure what that is).
No matter what you will need a proper PC. I would ditch the single core for at least a cheap quad core. That in itself is maybe 400$ these days. It doesn't really matter what video card you have if you are not using accelerated playback. The biggest factors will be processor speed, number of cores, and amount of memory you can use as buffer. Use one thread per stream to take the data from the network and put it in the buffer, and another thread per stream to playback from the buffer. Eight streams of 4KHz video should be plenty doable in software.
So I found a solution that works. I found a program at DirectShow Video Source Filter for JPEG and M-JPEG IP Cameras | Fooling Around which creates a DirectShow source filter from a URL stream. Then it's simply a matter of using a samplegrabber filter to capture and modify the image. All painting is performed by the video card, so it saves the PC a lot of work. Now that I have done one stream, it is time to test all of them.
Tags for this Thread