Video survillience robot matlab code

In this project we present an operational computer video system for real-time detection and tracking of motion. The system captures monocular video of a scene and identifies those moving objects. This serves as both a proof-of-concept and a verification of other existing algorithms for motion detection also. An approach to statistically modeling of motion developed with a pre-processing stage of image segmentation. This design allows a system that is robust with respect to occlusion, clutter, and extraneous motion. In this project we mainly concentrate on image segmentation for tracking motion and then detection of motion, with intimation of motion through a microcontroller based control system panel prototype.

As Background subtraction is a common computer vision task, we analyze the usual pixel-level approach. We develop an efficient adaptive algorithm system based on pixel comparison. Recursive comparison of pixel is used between the present frame and the reference frame. This algorithm is implemented using image processing in MatLAB Environment and we work on making it a real-time applicable tool for various possible applications. Thus an attempt to build a video system for real-time detection and tracking of motion which has the ability to minimize both false detections and missed detections, interfaced with a hardware unit based on microcontroller, communicating serially with the computer system as a control unit panel prototype.

Human quest for an automatic detection system of everyday occurrence lead to the necessity of inventing an intelligent surveillance system which will make lives easier as well as enable us to compete with tomorrows technology and on the other hand it pushes us to analyze the challenge of the automated video surveillance scenarios harder in view of the advanced artificial intelligence. Nowadays, it is seen that surveillance cameras are already prevalent in commercial establishments, with camera output being recorded to tapes that are either rewritten periodically or stored in video archives. To extract the maximum benefit from this recorded digital data, detect any moving object from the scene is needed without engaging any human eye to monitor things all the time. Real-time segmentation of moving regions in image sequences is a fundamental step in many vision systems. A typical method is background subtraction. Many background models have been introduced to deal with different problems.

Background subtraction technique was traditionally applied to detection of objects. We can get object regions only by doing subtraction between an observed image and the background image without requiring prior information about the objects. However, when simple background subtraction technique is applied for video-based surveillance which usually captures outdoor scenes, it often detects not only objects but also a lot of noise regions since it is quite sensitive to small intensity or color changes such as illumination changes.

There are many approaches to handle these back-ground changes. In our project we have introduced the usual pixel level approach, in which we compare the corresponding pixel values of the foreground with reference to standard background. So, whenever we find a change in the pixel values beyond a threshold level we consider it as a change which becomes the part of our extracted image.

Image background and foreground are needed to be separated, processed and analyzed. The data found from it is then used further to detect motion. In this project work robust routines for accurately detecting and tracking moving objects have been developed and analyzed.

While tracking is one section of this project we also detect any particular motion regarding an object in the given video and use a hardware unit which is a prototype of control unit panel to intimate the motion of an object which is detected. This hardware unit is interfaced to the system via serial port communication controlled by MatLAB coding. Hardware unit has a microcontroller embedded in it which is user programmed for the particular application. This unit comprises a LCD for displaying the status and a buzzer system which is an intimation of any motion detection as per as the input video is concerned.

2.1 Motion detection:

Motion detection in consequent images is nothing but the detection of the moving object in the

Scene. In video surveillance, motion detection refers to the capability of the surveillance system to detect motion and capture the events. Motion detection is usually a software-based monitoring algorithm which will signal the surveillance camera to begin capturing the event when it detects motions. This is also called activity detection. An advanced motion detection surveillance system can analyze the type of motion to see if it warrants an alarm. In this project, a camera fixed to its base has been placed and is set as an observer at the outdoor for surveillance. Any small movement with a level of tolerance it picks is detected as motion.

Aside from the intrinsic usefulness of being able to segment video streams into moving and

background components, detecting moving blobs provides a focus of attention for recognition,

classification, and activity analysis, making these later processes more efficient since only “moving” pixels need be considered.

There are three conventional approaches to moving object detection:

temporal differencing, background subtraction and optical flow. Temporal differencing is very

adaptive to dynamic environments, but generally does a poor job of extracting all relevant feature pixels. Background subtraction provides the most complete feature data, but is extremely sensitive to dynamic scene changes due to lighting and extraneous events. Optical flow can be used to detect independently moving objects in the presence of camera motion; however, most optical flow computation methods are computationally complex, and cannot be applied to full-frame video streams in real-time without specialized hardware.

2.2 Motion in real time environment: Problems

Video motion detection is fundamental in many autonomous video surveillance strategies. However, in outdoor scenes where inconsistent lighting and unimportant, but distracting, background movement is present, it is a challenging problem. In real time environment where scene is not under control situation is much worse and noisy. Light may change anytime which cause system output less meaningful to deal with. Recent research has produced several background modeling techniques, based on image differencing, that exhibit real-time performance and high accuracy for certain classes of scene.

Some of these background modeling techniques using video sequences of outdoor scenes where the weather introduces unpredictable variations in both lighting and background movement. The results are analyzed and reported, with the aim of identifying suitable directions for enhancing the robustness of motion detection techniques for outdoor video surveillance systems. Motion in indoor and other situations are considered and analyzed as well.

2.3 Video Surveillance:

An appliance that enables embedded image capture capabilities that allows video images or extracted information to be compressed, stored or transmitted over communication networks or digital data link. Digital video surveillance systems are used for any type of monitoring. Broadly,

video surveillance is the image sequences which are recorded to monitor the live activities of a particular scene. The importance of this digital evidence is given the first priority for any kind of

occurrence. This digital information is recently become the field of interest to the researchers on the field of AI, Robotics, Forensic Science and other major fields of science.

2.4 Impact of video surveillance in commercial areas:

“What are you looking at?” — Graffiti by Banksy commenting on the neighboring surveillance camera in a concrete subway underpass near Hyde Park in London. The greatest impact of computer-enabled surveillance is the large number of organizations involved in surveillance operations. The state and security services still have the most powerful surveillance systems, because they are enabled under the law. But today levels of state surveillance have increased, and using computers they are now able to draw together many different information sources to produce profiles of persons or groups in society. Many large corporations now use various form of "passive" surveillance. This is primarily a means of monitoring the activities of staff and for controlling public relations. But some large corporations actively use various forms of surveillance to monitor the activities of activists and campaign groups who may impact their operations. Many companies trade in information lawfully, buying and selling it from other companies or local government agencies that collect it. This data is usually bought by companies who wish to use it for marketing or advertising purposes. Personal information is obtained by many small groups and individuals. Some of this is for harmless purposes, but increasingly sensitive personal information is being obtained for criminal purposes, such as credit card and other types of fraud.

2.5 Video Surveillance Nowadays

Modern surveillance cannot be totally avoided. However, non-state groups may employ surveillance techniques against an organization, and some precautions can reduce their success. Some states are also legally limited in how extensively they can conduct general surveillance of people they have no particular reason to suspect. The constantly growing interest in the field of robotic vision is pushing the researchers hard to produce something which significantly fit in their requirement.

2.6 SOFTWARE OVERVIEW - MatLAB

MATLAB is a numerical computing environment and programming language. Created by The MathWorks, MATLAB allows easy matrix manipulation, plotting of functions and data, implementation of algorithms, creation of user interfaces, and interfacing with programs in other languages. Although it is numeric only, an optional toolbox interfaces with the Maple symbolic engine, allowing access to computer algebra capabilities.

History
Short for "Matrix Laboratory", MATLAB was invented in the late 1970s by Cleve Moler, then chairman of the computer science department at the University of New Mexico. He designed it to give his student’s access to LINPACK and EISPACK without having to learn Fortran. It soon spread to other universities and found a strong audience within the applied mathematics community. Jack Little, an engineer, was exposed to it during a visit Moler made to Stanford University in 1983. Recognizing its commercial potential, he joined with Moler and Steve Bangert. They rewrote MATLAB in C and founded The Math Works in 1984 to continue its development. These rewritten libraries were known as JACKPAC.

MATLAB was first adopted by control design engineers, Little's specialty, but quickly spread to many other domains. It is now also used in education, in particular the teaching of linear algebra and numerical analysis, and is popular amongst scientists involved with image processing.

Syntax
MATLAB is built around the MATLAB language, sometimes called M-code or simply M. The simplest way to execute M-code is to type it in at the prompt, >> , in the Command Window, one of the elements of the MATLAB Desktop. In this way, MATLAB can be used as an interactive mathematical shell. Sequences of commands can be saved in a text file, typically using the MATLAB Editor, as a script or encapsulated into a function, extending the commands available.

Variables

Variables are defined with the assignment operator, =. MATLAB is dynamically typed, meaning that variables can be assigned without declaring their type, and that their type can change. Values can come from constants, from computation involving values of other variables, or from the output of a function.

Vectors/Matrices

MATLAB is a "Matrix Laboratory", and as such it provides many convenient ways for creating matrices of various dimensions. In the MATLAB vernacular, a vector refers to a one dimensional (1×N or N×1) matrix, commonly referred to as an array in other programming languages. A matrix generally refers to a multi-dimensional matrix, that is, a matrix with more than one dimension, for instance, an N×M, an N×M×L, etc., where N, M, and L are greater than 1. In other languages, such a matrix might be referred to as an array of arrays, or array of arrays of arrays, or simply as a multidimensional array.

MATLAB provides a simple way to define simple arrays using the syntax: init:increment:terminator.

For instance:

>> array = 1:2:9

array =

 1 3 5 7 9

Defines a variable named array (or assigns a new value to an existing variable with the name array) which is an array consisting of the values 1, 3, 5, 7, and 9. That is, the array starts at 1, the init value, and each value increments from the previous value by 2 (the increment value), and stops once it reaches but not exceeding 9 (9 being the value of the terminator).

>> array = 1:3:9

array =

 1 4 7

the increment value can actually be left out of this syntax (along with one of the colons), to use a default value of 1.

>> ari = 1:5

ari = 1 2 3 4 5

Assigns to the variable named ari an array with the values 1, 2, 3, 4, and 5, since the default value of 1 is used as the incrementer.

Indexing is one-based, which is the usual convention for matrices in mathematics. This is atypical for programming languages, whose arrays more often start with zero.

Matrices can be defined by separating the elements of a row with blank space or comma and using a semicolon to terminate each row. The list of elements should be surrounded by square brackets []. Elements and sub arrays are accessed using parenthesis ().

>> A = [16 3 2 13; 5 10 11 8; 9 6 7 12; 4 15 14 1]

A =

 16    3    2    13

  5    10  11     8

  9     6    7    12

  4    15  14    1

>> A(2,3)

ans =

 11

 A square identity matrix of size n can be generated using the function eye, and matrices of any size with zeros or ones can be generated with the functions zeros and ones, respectively.

>> eye(3)

ans =

 1 0 0

 0 1 0

 0 0 1

>> zeros(2,3)

ans =

 0 0 0

 0 0 0

>> ones(2,3)

ans =

 1 1 1

 1 1 1

Most MATLAB functions can accept matrices and will apply themselves to each element. For example, mod(2*J,n) will multiply every element in "J" by 2, and then reduce each element modulo "n". MATLAB does include standard "for" and "while" loops, but using MATLAB's vectorized notation often produces code that is easier to read and faster to execute.

Semicolon

In many other languages, the semicolon is required to terminate commands. In MATLAB the semicolon is optional. If a statement is not terminated with a semicolon, then the result of the statement is displayed. A statement that does not explicitly return a result, for instance 'clc', will behave the same whether or not a semicolon is included.

Graphics

Function plot can be used to produce a graph from two vectors x and y. The code:
x=0:pi/100:2*pi;
y=sin(x);
plot(x,y)

Limitations

MATLAB, like FORTRAN, Visual Basic and Ada, uses parentheses, e.g. y = f(x), for both indexing into an array and calling a function. Although this syntax can facilitate a switch between a procedure and a lookup table, both of which correspond to mathematical functions, a careful reading of the code may be required to establish the intent, especially for C programmers who are more familiar with the use of square brackets for array indexing.

MATLAB lacks a package system, like those found in modern languages such as Java and Python, where classes can be resolved unambiguously, e.g. Java's java.lang.System.out.println(). In MATLAB, all functions share the global namespace, and precedence of functions with the same name is determined by the order in which they appear in the user's MATLAB path and other subtle rules. As such, two users may experience different results when executing what otherwise appears to be the same code when their path is different.

Many functions have a different behavior with matrix and vector arguments. Since vectors are matrices of one row or one column, this can give unexpected results. For instance, function sum(A) where A is a matrix gives a row vector containing the sum of each column of A, and sum(v) where v is a column or row vector gives the sum of its elements; hence the programmer must be careful if the matrix argument of sum can degenerate into a single-row array. While sum and many similar functions accept an optional argument to specify a direction, others, like plot, do not, and require additional checks. There are other cases where MATLAB's interpretation of code may not be consistently what the user intended (e.g. how spaces are handled inside brackets as separators where it makes sense but not where it doesn't, or backslash escape sequences which are interpreted by some functions like fprintf but not directly by the language parser because it wouldn't be convenient for Windows directories). What might be considered as a convenience for commands typed interactively where the user can check that MATLAB does what the user wants may be less supportive of the need to construct reusable code.

Though other data types are available, the default is a matrix of doubles. This array type does not include a way to attach attributes such as engineering units or sampling rates. Although time and date markers were added in R14SP3 with the time series object, sample rate is still lacking. Such attributes can be managed by the user via structures or other methods.

MATLAB is a proprietary product of The MathWorks, so users are subject to vendor lock-in. Some other numerical analysis software packages, however, are partially source compatible (like GNU Octave) or provide a simple migration path (like Scilab).

MatLAB comes with various toolboxes where each toolbox covers one region of study. Th toolboxes which have been used in this project most extensively are:

1. Image Processing Toolbox
2. Instrument Control Toolbox
3. Data Acquisition Toolbox.

2.6.1 Introduction to image processing in MATLAB

Image Processing Toolbox provides a comprehensive set of reference-standard algorithms and graphical tools for image processing, analysis, visualization, and algorithm development. You can restore noisy or degraded images, enhance images for improved intelligibility, extract features, analyze shapes and textures, and register two images. Most toolbox functions are written in the open MATLAB language, giving you the ability to inspect the algorithms, modify the source code, and create your own custom functions.

Image Processing Toolbox supports engineers and scientists in areas such as biometrics, remote sensing, surveillance, gene expression, microscopy, semiconductor testing, image sensor design, color science, and materials science. It also facilitates the learning and teaching of image processing techniques.

A digital image is composed of pixels which can be thought of as small dots on the screen. A digital image is an instruction of how to color each pixel. We will see in detail later on how this is done in practice. A typical size of an image is 512-by-512 pixels. Later on in the course you will see that it is convenient to let the dimensions of the image to be a power of 2. For example, 2⁹=512. In the general case we say that an image is of size m-by-n if it is composed of m pixels in the vertical direction and n pixels in the horizontal direction.

Let us say that we have an image on the format 512-by-1024 pixels. This means that the data for the image must contain information about 524288 pixels, which requires a lot of memory! Hence, compressing images is essential for efficient image processing. You will later on see how Fourier analysis and Wavelet analysis can help us to compress an image significantly. There are also a few "computer scientific" tricks (for example entropy coding) to reduce the amount of data required to store an image.

Image formats supported by MatLAB

The following image formats are supported by MatLAB:

• BMP
• HDF
• JPEG
• PCX
• TIFF
• XWB

Most images you find on the Internet are JPEG-images which is the name for one of the most widely used compression standards for images. If you have stored an image you can usually see from the suffix what format it is stored in. For example, an image named myimage.jpg is stored in the JPEG format and we will see later on that we can load an image of this format into MatLAB.

Working formats in MATLAB

If an image is stored as a JPEG-image on your disc we first read it into MatLAB. However, in order to start working with an image, for example perform a wavelet transform on the image, we must convert it into a different format. This section explains four common formats.

Intensity image (gray scale image)

This is the equivalent to a "gray scale image" and this is the image we will mostly work with in this course. It represents an image as a matrix where every element has a value corresponding to how bright/dark the pixel at the corresponding position should be colored. There are two ways to represent the number that represents the brightness of the pixel: The double class (or data type). This assigns a floating number ("a number with decimals") between 0 and 1 to each pixel. The value 0 corresponds to black and the value 1 corresponds to white. The other class is called uint8 which assigns an integer between 0 and 255 to represent the brightness of a pixel. The value 0 corresponds to black and 255 to white. The class uint8 only requires roughly 1/8 of the storage compared to the class double. On the other hand, many mathematical functions can only be applied to the double class. We will see later how to convert between double and uint8.

Binary image

This image format also stores an image as a matrix but can only color a pixel black or white (and nothing in between). It assigns a 0 for black and a 1 for white.

Indexed image

This is a practical way of representing color images. (In this course we will mostly work with gray scale images but once you have learned how to work with a gray scale image you will also know the principle how to work with color images.) An indexed image stores an image as two matrices. The first matrix has the same size as the image and one number for each pixel. The second matrix is called the color map and its size may be different from the image. The numbers in the first matrix is an instruction of what number to use in the color map matrix.

RGB image

This is another format for color images. It represents an image with three matrices of sizes matching the image format. Each matrix corresponds to one of the colors red, green or blue and gives an instruction of how much of each of these colors a certain pixel should use.

Multiframe image

In some applications we want to study a sequence of images. This is very common in biological and medical imaging where you might study a sequence of slices of a cell. For these cases, the multiframe format is a convenient way of working with a sequence of images. In case you choose to work with biological imaging later on in this course, you may use this format.

How to convert between different formats

Various formats can be converted to there corresponding ones i.e conversion between different formats given above is possible using various functions available in image processing toolbox such as rgb2gray() etc . All these commands require the Image processing tool box!

Variables in MATLAB :

We have three different classes of MATLAB data: floating point numbers, strings, and symbolic expressions. Variables include types double, uint and many more.

2.6.2 Image Acquisition Toolbox

Acquire images and video from industry-standard hardware

Image Acquisition Toolbox™ lets you acquire images and video directly into MATLAB® and Simulink® from PC-compatible imaging hardware. You can detect hardware automatically, configure hardware properties, preview an acquisition, and acquire images and video. With support for multiple hardware vendors, you can use a range of imaging devices, from inexpensive Web cameras or industrial frame grabbers to high-end scientific cameras that meet low-light, high-speed, and other challenging requirements. Together, MATLAB, Image Acquisition Toolbox, and Image Processing Toolbox™ provide a complete environment for developing customized imaging applications. You can acquire images and video, visualize data, develop processing algorithms and analysis techniques, and create graphical user interfaces. You can use Image Acquisition Toolbox with Simulink and Video and Image Processing Blockset™ to simulate and model real-time embedded imaging systems. Acquire images and video from industry-standard hardware

Key Features

■ Automatically detects image and video acquisition devices

■ Manages device configurations

■ Provides live video previewing

■ Acquires static images and continuous video

■ Enables in-the-loop image processing and analysis

■ Provides graphical user interface for working with devices

■ Supports devices for use with MATLAB and Simulink

■ Supports multiple hardware vendors

Working with Image Acquisition Toolbox

Image Acquisition Toolbox helps you connect to and configure your hardware, preview the acquisition, and acquire and visualize image data. You can use the toolbox from the Image Acquisition Tool, the MATLAB command line, or the From Video Device block within Simulink. This lets you control your image acquisition parameters and incorporate them into M-scripts, applications built within MATLAB, or Simulink models. Within the toolbox, the Image Acquisition Tool is a graphical user interface for working with image and video acquisition devices in MATLAB. With this tool, you can see all hardware available on your PC, change device settings, preview an acquisition, control acquisition parameters, and acquire image or video data. You can also record data directly to an AVI file or export hardware configuration settings to an M-file so that you can incorporate them into other MATLAB scripts. Connecting to Hardware Image Acquisition Toolbox automatically detects compatible image and video acquisition devices. The connection to your devices is encapsulated as an object, providing an

interface for configuration and acquisition. You can create multiple connection objects for simultaneous acquisition from as many devices as your PC and imaging hardware support.

Configuring Hardware

The toolbox provides a consistent interface across multiple hardware devices and vendors, simplifying the configuration process. You configure your hardware by using the Image Acquisition Tool or by modifying the properties of the object associated with the hardware on the MATLAB command line. The toolbox also supports camera files from hardware vendors. You can set base properties that are common to all supported hardware. These properties can include video format, resolution, region of interest (ROI), and returned color space. You can also set device-specific properties, such as hue, saturation, brightness, frame rate, contrast, and video sync, if your device supports these properties.

Previewing the Acquisition

The Image Acquisition Toolbox video preview window helps you verify and optimize your acquisition parameters. It instantly reflects any adjustments that you make to acquisition properties. The Image Acquisition Tool has a built-in preview window, and you can add one to any application built with MATLAB. A typical Acquiring Image Data Image Acquisition Toolbox can continuously acquire image data while you are processing the acquired data in MATLAB or Simulink. The toolbox automatically buffers acquired data into memory, handles memory and buffer management, and enables acquisition from an ROI. Data can be acquired in

a wide range of data types, including signed or unsigned 8-, 16-, and 32-bit integers and single- or double-precision floating point. The toolbox supports any color space provided by the image acquisition device, such as RGB, YUV, or grayscale. Raw sensor data in a Bayer pattern can be automatically converted into RGB data. The toolbox supports any frame rate and video resolution supported by your PC and imaging hardware.

Advanced Acquisition Features

Image Acquisition Toolbox supports three trigger types: immediate, manual, and hardware. Hardware triggers, which are device-specific, let you synchronize your acquisition to an external signal. You can log data to disk, memory, or both simultaneously. Image Acquisition Toolbox

lets you:

• Log each image frame or log frames at specified intervals

• Log data to disk as compressed or uncompressed AVI streams

• Extract single images from a video streamand store them in standard formats, including

BMP, JPEG, and TIFF

For advanced sequencing of your acquisition application, you can create callback functions

that are automatically executed whenever events occur, such as acquisition started or stopped, trigger occurred, and a set number of frames acquired.

2.6.3 Instrument Control Toolbox

Control and communicate with test and measurement instruments

KEY FEATURES

■ Instrument driver support for IVI, VXIplug&play, and MATLAB instrument drivers

■ Support for GPIB and VISA standard protocols (GPIB, GPIB-VXI, VXI, USB, TCP/IP, serial)

■ Support for networked instruments using the TCP/IP and UDP protocols

■ Graphical user interface for identifying, configuring, and com­municating with instruments

■ Hardware availability, management, and configuration tools

■ Instrument driver development and testing tools

■ Functions for reading and writing binary and text (ASCII) data

■ Synchronous and asynchronous (blocking and nonblocking) read-and-write operations

■ Event handling for time-out, bytes read, data written, and other events

■ Recording of data transferred to and from instruments

The Instrument Control Toolbox lets you com­municate with instruments, such as oscilloscopes, function generators, and analytical instruments, directly from MATLAB®. With the toolbox, you can generate data in MATLAB to send out to an instrument, or read data into MATLAB for analysis and visualization.

The toolbox provides a consistent interface to all devices independent of hardware manufac­turer, protocol, or driver. The toolbox supports IVI, VXIplug&play, and MATLAB instrument drivers. Support is also provided for GPIB, VISA, TCP/IP, and UDP communication protocols.

Working with the Instrument Control Toolbox

The Instrument Control Toolbox provides a variety of ways to communicate with instru­ments, including:

• Instrument drivers

• Communication protocols

• Graphical user interface (TMTool)

The Instrument Control Toolbox is based on MATLAB object technology. The toolbox includes functions for creating objects that contain properties related to your instrument and to your instrument control session.

Instrument Drivers

Instrument drivers let you communicate with an instrument independent of device protocol. As a result, you can use common MATLAB terminology to communicate with instruments without learning instrument-specific commands, such as Standard Commands for Programmable Instruments (SCPI).

The toolbox lets you work with VXIplug&play, IVI, and MATLAB instrument drivers. VXIplug&play and IVI instrument drivers often ship with your instrument and are also available from the instrument manufacturers’ Web sites. You can also create MATLAB instru­ment drivers with driver development tools provided in the Instrument Control Toolbox.

2.7 APPLICATIONS

The ability to reliably detect and track human motion is a useful tool for higher-level applications that rely on visual input. Interacting with humans and understanding their activities are at the core of many problems in intelligent systems, such as human-computer interaction and robotics. An algorithm for human motion detection digests high-band width video into a compact description of the human presence in that scene. This high-level description can then be put to use in other applications.

Some examples of applications that could be realized human motion detection and tracking are:

Ø Automated surveillance for security-conscious venues such as airports, casinos, museums, and government installations: Intelligent software could monitor security cameras and detect suspicious behavior. Furthermore, human operators could search archived video for classes of activity that they specify without requiring manual viewing of each sequence. Having automated surveillance vastly increases the productivity of the human operator and increases coverage of the surveillance.

Ø Human interaction for mobile robotics : Autonomous mobile robots in the workplace or home could interact more seamlessly with the humans in their environment if they could reliably detect their presence. For example, robots to assist the elderly would know when assistance is needed based on the motion of a person.

Ø Safety devices for pedestrian detection motor vehicles: Intelligent software on a camera-equipped car could detect pedestrians and warn the driver.

Ø Automatic motion capture for film and television: Producing computer-generated imagery of realistic motion currently requires the use of a motion-capture system that stores the exact 2-D or 3-D motion of a human body using visual or radio markers attached to each limb of an actor. With accurate algorithms for human motion tracking, the same data could be acquired from any video without any additional equipment.

Currently, no algorithm exists that can perform human motion detection reliably and efficiently enough for the above applications to be realized.

This is an attempt to build up on

One of its application in terms of this project’s foreground extraction code output is shown below:

Methodology:

Reliable tracking of moving humans is essential to motion estimation, video surveillance and human-computer interface. This project presents a new approach to human motion tracking that combines view-based and model-based techniques. Monocular video is processed at both pixel level and object level. At the pixel level we compare each pixel value and built up a new object that is termed as the foreground when the new image is compared with the standard background image.

The field of computer vision concerned with problems that involve interfacing computers with their surrounding environment through visual means. One such problem, object recognition, involves the presence of a known object in an image, given some knowledge about what that object should like. As humans, we take this ability for granted, as our brains are extraordinarily proficient at both learning new objects and recognizing them later. However, in computer vision, this same problem has proven to be one of the most difficult and computationally intensive of the field. Given the current state of the art, a successful for object recognition requires one to define the problem with a more specific focus.

In this project, we consider a sub-problem of object recognition: human motion detection, in which we are interested in recognizing humans based solely on the characteristic patterns of motion that they exhibit. This approach differs from other techniques for human detection, such as those that recognize humans based on shape, color, texture, or surface features. This project attempts to presents a fully realized system for human motion detection that can be deployed in the field. Its characteristics include real-time performance, insensitivity to background clutter and movement, and a modular design that can be generalized to other types of motion.

In this project we implement our knowledge in image processing based on MATLAB environment to extract the foreground part of the object from a standard background present as reference. This is an attempt to detect an unwanted foreign object (human) in a restricted zone and intimate by using an alarm system or any kind of security control that is desired.

In this project we divide our work into modules which adds up to provide the output on a whole for the project. These modules deals with the challenges like the human that is the foreground extraction from the standard background, interfacing a video camera to the system which provides the data to work on with and a alarming system interfaced to the system responding as per as the system code output.

A real time motion detector is aimed to build a system that detects any moving object. In the process it generate the moving pattern of the moving object, there by properly tracking its motion all the way down. When the world is running behind atomization the need for proper security that is an automatic security system is the need of the hour. Thus a dedicated system for motion detection is the aim of this project.

For the process of tracking motion and intimating the detection of any kind of motion we develop this system which can broadly be separated into three categories:

Video Capturing System:

This section or module basically consists of a camera, which captures a frame as the reference picture or the desired background condition which thus forms the reference for any kind of motion tracking or foreground extraction. After capturing a snapshot of the standard background the camera is programmed to capture a video of user defined number of frames. The captured video is to be processed for detecting any motion with the reference frame snapshot as the reference condition.

Here in this project we go for a camera that is controlled by MatLAB coding itself. When it comes to camera controlling through MatLAB coding we make use of the Image Acquisition Toolbox available with MatLAB, for the various options available for controlling a camera connected to the personal computer. By using the various functions provided by the image acquisition toolbox we can use the camera to take snapshots and even a sequence of images which is a video. This module can be separated into few blocks as shown bellow:

Install Your Image Acquisition Device

Follow the setup instructions that come with your image acquisition device. Setup typically involves Installing the frame grabber board in your computer. Installing any software drivers required by the device. These are supplied by the device vendor. Connecting a camera to a connector on the frame grabber board. Verifying that the camera is working properly by running the application software that came with the camera and viewing a live video stream.

Retrieve Hardware Information

In this step, one get several pieces of information that the toolbox needs to uniquely identify the image acquisition device one want to access. One use this information when they create an image acquisition object.

The following table lists this information.

Device Information	Description
Adaptor name	An adaptor is the software that the toolbox uses to communicate with an image acquisition device via its device driver. The toolbox includes adaptors for certain vendors of image acquisition equipment and for particular classes of image acquisition devices. See Determining the Adaptor Name for more information.
Device ID	The device ID is a number that the adaptor assigns to uniquely identify each image acquisition device with which it can communicate. See Determining the Device ID for more information. Note: Specifying the device ID is optional; the toolbox uses the first available device ID as the default.
Video format	The video format specifies the image resolution (width and height) and other aspects of the video stream. Image acquisition devices typically support multiple video formats. See Determining the Supported Video Formats for more information. Note: Specifying the video format is optional; the toolbox uses one of the supported formats as the default.

The above information regarding the camera interfaced with the personal computer is gathered for controlling the camera in acquiring the video that has to be processed by the code. The set of code that collects and displays all the required information regarding the camera is listed bellow..

>> imaqInfo = imaqhwinfo % information regarding the adaptor is acquired.

>>imaqInfo.InstalledAdaptors

>>matroxInfo = imaqhwinfo('winvideo')

>>matroxInfo.DeviceInfo

>>device1.DeviceName % device name

>>device1.DeviceID %device ID

>>device1.DefaultFormat % information regarding the default format of the camera in use

>>device1.SupportedFormats %all the supported formats.

Create a Video Input Object

In this step one create the video input object that the toolbox uses to represent the connection between MATLAB and an image acquisition device. Using the properties of a video input object, you can control many aspects of the image acquisition process.

To create a video input object, use the video input function at the MATLAB prompt. The DeviceInfo structure returned by the imaqhwinfo function contains the default videoinput function syntax for a device in the ObjectConstructor field.

Preview the Video Stream

After you create the video input object, MATLAB is able to access the image acquisition device and is ready to acquire data. However, before you begin, you might want to see a preview of the video stream to make sure that the image is satisfactory. For example, you might want to change the position of the camera, change the lighting, correct the focus, or make some other change to your image acquisition setup.

Acquire Image Data

After you create the video input object and configure its properties, you can acquire data. This is typically the core of any image acquisition application, and it involves these steps:

Starting the video input object -- You start an object by calling the start function. Starting an object prepares the object for data acquisition. For example, starting an object locks the values of certain object properties (they become read only). Starting an object does not initiate the acquiring of image frames, however. The initiation of data logging depends on the execution of a trigger.

Triggering the acquisition -- To acquire data, a video input object must execute a trigger. Triggers can occur in several ways, depending on how the TriggerType property is configured. For example, if you specify an immediate trigger, the object executes a trigger automatically, immediately after it starts. If you specify a manual trigger, the object waits for a call to the trigger function before it initiates data acquisition.

Bringing data into the MATLAB workspace -- The toolbox stores acquired data in a memory buffer, a disk file, or both, depending on the value of the video input object LoggingMode property. To work with this data, you must bring it into the MATLAB workspace. To bring multiple frames into the workspace, use the getdata function. Once the data is in the MATLAB workspace, you can manipulate it as you would any other data.

Thus to perform listed operations above there required code used is shown bellow

>>vidobj = videoinput('winvideo', 1);% creating video object.

>>preview(vidobj)%starting the preview window

>>set(vid.source,’Brightness’,100);%brightness adjustment

>>r=input('ready to capture the standard background ( if yes press a numeric key)');

>>pic = getsnapshot(vidobj);% acquire the standard background image.

>>r=input('ready to capture video ( if yes press a numeric key)');

>>set(vidobj, 'FramesPerTrigger', 50);%capture 50 frames

>>start(vidobj)%starting the video capture.

>>pause(5)

>>numAvail = vidobj.FramesAvailable

>>imageData = getdata(vidobj, numAvail);%imagedata carries the acquired video.

These few lines of code allows us to control a camera that is an image acquisition device to capture a video which is taken as the input for the processing required to construct information regarding the foreground moving object along with proper detection motion. Any motion detected is forwarded to the hardware unit to intimate through an alarming or a control system section.

Here in this project we make use of a web-cam which is installed with the personal computer. This web-cam is the source of video for this project controlled by the functions used in MatLAB made available by the Instrument Control Toolbox in MatLAB.

3.3 Foreground Extraction:

This is the most important section of the project. Here we reconstruct the foreground object removing the background elements. For doing this we first need to build up a constructive background image which can be a perfect reference image for the processing of image segmentation.

Before we can proceed to identifying the foreground section of the images with the background image as the reference, we first need to perform some conditioning process on these images that is the continues frames of a video.

It is always very difficult to construct a robust system which will suit various light conditions and more difficult is to remove the noise signals from various sources and thus few steps are taken to make things more real-time oriented.

First and foremost each image frames is converted to grayscale image. This processing makes things easier to work on as a grayscale image, which is a 2-dimensional unit when compared to a RGB image which is a 3-dimensional. As we go for a pixel to pixel comparison for identifying the foreground image a 2-D unit of data will be faster and easier to process upon when compared to a 3-D unit. Thus a grayscale form of these image frames are utilized for foreground extraction.

In order to go about with the foreground extraction we use MatLAB coding. We make use of the various functions available with Image Processing Toolbox available with MatLAB. The various functions in this toolbox gives us an option analyze the images to the fullest level. This helps us to gather maximum information in the smallest span of time increasing the processing time along with the accuracy level of the image processing that is required to reconstruct only the moving object.

Image Processing Toolbox:

The Image Processing Toolbox is a collection of functions that extend the capability of the MATLAB numeric computing environment. The toolbox supports a wide range of image processing operations:

· including Spatial image transformations

· Morphological operations

· Neighborhood and block operations

· Linear filtering and filter design

· Transforms

· Image analysis and enhancement

· Image registration

· Deblurring

· Region of interest operations

The Video and Image Processing Blockset extends Simulink® with a rich, customizable framework for the rapid design, simulation, implementation, and verification of video and image processing algorithms and systems. It includes basic primitives and advanced algorithms for designing embedded imaging systems in a wide range of applications in aerospace and defense, automotive, commu­nications, consumer electronics, education, and medical electronics industries.

For the above mentioned function of image conversion to grayscale, Image Processing Toolbox available with MatLAB provides the functions like ‘rgb2gray’ and many other conversion tools required for handling various types of images in process along with various formats of images available. The task of foreground extraction is the most critical part of this project where each and every pixels of each frame is taken into account for gathering information regarding the foreground image. For gathering this information we make use of a few filters available in MatLAB for more and more proper definition of edges and proper weightage for each pixel value of the image under process. In case of this project we use a low pass Gaussian filter. This filter sharpens the image visibility in this process providing us a image with much sharper and well defined structures.

>> nback=uint8(filter2(fspecial('gaussian'),nback));

As we have discussed about various image processing done on these images let us discuss how can we make the processing faster. As we desire for spontaneous results for any action, steps are to be taken to build up a fast system which gathers information at a brisk pace to provide faster results at the same time not compromising in terms of accuracy and precision. As it is based on pixel to pixel comparison, faster processing is possible when the number of pixels is reduced. This is possible when we can resize the image to a smaller size which not only maintains the quality of images at the same time reduces the number of pixels to be processed. Resizing of images is possible by using the inbuilt functions available in this toolbox. ‘ imresize ‘ is one of such functions which can be used to resize the image to the desired without reducing the quality of the image in use.

>> B = imresize(A,m)

Returns image B that is m times the size of A.A can be an indexed image, grayscale image, RGB, or binary image. If m is between 0 and 1.0, B is smaller than A. If m is greater than 1.0, B is larger than A. When resizing the image, imresize uses nearest-neighbor interpolation.

3.3.2 Algorithm

The detailed algorithm of background model construction and foreground object detection is summarized as follows:

1. When a new pixel in the new image is observed the probability that the same pixel value is present in the background image is taken into account.

2. If the pixel value is greater than the threshold, i.e. if the pixel value is greater than the tolerance range when compared to the same position pixel value of the background image then the pixel is recorded marking it as a part of the foreground image. This is done as a background image pixel is expected to repeat its value for a long time. Atleast it is expected to be with in a tolerance range. A very constructive change marks the pixel as a part of the foreground object that is need to be tracked and segmented out.

3. The observed pixel if lies within the tolerance then it is considered as a part of the background and is replaced with a zero. At the same time if any significant change in the pixel value is observed the pixel particular value is retained as it is assumed to be the part of the foreground object.

This probability algorithm allows us to reconstruct an image which retains the pixel values of the pixels which is the part of the foreground object and removing the pixels which is a part of the background image and is not required. This is the process that takes place with in the foreground extraction module of this project.

This algorithm is to construct the foreground image using the standard background image as reference. This algorithm sums up one of the module of our project that deals with image processing in MATLAB environment.

Interms of MatLAB code the above algorithm can be implemented in the form described bellow. This code constructs the moving object segmenting out from the video that involves the common background. As it is evident from the algorithm the code goes for a pixel to pixel comparison which is done with reference to the standard background.

>>[m n]=size(nback); %size of the image as a matrix.

% now the challenge is to consider each and every pixel value of the image matrix compare the pixels of the two image and consider a the pixel if it is different when compared to the pixel at same location.

% i am considering a threshold or a tollerence level of +/- 40.

>>frame=frame+1; % for intimating the number of frames under process.

>>for i= 1: m

>> for j= 1:n

h=mean2(nbackground(i,j));

l=mean2(background(i,j));

p=h-l;

if fback(i,j)==nback(i,j)

foreground(i,j)=0;

elseif (double(fback(i,j))-20)

foreground(i,j)=0;

else

foreground(i,j)=uint8(nback(i,j));

>> end

>> end

>> end

As it is a probability function it is expected to have plenty of noise signals as an attribute after the processing of the image under the above algorithm. These noise signals should be removed in order to have more appropriate segmentation with properly defined foreground object boundaries. There are various functions that can be utilized available with MatLAB image processing toolbox for removing these noise signals.

Among various tools available with image processing toolbox one of the most significant category or feature that comes with this toolbox is the morphological functions which gives image processing a new dimension, that is useful in various aspects.

BWAREAOPEN

Binary area open; remove small objects

BW2 = bwareaopen(BW,P) removes from a binary image all connected components (objects) that have fewer than P pixels, producing another binary image, BW2. The default connectivity is 8 for two dimensions, 26 for three dimensions, and conndef(ndims(BW),'maximal') for higher dimensions

3.4 Motion Detection

Once we have acquired the video from an image acquisition device interfaced with the system, controlled by MatLAB coding we go for foreground extraction which provides the segmented video of the moving object in the foreground image. Done with all this next task is the process of identifying any kind of motion automatically with respect to the standard image or the stagnated section of the video. This module thus deals with the detection motion by implementing the algorithm for motion detection.

3.4.1 Algorithm :

1. We consider the standard background image or the first frame of the video as the reference image for comparing the other frames of the video in order to detect any kind of motion that is there in terms of any object is concerned.
2. Here we go for reference or background to foreground subtraction to identify any sufficient change the pixel values. If the change in the pixel value is sufficient enough such that the change can be marked as any motion is what the difference of the two frames is observed for.
3. The difference image that is constructed when the difference between the two images that is the reference image and the current frame of the video, is first converted to binary and then the total number of 1’s will identify the change weather it is good enough value to declare motion detected or the change is too small to be considered as a motion of significant amount.
4. Here as one can understand that the image has a considerable amount of movement or not. If there is initially some motion that is detected and in the case when the diff is found to be equal for few consecutive frames. This would indicate that the moving object has stopped again. And hence it is this frame which is considered as the new reference frame for detecting any kind of motion.

These above steps are coded in MatLAB and executed to detect any kind of motion with respect to the input video. The motion tracking continues throughout the video with respect to the standard background which changes as per the motion of the moving object in the captured video. This satisfies the aim of the project where we consider an indoor surveillance system. This system aims to find any kind unwanted disturbances in a highly secure zone such as bank vaults or military weapon hanger, and many other such highly secure zone.

>>bac=vid{1};

for i=1:length(vid)

if count>4

bac=vid{i-2};

count=0;

end

fore=vid{i};

imagesc(fore);

axis image off

drawnow;

pause(0.5);

diff=imabsdiff(fore,bac);

bw=im2bw(diff);

pix{i}=sum(bw);

pix{i}=sum(pix{i});

imshow(fore);

hold on;

if pix{i}>50

data=1;

disp('motion detected');

else

data=2;

disp('motion not detected');

end

if ((pix{i}>1) && (pix{i}==pix{i-1}))

count=count+1;

end

fprintf(s,'%d\n',data);

end

As we have seen how the three modules of our project is written in MatLAB and executed to derive the required or desired result that is motion tracking. As we realize that a MatLAB environment can be provided in a personal computer, which executes the code and returns the desired result. When we consider its application the result restricted only to the personal computer system doesn’t meet the need. The result of the processing as per as the motion detection is considered has to be driven out and given to an external alarm or a public addressing section and at the same time can be used for various other operations such as, an automatic evacuation or an automatic sealing of the restricted area and even this data is desired at a desired when the control room is at a distance from the field of action. Such needs is satisfied by serial communication of the personal computer system with the external hardware unit. We send various kind of external signal as per the condition that is detected.

3.5 Serial Port Communication ( MatLAB )

This serial communication is possible between the personal computer and the instrument or the hardware module in MatLAB environment by making use of the various functions related to the serial communication available in Instrumentation Control Toolbox which comes with MatLAB. Used widely for instrument control through various communication protocols available. In this project we opt for serial communication. The bits transferred from the PC through the serial port communication is given as input to the microcontroller installed in the hardware module which takes action as per the coding of it is concerned.

Instrumentation Control Toolbox.

Control and communicate with test and measurement instruments

The Instrument Control Toolbox lets you com­municate with instruments, such as oscilloscopes, function generators, and analytical instruments, directly from MATLAB®. With the toolbox, you can generate data in MATLAB to send out to an instrument, or read data into MATLAB for analysis and visualization.

The toolbox provides a consistent interface to all devices independent of hardware manufac­turer, protocol, or driver. The toolbox supports IVI, VXIplug&play, and MATLAB instrument drivers. Support is also provided for GPIB, VISA, TCP/IP, and UDP communication protocols.

Thus the hardware is interfaced with the personal computer through serial communication using the functions available with the above mentioned toolbox. As per as the conditioned achived when processed for motion detection we send numerical data through the serial port to the hardware unit which is received by the microcontroller through MAX232 and the required action is taken as per as the microcontroller code is concerned. Let us look upon some details that is required to develop any communication using serial communication.

What Is Serial Communication?

Serial communication is the most common low-level protocol for communicating between two or more devices. Normally, one device is a computer, while the other device can be a modem, a printer, another computer, or a scientific instrument such as an oscilloscope or a function generator.

As the name suggests, the serial port sends and receives bytes of information in a serial fashion -- one bit at a time. These bytes are transmitted using either a binary format or a text (ASCII) format.

The Serial Port Interface Standard

Over the years, several serial port interface standards for connecting computers to peripheral devices have been developed. These standards include RS-232, RS-422, and RS-485 -- all of which are supported by the serial port object. Of these, the most widely used standard is RS-232, which stands for Recommended Standard number 232.

The current version of this standard is designated as TIA/EIA-232C, which is published by the Telecommunications Industry Association. However, the term "RS-232" is still in popular use, and is used in this guide when referring to a serial communication port that follows the TIA/EIA-232 standard. RS-232 defines these serial port characteristics: The maximum bit transfer rate and cable length The names, electrical characteristics, and functions of signals The mechanical connections and pin assignments Primary communication is accomplished using three pins: the Transmit Data pin, the Receive Data pin, and the Ground pin. Other pins are available for data flow control, but are not required.

Connecting Two Devices with a Serial Cable

The RS-232 standard defines the two devices connected with a serial cable as the Data Terminal Equipment (DTE) and Data Circuit-Terminating Equipment (DCE). This terminology reflects the RS-232 origin as a standard for communication between a computer terminal and a modem.

Throughout this guide, your computer is considered a DTE, while peripheral devices such as modems and printers are considered DCEs. Note that many scientific instruments function as DTEs.
Because RS-232 mainly involves connecting a DTE to a DCE, the pin assignments are defined such that straight-through cabling is used, where pin 1 is connected to pin 1, pin 2 is connected to pin 2, and so on. A DTE to DCE serial connection using the transmit data (TD) pin and the receive data (RD) pin is shown below

3.6.1 Why Microcontroller?

Meeting the computing needs of the task at hand efficiently and cost effectively

Ø Speed

Ø Packaging

Ø Power consumption

Ø The amount of RAM and ROM on chip

Ø The number of I/O pins and the timer on chip

Ø How easy to upgrade to higher-performance or lower power-consumption versions

Ø Cost per unit

· Availability of software development tools, such as compilers, assemblers, and debuggers

The 8051 family has the largest number of diversified (multiple source) suppliers

Ø Intel (original)

Ø Atmel

Ø Philips/Signetics

Ø AMD

Ø Infineon (formerly Siemens)

Ø Matra

Ø Dallas Semiconductor/Maxim

Intel introduced 8051, referred as MCS-51, in 1981

· The 8051 is an 8-bit processor

· The CPU can work on only 8 bits of data at a time

Specifications of 8051 are:

Ø 128 bytes of RAM

Ø 4K bytes of on-chip ROM

Ø Two timer

Ø One serial port

Ø Four I/O ports, each 8 bits wide

Ø 6 interrupt sources

The 8051 became widely popular after allowing other manufactures to make and market any flavor of the 8051, but remaining code-compatible

In this project we have used AT89C51 Microcontroller from Atmel Corporation which has the features like:

Ø Flash (erase before write)

· ROM burner that supports flash

· A separate eraser is not needed

The most widely used registers:

A (Accumulator) For all arithmetic and logic instructions

B, R0, R1, R2, R3, R4, R5, R6, R7

DPTR (data pointer), and PC (program counter)

Compilers produce hex files that is downloaded to ROM of microcontroller.

The size of hex file is the main concern

· Microcontrollers have limited on-chip ROM

· Code space for 8051 is limited to 64K bytes

The programming can be done either in Assembly language or C language.Programming in Assembly language is tedious and time consuming.C programming ,on the other hand, is less time consuming and much easier to writ,but the Hex file produced is much larger than if we used Assembly language.The following are some of the reasons for writing programs in C instead of Assembly language

1. It is easier and less time consuming to write in C than Assembly

2. C is easier to modify and update

3. You can use code available in function libraries

4. C code is portable to other microcontrollers with little or no modification

A good understanding of C data types for 8051 can help programmers to create smaller hex files

· Unsigned char

· Signed char

· Unsigned int

· Signed int

· Sbit (single bit)

· Bit and sfr

We should stick with the unsigned char unless the data needs to be represented as signed numbers Ex: temperature

HARDWARE

8051 family members (e.g, 8751, 89C51, 89C52, DS89C4x0) have 40 pins dedicated for various functions such as I/O, -RD, -WR, address, data, and interrupts

These come in different packages, such as

· DIP(dual in-line package),

· QFP(quad flat package), and

LLC(leadless chip carrier)

Object Identification:

This system although has the capability to track motion of an object it still cant discriminate the object as per its structure. The system tracks motion but it is any kind of motion need not be human motion only. An up gradation to tracking and identification of human motion will make the system more application oriented.

Robust Surveillance:

More over this system is based on internal or indoor surveillance. When it comes to outdoor surveillance the dynamic background offers a challenge of reconstructing the background using a fast algorithm. This enhancement in ability to construct the background would even provide much more better results in the present scenario too.

Faster Processing:

We look for a real-time system so enhancing the processing speed would give spontaneous results as per as the foreground extraction is concerned. For faster response we need to use better and faster processor and algorithms that process at a faster rate.

• DSP – Processor
• FPGA Kit

Faster & Better Communication:

For faster and more reliable communication we can go for better and faster protocols that will provide even parallel or wireless communication options. This makes the system faster and user oriented when we talk about wireless means of communication for real-time operations.

Proper Filtering:

Better filter usage will remove noise and unwanted pixels which creeps up during segmentation. Proper filters has to be used which can remove enough amount of noise without spoiling the required segment.

Effective Edge & Object Detection:

Proper edge detection is a technique that can be used for proper extraction of the foreground object. These will help to construct proper and sharp extracted images.

These developments will increase its field of application making this project more effective in real-time environment.

Higher-End Video Capturing Aid:

A proper high definition camera will also make the process more effective but using a higher end processor, as it will effective reduce the speed of the program as one will have more number of pixels to work on.