Tactical infrastructure like fencing, roads, and lights are critical to securing a nation’s border. However it alone is not enough to avoid the unlawful movement of individuals and contraband into a country.
“Technology will be the primary driver of all land, maritime, and air domain awareness – this can become only more apparent as [U.S. Customs and Border Protection (CBP)] faces future threats,” based on testimony from CBP officials with a Senate hearing on homeland security in 2015.
And machine vision’s fingerprints are over that technology. “The details obtained from fixed and mobile surveillance systems, ground sensors, imaging systems, and other advanced technologies enhances situational awareness and enables CBP to detect, identify, monitor, and appropriately react to threats inside the nation’s border regions,” the testimony states.
On the U.S.-Mexico border within the state of Arizona, as an example, Top Machine Vision Inspection System Manufacturer persistently detect and track so-called “items of interest.” Built to withstand its harsh desert surroundings, IFT is equipped with radar, commercial off-the-shelf daylight cameras and thermal imaging sensors, and microwave transmitters that send data to border agents at the Nogales station for analysis and decision-making.
On all 3 fronts of land, maritime, and aerial surveillance, machine vision companies are providing imaging systems – and, more often, research into the generated data – that meet government agencies’ objectives of flexibility, cost effectiveness, as well as simple deployment in border security applications.
Managing Diverse Conditions – The perennial problem with vision systems found in border surveillance applications is handling the diversity of an outdoor environment using its fluctuating lighting and weather conditions, as well as varied terrain. Inspite of the challenges, “you can find places that you can implement controls to improve upon the intelligence of the system,” says Dr. Rex Lee, president and CEO of Pyramid Imaging (Tampa, Florida). He points to customers who monitor trains along the southern border of the U.S. for illegal passengers.
“Those trains will need to go within a trellis, which can be equipped with the appropriate sensors and lighting to assist inspect the trains,” Dr. Lee says. Government departments given the job of border security use infrared cameras to detect targets during the night and in other low-light conditions, but thermal imaging does have its limits, too. “Infrared cameras work really well once you can make use of them in high-contrast conditions,” Dr. Lee says. “But when you’re seeking to pick up a human at 98.6°F on the desert floor which is 100°F, the desert is emitting radiation at nearly the identical area of the spectrum. So customers depend on other areas of the spectrum including shortwave infrared (SWIR) to try to catch the difference.”
Infrared imaging works well in monitoring motorized watercraft because the boat’s engine includes a thermal signature. “What’s nice about water is the fact it’s relatively uniform and it’s very easy to ‘wash out’ that background and see anomalies,” Dr. Lee says.
But the problem is that the oceans present a huge amount of area to protect. Says Dr. Lee, “To view all of it is a compromise between having a lot of systems monitoring the water or systems which can be rich in the sky, in which case you will find the problem of seeing something really tiny in a very large overall view.”
CMOS Surpasses CCD – One key change in imaging systems used in border surveillance applications is the shift from CCD to CMOS sensors as the latter is surpassing the standard and performance of the former. To accommodate this change, a couple of years ago Adimec Advanced Image Systems bv (Eindhoven, the Netherlands) integrated the newest generation of CMOS image sensors – that offer significant improvements in image quality and sensitivity – into its TMX series of rugged commercial off-the-shelf cameras for top-end security applications. TMX cameras keep a maximum frame rate of 60 fps or 30 fps for RGB color images at full HD resolution.
Furthermore, CMOS image sensors are emerging as a replacement for electron-multiplying CCDs (EMCCDs), says Leon van Rooijen, Business Line Director Global Security at Adimec. Thanks to their superior performance over CCDs in low-light conditions, EMCCDs often are deployed in applications like harbor or coastal surveillance.
But EMCCDs have distinct disadvantages. As an example, an EMCCD must be cooled in order to offer the most effective performance. “That is quite some challenge inside the sense of integrating power consumption as well as the fact that you have to provide high voltage to the sensors,” van Rooijen says. “And if you need to have systems operating for any long duration without maintenance, an EMCCD is not really the very best solution.”
To resolve these challenges, Adimec is concentrating on image processing “to get the most out of the newest generation CMOS in the future nearer to the performance global security customers are utilized to with EMCCD without all of the downsides from the cost, integration, and reliability,” van Rooijen says.
Adimec also is tackling the process of mitigating the turbulence that takes place with border surveillance systems over very long ranges, particularly as systems which were using analog video are now taking steps toward higher resolution imaging to cover the bigger areas.
“When imaging at long range, you may have atmospheric turbulence by the heat rising from the ground, as well as on sea level, rising or evaporated water creates problems regarding the haze,” van Rooijen says. “We are going to show turbulence mitigation within the low-latency hardware embedded in our platform and will work with system integrators to optimize it for land and sea applications since they possess the biggest difficulties with turbulence.”
A Lot More Than Pictures – Like machine vision systems deployed in industrial applications, border home security systems generate lots of data that needs analysis. “The surveillance industry traditionally has been a little slower to add analytics,” says Dr. Lee of Pyramid Imaging. “We percieve significant opportunity there and also have been working with a lot of our customers so that analytics tend to be more automated with regards to what is being detected and also to analyze that intrusion, and after that have the ability to have a proper response.”
Some companies have developed software that identifies anomalies in persistent monitoring. As an example, if a passenger on the airport suddenly abandons a suitcase, the software will detect that the object is unattended nefqnm everything around it consistently move.
Even with robust vision-based surveillance capabilities whatsoever points of entry, U.S. border patrol and homeland security have to contend with a much bigger threat. “America does a pretty good job checking people to arrive, but we all do a very poor job knowing if they ever leave,” Dr. Lee says. “We know how you can solve that problem using technology, but that produces its very own problems.
“The best place to get this done is at the Automated Vision Inspection Machines in the TSA line, where you can have a mechanism to record everybody,” Dr. Lee continues. “But that is going to be expensive because you need to do this at each and every airport in the usa. Monitoring and recording slows things down, and TSA is under a lot of pressure to speed things up.” Another surveillance option that government departments have discussed has taken noncontact fingerprints at TSA each and every time someone flies. “Most of the American public won’t tolerate that,” Dr. Lee says. “They will debate that fingerprinting is simply too much government oversight, which will result in a large amount of pressure and pushback.”