Security and surveillance, artificial intelligence (AI), automotive systems, consumer products—image sensing technology has a home within them all. As sensor research and development trudges forward, the need for new, advanced solutions for demanding recognition-based applications continues to grow.

The ECN Q&A below features Leo Bai, general manager of Artificial Intelligent BU at SmartSens Technology, where we discuss global CMOS image sensor design, industry trends, benefits for emerging applications, the SmartSens SC031GS product, and more.

ECN: How would you describe the current state of the global CMOS image sensor market in terms of product design and industry trends?

Bai: As advanced CMOS image sensor design technology, global shutter is trending and frequently used in the market. Global shutter CMOS image sensor (CIS) is designed to capture moving objects or objects in motion without any image distortion.

Only with high frame rate capacity, rolling shutter CIS can capture moving objects with little image distortion. However, there are several problems by doing so:

1. Rolling shutter CIS with high frame rate is hard to design and costs more.
2. The system has to use special platforms (e.g. high-end SoC/FPGA) with high bandwidth, in order to work with the high frame rate rolling shutter CIS.
3. High frame rate CIS can only be used for very short exposure time. For example, the CIS with 1,000 fps capacity can only be used for 1 ms as the maximum exposure time, otherwise, the frame rate will be reduced. Lower fps will result in more image distortion. Therefore, in order to get brighter images, the system has to use larger gain, which causes more noise.

All of these problems limit image sensor applications, but global shutter technology provides the solution.

Originally, global shutter CIS are mainly used in the field of factory automation (FA) and intelligent transportation systems (ITS). With the rapid development of AI and machine vision (MV) technology, more and more emerging applications (for example, drones, autopilot, robots, logistics, IoT, etc.) need global shutter CIS to improve system efficiency and intelligence levels.

However, the challenge is that only few global shutter products can be chosen by customers for various new applications for two reasons:

1. Although the high-end global shutter CIS that is mainly used for FA market has excellent performance, it also means higher cost, larger package form, and higher-level interface. So, high-end global shutter is not suitable for all applications in the AI and MV markets.
2. Low-end global shutter CIS has the cost advantage, but its performance cannot meet the data accuracy requirement of the AI and MV algorithm.

In order to meet the market needs, SmartSens has developed cost-effective, high-performance global shutter CIS for emerging AI and MV applications, i.e. the world’s first BSI-based global shutter CMOS image sensor family (SmartGS). Because of its excellent signal-to-noise ratio, high sensitivity, and dynamics, the SmartGS image sensors are widely used in FA, code reader, intelligent sensing, and other fields.

ECN: How do you think the design of CMOS image sensors will change within the next 3-5 years?

Bai: With rapid development of AI and MV technology, there will be a growing number of new applications that need CIS to get direct data for processing and analyzing. This will present much broader market opportunities for image sensors than ever before. This will also help CIS technology improve in sensitivity, SNR, dynamic range, cost, package form, and so on. It will also require dedicated CIS technology for different industry applications.

ECN: How can emerging technologies and a variety of applications benefit from new, intelligent imaging solutions?

Bai:  How to integrate AI technology with various application scenarios has become one of the hottest trends for almost all major industries. Every enterprise and R&D team is trying to provide new product form and better user experience with AI technology for new market demands and end users.

The essence of AI technology is to use intelligent algorithms for analysis, feature extraction, information comparison, and recognition based on acquired data, and then perform a series of corresponding automated processing flow based on analysis results.

The use of AI technology is inseparable from the processing of data, and image data is the most objective and direct expression of the real world, so it is closely linked to AI technology. Therefore, new intelligent imaging solutions such as CMOS image sensor technology will be able to provide high-quality image data for the AI technologies, so its algorithm can process more accurate image data for innovative user applications for different industries.

For example, the SmartGS series image sensor has been widely used in many intelligent sensing applications, including obstacle avoidance and optical flow in drone applications. It has also been used for vSLAM in robotic applications. This CMOS image sensor enables more accurate gesture recognition, face recognition, and iris recognition faster than ever before.

ECN: SmartSens recently launched the SC031GS CMOS image sensor. What are some notable product features that aren’t currently matched by competitors?

Bai: SC031GS is the world’s first commercial-grade 300,000-pixel global shutter CMOS image sensor based on BSI pixel technology. BSI technology makes SC031GS more sensitive. Derived from high-end industrial CIS, SmartGS design technology guarantees low system noise, high-shutter efficiency, single-frame HDR, among other features. Because of these high performances, SC031GS is becoming popular in intelligent sensing (drone, robotics) and IoT industries. Designed for intelligent applications like face to vSLAM, optical flow, 2D-decoder, and gesture recognition, SC031GS’s design is fully considered about system solution. So, SC031GS is better suited for intelligent applications than other competitors.

For comparison, Figure 1 shows a photo without the CMOS image sensor from the SmartSens SmartClarity™ family.

Figure 2 shows a photo with the CMOS image sensor with HDR feature from the SmartSens SmartClarity™ family.