Camera: Sony Image Sensor— AI Initiatives — ECCV2020

Session 1
Intelligent Vision Sensor

Sony has developed innovative image sensor called intelligent vision sensor. They are the first image sensors in the world to be equipped with AI processing functionally. The sensor was realized by utilizing Sony’s advanced stacking technology and original DSP for AI. Including AI processing functionality on the image sensor itself enables high-speed edge AI processing and extraction of only the necessary data, reduces data transmission latency, addresses privacy concerns, and reduces power consumption and communication costs in using cloud services. These products expand the opportunities to develop AI-equipped cameras, enabling a diverse range of applications in the retail and industrial equipment industries and contributing to building optimal systems that link with the cloud.

Session 2
Online and Offline 3D Mapping

Sony RDC is doing research in the field of Computer Vision. Among others, we have implemented state-of-the-art systems in 3D modelling; both: online and offline reconstruction. For rendering possible 360˚ reconstruction and sensing in real time with low power consumption, we have developed a processor specifically for vision sensing. This processor can connect up to 12 cameras, IMUs, etc., and performs stereo depth estimation, SLAM, occupancy mapping, etc., simultaneously. The focus of our offline 3D modelling system comprises the reconstruction of high-resolution as well as large sets of images. We have developed effective solutions on Multi-View Stereo, depth refinement and mesh reconstruction and show superior results on well-known academic benchmarks. Combining online and offline reconstruction enables a unique VR experience and extended robotic applications.

Session 3
Sony AI

Sony AI is a new organization, established in November 2019, with offices in Japan, the US and Europe. With a mission to unleash human imagination and creativity, we believe in researching and developing AI techniques that empower artists, makers and creators around the world. In this session, I will introduce Sony AI’s approach to AI research, and discuss some of our first projects.

Business use case

Case 1
Sony’s World’s First Intelligent Vision Sensors with AI Processing Functionality
Enabling High-Speed Edge AI Processing and Contributing to Building of Optimal Systems Linked with the Cloud

Sony Corporation announced two models of intelligent vision sensors, the first image sensors in the world to be equipped with AI processing functionality. Including AI processing functionality on the image sensor itself enables high-speed edge AI processing and extraction of only the necessary data, which, when using cloud services, reduces data transmission latency, addresses privacy concerns, and reduces power consumption and communication costs.

Case 2
Using imaging information
that humans can’t see in AI applications
Polarization image sensor

1) Polarization Image Sensor with Four-Directional on-chip Polarizer and global shutter function

Case 3
Eye AF in “Alpha” Series and “Xperia 1”
Focus on Shooting with Eye AF and Capturing the Decisive Moment of Kando

1) Certain Success through AF Technology The Strong Belief Supporting Advanced Technology

Case 4
Image Recognition Technology on aibo
~ Building Eyes of aibo to Live in Homes ~

aibo uses its eyes (cameras) to recognize the surroundings and modifies the behavior based on what it sees. It identifies owners and detects friends (other aibos) and toys to play with. Edge-friendly lightweight algorithm based on deep learning allows to recognize rapidly to interact with people. With the depth sensor, it can reach to the toys (ex. Pink balls/bones) while avoiding obstacles. A camera on the back realizes SLAM for building the map of the room and it can be used for go-to-charger and patrol application.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
For 4D Sensing, AR, AV/Robot, and AIoT

For 4D Sensing, AR, AV/Robot, and AIoT

Marc. Y. , Tech Leader. Focus on new techs such as 4D Sensing, AR, AV/Robot, and AIoT. More at http://4da.tech/