How Microsoft developed its Smart Surface camera

Microsoft has yet again carved a niche for itself by building its surface Hub 2 Smart camera. The company said it had planned to upgrade the device previously before this time and now the surface Hub 2 Smart camera is here with some mind-blowing and unique features one can’t resist best suitable for all individuals especially those in the corporate organization.

Steven Bathiche is in charge of all hardware Innovation for Microsoft devices. The Microsoft surprise $799.99 surface hub 2 smart camera debuted last week, Which came with an automatic refraining without the warping one usually see on other conference room cameras. It comes with Special Features that can detect faces and bodies which enables everyone in room visibility during a meeting, when taking a picture or whether the person is close to the camera or up to eight metres away.

The surface Hub 2 Smart camera can capture as much as an entire conference room due to its 136-degree field of view, making the people at the front of the camera focus alongside those in the bank. With this upgrade, the Microsoft surface hub 2 camera before Covid’19 put hybrid meetings into focus, the reason why it’s modular and can be separated from the top of the 55 or 85-inch displays. Steven Bathiche said ” We know we are going to evolve the experience, but we didn’t know exactly how, but we knew that was going to change and needed to change with people’s needs. The evolution of the conference room and how our culture will essentially adapt towards meetings.

The smart surface camera virtually captures everyone in a conference meeting with a traditional camera. Microsoft is coming up with this brilliant idea was mainly because they felt cameras are in need to handle bigger rooms. The team also created Microsoft’s optics, Artificial Intelligence model and edge computer to go into the Surface Hub 2 Smart camera. The power of this device is computational photography because it has onboard compute, 1 Teraflop of computing that accommodates a really large Artificial Intelligence Model that has been built.

The surface Hub 2 also include the auto framing application, it resides in the camera so that when used the image is just as a 4k image making it look exactly like a webcam to the Surface Hub. All of the Artificial Intelligence work on this device is done on the camera itself and never sent to the cloud or over the wire to the Surface Hub 2 to process. The device enables features like an AI model, Processes all the data, and makes the decision to crop the image accordingly. Also, the automatic framing can grasp everyone automatically in the conference room. The device also uses tilt compensation to set the image of the camera position and enhance more natural eye contact. What is, even more, cooler about this device is that you can remove the fisheye effect from wide lenses so people don’t look out of place or stretched inside the meeting room.

Steven Bathiche while explaining what makes the device unique stated that Microsoft had designed an 11 element, completely glass lens with a super sharp focus and close to the refraction limits. Behind the lens is a 12- megapixel sensor (4000×3000) with an F/1.8 aperture that generates the 4k cropped image. “The actual lens is a 184-degree field of view, so that camera can look behind itself and knowing how the device operates most of this hardware is nothing without the AI Models that powers the surface Smart camera. Although this project commenced before the pandemic, its AI model had to go through adequate and vigorous training during the pandemic, which is why this project was focused on how the world had evolved during the pandemic era in the area of having to work remotely and also have meetings at the comfort of one’s house. This project was done in New Zealand because that’s the only country with zero Covid’19 cases, and Microsoft had to open an office there to bring the surface Hub 2 Smart camera to life.

Steven Bathiche went further by giving details on how actors and actresses were hired to collect and collate data in all, which made their data set massive. Microsoft had trained his Artificial Intelligence model on faces and Bodies to ensure it fully detects people that are not always facing the camera, it also improves the diversity through the use of synthetic people and faces. “We have a cool internal Technology that can generate synthetic people and faces. One thing about this camera is that despite all its amazing features it isn’t trained to detect pets or animals. Microsoft ensures it applied its responsible AI principles to this project by including a committee and set of tools to enable fairness and artificial intelligence.

The robustness and inclusiveness are one of the special things which makes this camera unique which people might not necessarily get to see on the box, by our data it’s amazing across the board in term of disparity between various groups, gender, skin tone, hairstyle and race.

Steven Bathiche emphasises that Microsoft has carved a niche in his words “Microsoft sat and turned the heck” out of the auto framing capacity of its smart camera over the years to ensure it is not forward or lacking behind when it comes to content. Therefore every frame that the camera gets decides whether it’s worth it to move or receipt the image. He stated that the design wasn’t really designed to be a regular webcam but it was specifically designed specifically for the Surface Hub 2 Smart camera to suit its Special features.