Apple’s Spatial Reframing Produces an AI I Can Get Behind as a Photographer
We knew Apple’s Worldwide Developers Conference would be full of AI, but I didn’t expect to see an image feature that made me think: “This is wild.”
During the WWDC keynote on Monday, Apple showed off a few new editing features in its Photos app that I think will be really useful. In addition to the existing Cleanup tool, which can remove unwanted distractions from the image, we will also be able to stretch the edges of the image.
But it was Spatial Reframing, a feature that lets you adjust the composition of the image to reflect where you wish you were standing to take it, that caught my eye.
All these features are used generative AIand will be included in the new tools category in the editing area of the graphics application. The first developer beta of iOS 27 available now to registered developers.
More AI, less slop
Generative AI is a technology that photographers are avoiding (or should be), for everyone’s sake AI slop produced everywhere. And yes, that includes creations from Apple A photo playground operating system, i image generator that the company also exhibited during the event The highest number of WWDCs.
But generative AI doesn’t need to dictate full images created from text commands. When used in selected areas, such as wiping away a piece of trash near a subject’s feet, generative AI can do some of the menial work of changing pixels that photographers would spend time touching up in an app like Photoshop. Google Pixel phones install the same Magic Eraser tool.
Spatial Reframing is a great example of how technology can be used to enhance the real images you capture.
How Spatial Reframing will work
Apple’s Landscape Photos technology uses AI to find depth in a flat image, giving a responsive 3D effect when you tilt your phone or look at it Vision Pro headseteven if it wasn’t shot as a local shot. It can give depth to The iPhone turn off screenshots, too.
In fact, the quality of the result is very good. The separation between the subject of the image and the background is usually not surprising, and it does not have a “cut” look. But it’s mostly a neat gimmick on the iPhone (I don’t have Vision Pro to hear it in that place).
Spatial Reframing takes that technology and makes it practical. As shown in the main note demo, you will be able to drag the image to adjust the perspective of the shot. The background will adjust as if you took a physical step to the side or repositioned your camera to get a better angle.
The original image (left) undergoes a change of perspective (middle) and Scene Reconstruction fills in the background details using artificial intelligence (note the structure in the distance that was not visible in the original.
Image editing software such as Adobe Lightroom allows you to adjust the plane of the entire image, rotating it around a central axis for limited reframing, but at the cost of distorting the image.
After reframing the shot, the Photos app uses artificial intelligence to fill in any areas around the edges.
Apple says it uses on-device spatial modeling to detect depth, and its proprietary cloud computing architecture to process image generation.
“It produces new content only to fill the gaps when the perception has changed,” said Alok Deshpande, Apple’s director of Camera and Photos Software. “This ensures that the reconstructed image remains consistent with the original scene.”
The result is a picture from the place you want to move to when you take the picture.
Whether the edited images are actually as clean as they were downloaded remains to be seen. Sometimes I use an existing one Clean up feature in Photos, but it can be very hit or miss in terms of the quality of the pixels produced. With the new photo models promised in iOS 27, I hope the edited photos are ones any photographer would be proud to share — maybe not in a gallery or a contest, but informally with friends or on social media.
