Apple, in collaboration with researchers from the University of California, has released its own AI-based image editing model that and anybody can try.
The model has been dubbed “MGIE,” which stands for MLLM-Guided Image Editing and allows users to edit images based on natural language instructions. It leverages multimodal large language models (MLLMs), which combine various types of information, such as text, photos, and videos, to understand and generate human-like language.
While studies on MLLMs have shown promising results in understanding and generating images, they haven’t been implemented widely yet.
MGIE was presented in a paper at the International Conference on Learning Representations 2024. The paper showcases how MGIE can improve automatic metrics and human evaluation while also maintaining competitive inference efficiency.
MGIE utilizes MLLMs to derive expressive instructions from user input. Venture Beat offers an example: a user could tell MGIE to “make the sky more blue,” and MGIE would interpret this as “increase the saturation of the sky region by 20%.”
How MGIE interprets natural language | Image Credit: Apple, UC Santa Barbara
Then, once the instructions are derived from the natural-language input, it generates an edited version of the image. It can be used to create a wide range of edits, ranging from color adjustments to generating or removing parts of an image.
MGIE is open-source and available on GitHub for anyone to try. The GitHub page allows users to snag the code, data, and pre-trained models.
The speed of image generation will vary significantly based on hardware performance. Also, models such as this are incredibly RAM hungry.
Apple has been working behind the scenes on its AI-assisted features for some time now. In January, code in the iOS 17.4 beta suggested that Apple is working on a new AI-powered version of Siri.
Apple stock has taken a beating as of late, in part because analysts have loudly proclaimed that the company is behind Meta, Google, and Microsoft in generative AI implementation. It’s not clear why this wasn’t a problem when it wasn’t first to a mobile phone, a tablet, a smartwatch, or a VR headset, but is with generative AI.
Perhaps to dissuade those analysts, Apple CEO Tim Cook recently said that Apple is spending “a tremendous amount of time and effort” on AI features that will be announced in the coming months. He also noted that Apple has been working internally on generative AI.
This story originally appeared on Appleinsider
