NVIDIA’s AI system can create photorealistic images from your drawings

NVIDIA's AI system can create photorealistic images from your drawings

Credit: NVIDIA

For many, the difference between AI and the human brain lies in creativity. It’s hard to teach a machine to express itself, though that may be about to change.

NVIDIA has created an AI system that turns your scribbles into photorealistic landscape images. GauGAN Beta gives users a canvas resembling Microsoft Paint to draw onto. As if by artificial intelligence-powered magic, the machine creates an image on the right of your screen based on what you’ve cobbled together with the painting and filling tools.

The results are somewhat rough so far. You can, however, switch up the filter of your picture to create different effects. The tool demonstrates how far “machine drawing” has come in the last few years.

Creativity is a difficult thing for science and technology to analyse. Expression often comes from specific influence and human experience; it’s hard to quantify a particular magic about an artistic piece and the greatest pieces of artwork, whether musical, painted or otherwise, are often iconic, at least partly, thanks to a backstory. For years, there has been a debate about whether machines and computers – especially AI – can truly produce art to rival a human being.

NVIDIA’s latest attempt to create an algorithm that produces art is not strictly AI art, really. The tool relies on a user to draw a basic sketch first. It may be debatable as to whether AI can create art, but this system simply assists with photorealistic collages.

It is inevitable that programs like Adobe Photoshop will incorporate more AI capabilities. Artificial intelligence can be used to smartly erase pixels, generate parts of an image to save you time and provide clever edits based on how a picture looks. Is it taking the fun out of photo editing? Maybe. AI can make photo editing and drawing far more accessible to the average user, though.

How does AI assist with images?

Cameras obviously take photos by analysing what it sees, pixel by pixel. Most cameras don’t use artificial intelligence though. They simply take an aggregate colour sample for every pixel of their resolution.

This is where AI comes in. Whilst a camera will analyse surroundings to take a sample colour, AI creates the sample colour itself based on data. The data in the case of the NVIDIA system is whether you want to create a rock, tree, plant, river or something else.


Creativity is a difficult thing for science and technology to analyse.


In the case of Photoshop, Adobe introduced a Content-Aware tool a few years ago. Say you have a picture of a person standing on a beach. You can select the subject and use Content-Aware to delete them seamlessly from the image: AI is used by the Content-Aware tool to fill in the pixels with randomly generated colours, based on how the rest of the image looks. So if you’re erasing someone from a beach scene, the chances are that Content-Aware will fill in the space with similar blues and greens of the sea and similar yellows of the beach.

These kinds of tools usually employ Machine Learning (ML). A set of two or more ML algorithms work in conjunction with each other: one tries to produce a realistic image whilst the other one challenges the flaws of that image. It’s similar to the way that deepfake works and by learning from its mistakes, it manages to produce better images based on data.

The more data that you feed an algorithm, the more realistic your finished image will be. The NVIDIA AI system is a little rough so far and doesn’t create the most like-life images so far but it isn’t too much of a stretch to imagine more defined, authentic images with a more powerful processor.

How could this AI system be used?

The tales of what artificial intelligence could do are mind-boggling. There’s talk of how just the picture of an ear could be used to generate an entire face from AI and be used in criminal reconstructions. It’s possible to create extremely realistic computer-generated models for retail.

Creating realistic landscapes with quick sketches opens doors in the worlds of entertainment and digital art. Factor in the imminent rise of VR, thanks to 5G, and it might not be long before we are able to build three-dimensional worlds simply by drawing them out in a Paint-like program first.

In the world of design, AI could be groundbreaking. Architects could use systems like the NVIDIA GauGAN to create designs with realistic textures in a flash. Interior designers could do a similar thing, utilising the speed and smartness of AI to bring in textures to their designs easily. If you’re looking to create a mock-up of anything, from a building to a complete garden, artificial intelligence could well become the default starting block for designers of all walks of life.

For now however, GauGAN is a tool for messing around and having fun. AI might not be inherently creative itself, but there’s no reason that you can’t get creative with it.

Luke Conrad

Technology & Marketing Enthusiast

The Future of Smart Buildings: Trends in Occupancy Monitoring

Khai Zin Thein • 12th June 2024

Occupancy monitoring technology is revolutionising building management with advancements in AI and IoT. AI algorithms analyse data from IoT sensors, enabling automated adjustments in lighting, HVAC, and security systems based on occupancy levels. Modern systems leverage big data and AI to optimise space usage and resource management, reducing energy consumption and promoting sustainability. Enhanced encryption...

The need to weave agility throughout the business

John Craig Swartz SVP at POWWR • 11th June 2024

With geopolitical tensions, more extreme weather events and the legacy of a global pandemic, it is more difficult for energy suppliers to preserve their margins and remain competitive than ever before. To thrive in the current climate, it is imperative that a supplier makes marginal gains wherever they can. Profitability within the sector today hinges...

Artificial general intelligence is closer than expected

AI expert Stuart Fenton • 10th June 2024

Whilst most of the attention around artificial intelligence (AI) thus far has been on ChatGPT, it is just the tip of the iceberg. In many ways, ChatGPT shouldn’t be thought of as true AI as it is – at its heart – just generative, learned behaviour. The future of AI, in contrast, is a system...

The State of Data Streaming

Confluent • 06th June 2024

Confluent survey: 90% of respondents say data streaming platforms can lead to more product and service innovation in AI and ML development 86% of respondents cite data streaming as a strategic or important priority for IT investments in 2024 For 91% of respondents, data streaming platforms are critical or important for achieving data-related goals

The State of Data Streaming

Confluent • 06th June 2024

Confluent survey: 90% of respondents say data streaming platforms can lead to more product and service innovation in AI and ML development 86% of respondents cite data streaming as a strategic or important priority for IT investments in 2024 For 91% of respondents, data streaming platforms are critical or important for achieving data-related goals

Grant Funding Awarded to Advance Cancer Therapeutics Discovery

Dr Alan Roth • 04th June 2024

The CRUK (Cancer Research UK) Scotland Institute and Oxford Drug Design, a biotechnology company with core expertise in AI drug discovery, have announced that their joint application for the MRC (UK Medical Research Council) National Mouse Genetics Network (NMGN) Business Engagement Fund has been awarded.