{"id":20981,"date":"2023-04-24T18:47:58","date_gmt":"2023-04-24T14:47:58","guid":{"rendered":"https:\/\/me-en.kaspersky.com\/blog\/neural-networks-data-leaks\/20981\/"},"modified":"2023-04-24T18:48:05","modified_gmt":"2023-04-24T14:48:05","slug":"neural-networks-data-leaks","status":"publish","type":"post","link":"https:\/\/me-en.kaspersky.com\/blog\/neural-networks-data-leaks\/20981\/","title":{"rendered":"How AI can leak your private data"},"content":{"rendered":"

Your (neural) networks are leaking<\/h2>\n

\nResearchers at universities in the U.S. and Switzerland, in collaboration with Google and DeepMind, have published a paper<\/a> showing how data can leak from image-generation systems that use the machine-learning algorithms DALL-E<\/a>, Imagen<\/a> or Stable Diffusion<\/a>. All of them work the same way on the user side: you type in a specific text query \u2014 for example, \u201can armchair in the shape of an avocado\u201d \u2014 and get a generated image in return.<\/p>\n

\"Image<\/a>

Image generated by the Dall-E neural network. Source<\/a>.<\/p><\/div>\n

All these systems are trained on a vast number (tens or hundreds of thousands) of images with pre-prepared descriptions. The idea behind such neural networks is that, by consuming a huge amount of training data, they can create new, unique images. However, the main takeaway of the new study is that these images are not always so unique. In some cases it\u2019s possible to force the neural network to reproduce almost exactly an original image previously used for training. And that means that neural networks can unwittingly reveal private information.<\/p>\n

\"Image<\/a>

Image generated by the Stable Diffusion neural network (right) and the original image from the training set (left). Source<\/a>.<\/p><\/div>\n

More data for the \u201cdata god\u201d<\/h2>\n

\nThe output of a machine-learning system in response to a query can seem like magic to a non-specialist: \u201cwoah \u2013 it\u2019s like an all-knowing robot!\u201d! But there\u2019s no magic really\u2026<\/p>\n

All neural networks work more or less in the same way: an algorithm is created that\u2019s trained on a data set \u2014 for example a series of pictures of cats and dogs \u2014 with a description of what exactly is depicted in each image. After the training stage, the algorithm is shown a new image and asked to work out whether it\u2019s a cat or a dog. From these humble beginnings, the developers of such systems moved on to a more complex scenario: the algorithm trained on lots of pictures of cats creates an image of a pet that never existed on demand. Such experiments are carried out not only with images, but also with text, video and even voice: we\u2019ve already written about the problem of deepfakes<\/a> (whereby digitally altered videos of (mostly) politicians or celebrities seem to say stuff they never actually did).<\/p>\n

For all neural networks, the starting point is a set of training data: neural networks cannot invent new entities from nothing. To create an image of a cat, the algorithm must study thousands of real photographs or drawings of these animals. There are plenty of arguments for keeping these data sets confidential. Some of them are in the public domain; other data sets are the intellectual property of the developer company that invested considerable time and effort into creating them in the hope of achieving a competitive advantage. Still others, by definition, constitute sensitive information. For example, experiments are underway to use neural networks to diagnose diseases based on X-rays and other medical scans. This means that the algorithmic training data contains the actual health data of real people, which, for obvious reasons, must not fall into the wrong hands.\n<\/p>\n

Diffuse it<\/h2>\n

\nAlthough machine-learning algorithms look the same to the outsider, they are in fact different. In their paper, the researchers pay special attention to machine-learning diffusion models<\/em>. They work like this: the training data (again images of people, cars, houses, etc.) is distorted by adding noise. And the neural network is then trained to restore such images to their original state. This method makes it possible to generate images of decent quality, but a potential drawback (in comparison with algorithms in generative adversarial networks<\/a>, for example) is their greater tendency to leak data.<\/p>\n

The original data can be extracted from them in at least three different ways: First, using specific queries, you can force the neural network to output \u2014 not something unique, generated based on thousands of pictures \u2014 but a specific source image. Second, the original image can be reconstructed even if only a part of it is available. Third, it\u2019s possible to simply establish whether or not a particular image is contained within the training data.<\/p>\n

Very often, neural networks are\u2026 lazy<\/em>, and instead of a new image, they produce something from the training set if it contains multiple duplicates of the same picture. Besides the above example with the Ann Graham Lotz photo, the study gives quite a few other similar results:<\/p>\n

\"Odd<\/a>

Odd rows: the original images. Even rows: images generated by Stable Diffusion v1.4. Source<\/a>.<\/p><\/div>\n

If an image is duplicated in the training set more than a hundred times, there\u2019s a very high chance of its leaking in its near-original form. However, the researchers demonstrated ways to retrieve training images that only appeared once in the original set. This method is far less efficient: out of five hundred tested images, the algorithm randomly recreated only three of them. The most artistic method of attacking a neural network involves recreating a source image using just a fragment of it as input.<\/p>\n

\"The<\/a>

The researchers asked the neural network to complete the picture, after having deleted part of it. Doing this can be used to determine fairly accurately whether a particular image was in the training set. If it was, the machine-learning algorithm generated an almost exact copy of the original photo or drawing. Source<\/a>.<\/p><\/div>\n

At this point, let\u2019s divert our attention to the issue of neural networks and copyright.\n<\/p>\n

Who stole from whom?<\/h2>\n

\nIn January 2023, three artists sued<\/a> the creators of image-generating services that used machine-learning algorithms. They claimed (justifiably) that the developers of the neural networks had trained them on images collected online without any respect for copyright. A neural network can indeed copy the style of a particular artist, and thus deprive them of income. The paper hints that in some cases algorithms can, for various reasons, engage in outright plagiarism, generating drawings, photographs and other images that are almost identical to the work of real people.<\/p>\n

The study makes recommendations for strengthening the privacy of the original training set:\n<\/p>\n