What is the easiest way to find a threat (either phishing or spam) in your e-mail? A variety of technical headers and other indirect markers of an unwanted message can point the way, but we shouldn\u2019t forget the most obvious bit \u2014 the message text. One might think it\u2019s the first thing to analyze; after all, the text is what cybercriminals or unscrupulous advertisers use to manipulate recipients. The task isn\u2019t quite that simple, though; whereas signature analysis could cope with the task in the past, it is now necessary to analyze the text using machine-learning algorithms. And if the machine learning model is to be trained to classify messages correctly, it needs to be fed messages in significant quantities \u2014 and that is not always practical, for privacy reasons. We found a solution.<\/p>\n
Ten years ago, catching a huge proportion of unwanted e-mail based purely on message text was relatively easy because cybercriminals used the same templates \u2014 the text of spam (and phishing) messages hardly changed. Today, cybercriminals continually improve the efficiency of their mailings, and they use millions of hooks: new video games, TV series, or smartphone models; political news; even emergencies (take, for example, the abundance of phishing and spam related to COVID-19). The massive variety of topics complicates the detection process. Moreover, attackers can even vary the text within one mailing wave to elude e-mail filters.<\/p>\n
Of course, signature-based approaches are still in use, though their success basically relies on encountering text that someone has already classified as unwanted or harmful. They can\u2019t work proactively because spammers can bypass them by making changes to mailing text. The only way to deal with this problem is through machine learning.<\/p>\n
In recent years, machine-learning methods have shown good results in solving many problems. By analyzing a large amount of data, models learn to make decisions and find nontrivial common features in an information stream.\u00a0 We use neural networks trained on technical e-mail headers, together with DMARC, to detect e-mail threats. So, why can\u2019t we just do the same thing with message text?<\/p>\n
As mentioned above, models need a huge amount of data. In this case, the data consists of e-mails, and not only malicious ones \u2014 we need legitimate messages as well. Without them, teaching the model to distinguish an attack from legitimate correspondence would be impossible. We have numerous e-mail traps that catch all sorts of unwanted e-mails (we use them to make signatures) but obtaining legitimate letters for learning is a more complicated task.<\/p>\n
Typically, data is collected on servers for centralized learning. But when we are talking about text, additional difficulties arise: E-mails can contain private data, so storing and processing them in their original form would be unacceptable. So, how can we obtain a large enough collection of legitimate e-mails?<\/p>\n
We solve that problem by using the federated learning method, neatly eliminating the need to collect legitimate e-mails and instead training models in a decentralized way. Model training takes place directly on the client\u2019s mail servers, and the central server receives only the trained weights of the machine-learning models, not message text. At the central server, algorithms combine the data with the resulting version of the model, and then we send it back to client\u2019s solutions, where model again proceeds to analyze the stream of e-mails.<\/p>\n
That\u2019s a slightly simplified picture: Before the newly trained model is set loose on real letters, it goes through several iterations of additional training. In other words, two models work simultaneously on the e-mail server: one in training mode, the other in active mode. After several trips to the central server, the retrained model replaces the active one.<\/p>\n
It\u2019s impossible to recover the text of specific e-mails from the model weights; thus its privacy during processing is assured. Nevertheless, training on real e-mails significantly improves the detection model\u2019s quality.<\/p>\n