The AI world of today and the future is based on data acquisition for continued learning. It is becoming increasingly “intelligent” thanks to machine learning with deep neural networks via the internet and the cloud. Then here comes Federated Learning, which allows our Artificial Intelligence to be trained without uploading data to servers. So, what is Federated Learning, and why should you care in a day and age where everything seems to be so reliant on the internet’s deep networks and databases?
Federated Learning in AI is important because of the privacy concerns still prevalent in today’s digital landscape. Federated Learning doesn’t require your personal data to be uploaded in the cloud or a server for Artificial Intelligence to train. As such, it is a safer and more private way for machines to learn.
Even though it seems personal and public information is pretty much all over the internet today, privacy is still very much a concern, especially when it comes to your personal information and other sensitive data. After all, even the cloud servers can get hacked, and if that doesn’t convince you, let us show you and talk more about why we need federated learning in our AI.
What is Federated Learning?
Federated Learning is still in its early stages and is a privacy-focused form of machine learning. Federated Learning enables devices to learn while keeping all the training data on the device. This machine learning technique alleviates the need to store all of its data on one machine or in the cloud or a data center.
This algorithmic structure is different than traditional consolidated machine learning systems where all the local data is stored on one server. Federated Learning is utilized on millions of heterogeneous devices for numerous applications.
In Federated Learning model training can use much less communication while allowing greater privacy and security. It also has the ability in certain applications to lower latency and utilize less power consumption.
Critical Features of Artificial Intelligence
Arguably the most critical feature of artificial intelligence or an AI is its ability to learn and adapt depending on the data that it gathers. In a sense, AI learns and trains itself using a lot of data that is pretty easy to access today, thanks to how the internet serves as a deep neural network that the AI in our computers and machines can use.
So, what happens is our computers, Smartphones, IoT, machines, and peripheral devices serve as terminals that allow them to connect to a computer server that uses AI algorithms to learn and adapt.
But do we always need to be connected to some server through the internet for our computers and devices to learn and train? Is it entirely necessary for machines to use servers as a way for them to continue to improve? Well, not necessarily, because that is where Federated Learning comes in.
For instance, in the past, most mobile phones, computers, and devices utilized fewer terminals that connect to a centralized mainframe or server, which does all of the thinking and data analysis. That’s why you needed to be connected to the internet for you to make use of Siri. But something changed during the latter part of the 2010s when newer powerful chips were introduced into smartphones, and the game changed.
So, thanks to the more powerful chips that our phones have today, machine learning has become a shared responsibility among powerful machines, computers, peripherals, and devices instead of leaving the computing over to the larger engines found in centralized servers.
The Definition of Federated
Think of Federated Learning as similar to how the U.S. Federal government functions. There is a centralized form of government that’s rested on the president’s powers. Federated is “connected and treated as one,” according to www.yourdictionary.com.
However, each state is allowed to have independence from one another and, to some degree, from the central government. But all of the states combined to form one nation regardless of how independently or ‘decentralized” they may act from one another. A mobile device that takes part in Federated Learning enjoys the same kind of “independence” that a state in a federal government wants.
What happens in Federated Learning is that a device downloads a model that is supposed to run on smartphones taking part in federated learning. The model runs locally and independently on that one device and then uses the data stored in that smartphone to learn and train all without relying on a centralized mainframe or server that it can connect to using the internet.
The update is sent over to the central server or the cloud, where it is averaged with the update collected from all of the other devices to improve the model’s overall performance. The model will then try to strengthen itself by collecting all the changes it learned from the different Federated Learning devices and then averages it before updating itself. And the essential part of it all is that the individual data that the model collected will not get sent over together with the update and will remain on personal smartphones or devices.
Why Do We Need Federated Learning?
Now that you know how Federated Learning works, you may wonder why we need to use Federated Learning instead of relying on the traditional machine learning models that use a centralized machine or mainframe that does all of the computing and analysis.
Well, for starters, the most crucial reason for relying on Federated Learning is that it keeps things private on your part. This is because of how your machine or your mobile device continues to learn and adapt without the need to make use of the data sent over to it from the cloud. In that sense, your smartphone doesn’t send over your personal data and information to a central server as only the updates on the phone are sent to it.
That means that what is meant to stay on your mobile device will remain on your mobile device. This ability allows anyone to enjoy using their smartphone or other devices while minimizing the risk of their personal data and information being leaked online. Especially now that we live in an age where cyber-attacks are becoming more common as hackers and cyber-criminals are getting smarter and better at what they do.
Federated Learning Saving Time and Resources
Another reason why we need Federated Learning is that AI learning happens faster because the mobile device no longer needs to wait on what the server tells it to do. This process is necessary for a mobile device to perform and act according to what it has learned through experience and by analyzing the individual data stored on the machine.
This training allows you to enjoy a faster-personalized experience with your smartphone as your mobile device will adapt to your lifestyle and needs without the need for it to wait on the central server. There is more personalization when it comes to Federated Learning since your smartphone will only be basing its updates on the data collected from you.
Of course, the goal is to realize less power consumption with some forms of Federated Learning because your mobile device won’t always have to rely on the network provided to it by a central server for it to train and learn. This change allows you to enjoy a more efficient mobile device in its performance and power consumption.
Federated Learning Applications
Federated Learning is still in the early phases because its learning approach for training deep networks was only introduced in 2016 by Google AI researchers in a paper titled “Communication-Efficient Learning of Deep Networks from Decentralized Data.” However, as we go deeper into the 2020s, the possibility of Federated Learning becoming commonplace in many different applications is growing thanks to how developers have been taking advantage of it.
Google itself has been testing Federated Learning in the application of the Google Keyboard on Android phones. The central premise of how Federated Learning is used in the keyboard is that your phone locally stores the data regarding any suggested query that appears whenever you are using Google Keyboard.
The proprietary information of the keyboard gets stored on the mobile device so that your phone will be able to improve its suggestion model in the future. So, in that regard, the device will learn and adjust its suggestions based on how you are using Google Keyboard and whether or not you are clicking on the recommendations previously suggested by your phone.
This algorithm allows your phone to function more efficiently personally as it learns and adapts to your lifestyle and needs without relying on a centralized server.
Federated Learning Future Uses
Digital Healthcare / Pharmaceuticals
Federated Learning has also found its way into digital health as patient privacy is quite important. In that sense, applications based on specific medical fields can use collaborative efforts using training algorithms shared by other devices to a central server without uploading and divulging sensitive patient data and information.
Companies like NVIDIA use Clara Train Federated Learning software that utilizes the SDK edge AI computing platform. Clara Train is used for AI applications in medical imaging, genetic analysis, and oncology, among other areas.
Pharmaceutical companies like Amgen, Astellas, AstraZeneca, Bayer, Merck KGaA, and Novartis have an operational Federated Learning predictive modeling platform for drug discovery. This machine learning process is still a work in progress as medical practitioners and developers themselves are still exploring the possibility of using Federated Learning in medicine to improve the safety and privacy of patient data.
Artificial Intelligence-Powered Voice Assistants
AI-Powered Voice Assistants like Google Assitant use Federated Learning with the voice recordings stored on user’s devices to help with modeling with their “Hey Google” detection. The Google Assistant learns how to adjust the modeling from the voice data it receives and then sends a condensed report of the data changes to Google’s servers.
Google and other companies such as Apple, NVIDIA, and IBM collect data summaries from many of their users in this manner to improve their products all of the time.
Peripheral Devices Like Phones, Watches and Fitness Trackers
Companies like Apple use Federate Learning with many of their Peripheral Devices like the iPhone, iPad, and Apple Watch. Statistical learning, data collection, and modeling use these devices while moving away from centralized data collection.
Companies using Federated Learning focus on privacy and protection against hacking and sensitive data disclosures.
Automotive Industry
The Automotive Industry will likely be using Federated Learning for its self-driving vehicles, with the constant menace of cybersecurity threats and the expansion of 5G and IoT edge devices into cars.
Numerous studies, scholarly papers, articles, and new platforms specifically address the automotive industry and federated learning. To privately access the volumes of user data and not have to load it from self-driving vehicles like onto servers and keep it in place will be priceless in the future.
Financial Industry
The financial industry relies heavily on AI for its capabilities, and the financial sector can become more accurate through Federated Learning. In an AI series published by the WeBank AI Group, Morgan & Claypool Publishing House, Federated Learning and its use in finance and computer vision are fundamental to the next generation of AI.
In traditional banking, fintech, credit scoring, and online payment platforms, the privacy concerns associated with the patriot act and private banking impose extreme limitations on AI. But with, the use of technologies like Federated Learning and leaving the user’s information in place instead of storing in a single server environment shows very much promise to these industries.
Federated Learning Limitations
As incredible as federated learning might sound when it comes to the future of AI in our mobile devices, there are still limitations that need to be addressed:
- Federated learning requires significant computing power on the part of the device taking part in it.
- There is a need for constant communication between nodes during the entire training, and you would also need high bandwidth connections due to such constant contact.
- Hackers may still be able to find their way into the training models in smartphones by using backdoors that will allow them to hijack the data found in individual mobile devices.
- There is no access to global training data, which will make it challenging to remove biases that shouldn’t even be taking part in the training data.
References:
- https://arxiv.org/pdf/1610.05492.pdf
- https://ai.oogleblog.com/2017/04/federated-learning-collaborative.htm
- https://arxiv.org/pdf/1909.11875.pdf
- Federated Learning for Mobile Keyboard Prediction – Google Research
- The future of digital health with federated learning
- A survey on federated learning – ScienceDirect
- [2106.06843] Federated Learning on Non-IID Data: A Survey
- (PDF) Deep Federated Learning for Autonomous Driving