Weather prediction using Machin Learning (ML) in Python

 

With a low budget, you cant create your 1st  AI/ML project, and in this blog, I will guide you throw how to create an AI weather forecasting prototype.

General Information before we start:

I.Preface of AI

II.Machine learning 5 core stages

I.Preface of AI:

AI is a simulation of human intelligence in a machine that acts like humans and mimics their actions.

Examples for famous AI:

Sofia the robot & HAL in HAL spaceship, Self-driving cars, Google assistant

Google-powered AI- predictions, Ridesharing App.s Uber & Careem...etc

There are three main types of AI:

1.Narrow AI:-

Narrow AI is mainly designed to perform one task at a time, which is referred to as ''weak AI'' also due to the fact that it focuses on one task, to simply improve something that already works, but can work better performed by AI.

Examples for it, probably all the existing AI systems.

2.General AI:-

General AI or as it's called Strong AI; is the purpose we still conduct research on AI, it's designed to think like humans and make decisions on its own without previous training, in other words, the machine teaches itself by itself. 

3.Conscious AI:-

Artificial consciousness (AC), also known as machine consciousness (MC) or synthetic consciousness, is a field related to AI and cognitive robots. The aim of the theory of artificial consciousness is to "Define that which would have to be synthesized were consciousness to be found in an engineered artifact".[1]

How to create your AI prototype:

1.Define the problem you want to solve using AI

2.Design and build your system, you can start with mind mapping your idea.

3.Collect your data and make sure it's enough, alongside downloading the dataset to improve your training.

4.Prepare your data

5.Decide the form of output from your AI system.

6.Choose the right algorithms to train your model.

7.Don't forget to be economizing as much as you can.👀

8.Choose an easy IDE to start programming, and of course an easy programming language.

...Ok after doing all of that how will you start? 

before we get to the exciting part 😁 you shall know how will you train your dragon ..oops your model😅

There are four types of learning:

1.Supervised

2. Unsupervised

3. Semi-supervised

4. Reinforced

1. Supervised Learning:

Is done by exposing the AI to data it's output is already known.

Your algorithm uses the output to decide either the answer is right or not.

2. Unsupervised Learning:

This type of learning is used to teach an AI with data that you don't know its output.

By exposing the algorithm to the data and let it decide what it sees by itself.

The system is not given a pre-determined set of outputs or correlations between inputs and outputs or a “correct answer.”

3. Semi-supervised Learning:

This type of learning falls between the 1st two types of learning and the learning process is done by using a dataset that half/some of it have a predetermined output and the other half doesn't have a pre-determined output.

4. Reinforced Learning:

It's a type of dynamic programming that uses reward and punishment as a learning technique. This learning algorithm learns only by interacting with the right environment,  It receives rewards by performing correctly and penalties for doing so incorrectly.

🚩 Mostly the learning type that is used is Supervised learning because we still didn't reach the strong AI that can teach itself by itself.

So Let's get to the model,...😃

The model target is to create a weather forecasting AI with different algorithms and compare the results.

II. Machine learning 5 core stages:

1. Get Data/Data collection:

Whatever means you choose to get your data you are free, also as I said before it's better to add another dataset pre-downloaded from websites that makes their dataset available for geeks 😁

If you want to know more about datasets and what they are and where to find them 

check DataQuest.

2. Clean & prepare your data/ Preprocessing:

Data preprocessing is the most basic important step actually in the whole process of creating your AI. Preprocessing means you prepare your data for the next step which is training, so what do you need to do?... well you start by eliminating the parameters you don't need in your dataset, then..import the libraries you will use, after importing libraries you need to also import the dataset as a CSV or "comma-separated value" then define the dependent and independent variables in our dataset, mostly they are referred to as X & Y. And then you start identifying and handling the missing values the last step is encoding the categorical data, which the information that has specific categories within the dataset.

3. Train your model:

Training your model is basically the core of the AI learning process, as the more you train and test differently each time the more intelligent AI you get.

And by train and test differently each time I mean, you feed different input data to your model. In this way, your model will build a learning pattern or better say thinking pattern finally the prediction of your model will be the closest to the correct value.

Training is done by feeding your input data to your algorithm to get an output.

4. Testing model:

In the testing stage, you start to give your AI model a new input and see how correct the answer will be if the answer was correct then the model is complete and your mission is accomplished.

5. Improving the model:

After all, an AI is a mimic of the human brain, it needs to be trained more and tested on new projects and ideas to improve its performance.

Also, there are more useful ways to improve the AI model accuracy I recommend you look at this useful article.

Ways to improve the accuracy of your model

Ok so this project is composed of two parts, 1st the hardware part where you collect your data, and the 2nd part is where you connect it to the AI.

this is a simple block diagram of the hardware:

______________________________

How can you plan it

1. The temperature/Humidity sensor will read the data which is the two physical parameters Temperature & Humidity.
2. Node MCU will read it, where I can read them and create a dataset for a      good period of time like for a month.
3. Save the dataset in an XLs sheet and then to programming the AI.  Or as mentioned before you can train using a separate dataset.
4. The IoT platform I used is Thing Speak.
5. AI model sends a request to Thing Speak to get the last reading, then make the prediction.

Hardware part:

It's the easiest part, you will start with connecting the DHT sensor to the node

 

Image from: "Electronics wings"

• VCC- 3.3 VCC
• GND-GND
• data - D2 "Any digital pin you wish"

💬Then you can start programming using whatever IDE you prefer if you are familiar with programming node MCU and sensors, and if you are not I recommend you start with Arduino IDE it's easy peasy lemon squeezy.

Programming The Node MCU (WiFi module)

For programming Node MCU there are different tools and languages to program it, but the easiest is Arduino IDE as it's user-friendly.

code explanation on github.

https://github.com/ZienabEsam/Weather-Forecasting-using-ML-and-Node-MCU-8266.

The most important part of the sensors reading is sending the post request to Thing Speak the IoT platform, for more details about how to create your channel see this link👇 

Matlab.

To send a request to a website you either GET data or POST data,

a request is composed of three main lines:

"as I am writing in  C language"

1.     client.print("POST /update HTTP/1.1\n");
2.     client.print("Host: api.thingspeak.com\n");
3.     client.print("Connection: close\n");

1st line contains what is the purpose of your request and that is more than a GET or POST request it can be more than this you can read more about it under "Web requests". then the IP version 1.1.

2nd line is the hostname

3rd line is what do you want to do with your connection after the request is complete.

AI Model programing

For programming the AI model I chose JupyterLab it's based on Jupyter Notebook, I also used conda prompt as the package manager, it's one of Anaconda project tools.

For installing any of these tools the links are inserted above👆.

So let's install the libraries we need,

you will need these libraries for your model to complete the 5 main stages,

1. Pandas, for data manipulation.
2. Numpy, for working with arrays.
3. Tkinter, for building the GUI.
4. Sklearn, for the training algorithms.
5. Matplotlib, for plotting the results and it's based on MatLab.
6. Thingspeak, to get the latest reading from the sensors.

Let's talk a bit about how will you code,

1st you have to import the libraries we mentioned before, but more specifically you have to know what training algorithm you will use,

like if the task you want your AI to do is a continuous predictions you will use a regressor algorithm like predicting weather or annual earning or in Gaming, but you have to use classifier algorithm for finding/predicting the label of the class.

I used the Ensemble training algorithm for the better prediction,

basically ensembling technique is based on using mutible training algorithm and making a better prediction.

Ensemble technique capable of performing both regression and classification tasks with the use of multiple decision trees and a 

technique called Bootstrap Aggregation, commonly known as bagging.

*Bagging is training each decision tree on diffrent data sample.

Algorithms I used for ensemble,

1.Decision tree regresion.

2.K-neerest neighbor (K-NN) regressor.

3.Randome Forest Regression.

4.Linear regression.

Connecting AI model to IoT platform 

As mentioned before you have to know what is the read and write keys to your channel on ThingSpeak, in the code

you have to define the read key of your channel and import ThingSpeak liberary in python code.

Complete Code on Git Hub:

https://github.com/ZienabEsam/Weather-Forecasting-using-ML-and-Node-MCU-8266