---
title: Build your perceptron neural net from scratch
date: 2017-08-12T16:45:01.889Z
edited: 2026-11-05T12:00:00.000Z
author: Ismail Ghallou (Smakosh)
canonical: https://smakosh.com/blog/build-your-perceptron
reading_time: 4 min (772 words)
Tags: ai
---

# Build your perceptron neural net from scratch

> In 1958, Cornell psychologist Frank Rosenblatt introduced the Perceptron, one of the first artificial neural networks that could learn from data.

Let’s start by defining the terms first: AI (Artificial intelligence), ANN (Artificial neural networks), Machine learning & Deep learning.

## AI

> AI is the area of computer science that builds systems able to perform tasks we associate with intelligence, such as recognizing patterns, understanding language and making decisions.

## ANN

> An ANN, or neural net, is a computing system loosely inspired by the biological neural networks in our brains, even though the two work very differently.

![ANN vs biological neuron](/assets/blog/nn.jpeg)

> On the left: a biological neuron. On the right: an artificial neural network

## Machine learning

> Machine learning is a field within AI that focuses on algorithms that learn patterns from examples, which we call “training data”, and then use them to make predictions on new input data.

## Deep learning

> Deep learning is a sub-field of machine learning with the same goal, using neural networks with many layers (deep neural networks).

![fields](/assets/blog/nvidia.png)

So what’s the perceptron then? The perceptron is an algorithm for supervised learning, which means we know the result we’re trying to get for every training example, like feeding the size & location of a house to predict its price. On the other hand, unsupervised learning draws conclusions from datasets made only of input data, without labeled responses.

A neuron receives input data, multiplies each input by a weight (random at first), adds a bias value and passes the sum through an activation function to get a result. Rosenblatt’s original perceptron used a step function as its activation and only changed its weights when it misclassified an example. In this article we’ll build its smooth cousin: a single neuron with a sigmoid activation, trained with gradient descent. After every example it measures how far the result is from the target and nudges the weights & bias in the direction that reduces that error.

![perceptron](/assets/blog/single_neuron.jpeg)

> It can be more than 2 inputs!

I know there are some terms there you didn’t get, so let’s go ahead and explain that slowly, let’s say you’re a farmer and you want to classify two types of flowers manually, the best way to do this manually is to take two measurements of each flower, like the length & width of its sepal, and plot them on a graph just like shown below:

![Iris Data](/assets/blog/iris_data.jpeg)

> Data was taken from <https://kaggle.com>

Notice the unknown flower (#55) whose type we’re trying to figure out: on the graph it lands among the Iris-versicolor flowers, which means it’s most likely an Iris-versicolor.

What you’ve just done manually is called linear classification: you plotted the flowers you already know as training data, noticed the pattern and drew a line (a decision boundary) between the two types. Don’t confuse it with linear regression, which predicts a continuous number such as a house price.

![Intuition](/assets/blog/intuition.jpeg)

So now let’s automate it with our own neuron built from scratch. The activation function we’ll be using is the sigmoid, which squashes any number into a value between 0 and 1, so we can read the output as “how likely is this an Iris-versicolor” (0 = Iris-setosa, 1 = Iris-versicolor). Here’s how it looks on a graph:

![Sigmoid function](/assets/blog/sigmoid.jpeg)

> Sigmoid function

Now let’s understand how gradient descent works. With a single neuron there’s only one layer, so “backpropagation” is simply the chain rule applied once. We will need:

- The cost function, also known as the squared error function

![cost function](/assets/blog/cost_function.jpeg)

- The derivative of the sigmoid function

![sigmoid derivative](/assets/blog/sigmoid_derivative.jpeg)

- The derivative of the cost function

![cost function derivative](/assets/blog/d_cost_function.jpeg)

- Let z = input_1 _ weight_1 + input_2 _ weight_2 + bias be the value we feed into the sigmoid. By the chain rule, the slope of the cost with respect to z is the derivative of the cost function multiplied by the derivative of the sigmoid evaluated at z.

- Learning rate: we will go with 0.2

- The derivative of z with respect to each weight is that weight’s input (the input is a constant when we vary its weight), and the derivative of z with respect to the bias is 1. Multiply each one by the slope above to get how much each weight & the bias contribute to the error.

- Then we get the new weights & bias values by taking a small step against that slope, like shown below:

![generating new weights & bias](/assets/blog/new_values.jpeg)

> Repeat this for many randomly picked examples and the weights settle on a line that separates the two types. The notebook does the same with all four measurements (sepal & petal length and width) instead of two, and classifies all 54 test flowers correctly.

Check the code on [Github](https://github.com/smakosh/Perceptron-neural-net-from-scratch/blob/master/Linear_regression_perceptron.ipynb) & [Kaggle](https://www.kaggle.com/smakosh/perceptron-neural-net-from-scratch/notebook)

---

_Originally published at https://smakosh.com/blog/build-your-perceptron_
