Use Python to Access Web Data


This post will discuss how to use Python to access web data.

  • Become familiar with the request and response cycle that your browser does to communicate with servers.
  • Become familiar with protocols that are happening when your browser is working to access data.
  • Know how to write Python programs that can access web data.

A Brief Discussion Regarding The Internet and Networking

The picture below describes the transport control protocol. It illustrates the basic method of how information goes back and forth from your computer and destination web servers.

TCP Protocol
The TCP layer of the network architecture serves to handle peer-to-peer connections between your computer and a web server.

Focus mainly on the transport layer of this architecture. This is the peer-to-peer connection between your computer and a web server. Think of it as a telephone call over the internet.

How The TCP Layer Relates to Python

When you talk to someone on your cell phone, you do not worry about how the connection is made. You simply become aware of the connection and start talking.

Use this cell phone cell phone analogy as a metaphor when making a socket inside your computer. A socket will allow Python to access web data.


When you talk to other applications on the internet, you have to know the specific port number of the application you wish to access. TCP port numbers allow multiple applications to exist on the same server.

You can think of port numbers as extensions within a phone number. There is an IP address, and within that are numbers for various applications that may exist on the same server.

Below is a picture of common TCP port numbers. The one you will use mostly with Python is port 80.

TCP Port Numbers
The most common TCP Port Number you will use with Python is 80.

The Python Socket Library

Python has a socket library that already contains all the code you need to access web data.

There are three lines of code to use when making a socket. These three lines accomplish the following:

  1. Import the library
  2. Establish a socket.
  3. Define the end server.
Sockets in Python
Use these three lines of code when you need to make a socket.
For more information get the book Introduction to Networking. You can also take an Internet History course.

Leave a Reply

Your email address will not be published. Required fields are marked *